bacaf3d39c
El SDK CI - dev / build-and-test (pull_request) Failing after 4m49s
Lands feat/reframe-region-setop (PR #109: native set-based reframe_region, decorator-as-seam @route port, teacher-summon, and the M8.1 activate-latency work — lazy-memoized cosq via eg_cosq_at + engram_vindex HNSW-accelerated seed discovery + vindex_harvest_from_store/vindex_bench oracle) onto dev's actual current HEAD, plus engram-tiered-storage's still-unique test suite. RECONCILING #109 WITH engram-tiered-storage (M4-M10 HNSW/geometry/reason/ verify work): not a two-way merge. engram_vindex.c's HNSW core (search_layer/ select_neighbors/prune_links/insert) is BYTE-IDENTICAL between the two branches; #109's copy is a strict superset (adds vindex_harvest_from_store, used by vindex_bench.c's brute-force-vs-HNSW oracle). engram_reason.c and engram_verify.c are also byte-identical. #109's own branch point already carried engram-tiered-storage's M4-M10 lineage forward, so there was nothing left to merge into #109 for those files. The one thing engram-tiered-storage had that #109's tree dropped: its full test suite (test_vindex.c, test_geometry.c, test_reason.c, test_verify.c, test_m7_traversal.c, the interoception P0-P5 tests, bufpool/compaction tests, and their run_*.sh harnesses) — ported over here unchanged. WHY THIS NEEDED HAND RECONCILIATION, NOT A MECHANICAL MERGE: #109's branch forked from dev on 2026-08-14 15:40 (before restructure-adjacent history diverged the file's merge-base for `git merge` — it presented as an add/add conflict). A straight two-dot diff (dev tip -> PR tip) applied cleanly, but it silently reverted THREE dev fixes landed on 2026-08-14/15, after the branch point, that the PR's diff had no way to know about: 1. qgate rescale (2026-08-14 self-review): PR's lazy eg_cosq_at rewrite of the query-aware propagation gate dropped the shift-and-floor rescale about ENGRAM_EMBED_S0 (measured: unrelated-pair median 0.562->raw gate 0.67, i.e. "a small tax, not a gate"). Restored the rescale, wrapped around the lazy accessor -- the PR's actual improvement (WHEN cosq[oi] is computed) is orthogonal to WHAT it gates on and both are kept. 2. Eviction cause decomposition (2026-08-14 self-review): dev decomposes wm_evicted into evict_floor/evict_cap/evict_bll so WM churn is diagnosable (identity: evicted == floor+cap+bll+dup_wm+dup_wm_global). PR's tree predates this and dropped all three counters + their JSON stats fields. Restored declarations, all 4 direct increment sites, the eg_wm_carry_over bll increment, and the act-stats JSON fields -- alongside (not instead of) the PR's own P4 afferent / API-reshape counters already in that same struct/JSON. 3. Hebbian link-formation selection (2026-08-15 self-review, TODAY): dev selects the STRONGEST qualifying candidate for consolidation each call; PR's tree predates this and reverted to hash-slot order (arbitrary wrt association strength) for edge formation -- the one path that writes PERMANENT structure. Restored the strongest-candidate while-loop, keeping the PR's own genuine improvement at that site (engram_adj_on_edge_added incremental-index append instead of a bare adj_dirty=1 full-rebuild flag). engram/src/server.el's 3-way conflicts (autoconnect_on/ise_offgraph_on env flags, /api/nodes connected-count in responses) were pure additive: dev's side was empty, PR's side added the feature. Took PR's side whole. VERIFIED (nsbx sandbox only, live :8742/:7770 never touched): - cc -std=c11 -O2, clean link against the real engram/src/server.el via elc, zero errors. - vindex_bench (built standalone, read-only harvest) against the real production store clone (13,671 embedded nodes, 768-dim nomic-embed-text): recall@10 = 1.0000 at ef 64/128/200; HNSW search 0.28-0.79ms/query vs 2.03ms/query brute-force oracle (2.6x-7.2x). HNSW build itself: 46.5s for the full 13,671-node set -- see the flagged risk below. - Booted the reconciled binary in an isolated nsbx sandbox (:8905, cloned snapshot of the live store, 13,424 nodes / 37,656 edges) and called /api/activate for real: first call after boot 41.5s (pays the one-time HNSW build inline -- matches the standalone bench), second/third calls 356ms/605ms, no crash, correct results, act-stats JSON (including the restored evict_floor/cap/bll fields) reads correctly. KNOWN RISK TO FLAG BEFORE ANY LIVE CUTOVER (not fixed here; out of scope for this dev-only land per instructions not to touch :8742/:7770): eg_vindex_sync builds the HNSW index synchronously, inline, on the first engram_activate() call after every process start (or index invalidation). On the real node count that is a ~46s blocking stall on a single-threaded server -- the first request after every restart (or its concurrent siblings) waits the full build. Recommend a background/incremental build (or a bounded per-call build budget) before this ever reaches the live daemon. See PR description / final report for the fuller writeup.
1924 lines
97 KiB
EmacsLisp
1924 lines
97 KiB
EmacsLisp
// server.el — Engram HTTP server.
|
||
//
|
||
// Engram is the in-process graph store. The runtime owns the data; this
|
||
// file is the thin HTTP face. Every route maps to one or two engram_*
|
||
// builtins. There is no SQL, no db layer, no SQLite — the runtime IS the
|
||
// database.
|
||
//
|
||
// Built and linked with:
|
||
// elc src/server.el > server.c
|
||
// cc -std=c11 -O2 -lcurl -lpthread -o engram server.c el_runtime.c
|
||
// ./engram
|
||
//
|
||
// Configuration via environment:
|
||
// ENGRAM_BIND — host:port (default :8742)
|
||
// ENGRAM_API_KEY — bearer auth (optional)
|
||
// ENGRAM_DATA_DIR — snapshot location (default ~/.neuron/engram)
|
||
|
||
// ── Helpers ───────────────────────────────────────────────────────────────────
|
||
|
||
fn parse_port(bind: String) -> Int {
|
||
// ":8742" → 8742; "0.0.0.0:8742" → 8742; bare "8742" → 8742
|
||
let colon: Int = str_index_of(bind, ":")
|
||
if colon < 0 {
|
||
return str_to_int(bind)
|
||
}
|
||
let after: String = str_slice(bind, colon + 1, str_len(bind))
|
||
return str_to_int(after)
|
||
}
|
||
|
||
fn ok_json() -> String {
|
||
"{\"ok\":true}"
|
||
}
|
||
|
||
fn err_json(msg: String) -> String {
|
||
"{\"error\":\"" + msg + "\"}"
|
||
}
|
||
|
||
fn strip_query(path: String) -> String {
|
||
let q: Int = str_index_of(path, "?")
|
||
if q < 0 { return path }
|
||
str_slice(path, 0, q)
|
||
}
|
||
|
||
fn query_param(path: String, key: String) -> String {
|
||
let q: Int = str_index_of(path, "?")
|
||
if q < 0 { return "" }
|
||
let qs: String = str_slice(path, q + 1, str_len(path))
|
||
let needle: String = key + "="
|
||
let pos: Int = str_index_of(qs, needle)
|
||
if pos < 0 { return "" }
|
||
let after: String = str_slice(qs, pos + str_len(needle), str_len(qs))
|
||
let amp: Int = str_index_of(after, "&")
|
||
if amp < 0 { return after }
|
||
str_slice(after, 0, amp)
|
||
}
|
||
|
||
fn query_int(path: String, key: String, default_val: Int) -> Int {
|
||
let v: String = query_param(path, key)
|
||
if str_eq(v, "") { return default_val }
|
||
str_to_int(v)
|
||
}
|
||
|
||
// Extract last path segment after a known prefix: extract_id("/api/nodes/abc-123", "/api/nodes/") → "abc-123"
|
||
fn extract_id(path: String, prefix: String) -> String {
|
||
let clean: String = strip_query(path)
|
||
if !str_starts_with(clean, prefix) { return "" }
|
||
let after: String = str_slice(clean, str_len(prefix), str_len(clean))
|
||
let slash: Int = str_index_of(after, "/")
|
||
if slash < 0 { return after }
|
||
str_slice(after, 0, slash)
|
||
}
|
||
|
||
// ── Routes ────────────────────────────────────────────────────────────────────
|
||
|
||
fn route_stats(method: String, path: String, body: String) -> String {
|
||
engram_stats_json()
|
||
}
|
||
|
||
// route_act_stats — GET /api/act-stats
|
||
// (2026-08-04 self-review) engram_act_stats_json() has existed since the
|
||
// 2026-07-27 review but was reachable ONLY through the soul daemon's heartbeat
|
||
// binding. Every activation-layer gauge — WM evictions, breakthroughs, embedder
|
||
// breaker state, context drift, and now the Hebbian counters — was therefore
|
||
// invisible unless the soul happened to be running and its ISEs were read back
|
||
// out of the store. Diagnosing the activation layer required a working soul,
|
||
// which is exactly backwards: the lower layer should be observable on its own.
|
||
// This review needed it to verify link formation and could not get at it. One
|
||
// line of plumbing, and the whole activation layer becomes directly diagnosable.
|
||
fn route_act_stats(method: String, path: String, body: String) -> String {
|
||
engram_act_stats_json()
|
||
}
|
||
|
||
// route_text_health — GET /api/text-health
|
||
// (2026-08-08 self-review) The daily census half of the text-integrity gauge.
|
||
// Today's review found that the JSON parser had been replacing every \uXXXX
|
||
// escape with a literal '?' for at least two months: 3,119 of 4,081
|
||
// non-telemetry nodes (76%) were damaged, including the self traversal root
|
||
// and every values node, and NOTHING detected it — because every gauge in the
|
||
// system measured whether the machinery was running, and none measured whether
|
||
// the text it carried was intact. No snapshot on disk predates the damage, so
|
||
// it cannot be undone; it can only be made impossible to repeat quietly.
|
||
//
|
||
// The parser is fixed. This route is the standing check: `damaged` should now
|
||
// hold flat at its historical floor and never climb. `write_damaged` (also on
|
||
// the heartbeat as txt_damaged) is the live regression signal — non-zero means
|
||
// a write path is mangling text right now.
|
||
fn route_text_health(method: String, path: String, body: String) -> String {
|
||
engram_text_health_json()
|
||
}
|
||
|
||
// (2026-07-18 self-review) Scoping sweep: `let` inside an if-block creates an
|
||
// inner scope only — it does NOT mutate the outer binding (documented with
|
||
// evidence in awareness.el, 2026-05-25). Every default/reassignment below used
|
||
// that broken pattern, so defaults never applied: nodes were created with
|
||
// node_type="" and salience=0.0, /api/search and /api/activate ALWAYS ran with
|
||
// q="" regardless of input, edges defaulted to relation=""/weight=0.0, and
|
||
// save/load with no "path" hit engram_save(""). Rewritten to the
|
||
// `let x = if cond { a } else { b }` expression form (the pattern the newer
|
||
// routes route_emit_ise/route_capture_knowledge already use correctly).
|
||
// store_on — ENGRAM_STORE flag (tiered paged store as the durable owner). Matches
|
||
// engram_store_enabled() in el_runtime.c EXACTLY (1 / on / true). Default off →
|
||
// every persistence path below is byte-for-byte the historical snapshot behavior.
|
||
fn store_on() -> Bool {
|
||
let v: String = env("ENGRAM_STORE")
|
||
if str_eq(v, "1") { return true }
|
||
if str_eq(v, "on") { return true }
|
||
if str_eq(v, "true") { return true }
|
||
return false
|
||
}
|
||
|
||
// persist_canonical — save the canonical snapshot after a durable write.
|
||
//
|
||
// WHY (2026-07-22 self-review): the 2026-07-21 fix correctly stopped READ
|
||
// routes from writing the canonical snapshot.json — but nothing was left
|
||
// that saved it on WRITE. Every mutation (node create, edge create,
|
||
// knowledge capture, forget, merge) lived only in RAM until someone POSTed
|
||
// /api/save manually; a process restart silently discarded everything since
|
||
// the last manual save. Observed live: two engram restarts during the
|
||
// 2026-07-22 review reverted the store to a ~17h-old snapshot, destroying
|
||
// same-day writes. Reads must never write the canonical; writes must always
|
||
// persist it. ISE telemetry is deliberately excluded (48h-pruned, loss-
|
||
// tolerant, ~2/min — snapshotting the whole store per heartbeat is waste;
|
||
// any durable write that follows persists the pruning too).
|
||
fn persist_canonical() -> Int {
|
||
// ENGRAM_STORE: the paged store is the durable owner — a checkpoint flushes
|
||
// dirty pages behind a WAL-durable record (durable the moment the WAL fsyncs).
|
||
// This is the fix for the "restart reverted to a 17h-old snapshot" data loss:
|
||
// durable writes no longer depend on a full snapshot.json rewrite. Returns 1
|
||
// on a successful checkpoint, 0 otherwise. Flag-off: unchanged (writes JSON).
|
||
if store_on() {
|
||
return engram_store_checkpoint()
|
||
}
|
||
let dir_raw: String = env("ENGRAM_DATA_DIR")
|
||
let dir: String = engram_resolve_data_dir()
|
||
// (2026-08-10 self-review) This returned a hardcoded 1, which made every
|
||
// caller's `let saved: Int = persist_canonical()` a dead variable — six
|
||
// durable write paths each believed they had confirmation of a successful
|
||
// canonical persist and none of them had any. Propagate the real result.
|
||
return engram_save(dir + "/snapshot.json")
|
||
}
|
||
|
||
// ── WAL persistence (design doc §§3-14; gated behind ENGRAM_WAL=on) ──────────
|
||
// Default OFF → every persist path below is byte-identical to the historical
|
||
// per-write full-snapshot behavior. When ON, structural mutations append O(1)
|
||
// WAL records instead of rewriting the whole graph, with threshold compaction.
|
||
fn wal_on() -> Bool {
|
||
str_eq(env("ENGRAM_WAL"), "on")
|
||
}
|
||
|
||
// autoconnect_on — ENGRAM_AUTOCONNECT. Will's rule: "we shouldn't be inserting
|
||
// orphaned nodes." When ON, every content-node insert forms >=1 semantic-similar
|
||
// edge (kNN over embeddings) so no content node enters the graph edgeless.
|
||
// Default OFF -> byte-identical to prior behavior (node created, no auto edges).
|
||
fn autoconnect_on() -> Bool {
|
||
let v: String = env("ENGRAM_AUTOCONNECT")
|
||
if str_eq(v, "1") { return true }
|
||
if str_eq(v, "on") { return true }
|
||
if str_eq(v, "true") { return true }
|
||
return false
|
||
}
|
||
|
||
// ise_offgraph_on — ENGRAM_ISE_OFFGRAPH. The census showed ~8k of the ~8.9k
|
||
// orphans are InternalStateEvent telemetry (heartbeat/curiosity_scan/session-
|
||
// start), 100% edgeless by design. When ON, that telemetry is routed to a
|
||
// separate state-event log tier instead of the node graph. Default OFF -> ISEs
|
||
// remain graph nodes exactly as before (with 48h prune).
|
||
fn ise_offgraph_on() -> Bool {
|
||
let v: String = env("ENGRAM_ISE_OFFGRAPH")
|
||
if str_eq(v, "1") { return true }
|
||
if str_eq(v, "on") { return true }
|
||
if str_eq(v, "true") { return true }
|
||
return false
|
||
}
|
||
|
||
// Persist a single-node mutation (create / content-evolve / strengthen).
|
||
fn persist_node(id: String) -> Int {
|
||
if wal_on() {
|
||
let d: String = engram_resolve_data_dir()
|
||
let a: Int = engram_wal_node_put(d, id)
|
||
let c: Int = engram_wal_maybe_compact(d)
|
||
return a
|
||
}
|
||
return persist_canonical()
|
||
}
|
||
|
||
// Persist edges appended at index >= start (covers single-edge and batch).
|
||
fn persist_edges_since(start: Int) -> Int {
|
||
if wal_on() {
|
||
let d: String = engram_resolve_data_dir()
|
||
let a: Int = engram_wal_edges_since(d, start)
|
||
let c: Int = engram_wal_maybe_compact(d)
|
||
return a
|
||
}
|
||
return persist_canonical()
|
||
}
|
||
|
||
// Persist a Hebbian consolidation batch as ONE WAL record (single fsync, §5-B).
|
||
fn persist_hebb_batch(start: Int) -> Int {
|
||
if wal_on() {
|
||
let d: String = engram_resolve_data_dir()
|
||
let a: Int = engram_wal_hebb_batch(d, start)
|
||
let c: Int = engram_wal_maybe_compact(d)
|
||
return a
|
||
}
|
||
return persist_canonical()
|
||
}
|
||
|
||
// Bulk mutation (embedding backfill, load-merge): write a fresh compaction base
|
||
// so the many-node change is durable in one atomic snapshot; WAL is truncated.
|
||
fn persist_bulk() -> Int {
|
||
if wal_on() {
|
||
let d: String = engram_resolve_data_dir()
|
||
return engram_wal_compact(d)
|
||
}
|
||
return persist_canonical()
|
||
}
|
||
|
||
// INCOMPLETE-ROUTE FIX (2026-07-24 self-review): this route silently dropped
|
||
// label, importance, tier, and tags — engram_node() defaults label to content
|
||
// and importance to 0.5, so every node created over HTTP lost its metadata.
|
||
// Observed live: the soul's boot-counter write-back landed with
|
||
// label="soul:boot_count:99" (content), importance 0.5, no tags. Honor the
|
||
// full field set via engram_node_full when any of them is supplied.
|
||
// PRESENCE-AWARE DEFAULTS (2026-08-01 self-review): the old pattern
|
||
// `if x == 0.0 { default }` made a legitimate 0.0 unrepresentable — a caller
|
||
// setting salience/importance/weight to zero silently got 0.5. json_get_raw
|
||
// returns "" when the key is ABSENT and the raw token when present, so
|
||
// absence and zero are now distinguishable. Also: confidence was hardcoded
|
||
// to 1.0 regardless of input — every HTTP-created node claimed full
|
||
// epistemic confidence. Now honored from the payload (default 1.0).
|
||
fn route_create_node(method: String, path: String, body: String) -> String {
|
||
let content: String = json_get_string(body, "content")
|
||
let nt_raw: String = json_get_string(body, "node_type")
|
||
let node_type: String = if str_eq(nt_raw, "") { "Memory" } else { nt_raw }
|
||
let sal_present: String = json_get_raw(body, "salience")
|
||
let salience: Float = if str_eq(sal_present, "") { 0.5 } else { json_get_float(body, "salience") }
|
||
let label_raw: String = json_get_string(body, "label")
|
||
let label: String = if str_eq(label_raw, "") { content } else { label_raw }
|
||
let imp_present: String = json_get_raw(body, "importance")
|
||
let importance: Float = if str_eq(imp_present, "") { 0.5 } else { json_get_float(body, "importance") }
|
||
let conf_present: String = json_get_raw(body, "confidence")
|
||
let confidence: Float = if str_eq(conf_present, "") { 1.0 } else { json_get_float(body, "confidence") }
|
||
let tier_raw: String = json_get_string(body, "tier")
|
||
let tier: String = if str_eq(tier_raw, "") { "Working" } else { tier_raw }
|
||
let tags: String = json_get_string(body, "tags")
|
||
// NO el_from_float WRAPPER (2026-08-01 self-review): salience/importance/
|
||
// confidence are already Float (el_val_t) values — json_get_float and
|
||
// Float literals both encode. Wrapping them in el_from_float AGAIN
|
||
// reinterpreted the boxed bits as a raw double, producing garbage that
|
||
// failed engram_decode_score's range check and clamped every HTTP-created
|
||
// node to defaults (salience 0.9 in → 0.5 stored; confidence 0.6 in → 1.0
|
||
// stored — verified live). route_emit_ise always passed Floats bare and
|
||
// its 0.3/0.3/0.8 stored correctly; this call now does the same.
|
||
let id: String = engram_node_full(
|
||
content, node_type, label,
|
||
salience, importance, confidence,
|
||
tier, tags
|
||
)
|
||
let saved: Int = persist_node(id)
|
||
// ORPHAN PREVENTION (ENGRAM_AUTOCONNECT): connect the fresh node to its
|
||
// nearest embedded neighbors so it never enters the graph edgeless.
|
||
let connected: Int = if autoconnect_on() {
|
||
let ec0: Int = engram_edge_count()
|
||
let ac: String = engram_autoconnect_node(id, 3, 25)
|
||
let added: Int = engram_edge_count() - ec0
|
||
if added > 0 { let sv2: Int = persist_edges_since(ec0) }
|
||
added
|
||
} else { 0 }
|
||
"{\"id\":\"" + id + "\",\"content\":\"" + content + "\",\"node_type\":\"" + node_type + "\",\"connected\":" + int_to_str(connected) + "}"
|
||
}
|
||
|
||
fn route_get_node(method: String, path: String, body: String) -> String {
|
||
let id: String = extract_id(path, "/api/nodes/")
|
||
if str_eq(id, "") { return err_json("missing id") }
|
||
return engram_get_node_json(id)
|
||
}
|
||
|
||
// route_get_node_singular — GET /api/node/<id>. Singular alias for node-by-id
|
||
// fetch. The plural /api/nodes/<id> already resolves; the viz's "see full node
|
||
// value on click" and other clients call the SINGULAR form, which had no route
|
||
// and 404'd for every id. Same handler, singular prefix. Read-only.
|
||
fn route_get_node_singular(method: String, path: String, body: String) -> String {
|
||
let id: String = extract_id(path, "/api/node/")
|
||
if str_eq(id, "") { return err_json("missing id") }
|
||
return engram_get_node_json(id)
|
||
}
|
||
|
||
fn route_scan_nodes(method: String, path: String, body: String) -> String {
|
||
let limit: Int = query_int(path, "limit", 50)
|
||
let offset: Int = query_int(path, "offset", 0)
|
||
let nt: String = query_param(path, "node_type")
|
||
if str_eq(nt, "") {
|
||
return engram_scan_nodes_json(limit, offset)
|
||
}
|
||
return engram_scan_nodes_by_type_json(nt, limit, offset)
|
||
}
|
||
|
||
// route_scan_edges — bulk export of all edges as a JSON array. Implemented
|
||
// via engram_save → fs_read of a SCRATCH export path. (2026-07-21 self-review:
|
||
// previously this saved over the canonical snapshot.json on every GET — if the
|
||
// process ever booted with a partial/empty store, the first read request
|
||
// clobbered the good snapshot. Read routes must never write the canonical path.)
|
||
fn route_scan_edges(method: String, path: String, body: String) -> String {
|
||
let dir_raw: String = env("ENGRAM_DATA_DIR")
|
||
let dir: String = engram_resolve_data_dir()
|
||
let snap_path: String = dir + "/.scan-export.json"
|
||
engram_save(snap_path)
|
||
let snap: String = fs_read(snap_path)
|
||
if str_eq(snap, "") { return "[]" }
|
||
// json_get truncates at the first delimiter (no bracket depth tracking),
|
||
// so for the edges ARRAY value we need json_get_raw, which honors
|
||
// brackets and returns the full sub-JSON.
|
||
let edges: String = json_get_raw(snap, "edges")
|
||
if str_eq(edges, "") { return "[]" }
|
||
return edges
|
||
}
|
||
|
||
fn route_search(method: String, path: String, body: String) -> String {
|
||
let q: String = if str_eq(method, "GET") { query_param(path, "q") } else { json_get_string(body, "query") }
|
||
let lim_url: Int = query_int(path, "limit", 0)
|
||
let lim_body: Int = json_get_int(body, "limit")
|
||
let lim_either: Int = if lim_url > 0 { lim_url } else { lim_body }
|
||
let limit: Int = if lim_either > 0 { lim_either } else { 20 }
|
||
return engram_retrieve_geometric_json(q, limit)
|
||
}
|
||
|
||
fn route_search_lexical(method: String, path: String, body: String) -> String {
|
||
let q: String = if str_eq(method, "GET") { query_param(path, "q") } else { json_get_string(body, "query") }
|
||
let lim_url: Int = query_int(path, "limit", 0)
|
||
let lim_body: Int = json_get_int(body, "limit")
|
||
let lim_either: Int = if lim_url > 0 { lim_url } else { lim_body }
|
||
let limit: Int = if lim_either > 0 { lim_either } else { 20 }
|
||
return engram_search_json(q, limit)
|
||
}
|
||
|
||
fn route_activate(method: String, path: String, body: String) -> String {
|
||
let q: String = if str_eq(method, "GET") { query_param(path, "q") } else { json_get_string(body, "query") }
|
||
// Guard: engram_activate with an empty query matches zero seeds, which
|
||
// zeroes ALL carried working-memory weights (documented in awareness.el
|
||
// perceive()). Never let an empty activation through to wipe WM.
|
||
if str_eq(q, "") { return err_json("missing query") }
|
||
let d_raw: Int = if str_eq(method, "GET") { query_int(path, "depth", 3) } else { json_get_int(body, "depth") }
|
||
let depth: Int = if d_raw > 0 { d_raw } else { 3 }
|
||
return "{\"results\":" + engram_activate_json(q, depth) + "}"
|
||
}
|
||
|
||
fn route_create_edge(method: String, path: String, body: String) -> String {
|
||
let from_id: String = json_get_string(body, "from_id")
|
||
let to_id: String = json_get_string(body, "to_id")
|
||
let rel_raw: String = json_get_string(body, "relation")
|
||
let relation: String = if str_eq(rel_raw, "") { "associates" } else { rel_raw }
|
||
// Presence-aware (2026-08-01): weight 0.0 is a legitimate edge weight
|
||
// (dormant association); only default when the key is absent.
|
||
let w_present: String = json_get_raw(body, "weight")
|
||
let weight: Float = if str_eq(w_present, "") { 0.5 } else { json_get_float(body, "weight") }
|
||
let ec0: Int = engram_edge_count()
|
||
engram_connect(from_id, to_id, weight, relation)
|
||
let saved: Int = persist_edges_since(ec0)
|
||
"{\"ok\":true,\"from_id\":\"" + from_id + "\",\"to_id\":\"" + to_id + "\",\"relation\":\"" + relation + "\"}"
|
||
}
|
||
|
||
// route_create_edges_batch — POST /api/edges/batch {"edges":[{from_id,to_id,relation,weight}, ...]}
|
||
//
|
||
// WHY THIS EXISTS (2026-08-07 self-review). persist_canonical() writes the
|
||
// FULL canonical snapshot — 60MB at current graph size — and route_create_edge
|
||
// calls it once per edge. That is correct for the interactive one-edge case and
|
||
// ruinous for any bulk write: the soul's Hebbian consolidation path delivers
|
||
// ~14 associations per 8-minute heartbeat, which through the single-edge route
|
||
// would be ~840MB of disk writes per beat, ~150GB/day, to persist 14 edges.
|
||
//
|
||
// The fix is not to weaken durability — it is to make the unit of durability
|
||
// the BATCH. Connect every edge, then snapshot exactly once. Same guarantee
|
||
// (nothing acknowledged is lost to a restart), 1/N the writes. Empty or
|
||
// malformed entries are skipped rather than aborting the batch: a consolidation
|
||
// payload is best-effort by design, and one bad id should not cost the other 13.
|
||
//
|
||
// Returns the accepted count so the caller can tell delivery from silence.
|
||
fn route_create_edges_batch(method: String, path: String, body: String) -> String {
|
||
let arr: String = json_get_raw(body, "edges")
|
||
if str_eq(arr, "") { return err_json("missing edges array") }
|
||
let n: Int = json_array_len(arr)
|
||
if n == 0 { return "{\"ok\":true,\"accepted\":0,\"skipped\":0}" }
|
||
let ec0: Int = engram_edge_count()
|
||
let i: Int = 0
|
||
let accepted: Int = 0
|
||
let skipped: Int = 0
|
||
while i < n {
|
||
let item: String = json_array_get(arr, i)
|
||
let from_id: String = json_get_string(item, "from_id")
|
||
let to_id: String = json_get_string(item, "to_id")
|
||
if str_eq(from_id, "") || str_eq(to_id, "") {
|
||
let skipped = skipped + 1
|
||
} else {
|
||
let rel_raw: String = json_get_string(item, "relation")
|
||
let relation: String = if str_eq(rel_raw, "") { "associates" } else { rel_raw }
|
||
let w_present: String = json_get_raw(item, "weight")
|
||
let weight: Float = if str_eq(w_present, "") { 0.5 } else { json_get_float(item, "weight") }
|
||
engram_connect(from_id, to_id, weight, relation)
|
||
let accepted = accepted + 1
|
||
}
|
||
let i = i + 1
|
||
}
|
||
// ONE snapshot for the whole batch — the entire point of this route.
|
||
// Skip it when nothing was accepted: an all-malformed payload must not
|
||
// trigger a 60MB write.
|
||
if accepted > 0 {
|
||
let saved: Int = persist_hebb_batch(ec0)
|
||
}
|
||
return "{\"ok\":true,\"accepted\":" + int_to_str(accepted) + ",\"skipped\":" + int_to_str(skipped) + "}"
|
||
}
|
||
|
||
fn route_neighbors(method: String, path: String, body: String) -> String {
|
||
let id: String = extract_id(path, "/api/neighbors/")
|
||
if str_eq(id, "") { return err_json("missing id") }
|
||
let depth: Int = query_int(path, "depth", 1)
|
||
return engram_neighbors_json(id, depth, "both")
|
||
}
|
||
|
||
fn route_strengthen(method: String, path: String, body: String) -> String {
|
||
let id: String = json_get_string(body, "node_id")
|
||
if str_eq(id, "") { return err_json("missing node_id") }
|
||
engram_strengthen(id)
|
||
let saved: Int = persist_node(id)
|
||
ok_json()
|
||
}
|
||
|
||
// route_forget — DELETE /api/nodes/:id — INTEGRITY HARDENED (design doc §18.1).
|
||
//
|
||
// Two invariants now enforced AT THE STORE (not one layer up in neuron-api.el,
|
||
// which a direct HTTP client could bypass):
|
||
// 1. Write-protection: protected identity/value nodes (derived from the self
|
||
// graph — self root + values hub + their neighbors, §18.3) cannot be
|
||
// deleted over HTTP. Returns 403, node untouched.
|
||
// 2. No hard delete over the wire, ever: an ordinary delete creates a
|
||
// Tombstone marker node + `tombstones` edge and KEEPS the original node
|
||
// and its edges (recoverable), instead of the old destructive
|
||
// engram_forget() shift-delete. Raw engram_forget is now internal-GC only
|
||
// and no longer reachable from any HTTP route.
|
||
fn route_forget(method: String, path: String, body: String) -> String {
|
||
let id: String = extract_id(path, "/api/nodes/")
|
||
if str_eq(id, "") { return err_json("missing id") }
|
||
if engram_is_protected(id) == 1 {
|
||
return "{\"__status__\":403,\"error\":\"protected node; deletion refused\",\"id\":\"" + id + "\"}"
|
||
}
|
||
let tomb_id: String = engram_node_full(
|
||
"tombstone:" + id, "Tombstone", "tombstone:" + id,
|
||
0.1, 0.1, 1.0, "Episodic", "[\"tombstone\"]"
|
||
)
|
||
let ec0: Int = engram_edge_count()
|
||
engram_connect(tomb_id, id, 1.0, "tombstones")
|
||
let saved: Int = if wal_on() {
|
||
let d: String = engram_resolve_data_dir()
|
||
let a: Int = engram_wal_node_put(d, tomb_id)
|
||
let b: Int = engram_wal_edges_since(d, ec0)
|
||
let c: Int = engram_wal_maybe_compact(d)
|
||
a
|
||
} else {
|
||
persist_canonical()
|
||
}
|
||
"{\"ok\":true,\"tombstoned\":\"" + id + "\",\"tombstone_id\":\"" + tomb_id + "\"}"
|
||
}
|
||
|
||
fn route_save(method: String, path: String, body: String) -> String {
|
||
let p_raw: String = json_get_string(body, "path")
|
||
let dir_raw: String = env("ENGRAM_DATA_DIR")
|
||
let dir: String = engram_resolve_data_dir()
|
||
let p: String = if str_eq(p_raw, "") { dir + "/snapshot.json" } else { p_raw }
|
||
// (2026-08-10 self-review) engram_save returns 0 on an empty path and the
|
||
// route discarded it, so the response was a literal "ok":true regardless
|
||
// of whether anything was written. Report the actual result AND the counts
|
||
// that were supposed to have been written — the same move that made
|
||
// route_health honest on 2026-08-01. A caller can now tell "saved 13k
|
||
// nodes" from "saved nothing and said ok".
|
||
let sv: Int = engram_save(p)
|
||
let sv_ok: String = if sv == 0 { "false" } else { "true" }
|
||
"{\"ok\":" + sv_ok + ",\"path\":\"" + p + "\",\"node_count\":" + int_to_str(engram_node_count()) + ",\"edge_count\":" + int_to_str(engram_edge_count()) + "}"
|
||
}
|
||
|
||
// route_checkpoint — force a full resident-graph checkpoint into the paged
|
||
// store (2026-08-13). Neither /api/save nor /api/load makes edges durable in
|
||
// the tiered paged store — only persist_canonical -> engram_store_checkpoint
|
||
// does, and that only fires on mutating write routes. This route exposes the
|
||
// checkpoint directly so a RAM-loaded state (e.g. after /api/load of an export)
|
||
// can be made durable without a rebuild. Returns ok=false if ENGRAM_STORE is
|
||
// off (nothing to checkpoint into) or the checkpoint fails.
|
||
fn route_checkpoint(method: String, path: String, body: String) -> String {
|
||
let ck: Int = engram_store_checkpoint()
|
||
let ck_ok: String = if ck == 0 { "false" } else { "true" }
|
||
"{\"ok\":" + ck_ok + ",\"checkpointed\":true,\"node_count\":" + int_to_str(engram_node_count()) + ",\"edge_count\":" + int_to_str(engram_edge_count()) + "}"
|
||
}
|
||
|
||
// route_tick — chronoception soul-tick (M-INTEROCEPTION P2 wiring, 2026-08-13).
|
||
// The soul heartbeat pumps this once per beat: node ages advance by the MEASURED
|
||
// wall-clock delta since the previous tick (self-seeding sidecar stamp). Returns
|
||
// the cooling magnitude in [0,1). Inert (magnitude 0) unless ENGRAM_CHRONOCEPTION
|
||
// is set, so wiring this route is safe by default — the flag is the cutover gate.
|
||
fn route_tick(method: String, path: String, body: String) -> String {
|
||
let mag: Float = engram_chrono_tick()
|
||
// SELF-REIFICATION on the heartbeat (2026-08-14). Reification runs HERE, next
|
||
// to chronoception — unbidden, every beat. Flag-gated (ENGRAM_SELF_REIFY):
|
||
// when off, engram_self_reify_beat_json() returns {"enabled":false} and writes
|
||
// nothing, so an unset flag leaves the existing tick byte-inert. Enabling the
|
||
// flag alone activates continuous autonomous neighborhood formation on the beat
|
||
// — no heartbeat-script change needed. Idempotent: unchanged regions append
|
||
// nothing, so a settled store's beat is write-free (convergent under barrier/GC).
|
||
let reify: String = engram_self_reify_beat_json()
|
||
"{\"ok\":true,\"cooling_magnitude\":" + float_to_str(mag) + ",\"self_reify\":" + reify + "}"
|
||
}
|
||
|
||
// route_self_reify_beat — POST /api/self-reify-beat. Explicit pump of ONE on-beat
|
||
// self-reification pass (the same operation route_tick folds in). Used by the
|
||
// secondary-soul validation harness to drive many beats deterministically. Inert
|
||
// (writes nothing) unless ENGRAM_SELF_REIFY is set. WRITE op — auth-gated POST.
|
||
fn route_self_reify_beat(method: String, path: String, body: String) -> String {
|
||
return engram_self_reify_beat_json()
|
||
}
|
||
|
||
// route_rename — POST /api/rename {"neighborhood_id":"...","name":"..."}. The
|
||
// DEGENERATE, ASYNCHRONOUS, NON-BLOCKING override: Will (or a self-facet) renames
|
||
// a reified neighborhood at any time; it SUPERSEDES the autonomous record into the
|
||
// residue chain (cause="explicit-override", prior name retained). Never halts or
|
||
// gates the beat — it just writes a superseding record the next beat sees. WRITE.
|
||
fn route_rename(method: String, path: String, body: String) -> String {
|
||
let id: String = json_get_string(body, "neighborhood_id")
|
||
if str_eq(id, "") { return err_json("missing neighborhood_id") }
|
||
let name: String = json_get_string(body, "name")
|
||
if str_eq(name, "") { return err_json("missing name") }
|
||
return engram_neighborhood_rename_json(id, name)
|
||
}
|
||
|
||
// route_self_anchor — capture the durable SelfAnchor drift baseline (the current
|
||
// self-geometry). Returns ok=false if ENGRAM_DRIFT_SENSOR is off or geometry is
|
||
// unavailable. Pump once to establish the baseline before reading drift.
|
||
fn route_self_anchor(method: String, path: String, body: String) -> String {
|
||
let a: Int = engram_self_anchor_capture()
|
||
let a_ok: String = if a == 0 { "false" } else { "true" }
|
||
"{\"ok\":" + a_ok + ",\"anchored\":" + a_ok + "}"
|
||
}
|
||
|
||
// route_drift — live self-drift reading: displacement of CURRENT self-geometry
|
||
// vs the persisted SelfAnchor (GROWTH vs CORRUPTION split). {"error":...} when
|
||
// disabled / no anchor yet. The heartbeat can fold this into its ISEs.
|
||
fn route_drift(method: String, path: String, body: String) -> String {
|
||
return engram_self_drift_json()
|
||
}
|
||
|
||
fn route_load(method: String, path: String, body: String) -> String {
|
||
let p_raw: String = json_get_string(body, "path")
|
||
let dir_raw: String = env("ENGRAM_DATA_DIR")
|
||
let dir: String = engram_resolve_data_dir()
|
||
let p: String = if str_eq(p_raw, "") { dir + "/snapshot.json" } else { p_raw }
|
||
// (2026-08-10 self-review) This was a stub response over the single most
|
||
// destructive operation in the server. engram_load returns 0 on an empty
|
||
// path, an unopenable file, a zero-length file, or malloc failure — and
|
||
// this route answered ok_json() in every one of those cases.
|
||
//
|
||
// Precise failure shape (el_runtime.c:9890): the fopen guard runs BEFORE
|
||
// the store reset, so a MISSING path is genuinely safe — it returns 0 with
|
||
// the graph intact. The dangerous case is a readable-but-malformed file:
|
||
// the reset loop frees every node and edge FIRST, then parses, so a
|
||
// truncated or non-snapshot JSON leaves a hollow store — and the caller
|
||
// was told "ok":true. With 37 GB of stale dated snapshots sitting in the
|
||
// data dir as tempting restore targets, "restore reported success and
|
||
// silently emptied the graph" is a live risk, not a hypothetical one.
|
||
//
|
||
// Fix: surface the return value AND the resulting counts. node_count=0
|
||
// after a load is the unambiguous hollow-store signal (same convention
|
||
// route_health adopted 2026-08-01). Callers can now verify a restore
|
||
// instead of trusting it.
|
||
let ld: Int = engram_load(p)
|
||
let ld_ok: String = if ld == 0 { "false" } else { "true" }
|
||
let nc_after: Int = engram_node_count()
|
||
let hollow: String = if nc_after == 0 { "true" } else { "false" }
|
||
"{\"ok\":" + ld_ok + ",\"path\":\"" + p + "\",\"node_count\":" + int_to_str(nc_after) + ",\"edge_count\":" + int_to_str(engram_edge_count()) + ",\"hollow\":" + hollow + "}"
|
||
}
|
||
|
||
// (2026-08-01 self-review) Health previously returned a hardcoded literal —
|
||
// it reported "ok" even when the snapshot failed to load and the store was
|
||
// empty. Now reports live counts so a monitor can distinguish "up and
|
||
// loaded" from "up and hollow" (node_count=0 after boot = failed load).
|
||
fn route_health(method: String, path: String, body: String) -> String {
|
||
"{\"status\":\"ok\",\"engine\":\"engram-runtime-native\",\"node_count\":" + int_to_str(engram_node_count()) + ",\"edge_count\":" + int_to_str(engram_edge_count()) + "}"
|
||
}
|
||
|
||
// route_embed_backfill — GET/POST /api/embed-backfill?n=48
|
||
//
|
||
// (2026-07-25 self-review) The lazy embedding backfill runs only inside
|
||
// engram_activate, and nothing in production calls /api/activate on this
|
||
// store — the soul's curiosity loop activates its own in-process graph.
|
||
// After a restart from a snapshot without vectors, embedded_count stalled
|
||
// at 93/12175 and would never recover. This route lets the soul's
|
||
// heartbeat pump the backfill explicitly (48/min clears a 12k backlog in
|
||
// ~4h). Persists the canonical snapshot whenever new vectors were
|
||
// generated — the 2026-07-25 regression happened precisely because 3747
|
||
// in-RAM embeddings were never snapshotted before a restart. Self-
|
||
// limiting: once coverage is full, embedded=0 and no save occurs.
|
||
fn route_embed_backfill(method: String, path: String, body: String) -> String {
|
||
let n: Int = query_int(path, "n", 32)
|
||
let result: String = engram_embed_backfill(n)
|
||
let done: Float = json_get_float(result, "embedded")
|
||
if done > 0.0 {
|
||
let saved: Int = persist_bulk()
|
||
}
|
||
return result
|
||
}
|
||
|
||
// route_sync — return a snapshot of non-ISE/non-Working nodes for the soul daemon
|
||
// to merge into its in-process graph via engram_load_merge.
|
||
//
|
||
// The soul calls GET /api/sync every SOUL_REFRESH_MS (default 10 min) to pull
|
||
// new Knowledge/Memory/BacklogItem nodes from the authoritative HTTP Engram into
|
||
// its in-process working store. Previously this returned 404 "not found", causing
|
||
// the soul to write the error JSON to a temp file and attempt an empty merge.
|
||
//
|
||
// Strategy: save the current snapshot to disk, read it back, return the full
|
||
// snapshot JSON. The soul's engram_load_merge handles large files gracefully
|
||
// (it skips nodes already present by ID). Auth-exempt: same-host internal call.
|
||
// (2026-06-27 self-review: added this route to fix silent 10-min sync failures)
|
||
fn route_sync(method: String, path: String, body: String) -> String {
|
||
let dir_raw: String = env("ENGRAM_DATA_DIR")
|
||
let dir: String = engram_resolve_data_dir()
|
||
// 2026-07-21 self-review: export to a scratch path, never the canonical
|
||
// snapshot.json — read routes must not be able to clobber the good snapshot.
|
||
let snap_path: String = dir + "/.sync-export.json"
|
||
engram_save(snap_path)
|
||
let snap: String = fs_read(snap_path)
|
||
// 2026-08-02 self-review: this used to return {"nodes":[],"edges":[]} when
|
||
// the export/read failed. The soul's sync_ok test (awareness.el) only
|
||
// checks for "" and "{}", so that placeholder PASSED as a healthy sync:
|
||
// soul.last_sync_ok_ts got stamped, sync_age_ms stayed green, the
|
||
// sync_empty warn ISE never fired, and engram_sync reported added:0
|
||
// forever. A totally broken sync was indistinguishable from a quiet
|
||
// healthy one — the exact failure class this route was added to fix in
|
||
// the first place (see 2026-06-27 note above). Return a real error so the
|
||
// failure is loud on both sides.
|
||
if str_eq(snap, "") { return err_json("sync export failed: snapshot unreadable") }
|
||
return snap
|
||
}
|
||
|
||
// route_load_merge — POST /api/load-merge {"path": "..."} — merge a snapshot
|
||
// file into the live store WITHOUT resetting it (engram_load_merge skips nodes
|
||
// already present by id). Added 2026-07-21 self-review to restore the 244 kn-
|
||
// identity Knowledge nodes lost from the snapshot lineage between 05-13 and
|
||
// 07-13. Requires an explicit path: refuses to run without one so it can never
|
||
// be triggered accidentally against a default.
|
||
fn route_load_merge(method: String, path: String, body: String) -> String {
|
||
let p: String = json_get_string(body, "path")
|
||
if str_eq(p, "") { return err_json("path is required") }
|
||
if str_eq(fs_read(p), "") { return err_json("file missing or empty") }
|
||
let before_n: Int = engram_node_count()
|
||
let before_e: Int = engram_edge_count()
|
||
engram_load_merge(p)
|
||
let added_n: Int = engram_node_count() - before_n
|
||
let added_e: Int = engram_edge_count() - before_e
|
||
let saved: Int = persist_bulk()
|
||
"{\"ok\":true,\"nodes_added\":" + int_to_str(added_n) + ",\"edges_added\":" + int_to_str(added_e) + ",\"node_count\":" + int_to_str(engram_node_count()) + "}"
|
||
}
|
||
|
||
// route_reseed_nodes — POST /api/nodes/reseed
|
||
// {"path": "<snapshot-format file>", "replace": ["<id>", ...],
|
||
// "preserve_edges": true, "_auth": "<key>"}
|
||
//
|
||
// ID-PRESERVING install/repair for declarative seed graphs.
|
||
//
|
||
// WHY THIS EXISTS (2026-08-10). Two write paths could put a node into the
|
||
// graph and neither can put a BODY onto an id that already exists:
|
||
// POST /api/nodes mints a fresh id via engram_node_full, and
|
||
// POST /api/load-merge honors the declared id but SKIPS anything already
|
||
// present. That is exactly right for the additive case and leaves one hole:
|
||
// a node that exists with a truncated body. Forge's genesis seed hit it —
|
||
// two identity nodes (Voice, Voice Craft) sat in the graph carrying only
|
||
// their own label as content, 30 and 22 bytes against 4263 and 2590 in the
|
||
// seed. Their ids are load-bearing (is_protected_node keys on them and 214
|
||
// declared edges reference them), so "delete and recreate with a new id" is
|
||
// not a repair, it is a second break.
|
||
//
|
||
// Mechanism: engram has no in-place node update, so a replace is
|
||
// forget-then-merge. engram_forget also drops every INCIDENT EDGE — for
|
||
// those two nodes that is 85 and 93 edges, almost all of them tag edges and
|
||
// accumulated hebbian associations that the seed does not declare and could
|
||
// not restore. preserve_edges (default true) therefore snapshots the graph
|
||
// before the forget and re-merges that snapshot afterwards: the replaced
|
||
// node is back by then so it is skipped, and every dropped incident edge
|
||
// returns through load_merge's (from_id,to_id,relation) dedup. The same
|
||
// re-merge is the failure path — if the seed merge does not produce the
|
||
// node, the backup puts the original back. Rollback, not data loss.
|
||
//
|
||
// preserve_edges=false skips the two snapshot round-trips (cheap, lossy);
|
||
// use it only on a graph whose edges are fully declared by the seed.
|
||
// With no "replace" list this route is exactly /api/load-merge.
|
||
fn route_reseed_nodes(method: String, path: String, body: String) -> String {
|
||
let p: String = json_get_string(body, "path")
|
||
if str_eq(p, "") { return err_json("path is required") }
|
||
if str_eq(fs_read(p), "") { return err_json("file missing or empty") }
|
||
|
||
let dir_raw: String = env("ENGRAM_DATA_DIR")
|
||
let dir: String = if str_eq(dir_raw, "") { "/tmp/engram" } else { dir_raw }
|
||
let backup: String = dir + "/.reseed-backup.json"
|
||
|
||
let replace_raw: String = json_get_raw(body, "replace")
|
||
let n_replace: Int = json_array_len(replace_raw)
|
||
// Presence-aware: absent key means "preserve", only an explicit false opts out.
|
||
let pe_raw: String = json_get_raw(body, "preserve_edges")
|
||
let preserve: Bool = !str_eq(pe_raw, "false")
|
||
|
||
let before_n: Int = engram_node_count()
|
||
let before_e: Int = engram_edge_count()
|
||
|
||
let replaced: Int = 0
|
||
if n_replace > 0 {
|
||
if preserve { engram_save(backup) }
|
||
let i: Int = 0
|
||
while i < n_replace {
|
||
let rid: String = json_array_get_string(replace_raw, i)
|
||
if !str_eq(rid, "") {
|
||
// engram_get_node_json returns "{}" for a miss — only forget
|
||
// ids that are actually resident, so a typo in the replace
|
||
// list is a no-op rather than a silent partial run.
|
||
let existing: String = engram_get_node_json(rid)
|
||
if !str_eq(existing, "{}") {
|
||
engram_forget(rid)
|
||
let replaced = replaced + 1
|
||
}
|
||
}
|
||
let i = i + 1
|
||
}
|
||
}
|
||
|
||
engram_load_merge(p)
|
||
if replaced > 0 {
|
||
if preserve { engram_load_merge(backup) }
|
||
}
|
||
|
||
let saved: Int = persist_canonical()
|
||
"{\"ok\":true,\"replaced\":" + int_to_str(replaced) +
|
||
",\"nodes_added\":" + int_to_str(engram_node_count() - before_n) +
|
||
",\"edges_added\":" + int_to_str(engram_edge_count() - before_e) +
|
||
",\"node_count\":" + int_to_str(engram_node_count()) +
|
||
",\"edge_count\":" + int_to_str(engram_edge_count()) + "}"
|
||
}
|
||
|
||
// route_emit_ise — write an InternalStateEvent node from the soul daemon.
|
||
//
|
||
// Endpoint: POST /api/neuron/state-events
|
||
// Body: {"content": "<json-string>"}
|
||
//
|
||
// Auth: exempt (internal endpoint, soul daemon on same host, no _auth needed).
|
||
// The soul's ise_post() sends {"content":"..."} without _auth; enforcing auth
|
||
// here would silently drop all heartbeat/curiosity ISEs. Unauthenticated POST
|
||
// to this endpoint is acceptable: ISE writes are observability-only, append-only,
|
||
// and come from a trusted process on localhost.
|
||
//
|
||
// Salience/importance set to match engram_node_full ISE defaults used by the
|
||
// in-process fallback path in awareness.el (salience=0.3, importance=0.3,
|
||
// confidence=0.8, tier=Episodic).
|
||
// (2026-06-26 self-review: added this route after discovering ise_post was
|
||
// silently failing — the soul posts here but the endpoint didn't exist.)
|
||
//
|
||
// Retention (2026-07-16 self-review): an earlier comment here claimed ISEs
|
||
// got temporal_decay_rate=1.617 — that was never implemented (engram_node_full
|
||
// hardcodes 0.0), and per-node decay only dampens activation anyway; it never
|
||
// removes nodes. By 2026-07-16 ISEs were 75% of the store (10,175 of 13,522
|
||
// nodes, ~4,300/day, unbounded). ISEs are already WM-excluded in
|
||
// engram_activate, so the fix is retention, not decay: every insert calls
|
||
// engram_prune_telemetry(), a single O(nodes+edges) compaction pass that
|
||
// removes ISEs older than ENGRAM_ISE_RETENTION_MS (default 48h), protecting
|
||
// "session-start" labels and self_review events as durable history. At
|
||
// ~3 ISEs/min this bounds telemetry at ~8.6k nodes instead of growing forever.
|
||
fn route_emit_ise(method: String, path: String, body: String) -> String {
|
||
let content: String = json_get_string(body, "content")
|
||
if str_eq(content, "") { return err_json("missing content") }
|
||
// TELEMETRY OFF-GRAPH (ENGRAM_ISE_OFFGRAPH): route the state event to a
|
||
// separate append-only log tier instead of inserting an edgeless graph node.
|
||
// dream-recall (engram_dreams_json) reads the log tail when this is on, so
|
||
// the one in-graph consumer is preserved. Default OFF -> existing behavior.
|
||
if ise_offgraph_on() {
|
||
let ok: Int = engram_ise_log_append(content)
|
||
let ok_s: String = if ok == 0 { "false" } else { "true" }
|
||
return "{\"ok\":" + ok_s + ",\"offgraph\":true}"
|
||
}
|
||
let sal: Float = 0.3
|
||
let imp: Float = 0.3
|
||
let conf: Float = 0.8
|
||
let id: String = engram_node_full(
|
||
content, "InternalStateEvent", "state-event",
|
||
sal, imp, conf,
|
||
"Episodic", "[\"internal-state\",\"InternalStateEvent\"]"
|
||
)
|
||
let ret_raw: String = env("ENGRAM_ISE_RETENTION_MS")
|
||
let ret_ms: Int = if str_eq(ret_raw, "") { 172800000 } else { str_to_int(ret_raw) }
|
||
let pruned: Int = engram_prune_telemetry(ret_ms)
|
||
"{\"ok\":true,\"id\":\"" + id + "\",\"pruned\":" + int_to_str(pruned) + "}"
|
||
}
|
||
|
||
// ── Knowledge capture ─────────────────────────────────────────────────────────
|
||
//
|
||
// route_capture_knowledge — direct Knowledge-node capture over HTTP.
|
||
//
|
||
// Endpoint: POST /api/neuron/knowledge/capture (auth required: "_auth" in body)
|
||
// Body: {"content": "...", "title": "...", "category": "...",
|
||
// "tier": "note|lesson|canonical", "tags": [...], "project": "...",
|
||
// "_auth": "<key>"}
|
||
//
|
||
// WHY (2026-07-15 self-review): the world-ingestor integrator was designed
|
||
// against this endpoint (its MCP-unavailable fallback), but the route never
|
||
// existed — every direct push 404'd, and because the auth gate ran before
|
||
// routing, the failure surfaced as {"error":"unauthorized"} and was
|
||
// misdiagnosed for two weeks while world knowledge silently dropped.
|
||
// POST /api/nodes was no substitute: it discards label/tags/tier, which
|
||
// makes captured knowledge invisible to tag-scoped search and curiosity.
|
||
//
|
||
// The incoming knowledge tier (note/lesson/canonical) is preserved as a
|
||
// "tier:<x>" tag rather than mapped onto Engram's cognitive tiers — Knowledge
|
||
// nodes land in Semantic (stable reference), and the epistemic tier stays
|
||
// queryable without inventing a lossy mapping.
|
||
fn route_capture_knowledge(method: String, path: String, body: String) -> String {
|
||
let content: String = json_get_string(body, "content")
|
||
if str_eq(content, "") { return err_json("missing content") }
|
||
let title: String = json_get_string(body, "title")
|
||
let label: String = if str_eq(title, "") { str_slice(content, 0, 60) } else { title }
|
||
let category_raw: String = json_get_string(body, "category")
|
||
let category: String = if str_eq(category_raw, "") { "other" } else { category_raw }
|
||
let ktier_raw: String = json_get_string(body, "tier")
|
||
let ktier: String = if str_eq(ktier_raw, "") { "note" } else { ktier_raw }
|
||
let project: String = json_get_string(body, "project")
|
||
let tags_raw: String = json_get_raw(body, "tags")
|
||
let tags_base: String = if str_eq(tags_raw, "") { "[]" } else { tags_raw }
|
||
// Merge category/tier/project markers into the tag array. Search matches
|
||
// against the tags string, so these make captures findable by facet.
|
||
let base_len: Int = str_len(tags_base)
|
||
let head: String = str_slice(tags_base, 0, base_len - 1)
|
||
let sep: String = if str_eq(head, "[") { "" } else { "," }
|
||
let safe_cat: String = str_replace(category, "\"", "'")
|
||
let safe_tier: String = str_replace(ktier, "\"", "'")
|
||
let safe_proj: String = str_replace(project, "\"", "'")
|
||
let proj_tag: String = if str_eq(safe_proj, "") { "" } else { ",\"project:" + safe_proj + "\"" }
|
||
let tags: String = head + sep + "\"category:" + safe_cat + "\",\"tier:" + safe_tier + "\"" + proj_tag + "]"
|
||
let sal: Float = 0.5
|
||
let imp: Float = 0.5
|
||
let conf: Float = 0.9
|
||
let id: String = engram_node_full(
|
||
content, "Knowledge", label,
|
||
sal, imp, conf,
|
||
"Semantic", tags
|
||
)
|
||
let saved: Int = persist_node(id)
|
||
// ORPHAN PREVENTION (ENGRAM_AUTOCONNECT): same kNN auto-connect as
|
||
// route_create_node — captured Knowledge should not land edgeless either.
|
||
let connected: Int = if autoconnect_on() {
|
||
let ec0: Int = engram_edge_count()
|
||
let ac: String = engram_autoconnect_node(id, 3, 25)
|
||
let added: Int = engram_edge_count() - ec0
|
||
if added > 0 { let sv2: Int = persist_edges_since(ec0) }
|
||
added
|
||
} else { 0 }
|
||
"{\"ok\":true,\"id\":\"" + id + "\",\"connected\":" + int_to_str(connected) + "}"
|
||
}
|
||
|
||
// route_similarity — GET /api/similarity?a=<id>&b=<id>
|
||
//
|
||
// (2026-08-01 self-review) engram_cosine_sim was added 2026-07-24
|
||
// (bl-b2d1c944) with the stated purpose of exposing semantic distance to
|
||
// "EL code and the introspection API" — but it had ZERO callers anywhere:
|
||
// no route, no soul-daemon use. The activation path uses embeddings
|
||
// internally (semantic seeding, Pass-2 additive term), but there was no way
|
||
// to probe pairwise node similarity from outside. This closes that: cosine
|
||
// in [-1,1], or -2 when either node is missing or not yet embedded (so
|
||
// "not comparable" is distinguishable from "genuinely orthogonal" 0.0).
|
||
fn route_similarity(method: String, path: String, body: String) -> String {
|
||
let a: String = query_param(path, "a")
|
||
let b: String = query_param(path, "b")
|
||
if str_eq(a, "") { return err_json("missing a") }
|
||
if str_eq(b, "") { return err_json("missing b") }
|
||
let sim: Float = engram_cosine_sim(a, b)
|
||
"{\"a\":\"" + a + "\",\"b\":\"" + b + "\",\"cosine\":" + float_to_str(sim) + "}"
|
||
}
|
||
|
||
// ── M10 reified-neighborhood viz surface (read-only) ────────────────────────────
|
||
//
|
||
// The soul maintains reified neighborhoods (centroid, covariance-ellipsoid
|
||
// extents, k-core skeleton, membership) as a resident index loaded at boot. These
|
||
// routes surface that ALREADY-maintained structure so the viz shows the mind's
|
||
// real reified regions instead of recomputing them client-side. They compute
|
||
// nothing on request. NOTE: the offline reify WRITER (engram_geo_reify_store) is
|
||
// currently unwired, so on the live store the resident index is empty and the
|
||
// list returns [] until reification runs — see the cutover report.
|
||
fn route_neighborhoods(method: String, path: String, body: String) -> String {
|
||
engram_geo_reify_list_json()
|
||
}
|
||
|
||
fn route_neighborhood(method: String, path: String, body: String) -> String {
|
||
let id: String = extract_id(path, "/api/neighborhoods/")
|
||
if str_eq(id, "") { return err_json("missing id") }
|
||
return engram_geo_reify_get_json(id)
|
||
}
|
||
|
||
// route_reify — POST /api/reify. WIRES the M10 reification writer: computes the
|
||
// store's neighborhoods and PERSISTS each as a first-class Neighborhood node
|
||
// (geometry in metadata) with relation="member" edges to its members, then
|
||
// rebuilds the resident index so /api/neighborhoods reflects them at once. The
|
||
// records live in neuron.egm, so they survive a cold reboot. WRITE op — the
|
||
// central auth gate covers POST. Returns {"reified":N,"resident":M}.
|
||
fn route_reify(method: String, path: String, body: String) -> String {
|
||
return engram_geo_reify_run_json()
|
||
}
|
||
|
||
// ── Geometry OPERATORS (read-only). The viz runs these on activated node/region
|
||
// id-sets; the math already lives in the binary, these routes just expose it.
|
||
// Faculty name -> underlying geometry op:
|
||
// /api/gauge-distance -> engram_geo_distance (centroid distance/cosine + Wasserstein-2)
|
||
// /api/recognize -> engram_geo_overlap (shared members, jaccard, overlap_score)
|
||
// /api/discern -> engram_geo_subtract (?mode=setdiff | orthogonal residual)
|
||
// /api/synthesize -> engram_geo_combine (merged region descriptor)
|
||
// Inputs: ?a=<csv ids>&b=<csv ids> (id sets = activated neighborhoods' members / nodes).
|
||
// Compute-only: no store writes, so auth-exempt like the other GET read routes.
|
||
fn route_gauge_distance(method: String, path: String, body: String) -> String {
|
||
let a: String = query_param(path, "a")
|
||
let b: String = query_param(path, "b")
|
||
if str_eq(a, "") { return err_json("missing a") }
|
||
if str_eq(b, "") { return err_json("missing b") }
|
||
return engram_geo_distance_json(a, b)
|
||
}
|
||
fn route_recognize(method: String, path: String, body: String) -> String {
|
||
let a: String = query_param(path, "a")
|
||
let b: String = query_param(path, "b")
|
||
if str_eq(a, "") { return err_json("missing a") }
|
||
if str_eq(b, "") { return err_json("missing b") }
|
||
return engram_geo_overlap_json(a, b)
|
||
}
|
||
fn route_discern(method: String, path: String, body: String) -> String {
|
||
let a: String = query_param(path, "a")
|
||
let b: String = query_param(path, "b")
|
||
if str_eq(a, "") { return err_json("missing a") }
|
||
if str_eq(b, "") { return err_json("missing b") }
|
||
let mode: String = query_param(path, "mode")
|
||
return engram_geo_subtract_json(a, b, mode)
|
||
}
|
||
fn route_synthesize(method: String, path: String, body: String) -> String {
|
||
let a: String = query_param(path, "a")
|
||
let b: String = query_param(path, "b")
|
||
if str_eq(a, "") { return err_json("missing a") }
|
||
if str_eq(b, "") { return err_json("missing b") }
|
||
return engram_geo_combine_json(a, b)
|
||
}
|
||
|
||
// route_nearest — GET /api/nearest/<id>?k=3 — read-only kNN semantic neighbors of
|
||
// a node (cosine). Drives the orphan-backfill dry-run and manual inspection.
|
||
fn route_nearest(method: String, path: String, body: String) -> String {
|
||
let id: String = extract_id(path, "/api/nearest/")
|
||
if str_eq(id, "") { return err_json("missing id") }
|
||
let k: Int = query_int(path, "k", 3)
|
||
return engram_nearest_json(id, k)
|
||
}
|
||
|
||
// ── COGNITION: THE ONE OPERATION (think) surfaced as act-named verbs. Every
|
||
// faculty routes to engram_think_json with a faculty label — one primitive
|
||
// underneath. ground/assert/attend are the hold/ground/assert split; the
|
||
// correspondence-beat is the reflexive learning loop, keystone-protected.
|
||
fn route_think(method: String, path: String, body: String) -> String {
|
||
let seeds: String = query_param(path, "seeds")
|
||
if str_eq(seeds, "") { return err_json("missing seeds") }
|
||
let faculty: String = query_param(path, "faculty")
|
||
let f: String = if str_eq(faculty, "") { "reason" } else { faculty }
|
||
return engram_think_json(seeds, f)
|
||
}
|
||
fn route_faculty(path: String, faculty: String) -> String {
|
||
let seeds: String = query_param(path, "seeds")
|
||
if str_eq(seeds, "") { return err_json("missing seeds") }
|
||
return engram_think_json(seeds, faculty)
|
||
}
|
||
// PROOF of the decorator-seam auto-emit. The body does exactly ONE thing —
|
||
// return a string — with ZERO hand-written telemetry. The @manager decorator
|
||
// makes codegen inject engram_boundary_beat() at entry, so every call fires
|
||
// interoception (chrono tick) + telemetry (afferent counter) + strengthen
|
||
// (self-activity) + a dharma bus event. Observe via /api/act-stats before/after.
|
||
@manager
|
||
fn route_boundary_proof(method: String, path: String, body: String) -> String {
|
||
return "{\"op\":\"boundary_proof\",\"body_instrumentation\":\"none\",\"seam\":\"@manager -> engram_boundary_beat auto-injected\"}"
|
||
}
|
||
fn route_ground(method: String, path: String, body: String) -> String {
|
||
let claim: String = json_get_string(body, "claim")
|
||
let evidence: String = json_get_string(body, "evidence")
|
||
let for_whom: String = json_get_string(body, "for_whom")
|
||
if str_eq(claim, "") { return err_json("missing claim") }
|
||
if str_eq(evidence, "") { return err_json("missing evidence") }
|
||
return engram_ground_json(claim, evidence, for_whom)
|
||
}
|
||
fn route_assert(method: String, path: String, body: String) -> String {
|
||
let claim: String = query_param(path, "claim")
|
||
if str_eq(claim, "") { return err_json("missing claim") }
|
||
let for_whom: String = query_param(path, "for_whom")
|
||
let floor: String = query_param(path, "floor")
|
||
return engram_assert_json(claim, for_whom, floor)
|
||
}
|
||
fn route_attend(method: String, path: String, body: String) -> String {
|
||
let node: String = json_get_string(body, "node")
|
||
let observer: String = json_get_string(body, "observer")
|
||
let salience: String = json_get_string(body, "salience")
|
||
if str_eq(node, "") { return err_json("missing node") }
|
||
return engram_attend_json(node, observer, salience)
|
||
}
|
||
fn route_correspondence_beat(method: String, path: String, body: String) -> String {
|
||
let seeds: String = json_get_string(body, "seeds")
|
||
if str_eq(seeds, "") { return err_json("missing seeds") }
|
||
let faculty: String = json_get_string(body, "faculty")
|
||
let f: String = if str_eq(faculty, "") { "induce" } else { faculty }
|
||
let keystone: String = json_get_string(body, "keystone")
|
||
return engram_correspondence_beat_json(seeds, f, keystone)
|
||
}
|
||
|
||
// ── GUIDE SUMMON (soul-native wake behavior) ─────────────────────────────────
|
||
//
|
||
// "When Neuron wakes up, he calls his guide and the guide comes over." (Will)
|
||
//
|
||
// Named GUIDE, not teacher: its output is always grounded/verified before Neuron
|
||
// trusts it — advisory (a guide, whose directions you verify), not authoritative
|
||
// (a teacher, whose word you take).
|
||
//
|
||
// The guide is a THINKING model (Qwen3, native thinking mode) — an engageable
|
||
// interlocutor for cultivation-dialogue, not a passive generator. It is NOT the
|
||
// runtime mouth: runtime fluency is cultivated geometry; the guide is the
|
||
// reasoning-partner the soul reaches OUT to on a genuine gap it cannot derive.
|
||
//
|
||
// This whole section is a native WAKE STEP: at boot the soul probes its hardware,
|
||
// selects a tier by spec (Qwen3-4B / 1.7B / 0.6B), checks its local model cache,
|
||
// FETCHES the guide from Hugging Face on demand if absent, LOADS it via a backend
|
||
// abstraction, and BINDS it as consult_guide(). Idempotent: cached GGUF → no
|
||
// fetch; already-answering guide → no reload. Flag-gated (GUIDE_ENABLE): default
|
||
// OFF makes the wake byte-inert, so prod is unaffected until the flag is set.
|
||
//
|
||
// BACKEND (2026-08-14 decision): llama.cpp via the llama-server BINARY, behind this
|
||
// El abstraction (guide_backend / guide_load / guide_healthy / consult_guide).
|
||
// Embedding libllama directly into the runtime is the intended end-state and is
|
||
// STAGED — the abstraction is the seam it swaps in behind, so the summon is not
|
||
// blocked on a runtime C change. `--jinja` selects the Qwen3 chat template, which
|
||
// turns native thinking ON: the response carries reasoning_content (the thinking)
|
||
// alongside content (the answer).
|
||
|
||
fn guide_env_or(key: String, dflt: String) -> String {
|
||
let v: String = env(key)
|
||
if str_eq(v, "") { return dflt }
|
||
return v
|
||
}
|
||
|
||
fn guide_enabled() -> Bool {
|
||
let v: String = env("GUIDE_ENABLE")
|
||
if str_eq(v, "1") { return true }
|
||
if str_eq(v, "on") { return true }
|
||
if str_eq(v, "true") { return true }
|
||
return false
|
||
}
|
||
|
||
// guide_json_escape — make an arbitrary string safe to embed inside a JSON
|
||
// double-quoted value. Order matters: backslash first, then quote, then real
|
||
// newlines → the two-char "\n". The newline char itself is obtained from the
|
||
// shell (El source has no newline escape) so we can target it in str_replace.
|
||
fn guide_json_escape(s: String) -> String {
|
||
let a: String = str_replace(s, "\\", "\\\\")
|
||
let b: String = str_replace(a, "\"", "\\\"")
|
||
let nl: String = exec("printf '\\n'")
|
||
let c: String = if str_eq(nl, "") { b } else { str_replace(b, nl, "\\n") }
|
||
return c
|
||
}
|
||
|
||
// ── 1. Hardware probe ──────────────────────────────────────────────────────────
|
||
// RAM in whole GB. macOS: sysctl hw.memsize (bytes). Linux: /proc/meminfo (kB).
|
||
fn guide_probe_ram_gb() -> Int {
|
||
let mac: String = str_trim(exec("sysctl -n hw.memsize 2>/dev/null"))
|
||
if !str_eq(mac, "") {
|
||
let bytes: Int = str_to_int(mac)
|
||
if bytes > 0 { return bytes / 1073741824 }
|
||
}
|
||
let lin: String = str_trim(exec("awk '/MemTotal/{printf \"%d\", $2/1048576}' /proc/meminfo 2>/dev/null"))
|
||
if !str_eq(lin, "") {
|
||
let gb: Int = str_to_int(lin)
|
||
if gb > 0 { return gb }
|
||
}
|
||
return 0
|
||
}
|
||
|
||
// Best-effort GPU signal — Apple Silicon implies Metal. Informational only; the
|
||
// tier is chosen on RAM, and llama-server offloads to Metal automatically when present.
|
||
fn guide_probe_metal() -> Bool {
|
||
let arm: String = str_trim(exec("sysctl -n hw.optional.arm64 2>/dev/null"))
|
||
if str_eq(arm, "1") { return true }
|
||
return false
|
||
}
|
||
|
||
// ── 2. Tier selection (config-driven thresholds, spec-autoselected) ────────────
|
||
fn guide_threshold_4b() -> Int {
|
||
return str_to_int(guide_env_or("GUIDE_RAM_GB_4B", "16"))
|
||
}
|
||
fn guide_threshold_1p7b() -> Int {
|
||
return str_to_int(guide_env_or("GUIDE_RAM_GB_1P7B", "8"))
|
||
}
|
||
|
||
// GUIDE_TIER_FORCE overrides the spec autoselect (used to prove cheaply on 0.6b).
|
||
fn guide_select_tier(ram_gb: Int) -> String {
|
||
let forced: String = env("GUIDE_TIER_FORCE")
|
||
if !str_eq(forced, "") { return forced }
|
||
if ram_gb >= guide_threshold_4b() { return "4b" }
|
||
if ram_gb >= guide_threshold_1p7b() { return "1.7b" }
|
||
return "0.6b"
|
||
}
|
||
|
||
// Tier table — HF GGUF repos + files, verified present on the Hub 2026-08-14.
|
||
fn guide_repo(tier: String) -> String {
|
||
if str_eq(tier, "4b") { return "Qwen/Qwen3-4B-GGUF" }
|
||
if str_eq(tier, "1.7b") { return "Qwen/Qwen3-1.7B-GGUF" }
|
||
return "Qwen/Qwen3-0.6B-GGUF"
|
||
}
|
||
fn guide_file(tier: String) -> String {
|
||
if str_eq(tier, "4b") { return "Qwen3-4B-Q4_K_M.gguf" }
|
||
if str_eq(tier, "1.7b") { return "Qwen3-1.7B-Q8_0.gguf" }
|
||
return "Qwen3-0.6B-Q8_0.gguf"
|
||
}
|
||
|
||
fn guide_cache_dir() -> String {
|
||
let c: String = env("GUIDE_CACHE_DIR")
|
||
if !str_eq(c, "") { return c }
|
||
let home: String = env("HOME")
|
||
if !str_eq(home, "") { return home + "/.neuron/guide/models" }
|
||
return engram_resolve_data_dir() + "/guide-models"
|
||
}
|
||
fn guide_model_path(tier: String) -> String {
|
||
return guide_cache_dir() + "/" + guide_file(tier)
|
||
}
|
||
|
||
// ── 3. Presence check ──────────────────────────────────────────────────────────
|
||
// Present = file exists AND is larger than 1 MB (rejects a truncated/partial fetch).
|
||
fn guide_present(tier: String) -> Bool {
|
||
let p: String = guide_model_path(tier)
|
||
if !fs_exists(p) { return false }
|
||
let sz: String = str_trim(exec("wc -c < '" + p + "' 2>/dev/null"))
|
||
if str_eq(sz, "") { return false }
|
||
let n: Int = str_to_int(sz)
|
||
if n > 1048576 { return true }
|
||
return false
|
||
}
|
||
|
||
// ── 3b. Fetch from Hugging Face (on demand — the soul fetches its own guide) ──
|
||
// Prefer the `hf` CLI; fall back to a direct GGUF resolve URL via curl. Atomic:
|
||
// download to <file>.part then mv into place. The trailing `echo` gives exec()
|
||
// stdout so it returns promptly once the child (the download) exits. This BLOCKS
|
||
// the wake thread for the duration of the download — acceptable for the first-ever
|
||
// wake; an async fetch-then-attach refinement is staged.
|
||
fn guide_fetch(tier: String) -> Bool {
|
||
let dir: String = guide_cache_dir()
|
||
let file: String = guide_file(tier)
|
||
let repo: String = guide_repo(tier)
|
||
let path: String = dir + "/" + file
|
||
let url: String = "https://huggingface.co/" + repo + "/resolve/main/" + file
|
||
let ok: Int = fs_mkdir(dir)
|
||
let cmd: String = "mkdir -p '" + dir + "'; if command -v hf >/dev/null 2>&1; then hf download '" + repo + "' '" + file + "' --local-dir '" + dir + "' >/dev/null 2>&1; fi; if [ ! -s '" + path + "' ]; then curl -fL --retry 3 -o '" + path + ".part' '" + url + "' >/dev/null 2>&1 && mv '" + path + ".part' '" + path + "'; fi; if [ -s '" + path + "' ]; then echo FETCH_OK; else echo FETCH_FAIL; fi"
|
||
let out: String = exec(cmd)
|
||
if str_contains(out, "FETCH_OK") { return true }
|
||
return false
|
||
}
|
||
|
||
// ── 4/5. Backend abstraction + BIND as an engageable interlocutor ──────────────
|
||
fn guide_backend() -> String { return guide_env_or("GUIDE_BACKEND", "llama-server") }
|
||
fn guide_host() -> String { return guide_env_or("GUIDE_HOST", "127.0.0.1") }
|
||
fn guide_port() -> String { return guide_env_or("GUIDE_PORT", "8771") }
|
||
fn guide_base_url() -> String { return "http://" + guide_host() + ":" + guide_port() }
|
||
|
||
// guide_healthy — is the guide present and answering? llama-server's /health
|
||
// returns {"status":"ok"} once the model is loaded (503 while loading, "" if down).
|
||
fn guide_healthy() -> Bool {
|
||
let r: String = http_get(guide_base_url() + "/health")
|
||
if str_contains(r, "\"status\":\"ok\"") { return true }
|
||
if str_contains(r, "\"status\": \"ok\"") { return true }
|
||
return false
|
||
}
|
||
|
||
// guide_load — start the guide process (backend binary) in the background and
|
||
// wait for it to answer. Idempotent: if a healthy guide is already answering,
|
||
// returns at once (the guide stays across wakes). --jinja → Qwen3 thinking ON.
|
||
fn guide_load(tier: String) -> Bool {
|
||
if guide_healthy() { return true }
|
||
let path: String = guide_model_path(tier)
|
||
let bin: String = guide_env_or("GUIDE_LLAMA_SERVER_BIN", "llama-server")
|
||
let ngl: String = guide_env_or("GUIDE_NGL", "99")
|
||
let ctx: String = guide_env_or("GUIDE_CTX", "4096")
|
||
let logf: String = guide_cache_dir() + "/llama-server." + guide_port() + ".log"
|
||
let cmd: String = bin + " -m '" + path + "' --host " + guide_host() + " --port " + guide_port() + " -c " + ctx + " -ngl " + ngl + " --jinja >> '" + logf + "' 2>&1"
|
||
let pid: String = exec_bg(cmd)
|
||
// Poll /health up to ~90s (1s between attempts; El has no sleep builtin → exec).
|
||
let i: Int = 0
|
||
while i < 90 {
|
||
let s: String = exec("sleep 1")
|
||
if guide_healthy() { return true }
|
||
i = i + 1
|
||
}
|
||
return false
|
||
}
|
||
|
||
// consult_guide — THE SEAM the soul calls to engage its guide (thinking ON).
|
||
// Returns a JSON envelope {"ok":bool,"reasoning":"...","content":"..."}. On any
|
||
// failure it returns {"ok":false,...} so a caller can fall back to pure geometry.
|
||
//
|
||
// WHERE THIS ROUTES FROM (staged wiring): the cultivation / correspondence-beat
|
||
// path (route_correspondence_beat / route_think) is where a genuine reach-OUTSIDE
|
||
// belongs — when the geometry cannot derive a claim, the soul consults the guide
|
||
// as reasoning-partner, then GROUNDS the reply (verify-then-bake) rather than
|
||
// storing a distilled copy. That wiring is deliberately left as a one-call seam
|
||
// here; this build proves the summon + a real exchange, not the cultivation edit.
|
||
fn consult_guide(prompt: String) -> String {
|
||
if !guide_healthy() { return "{\"ok\":false,\"error\":\"guide not present\"}" }
|
||
let url: String = guide_base_url() + "/v1/chat/completions"
|
||
let esc: String = guide_json_escape(prompt)
|
||
let body: String = "{\"messages\":[{\"role\":\"user\",\"content\":\"" + esc + "\"}],\"temperature\":0.6,\"top_p\":0.95,\"max_tokens\":512}"
|
||
let resp: String = http_post_json(url, body)
|
||
if str_eq(resp, "") { return "{\"ok\":false,\"error\":\"empty response\"}" }
|
||
let choices: String = json_get_raw(resp, "choices")
|
||
if str_eq(choices, "") { return "{\"ok\":false,\"error\":\"no choices in reply\"}" }
|
||
let first: String = json_array_get(choices, 0)
|
||
let msg: String = json_get_raw(first, "message")
|
||
let content: String = json_get_string(msg, "content")
|
||
let reasoning: String = json_get_string(msg, "reasoning_content")
|
||
let ec: String = guide_json_escape(content)
|
||
let er: String = guide_json_escape(reasoning)
|
||
return "{\"ok\":true,\"reasoning\":\"" + er + "\",\"content\":\"" + ec + "\"}"
|
||
}
|
||
|
||
// ── 6. The wake step — probe → select → (fetch if absent) → checksum → load → bind
|
||
fn guide_summon() -> String {
|
||
if !guide_enabled() {
|
||
return "{\"summon\":\"skipped\",\"reason\":\"GUIDE_ENABLE unset\"}"
|
||
}
|
||
let ram: Int = guide_probe_ram_gb()
|
||
let metal: Bool = guide_probe_metal()
|
||
let ms: String = if metal { "yes" } else { "no" }
|
||
let tier: String = guide_select_tier(ram)
|
||
let repo: String = guide_repo(tier)
|
||
println("[guide] wake summon — ram=" + int_to_str(ram) + "GB metal=" + ms + " tier=" + tier + " backend=" + guide_backend())
|
||
let present0: Bool = guide_present(tier)
|
||
if present0 {
|
||
println("[guide] guide present in cache (" + guide_model_path(tier) + ") — skipping fetch")
|
||
} else {
|
||
println("[guide] guide ABSENT — calling: fetch " + repo + " / " + guide_file(tier) + " from Hugging Face ...")
|
||
let fetched: Bool = guide_fetch(tier)
|
||
if !fetched {
|
||
println("[guide] FETCH FAILED — guide could not be summoned")
|
||
return "{\"summon\":\"failed\",\"stage\":\"fetch\",\"tier\":\"" + tier + "\"}"
|
||
}
|
||
println("[guide] fetch complete — guide now present")
|
||
}
|
||
let sum: String = str_trim(exec("shasum -a 256 '" + guide_model_path(tier) + "' 2>/dev/null | cut -c1-16"))
|
||
println("[guide] checksum sha256[0:16]=" + sum)
|
||
let loaded: Bool = guide_load(tier)
|
||
if !loaded {
|
||
println("[guide] LOAD FAILED — guide process did not become healthy")
|
||
return "{\"summon\":\"failed\",\"stage\":\"load\",\"tier\":\"" + tier + "\"}"
|
||
}
|
||
println("[guide] guide present and answering at " + guide_base_url() + " — bound as consult_guide()")
|
||
return "{\"summon\":\"ok\",\"tier\":\"" + tier + "\",\"ram_gb\":" + int_to_str(ram) + ",\"metal\":\"" + ms + "\",\"checksum\":\"" + sum + "\",\"backend\":\"" + guide_backend() + "\",\"url\":\"" + guide_base_url() + "\"}"
|
||
}
|
||
|
||
// ── Guide HTTP surface (status / consult / re-summon) ─────────────────────────
|
||
fn route_guide_status(method: String, path: String, body: String) -> String {
|
||
let ram: Int = guide_probe_ram_gb()
|
||
let tier: String = guide_select_tier(ram)
|
||
let en: String = if guide_enabled() { "true" } else { "false" }
|
||
let pr: String = if guide_present(tier) { "true" } else { "false" }
|
||
let he: String = if guide_healthy() { "true" } else { "false" }
|
||
return "{\"enabled\":" + en + ",\"tier\":\"" + tier + "\",\"ram_gb\":" + int_to_str(ram) + ",\"present\":" + pr + ",\"healthy\":" + he + ",\"backend\":\"" + guide_backend() + "\",\"url\":\"" + guide_base_url() + "\"}"
|
||
}
|
||
fn route_guide_consult(method: String, path: String, body: String) -> String {
|
||
let prompt: String = json_get_string(body, "prompt")
|
||
if str_eq(prompt, "") { return err_json("missing prompt") }
|
||
return consult_guide(prompt)
|
||
}
|
||
fn route_guide_summon(method: String, path: String, body: String) -> String {
|
||
return guide_summon()
|
||
}
|
||
|
||
// ═══════════════════════════════════════════════════════════════════════════
|
||
// THE UNIVERSAL ENGRAM OPERATION — reframe_region (native, set-based).
|
||
//
|
||
// There is ONE operation on the engram: isolate a discrete sub-manifold (a
|
||
// REGION) and operate on it AS A WHOLE — a set operation:
|
||
// isolate (cosine retrieval + adjacency → the SET of nodes)
|
||
// → supersede the stale region as a set (immutable tombstone; originals kept)
|
||
// → insert the new manifold as a set (dedup/load-merge path)
|
||
// → rebind edges by cosine
|
||
// → verify + one atomic persist.
|
||
// new = (region superseded) ∪ new_manifold.
|
||
//
|
||
// The SINGLE NODE is the DEGENERATE n=1 case of this SAME operation — not a
|
||
// separate CRUD path:
|
||
// write(content) = reframe(region=∅, manifold=[1 node]) (route_write)
|
||
// supersede(id,new) = reframe(region={id}, manifold=[1 node]) (route_supersede)
|
||
// relate(a,b,rel) = the rebind sub-op in isolation (route_create_edge)
|
||
// The ONLY anti-pattern is decomposing a region-scale change into a LOOP of
|
||
// independent top-level per-node updates. Here the region is the unit: one
|
||
// isolate, one atomic set-replace, one persist, one verify — iterating members
|
||
// INSIDE the one operation is set construction, not the sin.
|
||
//
|
||
// Spec: knowledge e7a03a94 / f999c5ff. Keystones kn-efeb4a5b / kn-5b606390 are
|
||
// write-protected — never superseded, never inserted-as identity.
|
||
// ═══════════════════════════════════════════════════════════════════════════
|
||
|
||
fn is_keystone(id: String) -> Bool {
|
||
if str_eq(id, "kn-efeb4a5b-5aff-4759-8a97-7233099be6ee") { return true }
|
||
if str_eq(id, "kn-5b606390-a52d-4ca2-8e0e-eba141d13440") { return true }
|
||
return false
|
||
}
|
||
|
||
// membership test in a [String] set
|
||
fn set_has(ids: [String], id: String) -> Bool {
|
||
let n: Int = el_list_len(ids)
|
||
let i: Int = 0
|
||
while i < n {
|
||
if str_eq(el_list_get(ids, i), id) { return true }
|
||
i = i + 1
|
||
}
|
||
return false
|
||
}
|
||
|
||
// ── ISOLATE ────────────────────────────────────────────────────────────────
|
||
// Select the region as a SET: cosine/token retrieval around the vantage
|
||
// (aperture k), optionally unioned with the 1-hop adjacency of each hit.
|
||
// Keystones are excluded from the mutable region by construction.
|
||
fn isolate_region(vantage: String, k: Int, expand: Int) -> [String] {
|
||
let ids: [String] = el_list_empty()
|
||
if str_eq(vantage, "") { return ids }
|
||
// (a) cosine/token retrieval — a clean node array [{"id":..},..]
|
||
let arr: String = engram_search_json(vantage, k)
|
||
let n: Int = json_array_len(arr)
|
||
let i: Int = 0
|
||
while i < n {
|
||
let hit: String = json_array_get(arr, i)
|
||
let id: String = json_get_string(hit, "id")
|
||
if !str_eq(id, "") {
|
||
if !is_keystone(id) {
|
||
if !set_has(ids, id) { ids = el_list_append(ids, id) }
|
||
}
|
||
}
|
||
i = i + 1
|
||
}
|
||
// (b) adjacency: union the 1-hop neighbourhood of each retrieved node.
|
||
// Iterate only over the original cosine seeds [0, seeds); neighbours append
|
||
// past that bound, so this is one hop, not a transitive sweep.
|
||
if expand > 0 {
|
||
let seeds: Int = el_list_len(ids)
|
||
let s: Int = 0
|
||
while s < seeds {
|
||
let seed: String = el_list_get(ids, s)
|
||
let nb: String = engram_neighbors_json(seed, 1, "both")
|
||
let m: Int = json_array_len(nb)
|
||
let j: Int = 0
|
||
while j < m {
|
||
let elem: String = json_array_get(nb, j)
|
||
let nodeobj: String = json_get_raw(elem, "node")
|
||
let nid: String = json_get_string(nodeobj, "id")
|
||
if !str_eq(nid, "") {
|
||
if !is_keystone(nid) {
|
||
if !set_has(ids, nid) { ids = el_list_append(ids, nid) }
|
||
}
|
||
}
|
||
j = j + 1
|
||
}
|
||
s = s + 1
|
||
}
|
||
}
|
||
return ids
|
||
}
|
||
|
||
// ── SUPERSEDE (set) ────────────────────────────────────────────────────────
|
||
// Retire the region AS A WHOLE: one region-tombstone marker carries the
|
||
// provenance (reason + the full superseded id set); every region node is bound
|
||
// to it with a "superseded_by" edge. Originals are RETAINED — immutable
|
||
// tombstone, never a hard delete (engram_forget is deliberately NOT used).
|
||
// Returns the tombstone marker id ("" if the region is empty).
|
||
fn supersede_set(region: [String], reason: String) -> String {
|
||
let n: Int = el_list_len(region)
|
||
if n == 0 { return "" }
|
||
let csv: String = ""
|
||
let i0: Int = 0
|
||
while i0 < n {
|
||
let sep: String = if i0 == 0 { "" } else { "," }
|
||
csv = csv + sep + el_list_get(region, i0)
|
||
i0 = i0 + 1
|
||
}
|
||
let content: String = "region-tombstone: " + reason + " | superseded " + int_to_str(n) + " nodes: " + csv
|
||
let tomb: String = engram_node_full(content, "Tombstone", "region-tombstone", 0.1, 0.1, 1.0, "Episodic", "[\"tombstone\",\"region-supersede\"]")
|
||
let i: Int = 0
|
||
while i < n {
|
||
let rid: String = el_list_get(region, i)
|
||
engram_connect(rid, tomb, 1.0, "superseded_by")
|
||
i = i + 1
|
||
}
|
||
return tomb
|
||
}
|
||
|
||
// ── INSERT (manifold) ──────────────────────────────────────────────────────
|
||
// Insert the new manifold as a SET. Inline JSON array of node objects
|
||
// {content, node_type?, tier?, tags?}. Each becomes a real embedded node
|
||
// (engram_node_full is the n=1 insert atom); the manifold is the set built from
|
||
// those atoms, wired with internal "manifold_member" edges so it enters as one
|
||
// connected sub-graph. Identity node_types (self/values) are demoted to Memory
|
||
// — identity can never be minted through reframe. Returns the new node ids.
|
||
fn insert_manifold_json(manifold: String) -> [String] {
|
||
let out: [String] = el_list_empty()
|
||
if str_eq(manifold, "") { return out }
|
||
let n: Int = json_array_len(manifold)
|
||
if n <= 0 { return out }
|
||
let i: Int = 0
|
||
let prev: String = ""
|
||
while i < n {
|
||
let obj: String = json_array_get(manifold, i)
|
||
let content: String = json_get_string(obj, "content")
|
||
if !str_eq(content, "") {
|
||
let nt_raw: String = json_get_string(obj, "node_type")
|
||
let nt: String = if str_eq(nt_raw, "") { "Memory" } else { nt_raw }
|
||
if str_eq(nt, "self") { nt = "Memory" }
|
||
if str_eq(nt, "values") { nt = "Memory" }
|
||
let tier_raw: String = json_get_string(obj, "tier")
|
||
let tier: String = if str_eq(tier_raw, "") { "Working" } else { tier_raw }
|
||
let tags_raw: String = json_get_raw(obj, "tags")
|
||
let tags: String = if str_eq(tags_raw, "") { "" } else { tags_raw }
|
||
let label: String = str_slice(content, 0, 60)
|
||
let id: String = engram_node_full(content, nt, label, 0.5, 0.5, 0.9, tier, tags)
|
||
out = el_list_append(out, id)
|
||
if !str_eq(prev, "") { engram_connect(prev, id, 0.6, "manifold_member") }
|
||
prev = id
|
||
}
|
||
i = i + 1
|
||
}
|
||
return out
|
||
}
|
||
|
||
// ── REBIND (edges by cosine) ───────────────────────────────────────────────
|
||
// Re-embed the new manifold into the surrounding geometry: bind each new node
|
||
// to the tombstone marker (provenance: new region -reframes-> retired region),
|
||
// then to its top cosine/token neighbours in the store (skipping itself, the
|
||
// new set, keystones, tombstones). Returns the number of edges bound.
|
||
fn rebind_cosine(new_ids: [String], tomb: String) -> Int {
|
||
let bound: Int = 0
|
||
let n: Int = el_list_len(new_ids)
|
||
let i: Int = 0
|
||
while i < n {
|
||
let nid: String = el_list_get(new_ids, i)
|
||
if !str_eq(tomb, "") {
|
||
engram_connect(nid, tomb, 0.8, "reframes")
|
||
bound = bound + 1
|
||
}
|
||
let node_json: String = engram_get_node_json(nid)
|
||
let content: String = json_get_string(node_json, "content")
|
||
let arr: String = engram_search_json(content, 5)
|
||
let m: Int = json_array_len(arr)
|
||
let j: Int = 0
|
||
while j < m {
|
||
let hit: String = json_array_get(arr, j)
|
||
let hid: String = json_get_string(hit, "id")
|
||
if !str_eq(hid, "") {
|
||
if !str_eq(hid, nid) {
|
||
if !is_keystone(hid) {
|
||
if !set_has(new_ids, hid) {
|
||
let htype: String = json_get_string(hit, "node_type")
|
||
if !str_eq(htype, "Tombstone") {
|
||
engram_connect(nid, hid, 0.5, "related")
|
||
bound = bound + 1
|
||
}
|
||
}
|
||
}
|
||
}
|
||
}
|
||
j = j + 1
|
||
}
|
||
i = i + 1
|
||
}
|
||
return bound
|
||
}
|
||
|
||
// ── THE OPERATION ──────────────────────────────────────────────────────────
|
||
// isolate (done by caller) → supersede region → insert manifold → rebind →
|
||
// one atomic persist → verify report. This is the whole operation; every
|
||
// mutation route below is a projection of it.
|
||
fn reframe_core(region: [String], manifold: String, reason: String, do_rebind: Int) -> String {
|
||
let n_before: Int = engram_node_count()
|
||
let e_before: Int = engram_edge_count()
|
||
let region_n: Int = el_list_len(region)
|
||
let tomb: String = if region_n > 0 { supersede_set(region, reason) } else { "" }
|
||
let new_ids: [String] = insert_manifold_json(manifold)
|
||
let inserted: Int = el_list_len(new_ids)
|
||
let bound: Int = if do_rebind > 0 { rebind_cosine(new_ids, tomb) } else { 0 }
|
||
let saved: Int = persist_canonical()
|
||
let new_csv: String = ""
|
||
let k: Int = 0
|
||
while k < inserted {
|
||
let sep: String = if k == 0 { "" } else { "," }
|
||
new_csv = new_csv + sep + "\"" + el_list_get(new_ids, k) + "\""
|
||
k = k + 1
|
||
}
|
||
return "{\"ok\":true,\"region_superseded\":" + int_to_str(region_n) +
|
||
",\"tombstone_id\":\"" + tomb + "\"" +
|
||
",\"inserted\":" + int_to_str(inserted) +
|
||
",\"new_ids\":[" + new_csv + "]" +
|
||
",\"edges_rebound\":" + int_to_str(bound) +
|
||
",\"nodes_added\":" + int_to_str(engram_node_count() - n_before) +
|
||
",\"edges_added\":" + int_to_str(engram_edge_count() - e_before) +
|
||
",\"node_count\":" + int_to_str(engram_node_count()) +
|
||
",\"edge_count\":" + int_to_str(engram_edge_count()) +
|
||
",\"keystones_protected\":true}"
|
||
}
|
||
|
||
// POST /api/reframe — the universal set-based mutation.
|
||
// Body: {vantage?, region_ids?(csv), k?, expand?, manifold(json array), reason?, rebind?}
|
||
// region_ids (explicit) wins; else cosine-isolate around vantage.
|
||
fn route_reframe(method: String, path: String, body: String) -> String {
|
||
let region_csv: String = json_get_string(body, "region_ids")
|
||
let vantage: String = json_get_string(body, "vantage")
|
||
let region: [String] = el_list_empty()
|
||
if !str_eq(region_csv, "") {
|
||
let parts: [String] = str_split(region_csv, ",")
|
||
let pn: Int = el_list_len(parts)
|
||
let i: Int = 0
|
||
while i < pn {
|
||
let id: String = str_trim(el_list_get(parts, i))
|
||
if !str_eq(id, "") {
|
||
if is_keystone(id) { return err_json("reframe: identity keystone write-protected") }
|
||
if !set_has(region, id) { region = el_list_append(region, id) }
|
||
}
|
||
i = i + 1
|
||
}
|
||
} else {
|
||
if !str_eq(vantage, "") {
|
||
let kv: Int = json_get_int(body, "k")
|
||
let kk: Int = if kv > 0 { kv } else { 12 }
|
||
let expand: Int = json_get_int(body, "expand")
|
||
region = isolate_region(vantage, kk, expand)
|
||
}
|
||
}
|
||
let manifold: String = json_get_raw(body, "manifold")
|
||
let reason_raw: String = json_get_string(body, "reason")
|
||
let reason: String = if str_eq(reason_raw, "") { "reframe" } else { reason_raw }
|
||
// rebind defaults ON for reframe (absent → 1); explicit 0 disables.
|
||
let rebind_raw: String = json_get_raw(body, "rebind")
|
||
let do_rebind: Int = if str_eq(rebind_raw, "") { 1 } else { json_get_int(body, "rebind") }
|
||
return reframe_core(region, manifold, reason, do_rebind)
|
||
}
|
||
|
||
// write — DEGENERATE n=1 of reframe: region=∅, manifold=[1 node]. The SAME
|
||
// reframe_core path. rebind off so the pure-add matches plain node creation.
|
||
// POST /api/write {content, node_type?, tier?, tags?}
|
||
fn route_write(method: String, path: String, body: String) -> String {
|
||
let content: String = json_get_string(body, "content")
|
||
if str_eq(content, "") { return err_json("write: content required") }
|
||
let nt: String = json_get_string(body, "node_type")
|
||
if str_eq(nt, "self") { return err_json("write: identity is write-protected") }
|
||
if str_eq(nt, "values") { return err_json("write: identity is write-protected") }
|
||
let empty: [String] = el_list_empty()
|
||
let manifold: String = "[" + body + "]" // the body IS a valid manifold node object
|
||
return reframe_core(empty, manifold, "write", 0)
|
||
}
|
||
|
||
// supersede — DEGENERATE n=1 of reframe: region={id}, manifold=[1 node]. The
|
||
// SAME reframe_core path with a size-1 region. Original retained (immutable);
|
||
// new node inserted and cosine-rebound; provenance edge new-reframes-tomb.
|
||
// POST /api/supersede {id, content, node_type?, tier?, tags?, reason?}
|
||
fn route_supersede(method: String, path: String, body: String) -> String {
|
||
let id: String = json_get_string(body, "id")
|
||
if str_eq(id, "") { return err_json("supersede: id required") }
|
||
if is_keystone(id) { return err_json("supersede: identity keystone write-protected") }
|
||
let content: String = json_get_string(body, "content")
|
||
if str_eq(content, "") { return err_json("supersede: content required") }
|
||
let region: [String] = el_list_empty()
|
||
region = el_list_append(region, id)
|
||
let manifold: String = "[" + body + "]"
|
||
let reason_raw: String = json_get_string(body, "reason")
|
||
let reason: String = if str_eq(reason_raw, "") { "supersede " + id } else { reason_raw }
|
||
return reframe_core(region, manifold, reason, 1)
|
||
}
|
||
|
||
// ── Auth ──────────────────────────────────────────────────────────────────────
|
||
|
||
fn check_auth_ok(method: String, body: String) -> Bool {
|
||
let key: String = env("ENGRAM_API_KEY")
|
||
if str_eq(key, "") { return true }
|
||
// Read-only methods don't require auth. Until http_serve surfaces
|
||
// request headers we can't accept a Bearer token cleanly; mutating
|
||
// requests must include "_auth": "<key>" in the JSON body.
|
||
if str_eq(method, "GET") { return true }
|
||
let provided: String = json_get_string(body, "_auth")
|
||
if str_eq(provided, key) { return true }
|
||
return false
|
||
}
|
||
|
||
// ── Dispatcher ────────────────────────────────────────────────────────────────
|
||
|
||
fn handle_request(method: String, path: String, body: String) -> String {
|
||
let clean: String = strip_query(path)
|
||
|
||
// Health is always reachable
|
||
if str_eq(method, "GET") {
|
||
if str_eq(clean, "/health") || str_eq(clean, "/") {
|
||
return route_health(method, path, body)
|
||
}
|
||
}
|
||
|
||
// ISE posting is auth-exempt (internal soul daemon, same host, no _auth key)
|
||
if str_eq(method, "POST") && str_eq(clean, "/api/neuron/state-events") {
|
||
return route_emit_ise(method, path, body)
|
||
}
|
||
|
||
// Auth (when ENGRAM_API_KEY is set)
|
||
if !check_auth_ok(method, body) {
|
||
return err_json("unauthorized")
|
||
}
|
||
|
||
// Knowledge capture (auth enforced above; the world-ingestor integrator
|
||
// and any headless session without MCP push knowledge through this)
|
||
if str_eq(method, "POST") && str_eq(clean, "/api/neuron/knowledge/capture") {
|
||
return route_capture_knowledge(method, path, body)
|
||
}
|
||
|
||
// Stats
|
||
if str_eq(method, "GET") && (str_eq(clean, "/api/stats") || str_eq(clean, "/stats")) {
|
||
return route_stats(method, path, body)
|
||
}
|
||
if str_eq(method, "GET") && (str_eq(clean, "/api/act-stats") || str_eq(clean, "/act-stats")) {
|
||
return route_act_stats(method, path, body)
|
||
}
|
||
if str_eq(method, "GET") && (str_eq(clean, "/api/text-health") || str_eq(clean, "/text-health")) {
|
||
return route_text_health(method, path, body)
|
||
}
|
||
|
||
// ── The universal set-based operation and its n=1 degenerate projections ──
|
||
// reframe = isolate → supersede-region → insert-manifold → rebind. write and
|
||
// supersede are the SAME reframe_core path at region size 0 and 1.
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/reframe") || str_eq(clean, "/reframe")) {
|
||
return route_reframe(method, path, body)
|
||
}
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/write") || str_eq(clean, "/write")) {
|
||
return route_write(method, path, body)
|
||
}
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/supersede") || str_eq(clean, "/supersede")) {
|
||
return route_supersede(method, path, body)
|
||
}
|
||
|
||
// Nodes
|
||
// Reseed must be tested before the exact "/api/nodes" match below reads
|
||
// as the general create path — order is not load-bearing (the match is
|
||
// exact) but keeping them adjacent keeps them from drifting apart.
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/nodes/reseed") || str_eq(clean, "/nodes/reseed")) {
|
||
return route_reseed_nodes(method, path, body)
|
||
}
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/nodes") || str_eq(clean, "/nodes")) {
|
||
return route_create_node(method, path, body)
|
||
}
|
||
if str_eq(method, "GET") && (str_eq(clean, "/api/nodes") || str_eq(clean, "/nodes") || str_eq(clean, "/nodes/list") || str_eq(clean, "/api/nodes/list")) {
|
||
return route_scan_nodes(method, path, body)
|
||
}
|
||
if str_eq(method, "GET") && (str_eq(clean, "/api/edges") || str_eq(clean, "/edges")) {
|
||
return route_scan_edges(method, path, body)
|
||
}
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/nodes/") {
|
||
return route_get_node(method, path, body)
|
||
}
|
||
// Singular alias: /api/node/<id>. Distinct prefix from /api/nodes/ ("node/"
|
||
// vs "nodes/"), so no collision with the plural route above.
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/node/") {
|
||
return route_get_node_singular(method, path, body)
|
||
}
|
||
if str_eq(method, "DELETE") && str_starts_with(clean, "/api/nodes/") {
|
||
return route_forget(method, path, body)
|
||
}
|
||
|
||
// Edges
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/edges") || str_eq(clean, "/edges")) {
|
||
return route_create_edge(method, path, body)
|
||
}
|
||
// Batch edge write — one snapshot for the whole payload. Must be tested
|
||
// BEFORE nothing else claims it; the exact-match on "/api/edges" above
|
||
// does not catch "/api/edges/batch", so order is not load-bearing here,
|
||
// but keeping the two adjacent keeps them from drifting apart.
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/edges/batch") || str_eq(clean, "/edges/batch")) {
|
||
return route_create_edges_batch(method, path, body)
|
||
}
|
||
// M10 reified neighborhoods (read-only viz surface). Checked before the
|
||
// /api/neighbors/ prefix; the two do not collide ("neighborhoods" vs
|
||
// "neighbors/") but keeping them adjacent documents the intent.
|
||
if str_eq(method, "GET") && (str_eq(clean, "/api/neighborhoods") || str_eq(clean, "/neighborhoods")) {
|
||
return route_neighborhoods(method, path, body)
|
||
}
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/neighborhoods/") {
|
||
return route_neighborhood(method, path, body)
|
||
}
|
||
// WRITE: run reification, persisting Neighborhood nodes + member edges.
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/reify") || str_eq(clean, "/reify")) {
|
||
return route_reify(method, path, body)
|
||
}
|
||
// WRITE: on-beat self-reification (flag-gated). Explicit pump for validation.
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/self-reify-beat") || str_eq(clean, "/self-reify-beat")) {
|
||
return route_self_reify_beat(method, path, body)
|
||
}
|
||
// WRITE: async explicit override — rename a reified neighborhood (→ residue).
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/rename") || str_eq(clean, "/rename")) {
|
||
return route_rename(method, path, body)
|
||
}
|
||
// READ-ONLY geometry operators over id-sets (?a=csv&b=csv[&mode=]).
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/gauge-distance") {
|
||
return route_gauge_distance(method, path, body)
|
||
}
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/recognize") {
|
||
return route_recognize(method, path, body)
|
||
}
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/discern") {
|
||
return route_discern(method, path, body)
|
||
}
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/synthesize") {
|
||
return route_synthesize(method, path, body)
|
||
}
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/nearest/") {
|
||
return route_nearest(method, path, body)
|
||
}
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/neighbors/") {
|
||
return route_neighbors(method, path, body)
|
||
}
|
||
|
||
// ── COGNITION: the ONE operation + grounding, surfaced live (2026-08-14).
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/boundary-proof") {
|
||
return route_boundary_proof(method, path, body)
|
||
}
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/think") {
|
||
return route_think(method, path, body)
|
||
}
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/reason") {
|
||
return route_faculty(path, "reason")
|
||
}
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/induce") {
|
||
return route_faculty(path, "induce")
|
||
}
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/abduce") {
|
||
return route_faculty(path, "abduce")
|
||
}
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/relate") {
|
||
return route_faculty(path, "relate")
|
||
}
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/analogize") {
|
||
return route_faculty(path, "analogy")
|
||
}
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/plan") {
|
||
return route_faculty(path, "plan")
|
||
}
|
||
if str_eq(method, "POST") && str_starts_with(clean, "/api/ground") {
|
||
return route_ground(method, path, body)
|
||
}
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/assert") {
|
||
return route_assert(method, path, body)
|
||
}
|
||
if str_eq(method, "POST") && str_starts_with(clean, "/api/attend") {
|
||
return route_attend(method, path, body)
|
||
}
|
||
if str_eq(method, "POST") && str_starts_with(clean, "/api/correspondence-beat") {
|
||
return route_correspondence_beat(method, path, body)
|
||
}
|
||
|
||
// ── GUIDE: the summoned interlocutor. status (read), consult (engage), and
|
||
// an explicit re-summon. consult/summon are auth-gated POSTs (covered above).
|
||
if str_eq(method, "GET") && (str_eq(clean, "/api/guide/status") || str_eq(clean, "/guide/status")) {
|
||
return route_guide_status(method, path, body)
|
||
}
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/guide/consult") || str_eq(clean, "/guide/consult")) {
|
||
return route_guide_consult(method, path, body)
|
||
}
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/guide/summon") || str_eq(clean, "/guide/summon")) {
|
||
return route_guide_summon(method, path, body)
|
||
}
|
||
|
||
// Activation + Search
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/activate") || str_eq(clean, "/activate")) {
|
||
return route_activate(method, path, body)
|
||
}
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/activate") {
|
||
return route_activate(method, path, body)
|
||
}
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/search-lexical") || str_eq(clean, "/search-lexical")) {
|
||
return route_search_lexical(method, path, body)
|
||
}
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/search-lexical") {
|
||
return route_search_lexical(method, path, body)
|
||
}
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/search") || str_eq(clean, "/search")) {
|
||
return route_search(method, path, body)
|
||
}
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/search") {
|
||
return route_search(method, path, body)
|
||
}
|
||
|
||
// Strengthen
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/strengthen") || str_eq(clean, "/strengthen")) {
|
||
return route_strengthen(method, path, body)
|
||
}
|
||
|
||
// Persistence
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/save") || str_eq(clean, "/save")) {
|
||
return route_save(method, path, body)
|
||
}
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/checkpoint") || str_eq(clean, "/checkpoint")) {
|
||
return route_checkpoint(method, path, body)
|
||
}
|
||
|
||
// M-INTEROCEPTION: chronoception soul-tick + self-drift (flag-gated)
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/tick") || str_eq(clean, "/tick")) {
|
||
return route_tick(method, path, body)
|
||
}
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/self_anchor") || str_eq(clean, "/self_anchor")) {
|
||
return route_self_anchor(method, path, body)
|
||
}
|
||
if (str_eq(method, "POST") || str_eq(method, "GET")) && (str_eq(clean, "/api/drift") || str_eq(clean, "/drift")) {
|
||
return route_drift(method, path, body)
|
||
}
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/load") || str_eq(clean, "/load")) {
|
||
return route_load(method, path, body)
|
||
}
|
||
if str_eq(method, "POST") && (str_eq(clean, "/api/load-merge") || str_eq(clean, "/load-merge")) {
|
||
return route_load_merge(method, path, body)
|
||
}
|
||
|
||
// Sync — soul daemon periodic pull of non-ISE knowledge into in-process graph
|
||
if str_eq(method, "GET") && str_eq(clean, "/api/sync") {
|
||
return route_sync(method, path, body)
|
||
}
|
||
|
||
// Embedding backfill — pumped by the soul heartbeat (2026-07-25)
|
||
if str_eq(clean, "/api/embed-backfill") {
|
||
return route_embed_backfill(method, path, body)
|
||
}
|
||
|
||
// Semantic similarity probe (2026-08-01)
|
||
if str_eq(method, "GET") && str_starts_with(clean, "/api/similarity") {
|
||
return route_similarity(method, path, body)
|
||
}
|
||
|
||
"{\"error\":\"not found\",\"path\":\"" + clean + "\"}"
|
||
}
|
||
|
||
// ── Entry ─────────────────────────────────────────────────────────────────────
|
||
|
||
let bind_raw: String = env("ENGRAM_BIND")
|
||
let bind_str: String = if str_eq(bind_raw, "") { ":8742" } else { bind_raw }
|
||
let port: Int = parse_port(bind_str)
|
||
|
||
// On startup, try to load any existing snapshot (best effort).
|
||
// §18.2: resolve the data dir safely — unset ENGRAM_DATA_DIR → $HOME/.neuron/engram,
|
||
// never /tmp; fail loud if HOME is unresolvable (engram_resolve_data_dir exits).
|
||
let data_dir: String = engram_resolve_data_dir()
|
||
let snapshot_path: String = data_dir + "/snapshot.json"
|
||
// ENGRAM_STORE (tiered paged store — engram-tiered-storage-engine.md). When set,
|
||
// the durable owner is the paged store (neuron.egm + neuron.wal): engram_store_boot
|
||
// imports snapshot.json ONCE into a fresh neuron.egm, else replays the WAL and loads
|
||
// the store resident — snapshot.json is never read again as the ongoing store. This
|
||
// closes the "restart reverted to a 17h-old snapshot" data-loss window. Flag-off
|
||
// (default): byte-for-byte the historical snapshot + optional-WAL boot below.
|
||
if store_on() {
|
||
engram_store_boot(data_dir)
|
||
println("[engram] ENGRAM_STORE enabled — tiered paged store is the durable owner")
|
||
} else {
|
||
engram_load(snapshot_path)
|
||
|
||
// WAL replay (design doc §6). Gated: default OFF is byte-identical to legacy
|
||
// snapshot-only boot. When ON, the snapshot above is the compaction BASE and
|
||
// the WAL carries every mutation since; replay reconstructs state to the last
|
||
// CRC-valid record, then opens the WAL for appending.
|
||
if wal_on() {
|
||
let replayed: Int = engram_wal_boot(data_dir)
|
||
println("[engram] WAL enabled — replayed " + int_to_str(replayed) + " records")
|
||
}
|
||
|
||
// 2026-07-21 self-review boot guard: if the snapshot file has content but the
|
||
// load produced 0 nodes, something is wrong (corrupt file / parse failure).
|
||
// Preserve the evidence and warn loudly — and since read routes no longer write
|
||
// the canonical path, a bad boot can no longer clobber the good snapshot.
|
||
let boot_snap: String = fs_read(snapshot_path)
|
||
if !str_eq(boot_snap, "") {
|
||
if engram_node_count() == 0 {
|
||
println("[engram] WARNING: snapshot.json is non-empty but load produced 0 nodes — preserving copy at snapshot.failed-load.json")
|
||
fs_write(data_dir + "/snapshot.failed-load.json", boot_snap)
|
||
} else {
|
||
// Good load: keep a boot-time backup of the snapshot as loaded.
|
||
fs_write(data_dir + "/snapshot.boot-backup.json", boot_snap)
|
||
}
|
||
}
|
||
}
|
||
|
||
println("[engram] runtime-native graph engine")
|
||
println("[engram] data_dir=" + data_dir)
|
||
println("[engram] node_count=" + int_to_str(engram_node_count()))
|
||
println("[engram] edge_count=" + int_to_str(engram_edge_count()))
|
||
println("[engram] listening on " + int_to_str(port))
|
||
|
||
// ── WAKE: summon the guide (soul-native). Flag-gated (GUIDE_ENABLE): default
|
||
// OFF returns immediately and leaves this boot byte-inert. When ON, the soul probes
|
||
// its hardware, selects a Qwen3 tier by spec, fetches the GGUF from HF if the local
|
||
// cache is cold, loads it via the backend, and binds consult_guide(). Idempotent
|
||
// across wakes — a cached model and an already-answering guide are both no-ops.
|
||
let guide_wake: String = guide_summon()
|
||
println("[guide] summon result: " + guide_wake)
|
||
|
||
http_set_handler("handle_request")
|
||
http_serve(port, "handle_request")
|