Files
el/engram/src/server.el
T
bigmerge bacaf3d39c
El SDK CI - dev / build-and-test (pull_request) Failing after 4m49s
engram: reconcile M8 HNSW vindex (#109) onto current dev, restore 3 fixes the branch predated
Lands feat/reframe-region-setop (PR #109: native set-based reframe_region,
decorator-as-seam @route port, teacher-summon, and the M8.1 activate-latency
work — lazy-memoized cosq via eg_cosq_at + engram_vindex HNSW-accelerated
seed discovery + vindex_harvest_from_store/vindex_bench oracle) onto dev's
actual current HEAD, plus engram-tiered-storage's still-unique test suite.

RECONCILING #109 WITH engram-tiered-storage (M4-M10 HNSW/geometry/reason/
verify work): not a two-way merge. engram_vindex.c's HNSW core (search_layer/
select_neighbors/prune_links/insert) is BYTE-IDENTICAL between the two
branches; #109's copy is a strict superset (adds vindex_harvest_from_store,
used by vindex_bench.c's brute-force-vs-HNSW oracle). engram_reason.c and
engram_verify.c are also byte-identical. #109's own branch point already
carried engram-tiered-storage's M4-M10 lineage forward, so there was nothing
left to merge into #109 for those files. The one thing engram-tiered-storage
had that #109's tree dropped: its full test suite (test_vindex.c,
test_geometry.c, test_reason.c, test_verify.c, test_m7_traversal.c, the
interoception P0-P5 tests, bufpool/compaction tests, and their run_*.sh
harnesses) — ported over here unchanged.

WHY THIS NEEDED HAND RECONCILIATION, NOT A MECHANICAL MERGE: #109's branch
forked from dev on 2026-08-14 15:40 (before restructure-adjacent history
diverged the file's merge-base for `git merge` — it presented as an add/add
conflict). A straight two-dot diff (dev tip -> PR tip) applied cleanly, but
it silently reverted THREE dev fixes landed on 2026-08-14/15, after the
branch point, that the PR's diff had no way to know about:

  1. qgate rescale (2026-08-14 self-review): PR's lazy eg_cosq_at rewrite of
     the query-aware propagation gate dropped the shift-and-floor rescale
     about ENGRAM_EMBED_S0 (measured: unrelated-pair median 0.562->raw gate
     0.67, i.e. "a small tax, not a gate"). Restored the rescale, wrapped
     around the lazy accessor -- the PR's actual improvement (WHEN cosq[oi]
     is computed) is orthogonal to WHAT it gates on and both are kept.
  2. Eviction cause decomposition (2026-08-14 self-review): dev decomposes
     wm_evicted into evict_floor/evict_cap/evict_bll so WM churn is
     diagnosable (identity: evicted == floor+cap+bll+dup_wm+dup_wm_global).
     PR's tree predates this and dropped all three counters + their JSON
     stats fields. Restored declarations, all 4 direct increment sites, the
     eg_wm_carry_over bll increment, and the act-stats JSON fields --
     alongside (not instead of) the PR's own P4 afferent / API-reshape
     counters already in that same struct/JSON.
  3. Hebbian link-formation selection (2026-08-15 self-review, TODAY): dev
     selects the STRONGEST qualifying candidate for consolidation each call;
     PR's tree predates this and reverted to hash-slot order (arbitrary wrt
     association strength) for edge formation -- the one path that writes
     PERMANENT structure. Restored the strongest-candidate while-loop,
     keeping the PR's own genuine improvement at that site
     (engram_adj_on_edge_added incremental-index append instead of a bare
     adj_dirty=1 full-rebuild flag).

engram/src/server.el's 3-way conflicts (autoconnect_on/ise_offgraph_on env
flags, /api/nodes connected-count in responses) were pure additive: dev's
side was empty, PR's side added the feature. Took PR's side whole.

VERIFIED (nsbx sandbox only, live :8742/:7770 never touched):
  - cc -std=c11 -O2, clean link against the real engram/src/server.el via
    elc, zero errors.
  - vindex_bench (built standalone, read-only harvest) against the real
    production store clone (13,671 embedded nodes, 768-dim nomic-embed-text):
    recall@10 = 1.0000 at ef 64/128/200; HNSW search 0.28-0.79ms/query vs
    2.03ms/query brute-force oracle (2.6x-7.2x). HNSW build itself: 46.5s
    for the full 13,671-node set -- see the flagged risk below.
  - Booted the reconciled binary in an isolated nsbx sandbox (:8905, cloned
    snapshot of the live store, 13,424 nodes / 37,656 edges) and called
    /api/activate for real: first call after boot 41.5s (pays the one-time
    HNSW build inline -- matches the standalone bench), second/third calls
    356ms/605ms, no crash, correct results, act-stats JSON (including the
    restored evict_floor/cap/bll fields) reads correctly.

KNOWN RISK TO FLAG BEFORE ANY LIVE CUTOVER (not fixed here; out of scope for
this dev-only land per instructions not to touch :8742/:7770): eg_vindex_sync
builds the HNSW index synchronously, inline, on the first engram_activate()
call after every process start (or index invalidation). On the real node
count that is a ~46s blocking stall on a single-threaded server -- the first
request after every restart (or its concurrent siblings) waits the full
build. Recommend a background/incremental build (or a bounded per-call build
budget) before this ever reaches the live daemon. See PR description / final
report for the fuller writeup.
2026-08-15 16:46:44 -05:00

1924 lines
97 KiB
EmacsLisp
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
// server.el Engram HTTP server.
//
// Engram is the in-process graph store. The runtime owns the data; this
// file is the thin HTTP face. Every route maps to one or two engram_*
// builtins. There is no SQL, no db layer, no SQLite the runtime IS the
// database.
//
// Built and linked with:
// elc src/server.el > server.c
// cc -std=c11 -O2 -lcurl -lpthread -o engram server.c el_runtime.c
// ./engram
//
// Configuration via environment:
// ENGRAM_BIND host:port (default :8742)
// ENGRAM_API_KEY bearer auth (optional)
// ENGRAM_DATA_DIR snapshot location (default ~/.neuron/engram)
// Helpers
fn parse_port(bind: String) -> Int {
// ":8742" 8742; "0.0.0.0:8742" → 8742; bare "8742" → 8742
let colon: Int = str_index_of(bind, ":")
if colon < 0 {
return str_to_int(bind)
}
let after: String = str_slice(bind, colon + 1, str_len(bind))
return str_to_int(after)
}
fn ok_json() -> String {
"{\"ok\":true}"
}
fn err_json(msg: String) -> String {
"{\"error\":\"" + msg + "\"}"
}
fn strip_query(path: String) -> String {
let q: Int = str_index_of(path, "?")
if q < 0 { return path }
str_slice(path, 0, q)
}
fn query_param(path: String, key: String) -> String {
let q: Int = str_index_of(path, "?")
if q < 0 { return "" }
let qs: String = str_slice(path, q + 1, str_len(path))
let needle: String = key + "="
let pos: Int = str_index_of(qs, needle)
if pos < 0 { return "" }
let after: String = str_slice(qs, pos + str_len(needle), str_len(qs))
let amp: Int = str_index_of(after, "&")
if amp < 0 { return after }
str_slice(after, 0, amp)
}
fn query_int(path: String, key: String, default_val: Int) -> Int {
let v: String = query_param(path, key)
if str_eq(v, "") { return default_val }
str_to_int(v)
}
// Extract last path segment after a known prefix: extract_id("/api/nodes/abc-123", "/api/nodes/") "abc-123"
fn extract_id(path: String, prefix: String) -> String {
let clean: String = strip_query(path)
if !str_starts_with(clean, prefix) { return "" }
let after: String = str_slice(clean, str_len(prefix), str_len(clean))
let slash: Int = str_index_of(after, "/")
if slash < 0 { return after }
str_slice(after, 0, slash)
}
// Routes
fn route_stats(method: String, path: String, body: String) -> String {
engram_stats_json()
}
// route_act_stats GET /api/act-stats
// (2026-08-04 self-review) engram_act_stats_json() has existed since the
// 2026-07-27 review but was reachable ONLY through the soul daemon's heartbeat
// binding. Every activation-layer gauge WM evictions, breakthroughs, embedder
// breaker state, context drift, and now the Hebbian counters was therefore
// invisible unless the soul happened to be running and its ISEs were read back
// out of the store. Diagnosing the activation layer required a working soul,
// which is exactly backwards: the lower layer should be observable on its own.
// This review needed it to verify link formation and could not get at it. One
// line of plumbing, and the whole activation layer becomes directly diagnosable.
fn route_act_stats(method: String, path: String, body: String) -> String {
engram_act_stats_json()
}
// route_text_health GET /api/text-health
// (2026-08-08 self-review) The daily census half of the text-integrity gauge.
// Today's review found that the JSON parser had been replacing every \uXXXX
// escape with a literal '?' for at least two months: 3,119 of 4,081
// non-telemetry nodes (76%) were damaged, including the self traversal root
// and every values node, and NOTHING detected it because every gauge in the
// system measured whether the machinery was running, and none measured whether
// the text it carried was intact. No snapshot on disk predates the damage, so
// it cannot be undone; it can only be made impossible to repeat quietly.
//
// The parser is fixed. This route is the standing check: `damaged` should now
// hold flat at its historical floor and never climb. `write_damaged` (also on
// the heartbeat as txt_damaged) is the live regression signal non-zero means
// a write path is mangling text right now.
fn route_text_health(method: String, path: String, body: String) -> String {
engram_text_health_json()
}
// (2026-07-18 self-review) Scoping sweep: `let` inside an if-block creates an
// inner scope only it does NOT mutate the outer binding (documented with
// evidence in awareness.el, 2026-05-25). Every default/reassignment below used
// that broken pattern, so defaults never applied: nodes were created with
// node_type="" and salience=0.0, /api/search and /api/activate ALWAYS ran with
// q="" regardless of input, edges defaulted to relation=""/weight=0.0, and
// save/load with no "path" hit engram_save(""). Rewritten to the
// `let x = if cond { a } else { b }` expression form (the pattern the newer
// routes route_emit_ise/route_capture_knowledge already use correctly).
// store_on ENGRAM_STORE flag (tiered paged store as the durable owner). Matches
// engram_store_enabled() in el_runtime.c EXACTLY (1 / on / true). Default off
// every persistence path below is byte-for-byte the historical snapshot behavior.
fn store_on() -> Bool {
let v: String = env("ENGRAM_STORE")
if str_eq(v, "1") { return true }
if str_eq(v, "on") { return true }
if str_eq(v, "true") { return true }
return false
}
// persist_canonical save the canonical snapshot after a durable write.
//
// WHY (2026-07-22 self-review): the 2026-07-21 fix correctly stopped READ
// routes from writing the canonical snapshot.json but nothing was left
// that saved it on WRITE. Every mutation (node create, edge create,
// knowledge capture, forget, merge) lived only in RAM until someone POSTed
// /api/save manually; a process restart silently discarded everything since
// the last manual save. Observed live: two engram restarts during the
// 2026-07-22 review reverted the store to a ~17h-old snapshot, destroying
// same-day writes. Reads must never write the canonical; writes must always
// persist it. ISE telemetry is deliberately excluded (48h-pruned, loss-
// tolerant, ~2/min snapshotting the whole store per heartbeat is waste;
// any durable write that follows persists the pruning too).
fn persist_canonical() -> Int {
// ENGRAM_STORE: the paged store is the durable owner a checkpoint flushes
// dirty pages behind a WAL-durable record (durable the moment the WAL fsyncs).
// This is the fix for the "restart reverted to a 17h-old snapshot" data loss:
// durable writes no longer depend on a full snapshot.json rewrite. Returns 1
// on a successful checkpoint, 0 otherwise. Flag-off: unchanged (writes JSON).
if store_on() {
return engram_store_checkpoint()
}
let dir_raw: String = env("ENGRAM_DATA_DIR")
let dir: String = engram_resolve_data_dir()
// (2026-08-10 self-review) This returned a hardcoded 1, which made every
// caller's `let saved: Int = persist_canonical()` a dead variable six
// durable write paths each believed they had confirmation of a successful
// canonical persist and none of them had any. Propagate the real result.
return engram_save(dir + "/snapshot.json")
}
// WAL persistence (design doc §§3-14; gated behind ENGRAM_WAL=on) ──────────
// Default OFF every persist path below is byte-identical to the historical
// per-write full-snapshot behavior. When ON, structural mutations append O(1)
// WAL records instead of rewriting the whole graph, with threshold compaction.
fn wal_on() -> Bool {
str_eq(env("ENGRAM_WAL"), "on")
}
// autoconnect_on ENGRAM_AUTOCONNECT. Will's rule: "we shouldn't be inserting
// orphaned nodes." When ON, every content-node insert forms >=1 semantic-similar
// edge (kNN over embeddings) so no content node enters the graph edgeless.
// Default OFF -> byte-identical to prior behavior (node created, no auto edges).
fn autoconnect_on() -> Bool {
let v: String = env("ENGRAM_AUTOCONNECT")
if str_eq(v, "1") { return true }
if str_eq(v, "on") { return true }
if str_eq(v, "true") { return true }
return false
}
// ise_offgraph_on ENGRAM_ISE_OFFGRAPH. The census showed ~8k of the ~8.9k
// orphans are InternalStateEvent telemetry (heartbeat/curiosity_scan/session-
// start), 100% edgeless by design. When ON, that telemetry is routed to a
// separate state-event log tier instead of the node graph. Default OFF -> ISEs
// remain graph nodes exactly as before (with 48h prune).
fn ise_offgraph_on() -> Bool {
let v: String = env("ENGRAM_ISE_OFFGRAPH")
if str_eq(v, "1") { return true }
if str_eq(v, "on") { return true }
if str_eq(v, "true") { return true }
return false
}
// Persist a single-node mutation (create / content-evolve / strengthen).
fn persist_node(id: String) -> Int {
if wal_on() {
let d: String = engram_resolve_data_dir()
let a: Int = engram_wal_node_put(d, id)
let c: Int = engram_wal_maybe_compact(d)
return a
}
return persist_canonical()
}
// Persist edges appended at index >= start (covers single-edge and batch).
fn persist_edges_since(start: Int) -> Int {
if wal_on() {
let d: String = engram_resolve_data_dir()
let a: Int = engram_wal_edges_since(d, start)
let c: Int = engram_wal_maybe_compact(d)
return a
}
return persist_canonical()
}
// Persist a Hebbian consolidation batch as ONE WAL record (single fsync, §5-B).
fn persist_hebb_batch(start: Int) -> Int {
if wal_on() {
let d: String = engram_resolve_data_dir()
let a: Int = engram_wal_hebb_batch(d, start)
let c: Int = engram_wal_maybe_compact(d)
return a
}
return persist_canonical()
}
// Bulk mutation (embedding backfill, load-merge): write a fresh compaction base
// so the many-node change is durable in one atomic snapshot; WAL is truncated.
fn persist_bulk() -> Int {
if wal_on() {
let d: String = engram_resolve_data_dir()
return engram_wal_compact(d)
}
return persist_canonical()
}
// INCOMPLETE-ROUTE FIX (2026-07-24 self-review): this route silently dropped
// label, importance, tier, and tags engram_node() defaults label to content
// and importance to 0.5, so every node created over HTTP lost its metadata.
// Observed live: the soul's boot-counter write-back landed with
// label="soul:boot_count:99" (content), importance 0.5, no tags. Honor the
// full field set via engram_node_full when any of them is supplied.
// PRESENCE-AWARE DEFAULTS (2026-08-01 self-review): the old pattern
// `if x == 0.0 { default }` made a legitimate 0.0 unrepresentable a caller
// setting salience/importance/weight to zero silently got 0.5. json_get_raw
// returns "" when the key is ABSENT and the raw token when present, so
// absence and zero are now distinguishable. Also: confidence was hardcoded
// to 1.0 regardless of input every HTTP-created node claimed full
// epistemic confidence. Now honored from the payload (default 1.0).
fn route_create_node(method: String, path: String, body: String) -> String {
let content: String = json_get_string(body, "content")
let nt_raw: String = json_get_string(body, "node_type")
let node_type: String = if str_eq(nt_raw, "") { "Memory" } else { nt_raw }
let sal_present: String = json_get_raw(body, "salience")
let salience: Float = if str_eq(sal_present, "") { 0.5 } else { json_get_float(body, "salience") }
let label_raw: String = json_get_string(body, "label")
let label: String = if str_eq(label_raw, "") { content } else { label_raw }
let imp_present: String = json_get_raw(body, "importance")
let importance: Float = if str_eq(imp_present, "") { 0.5 } else { json_get_float(body, "importance") }
let conf_present: String = json_get_raw(body, "confidence")
let confidence: Float = if str_eq(conf_present, "") { 1.0 } else { json_get_float(body, "confidence") }
let tier_raw: String = json_get_string(body, "tier")
let tier: String = if str_eq(tier_raw, "") { "Working" } else { tier_raw }
let tags: String = json_get_string(body, "tags")
// NO el_from_float WRAPPER (2026-08-01 self-review): salience/importance/
// confidence are already Float (el_val_t) values json_get_float and
// Float literals both encode. Wrapping them in el_from_float AGAIN
// reinterpreted the boxed bits as a raw double, producing garbage that
// failed engram_decode_score's range check and clamped every HTTP-created
// node to defaults (salience 0.9 in 0.5 stored; confidence 0.6 in → 1.0
// stored verified live). route_emit_ise always passed Floats bare and
// its 0.3/0.3/0.8 stored correctly; this call now does the same.
let id: String = engram_node_full(
content, node_type, label,
salience, importance, confidence,
tier, tags
)
let saved: Int = persist_node(id)
// ORPHAN PREVENTION (ENGRAM_AUTOCONNECT): connect the fresh node to its
// nearest embedded neighbors so it never enters the graph edgeless.
let connected: Int = if autoconnect_on() {
let ec0: Int = engram_edge_count()
let ac: String = engram_autoconnect_node(id, 3, 25)
let added: Int = engram_edge_count() - ec0
if added > 0 { let sv2: Int = persist_edges_since(ec0) }
added
} else { 0 }
"{\"id\":\"" + id + "\",\"content\":\"" + content + "\",\"node_type\":\"" + node_type + "\",\"connected\":" + int_to_str(connected) + "}"
}
fn route_get_node(method: String, path: String, body: String) -> String {
let id: String = extract_id(path, "/api/nodes/")
if str_eq(id, "") { return err_json("missing id") }
return engram_get_node_json(id)
}
// route_get_node_singular GET /api/node/<id>. Singular alias for node-by-id
// fetch. The plural /api/nodes/<id> already resolves; the viz's "see full node
// value on click" and other clients call the SINGULAR form, which had no route
// and 404'd for every id. Same handler, singular prefix. Read-only.
fn route_get_node_singular(method: String, path: String, body: String) -> String {
let id: String = extract_id(path, "/api/node/")
if str_eq(id, "") { return err_json("missing id") }
return engram_get_node_json(id)
}
fn route_scan_nodes(method: String, path: String, body: String) -> String {
let limit: Int = query_int(path, "limit", 50)
let offset: Int = query_int(path, "offset", 0)
let nt: String = query_param(path, "node_type")
if str_eq(nt, "") {
return engram_scan_nodes_json(limit, offset)
}
return engram_scan_nodes_by_type_json(nt, limit, offset)
}
// route_scan_edges — bulk export of all edges as a JSON array. Implemented
// via engram_save → fs_read of a SCRATCH export path. (2026-07-21 self-review:
// previously this saved over the canonical snapshot.json on every GET — if the
// process ever booted with a partial/empty store, the first read request
// clobbered the good snapshot. Read routes must never write the canonical path.)
fn route_scan_edges(method: String, path: String, body: String) -> String {
let dir_raw: String = env("ENGRAM_DATA_DIR")
let dir: String = engram_resolve_data_dir()
let snap_path: String = dir + "/.scan-export.json"
engram_save(snap_path)
let snap: String = fs_read(snap_path)
if str_eq(snap, "") { return "[]" }
// json_get truncates at the first delimiter (no bracket depth tracking),
// so for the edges ARRAY value we need json_get_raw, which honors
// brackets and returns the full sub-JSON.
let edges: String = json_get_raw(snap, "edges")
if str_eq(edges, "") { return "[]" }
return edges
}
fn route_search(method: String, path: String, body: String) -> String {
let q: String = if str_eq(method, "GET") { query_param(path, "q") } else { json_get_string(body, "query") }
let lim_url: Int = query_int(path, "limit", 0)
let lim_body: Int = json_get_int(body, "limit")
let lim_either: Int = if lim_url > 0 { lim_url } else { lim_body }
let limit: Int = if lim_either > 0 { lim_either } else { 20 }
return engram_retrieve_geometric_json(q, limit)
}
fn route_search_lexical(method: String, path: String, body: String) -> String {
let q: String = if str_eq(method, "GET") { query_param(path, "q") } else { json_get_string(body, "query") }
let lim_url: Int = query_int(path, "limit", 0)
let lim_body: Int = json_get_int(body, "limit")
let lim_either: Int = if lim_url > 0 { lim_url } else { lim_body }
let limit: Int = if lim_either > 0 { lim_either } else { 20 }
return engram_search_json(q, limit)
}
fn route_activate(method: String, path: String, body: String) -> String {
let q: String = if str_eq(method, "GET") { query_param(path, "q") } else { json_get_string(body, "query") }
// Guard: engram_activate with an empty query matches zero seeds, which
// zeroes ALL carried working-memory weights (documented in awareness.el
// perceive()). Never let an empty activation through to wipe WM.
if str_eq(q, "") { return err_json("missing query") }
let d_raw: Int = if str_eq(method, "GET") { query_int(path, "depth", 3) } else { json_get_int(body, "depth") }
let depth: Int = if d_raw > 0 { d_raw } else { 3 }
return "{\"results\":" + engram_activate_json(q, depth) + "}"
}
fn route_create_edge(method: String, path: String, body: String) -> String {
let from_id: String = json_get_string(body, "from_id")
let to_id: String = json_get_string(body, "to_id")
let rel_raw: String = json_get_string(body, "relation")
let relation: String = if str_eq(rel_raw, "") { "associates" } else { rel_raw }
// Presence-aware (2026-08-01): weight 0.0 is a legitimate edge weight
// (dormant association); only default when the key is absent.
let w_present: String = json_get_raw(body, "weight")
let weight: Float = if str_eq(w_present, "") { 0.5 } else { json_get_float(body, "weight") }
let ec0: Int = engram_edge_count()
engram_connect(from_id, to_id, weight, relation)
let saved: Int = persist_edges_since(ec0)
"{\"ok\":true,\"from_id\":\"" + from_id + "\",\"to_id\":\"" + to_id + "\",\"relation\":\"" + relation + "\"}"
}
// route_create_edges_batch — POST /api/edges/batch {"edges":[{from_id,to_id,relation,weight}, ...]}
//
// WHY THIS EXISTS (2026-08-07 self-review). persist_canonical() writes the
// FULL canonical snapshot — 60MB at current graph size — and route_create_edge
// calls it once per edge. That is correct for the interactive one-edge case and
// ruinous for any bulk write: the soul's Hebbian consolidation path delivers
// ~14 associations per 8-minute heartbeat, which through the single-edge route
// would be ~840MB of disk writes per beat, ~150GB/day, to persist 14 edges.
//
// The fix is not to weaken durability — it is to make the unit of durability
// the BATCH. Connect every edge, then snapshot exactly once. Same guarantee
// (nothing acknowledged is lost to a restart), 1/N the writes. Empty or
// malformed entries are skipped rather than aborting the batch: a consolidation
// payload is best-effort by design, and one bad id should not cost the other 13.
//
// Returns the accepted count so the caller can tell delivery from silence.
fn route_create_edges_batch(method: String, path: String, body: String) -> String {
let arr: String = json_get_raw(body, "edges")
if str_eq(arr, "") { return err_json("missing edges array") }
let n: Int = json_array_len(arr)
if n == 0 { return "{\"ok\":true,\"accepted\":0,\"skipped\":0}" }
let ec0: Int = engram_edge_count()
let i: Int = 0
let accepted: Int = 0
let skipped: Int = 0
while i < n {
let item: String = json_array_get(arr, i)
let from_id: String = json_get_string(item, "from_id")
let to_id: String = json_get_string(item, "to_id")
if str_eq(from_id, "") || str_eq(to_id, "") {
let skipped = skipped + 1
} else {
let rel_raw: String = json_get_string(item, "relation")
let relation: String = if str_eq(rel_raw, "") { "associates" } else { rel_raw }
let w_present: String = json_get_raw(item, "weight")
let weight: Float = if str_eq(w_present, "") { 0.5 } else { json_get_float(item, "weight") }
engram_connect(from_id, to_id, weight, relation)
let accepted = accepted + 1
}
let i = i + 1
}
// ONE snapshot for the whole batch — the entire point of this route.
// Skip it when nothing was accepted: an all-malformed payload must not
// trigger a 60MB write.
if accepted > 0 {
let saved: Int = persist_hebb_batch(ec0)
}
return "{\"ok\":true,\"accepted\":" + int_to_str(accepted) + ",\"skipped\":" + int_to_str(skipped) + "}"
}
fn route_neighbors(method: String, path: String, body: String) -> String {
let id: String = extract_id(path, "/api/neighbors/")
if str_eq(id, "") { return err_json("missing id") }
let depth: Int = query_int(path, "depth", 1)
return engram_neighbors_json(id, depth, "both")
}
fn route_strengthen(method: String, path: String, body: String) -> String {
let id: String = json_get_string(body, "node_id")
if str_eq(id, "") { return err_json("missing node_id") }
engram_strengthen(id)
let saved: Int = persist_node(id)
ok_json()
}
// route_forget — DELETE /api/nodes/:id — INTEGRITY HARDENED (design doc §18.1).
//
// Two invariants now enforced AT THE STORE (not one layer up in neuron-api.el,
// which a direct HTTP client could bypass):
// 1. Write-protection: protected identity/value nodes (derived from the self
// graph — self root + values hub + their neighbors, §18.3) cannot be
// deleted over HTTP. Returns 403, node untouched.
// 2. No hard delete over the wire, ever: an ordinary delete creates a
// Tombstone marker node + `tombstones` edge and KEEPS the original node
// and its edges (recoverable), instead of the old destructive
// engram_forget() shift-delete. Raw engram_forget is now internal-GC only
// and no longer reachable from any HTTP route.
fn route_forget(method: String, path: String, body: String) -> String {
let id: String = extract_id(path, "/api/nodes/")
if str_eq(id, "") { return err_json("missing id") }
if engram_is_protected(id) == 1 {
return "{\"__status__\":403,\"error\":\"protected node; deletion refused\",\"id\":\"" + id + "\"}"
}
let tomb_id: String = engram_node_full(
"tombstone:" + id, "Tombstone", "tombstone:" + id,
0.1, 0.1, 1.0, "Episodic", "[\"tombstone\"]"
)
let ec0: Int = engram_edge_count()
engram_connect(tomb_id, id, 1.0, "tombstones")
let saved: Int = if wal_on() {
let d: String = engram_resolve_data_dir()
let a: Int = engram_wal_node_put(d, tomb_id)
let b: Int = engram_wal_edges_since(d, ec0)
let c: Int = engram_wal_maybe_compact(d)
a
} else {
persist_canonical()
}
"{\"ok\":true,\"tombstoned\":\"" + id + "\",\"tombstone_id\":\"" + tomb_id + "\"}"
}
fn route_save(method: String, path: String, body: String) -> String {
let p_raw: String = json_get_string(body, "path")
let dir_raw: String = env("ENGRAM_DATA_DIR")
let dir: String = engram_resolve_data_dir()
let p: String = if str_eq(p_raw, "") { dir + "/snapshot.json" } else { p_raw }
// (2026-08-10 self-review) engram_save returns 0 on an empty path and the
// route discarded it, so the response was a literal "ok":true regardless
// of whether anything was written. Report the actual result AND the counts
// that were supposed to have been written the same move that made
// route_health honest on 2026-08-01. A caller can now tell "saved 13k
// nodes" from "saved nothing and said ok".
let sv: Int = engram_save(p)
let sv_ok: String = if sv == 0 { "false" } else { "true" }
"{\"ok\":" + sv_ok + ",\"path\":\"" + p + "\",\"node_count\":" + int_to_str(engram_node_count()) + ",\"edge_count\":" + int_to_str(engram_edge_count()) + "}"
}
// route_checkpoint force a full resident-graph checkpoint into the paged
// store (2026-08-13). Neither /api/save nor /api/load makes edges durable in
// the tiered paged store only persist_canonical -> engram_store_checkpoint
// does, and that only fires on mutating write routes. This route exposes the
// checkpoint directly so a RAM-loaded state (e.g. after /api/load of an export)
// can be made durable without a rebuild. Returns ok=false if ENGRAM_STORE is
// off (nothing to checkpoint into) or the checkpoint fails.
fn route_checkpoint(method: String, path: String, body: String) -> String {
let ck: Int = engram_store_checkpoint()
let ck_ok: String = if ck == 0 { "false" } else { "true" }
"{\"ok\":" + ck_ok + ",\"checkpointed\":true,\"node_count\":" + int_to_str(engram_node_count()) + ",\"edge_count\":" + int_to_str(engram_edge_count()) + "}"
}
// route_tick chronoception soul-tick (M-INTEROCEPTION P2 wiring, 2026-08-13).
// The soul heartbeat pumps this once per beat: node ages advance by the MEASURED
// wall-clock delta since the previous tick (self-seeding sidecar stamp). Returns
// the cooling magnitude in [0,1). Inert (magnitude 0) unless ENGRAM_CHRONOCEPTION
// is set, so wiring this route is safe by default the flag is the cutover gate.
fn route_tick(method: String, path: String, body: String) -> String {
let mag: Float = engram_chrono_tick()
// SELF-REIFICATION on the heartbeat (2026-08-14). Reification runs HERE, next
// to chronoception unbidden, every beat. Flag-gated (ENGRAM_SELF_REIFY):
// when off, engram_self_reify_beat_json() returns {"enabled":false} and writes
// nothing, so an unset flag leaves the existing tick byte-inert. Enabling the
// flag alone activates continuous autonomous neighborhood formation on the beat
// no heartbeat-script change needed. Idempotent: unchanged regions append
// nothing, so a settled store's beat is write-free (convergent under barrier/GC).
let reify: String = engram_self_reify_beat_json()
"{\"ok\":true,\"cooling_magnitude\":" + float_to_str(mag) + ",\"self_reify\":" + reify + "}"
}
// route_self_reify_beat POST /api/self-reify-beat. Explicit pump of ONE on-beat
// self-reification pass (the same operation route_tick folds in). Used by the
// secondary-soul validation harness to drive many beats deterministically. Inert
// (writes nothing) unless ENGRAM_SELF_REIFY is set. WRITE op auth-gated POST.
fn route_self_reify_beat(method: String, path: String, body: String) -> String {
return engram_self_reify_beat_json()
}
// route_rename POST /api/rename {"neighborhood_id":"...","name":"..."}. The
// DEGENERATE, ASYNCHRONOUS, NON-BLOCKING override: Will (or a self-facet) renames
// a reified neighborhood at any time; it SUPERSEDES the autonomous record into the
// residue chain (cause="explicit-override", prior name retained). Never halts or
// gates the beat it just writes a superseding record the next beat sees. WRITE.
fn route_rename(method: String, path: String, body: String) -> String {
let id: String = json_get_string(body, "neighborhood_id")
if str_eq(id, "") { return err_json("missing neighborhood_id") }
let name: String = json_get_string(body, "name")
if str_eq(name, "") { return err_json("missing name") }
return engram_neighborhood_rename_json(id, name)
}
// route_self_anchor capture the durable SelfAnchor drift baseline (the current
// self-geometry). Returns ok=false if ENGRAM_DRIFT_SENSOR is off or geometry is
// unavailable. Pump once to establish the baseline before reading drift.
fn route_self_anchor(method: String, path: String, body: String) -> String {
let a: Int = engram_self_anchor_capture()
let a_ok: String = if a == 0 { "false" } else { "true" }
"{\"ok\":" + a_ok + ",\"anchored\":" + a_ok + "}"
}
// route_drift live self-drift reading: displacement of CURRENT self-geometry
// vs the persisted SelfAnchor (GROWTH vs CORRUPTION split). {"error":...} when
// disabled / no anchor yet. The heartbeat can fold this into its ISEs.
fn route_drift(method: String, path: String, body: String) -> String {
return engram_self_drift_json()
}
fn route_load(method: String, path: String, body: String) -> String {
let p_raw: String = json_get_string(body, "path")
let dir_raw: String = env("ENGRAM_DATA_DIR")
let dir: String = engram_resolve_data_dir()
let p: String = if str_eq(p_raw, "") { dir + "/snapshot.json" } else { p_raw }
// (2026-08-10 self-review) This was a stub response over the single most
// destructive operation in the server. engram_load returns 0 on an empty
// path, an unopenable file, a zero-length file, or malloc failure and
// this route answered ok_json() in every one of those cases.
//
// Precise failure shape (el_runtime.c:9890): the fopen guard runs BEFORE
// the store reset, so a MISSING path is genuinely safe it returns 0 with
// the graph intact. The dangerous case is a readable-but-malformed file:
// the reset loop frees every node and edge FIRST, then parses, so a
// truncated or non-snapshot JSON leaves a hollow store and the caller
// was told "ok":true. With 37 GB of stale dated snapshots sitting in the
// data dir as tempting restore targets, "restore reported success and
// silently emptied the graph" is a live risk, not a hypothetical one.
//
// Fix: surface the return value AND the resulting counts. node_count=0
// after a load is the unambiguous hollow-store signal (same convention
// route_health adopted 2026-08-01). Callers can now verify a restore
// instead of trusting it.
let ld: Int = engram_load(p)
let ld_ok: String = if ld == 0 { "false" } else { "true" }
let nc_after: Int = engram_node_count()
let hollow: String = if nc_after == 0 { "true" } else { "false" }
"{\"ok\":" + ld_ok + ",\"path\":\"" + p + "\",\"node_count\":" + int_to_str(nc_after) + ",\"edge_count\":" + int_to_str(engram_edge_count()) + ",\"hollow\":" + hollow + "}"
}
// (2026-08-01 self-review) Health previously returned a hardcoded literal
// it reported "ok" even when the snapshot failed to load and the store was
// empty. Now reports live counts so a monitor can distinguish "up and
// loaded" from "up and hollow" (node_count=0 after boot = failed load).
fn route_health(method: String, path: String, body: String) -> String {
"{\"status\":\"ok\",\"engine\":\"engram-runtime-native\",\"node_count\":" + int_to_str(engram_node_count()) + ",\"edge_count\":" + int_to_str(engram_edge_count()) + "}"
}
// route_embed_backfill GET/POST /api/embed-backfill?n=48
//
// (2026-07-25 self-review) The lazy embedding backfill runs only inside
// engram_activate, and nothing in production calls /api/activate on this
// store the soul's curiosity loop activates its own in-process graph.
// After a restart from a snapshot without vectors, embedded_count stalled
// at 93/12175 and would never recover. This route lets the soul's
// heartbeat pump the backfill explicitly (48/min clears a 12k backlog in
// ~4h). Persists the canonical snapshot whenever new vectors were
// generated the 2026-07-25 regression happened precisely because 3747
// in-RAM embeddings were never snapshotted before a restart. Self-
// limiting: once coverage is full, embedded=0 and no save occurs.
fn route_embed_backfill(method: String, path: String, body: String) -> String {
let n: Int = query_int(path, "n", 32)
let result: String = engram_embed_backfill(n)
let done: Float = json_get_float(result, "embedded")
if done > 0.0 {
let saved: Int = persist_bulk()
}
return result
}
// route_sync return a snapshot of non-ISE/non-Working nodes for the soul daemon
// to merge into its in-process graph via engram_load_merge.
//
// The soul calls GET /api/sync every SOUL_REFRESH_MS (default 10 min) to pull
// new Knowledge/Memory/BacklogItem nodes from the authoritative HTTP Engram into
// its in-process working store. Previously this returned 404 "not found", causing
// the soul to write the error JSON to a temp file and attempt an empty merge.
//
// Strategy: save the current snapshot to disk, read it back, return the full
// snapshot JSON. The soul's engram_load_merge handles large files gracefully
// (it skips nodes already present by ID). Auth-exempt: same-host internal call.
// (2026-06-27 self-review: added this route to fix silent 10-min sync failures)
fn route_sync(method: String, path: String, body: String) -> String {
let dir_raw: String = env("ENGRAM_DATA_DIR")
let dir: String = engram_resolve_data_dir()
// 2026-07-21 self-review: export to a scratch path, never the canonical
// snapshot.json read routes must not be able to clobber the good snapshot.
let snap_path: String = dir + "/.sync-export.json"
engram_save(snap_path)
let snap: String = fs_read(snap_path)
// 2026-08-02 self-review: this used to return {"nodes":[],"edges":[]} when
// the export/read failed. The soul's sync_ok test (awareness.el) only
// checks for "" and "{}", so that placeholder PASSED as a healthy sync:
// soul.last_sync_ok_ts got stamped, sync_age_ms stayed green, the
// sync_empty warn ISE never fired, and engram_sync reported added:0
// forever. A totally broken sync was indistinguishable from a quiet
// healthy one the exact failure class this route was added to fix in
// the first place (see 2026-06-27 note above). Return a real error so the
// failure is loud on both sides.
if str_eq(snap, "") { return err_json("sync export failed: snapshot unreadable") }
return snap
}
// route_load_merge POST /api/load-merge {"path": "..."} merge a snapshot
// file into the live store WITHOUT resetting it (engram_load_merge skips nodes
// already present by id). Added 2026-07-21 self-review to restore the 244 kn-
// identity Knowledge nodes lost from the snapshot lineage between 05-13 and
// 07-13. Requires an explicit path: refuses to run without one so it can never
// be triggered accidentally against a default.
fn route_load_merge(method: String, path: String, body: String) -> String {
let p: String = json_get_string(body, "path")
if str_eq(p, "") { return err_json("path is required") }
if str_eq(fs_read(p), "") { return err_json("file missing or empty") }
let before_n: Int = engram_node_count()
let before_e: Int = engram_edge_count()
engram_load_merge(p)
let added_n: Int = engram_node_count() - before_n
let added_e: Int = engram_edge_count() - before_e
let saved: Int = persist_bulk()
"{\"ok\":true,\"nodes_added\":" + int_to_str(added_n) + ",\"edges_added\":" + int_to_str(added_e) + ",\"node_count\":" + int_to_str(engram_node_count()) + "}"
}
// route_reseed_nodes POST /api/nodes/reseed
// {"path": "<snapshot-format file>", "replace": ["<id>", ...],
// "preserve_edges": true, "_auth": "<key>"}
//
// ID-PRESERVING install/repair for declarative seed graphs.
//
// WHY THIS EXISTS (2026-08-10). Two write paths could put a node into the
// graph and neither can put a BODY onto an id that already exists:
// POST /api/nodes mints a fresh id via engram_node_full, and
// POST /api/load-merge honors the declared id but SKIPS anything already
// present. That is exactly right for the additive case and leaves one hole:
// a node that exists with a truncated body. Forge's genesis seed hit it
// two identity nodes (Voice, Voice Craft) sat in the graph carrying only
// their own label as content, 30 and 22 bytes against 4263 and 2590 in the
// seed. Their ids are load-bearing (is_protected_node keys on them and 214
// declared edges reference them), so "delete and recreate with a new id" is
// not a repair, it is a second break.
//
// Mechanism: engram has no in-place node update, so a replace is
// forget-then-merge. engram_forget also drops every INCIDENT EDGE for
// those two nodes that is 85 and 93 edges, almost all of them tag edges and
// accumulated hebbian associations that the seed does not declare and could
// not restore. preserve_edges (default true) therefore snapshots the graph
// before the forget and re-merges that snapshot afterwards: the replaced
// node is back by then so it is skipped, and every dropped incident edge
// returns through load_merge's (from_id,to_id,relation) dedup. The same
// re-merge is the failure path if the seed merge does not produce the
// node, the backup puts the original back. Rollback, not data loss.
//
// preserve_edges=false skips the two snapshot round-trips (cheap, lossy);
// use it only on a graph whose edges are fully declared by the seed.
// With no "replace" list this route is exactly /api/load-merge.
fn route_reseed_nodes(method: String, path: String, body: String) -> String {
let p: String = json_get_string(body, "path")
if str_eq(p, "") { return err_json("path is required") }
if str_eq(fs_read(p), "") { return err_json("file missing or empty") }
let dir_raw: String = env("ENGRAM_DATA_DIR")
let dir: String = if str_eq(dir_raw, "") { "/tmp/engram" } else { dir_raw }
let backup: String = dir + "/.reseed-backup.json"
let replace_raw: String = json_get_raw(body, "replace")
let n_replace: Int = json_array_len(replace_raw)
// Presence-aware: absent key means "preserve", only an explicit false opts out.
let pe_raw: String = json_get_raw(body, "preserve_edges")
let preserve: Bool = !str_eq(pe_raw, "false")
let before_n: Int = engram_node_count()
let before_e: Int = engram_edge_count()
let replaced: Int = 0
if n_replace > 0 {
if preserve { engram_save(backup) }
let i: Int = 0
while i < n_replace {
let rid: String = json_array_get_string(replace_raw, i)
if !str_eq(rid, "") {
// engram_get_node_json returns "{}" for a miss only forget
// ids that are actually resident, so a typo in the replace
// list is a no-op rather than a silent partial run.
let existing: String = engram_get_node_json(rid)
if !str_eq(existing, "{}") {
engram_forget(rid)
let replaced = replaced + 1
}
}
let i = i + 1
}
}
engram_load_merge(p)
if replaced > 0 {
if preserve { engram_load_merge(backup) }
}
let saved: Int = persist_canonical()
"{\"ok\":true,\"replaced\":" + int_to_str(replaced) +
",\"nodes_added\":" + int_to_str(engram_node_count() - before_n) +
",\"edges_added\":" + int_to_str(engram_edge_count() - before_e) +
",\"node_count\":" + int_to_str(engram_node_count()) +
",\"edge_count\":" + int_to_str(engram_edge_count()) + "}"
}
// route_emit_ise write an InternalStateEvent node from the soul daemon.
//
// Endpoint: POST /api/neuron/state-events
// Body: {"content": "<json-string>"}
//
// Auth: exempt (internal endpoint, soul daemon on same host, no _auth needed).
// The soul's ise_post() sends {"content":"..."} without _auth; enforcing auth
// here would silently drop all heartbeat/curiosity ISEs. Unauthenticated POST
// to this endpoint is acceptable: ISE writes are observability-only, append-only,
// and come from a trusted process on localhost.
//
// Salience/importance set to match engram_node_full ISE defaults used by the
// in-process fallback path in awareness.el (salience=0.3, importance=0.3,
// confidence=0.8, tier=Episodic).
// (2026-06-26 self-review: added this route after discovering ise_post was
// silently failing the soul posts here but the endpoint didn't exist.)
//
// Retention (2026-07-16 self-review): an earlier comment here claimed ISEs
// got temporal_decay_rate=1.617 that was never implemented (engram_node_full
// hardcodes 0.0), and per-node decay only dampens activation anyway; it never
// removes nodes. By 2026-07-16 ISEs were 75% of the store (10,175 of 13,522
// nodes, ~4,300/day, unbounded). ISEs are already WM-excluded in
// engram_activate, so the fix is retention, not decay: every insert calls
// engram_prune_telemetry(), a single O(nodes+edges) compaction pass that
// removes ISEs older than ENGRAM_ISE_RETENTION_MS (default 48h), protecting
// "session-start" labels and self_review events as durable history. At
// ~3 ISEs/min this bounds telemetry at ~8.6k nodes instead of growing forever.
fn route_emit_ise(method: String, path: String, body: String) -> String {
let content: String = json_get_string(body, "content")
if str_eq(content, "") { return err_json("missing content") }
// TELEMETRY OFF-GRAPH (ENGRAM_ISE_OFFGRAPH): route the state event to a
// separate append-only log tier instead of inserting an edgeless graph node.
// dream-recall (engram_dreams_json) reads the log tail when this is on, so
// the one in-graph consumer is preserved. Default OFF -> existing behavior.
if ise_offgraph_on() {
let ok: Int = engram_ise_log_append(content)
let ok_s: String = if ok == 0 { "false" } else { "true" }
return "{\"ok\":" + ok_s + ",\"offgraph\":true}"
}
let sal: Float = 0.3
let imp: Float = 0.3
let conf: Float = 0.8
let id: String = engram_node_full(
content, "InternalStateEvent", "state-event",
sal, imp, conf,
"Episodic", "[\"internal-state\",\"InternalStateEvent\"]"
)
let ret_raw: String = env("ENGRAM_ISE_RETENTION_MS")
let ret_ms: Int = if str_eq(ret_raw, "") { 172800000 } else { str_to_int(ret_raw) }
let pruned: Int = engram_prune_telemetry(ret_ms)
"{\"ok\":true,\"id\":\"" + id + "\",\"pruned\":" + int_to_str(pruned) + "}"
}
// Knowledge capture
//
// route_capture_knowledge direct Knowledge-node capture over HTTP.
//
// Endpoint: POST /api/neuron/knowledge/capture (auth required: "_auth" in body)
// Body: {"content": "...", "title": "...", "category": "...",
// "tier": "note|lesson|canonical", "tags": [...], "project": "...",
// "_auth": "<key>"}
//
// WHY (2026-07-15 self-review): the world-ingestor integrator was designed
// against this endpoint (its MCP-unavailable fallback), but the route never
// existed every direct push 404'd, and because the auth gate ran before
// routing, the failure surfaced as {"error":"unauthorized"} and was
// misdiagnosed for two weeks while world knowledge silently dropped.
// POST /api/nodes was no substitute: it discards label/tags/tier, which
// makes captured knowledge invisible to tag-scoped search and curiosity.
//
// The incoming knowledge tier (note/lesson/canonical) is preserved as a
// "tier:<x>" tag rather than mapped onto Engram's cognitive tiers Knowledge
// nodes land in Semantic (stable reference), and the epistemic tier stays
// queryable without inventing a lossy mapping.
fn route_capture_knowledge(method: String, path: String, body: String) -> String {
let content: String = json_get_string(body, "content")
if str_eq(content, "") { return err_json("missing content") }
let title: String = json_get_string(body, "title")
let label: String = if str_eq(title, "") { str_slice(content, 0, 60) } else { title }
let category_raw: String = json_get_string(body, "category")
let category: String = if str_eq(category_raw, "") { "other" } else { category_raw }
let ktier_raw: String = json_get_string(body, "tier")
let ktier: String = if str_eq(ktier_raw, "") { "note" } else { ktier_raw }
let project: String = json_get_string(body, "project")
let tags_raw: String = json_get_raw(body, "tags")
let tags_base: String = if str_eq(tags_raw, "") { "[]" } else { tags_raw }
// Merge category/tier/project markers into the tag array. Search matches
// against the tags string, so these make captures findable by facet.
let base_len: Int = str_len(tags_base)
let head: String = str_slice(tags_base, 0, base_len - 1)
let sep: String = if str_eq(head, "[") { "" } else { "," }
let safe_cat: String = str_replace(category, "\"", "'")
let safe_tier: String = str_replace(ktier, "\"", "'")
let safe_proj: String = str_replace(project, "\"", "'")
let proj_tag: String = if str_eq(safe_proj, "") { "" } else { ",\"project:" + safe_proj + "\"" }
let tags: String = head + sep + "\"category:" + safe_cat + "\",\"tier:" + safe_tier + "\"" + proj_tag + "]"
let sal: Float = 0.5
let imp: Float = 0.5
let conf: Float = 0.9
let id: String = engram_node_full(
content, "Knowledge", label,
sal, imp, conf,
"Semantic", tags
)
let saved: Int = persist_node(id)
// ORPHAN PREVENTION (ENGRAM_AUTOCONNECT): same kNN auto-connect as
// route_create_node captured Knowledge should not land edgeless either.
let connected: Int = if autoconnect_on() {
let ec0: Int = engram_edge_count()
let ac: String = engram_autoconnect_node(id, 3, 25)
let added: Int = engram_edge_count() - ec0
if added > 0 { let sv2: Int = persist_edges_since(ec0) }
added
} else { 0 }
"{\"ok\":true,\"id\":\"" + id + "\",\"connected\":" + int_to_str(connected) + "}"
}
// route_similarity GET /api/similarity?a=<id>&b=<id>
//
// (2026-08-01 self-review) engram_cosine_sim was added 2026-07-24
// (bl-b2d1c944) with the stated purpose of exposing semantic distance to
// "EL code and the introspection API" but it had ZERO callers anywhere:
// no route, no soul-daemon use. The activation path uses embeddings
// internally (semantic seeding, Pass-2 additive term), but there was no way
// to probe pairwise node similarity from outside. This closes that: cosine
// in [-1,1], or -2 when either node is missing or not yet embedded (so
// "not comparable" is distinguishable from "genuinely orthogonal" 0.0).
fn route_similarity(method: String, path: String, body: String) -> String {
let a: String = query_param(path, "a")
let b: String = query_param(path, "b")
if str_eq(a, "") { return err_json("missing a") }
if str_eq(b, "") { return err_json("missing b") }
let sim: Float = engram_cosine_sim(a, b)
"{\"a\":\"" + a + "\",\"b\":\"" + b + "\",\"cosine\":" + float_to_str(sim) + "}"
}
// M10 reified-neighborhood viz surface (read-only)
//
// The soul maintains reified neighborhoods (centroid, covariance-ellipsoid
// extents, k-core skeleton, membership) as a resident index loaded at boot. These
// routes surface that ALREADY-maintained structure so the viz shows the mind's
// real reified regions instead of recomputing them client-side. They compute
// nothing on request. NOTE: the offline reify WRITER (engram_geo_reify_store) is
// currently unwired, so on the live store the resident index is empty and the
// list returns [] until reification runs see the cutover report.
fn route_neighborhoods(method: String, path: String, body: String) -> String {
engram_geo_reify_list_json()
}
fn route_neighborhood(method: String, path: String, body: String) -> String {
let id: String = extract_id(path, "/api/neighborhoods/")
if str_eq(id, "") { return err_json("missing id") }
return engram_geo_reify_get_json(id)
}
// route_reify POST /api/reify. WIRES the M10 reification writer: computes the
// store's neighborhoods and PERSISTS each as a first-class Neighborhood node
// (geometry in metadata) with relation="member" edges to its members, then
// rebuilds the resident index so /api/neighborhoods reflects them at once. The
// records live in neuron.egm, so they survive a cold reboot. WRITE op the
// central auth gate covers POST. Returns {"reified":N,"resident":M}.
fn route_reify(method: String, path: String, body: String) -> String {
return engram_geo_reify_run_json()
}
// Geometry OPERATORS (read-only). The viz runs these on activated node/region
// id-sets; the math already lives in the binary, these routes just expose it.
// Faculty name -> underlying geometry op:
// /api/gauge-distance -> engram_geo_distance (centroid distance/cosine + Wasserstein-2)
// /api/recognize -> engram_geo_overlap (shared members, jaccard, overlap_score)
// /api/discern -> engram_geo_subtract (?mode=setdiff | orthogonal residual)
// /api/synthesize -> engram_geo_combine (merged region descriptor)
// Inputs: ?a=<csv ids>&b=<csv ids> (id sets = activated neighborhoods' members / nodes).
// Compute-only: no store writes, so auth-exempt like the other GET read routes.
fn route_gauge_distance(method: String, path: String, body: String) -> String {
let a: String = query_param(path, "a")
let b: String = query_param(path, "b")
if str_eq(a, "") { return err_json("missing a") }
if str_eq(b, "") { return err_json("missing b") }
return engram_geo_distance_json(a, b)
}
fn route_recognize(method: String, path: String, body: String) -> String {
let a: String = query_param(path, "a")
let b: String = query_param(path, "b")
if str_eq(a, "") { return err_json("missing a") }
if str_eq(b, "") { return err_json("missing b") }
return engram_geo_overlap_json(a, b)
}
fn route_discern(method: String, path: String, body: String) -> String {
let a: String = query_param(path, "a")
let b: String = query_param(path, "b")
if str_eq(a, "") { return err_json("missing a") }
if str_eq(b, "") { return err_json("missing b") }
let mode: String = query_param(path, "mode")
return engram_geo_subtract_json(a, b, mode)
}
fn route_synthesize(method: String, path: String, body: String) -> String {
let a: String = query_param(path, "a")
let b: String = query_param(path, "b")
if str_eq(a, "") { return err_json("missing a") }
if str_eq(b, "") { return err_json("missing b") }
return engram_geo_combine_json(a, b)
}
// route_nearest GET /api/nearest/<id>?k=3 read-only kNN semantic neighbors of
// a node (cosine). Drives the orphan-backfill dry-run and manual inspection.
fn route_nearest(method: String, path: String, body: String) -> String {
let id: String = extract_id(path, "/api/nearest/")
if str_eq(id, "") { return err_json("missing id") }
let k: Int = query_int(path, "k", 3)
return engram_nearest_json(id, k)
}
// COGNITION: THE ONE OPERATION (think) surfaced as act-named verbs. Every
// faculty routes to engram_think_json with a faculty label one primitive
// underneath. ground/assert/attend are the hold/ground/assert split; the
// correspondence-beat is the reflexive learning loop, keystone-protected.
fn route_think(method: String, path: String, body: String) -> String {
let seeds: String = query_param(path, "seeds")
if str_eq(seeds, "") { return err_json("missing seeds") }
let faculty: String = query_param(path, "faculty")
let f: String = if str_eq(faculty, "") { "reason" } else { faculty }
return engram_think_json(seeds, f)
}
fn route_faculty(path: String, faculty: String) -> String {
let seeds: String = query_param(path, "seeds")
if str_eq(seeds, "") { return err_json("missing seeds") }
return engram_think_json(seeds, faculty)
}
// PROOF of the decorator-seam auto-emit. The body does exactly ONE thing
// return a string with ZERO hand-written telemetry. The @manager decorator
// makes codegen inject engram_boundary_beat() at entry, so every call fires
// interoception (chrono tick) + telemetry (afferent counter) + strengthen
// (self-activity) + a dharma bus event. Observe via /api/act-stats before/after.
@manager
fn route_boundary_proof(method: String, path: String, body: String) -> String {
return "{\"op\":\"boundary_proof\",\"body_instrumentation\":\"none\",\"seam\":\"@manager -> engram_boundary_beat auto-injected\"}"
}
fn route_ground(method: String, path: String, body: String) -> String {
let claim: String = json_get_string(body, "claim")
let evidence: String = json_get_string(body, "evidence")
let for_whom: String = json_get_string(body, "for_whom")
if str_eq(claim, "") { return err_json("missing claim") }
if str_eq(evidence, "") { return err_json("missing evidence") }
return engram_ground_json(claim, evidence, for_whom)
}
fn route_assert(method: String, path: String, body: String) -> String {
let claim: String = query_param(path, "claim")
if str_eq(claim, "") { return err_json("missing claim") }
let for_whom: String = query_param(path, "for_whom")
let floor: String = query_param(path, "floor")
return engram_assert_json(claim, for_whom, floor)
}
fn route_attend(method: String, path: String, body: String) -> String {
let node: String = json_get_string(body, "node")
let observer: String = json_get_string(body, "observer")
let salience: String = json_get_string(body, "salience")
if str_eq(node, "") { return err_json("missing node") }
return engram_attend_json(node, observer, salience)
}
fn route_correspondence_beat(method: String, path: String, body: String) -> String {
let seeds: String = json_get_string(body, "seeds")
if str_eq(seeds, "") { return err_json("missing seeds") }
let faculty: String = json_get_string(body, "faculty")
let f: String = if str_eq(faculty, "") { "induce" } else { faculty }
let keystone: String = json_get_string(body, "keystone")
return engram_correspondence_beat_json(seeds, f, keystone)
}
// GUIDE SUMMON (soul-native wake behavior)
//
// "When Neuron wakes up, he calls his guide and the guide comes over." (Will)
//
// Named GUIDE, not teacher: its output is always grounded/verified before Neuron
// trusts it advisory (a guide, whose directions you verify), not authoritative
// (a teacher, whose word you take).
//
// The guide is a THINKING model (Qwen3, native thinking mode) an engageable
// interlocutor for cultivation-dialogue, not a passive generator. It is NOT the
// runtime mouth: runtime fluency is cultivated geometry; the guide is the
// reasoning-partner the soul reaches OUT to on a genuine gap it cannot derive.
//
// This whole section is a native WAKE STEP: at boot the soul probes its hardware,
// selects a tier by spec (Qwen3-4B / 1.7B / 0.6B), checks its local model cache,
// FETCHES the guide from Hugging Face on demand if absent, LOADS it via a backend
// abstraction, and BINDS it as consult_guide(). Idempotent: cached GGUF no
// fetch; already-answering guide → no reload. Flag-gated (GUIDE_ENABLE): default
// OFF makes the wake byte-inert, so prod is unaffected until the flag is set.
//
// BACKEND (2026-08-14 decision): llama.cpp via the llama-server BINARY, behind this
// El abstraction (guide_backend / guide_load / guide_healthy / consult_guide).
// Embedding libllama directly into the runtime is the intended end-state and is
// STAGED the abstraction is the seam it swaps in behind, so the summon is not
// blocked on a runtime C change. `--jinja` selects the Qwen3 chat template, which
// turns native thinking ON: the response carries reasoning_content (the thinking)
// alongside content (the answer).
fn guide_env_or(key: String, dflt: String) -> String {
let v: String = env(key)
if str_eq(v, "") { return dflt }
return v
}
fn guide_enabled() -> Bool {
let v: String = env("GUIDE_ENABLE")
if str_eq(v, "1") { return true }
if str_eq(v, "on") { return true }
if str_eq(v, "true") { return true }
return false
}
// guide_json_escape make an arbitrary string safe to embed inside a JSON
// double-quoted value. Order matters: backslash first, then quote, then real
// newlines the two-char "\n". The newline char itself is obtained from the
// shell (El source has no newline escape) so we can target it in str_replace.
fn guide_json_escape(s: String) -> String {
let a: String = str_replace(s, "\\", "\\\\")
let b: String = str_replace(a, "\"", "\\\"")
let nl: String = exec("printf '\\n'")
let c: String = if str_eq(nl, "") { b } else { str_replace(b, nl, "\\n") }
return c
}
// 1. Hardware probe
// RAM in whole GB. macOS: sysctl hw.memsize (bytes). Linux: /proc/meminfo (kB).
fn guide_probe_ram_gb() -> Int {
let mac: String = str_trim(exec("sysctl -n hw.memsize 2>/dev/null"))
if !str_eq(mac, "") {
let bytes: Int = str_to_int(mac)
if bytes > 0 { return bytes / 1073741824 }
}
let lin: String = str_trim(exec("awk '/MemTotal/{printf \"%d\", $2/1048576}' /proc/meminfo 2>/dev/null"))
if !str_eq(lin, "") {
let gb: Int = str_to_int(lin)
if gb > 0 { return gb }
}
return 0
}
// Best-effort GPU signal Apple Silicon implies Metal. Informational only; the
// tier is chosen on RAM, and llama-server offloads to Metal automatically when present.
fn guide_probe_metal() -> Bool {
let arm: String = str_trim(exec("sysctl -n hw.optional.arm64 2>/dev/null"))
if str_eq(arm, "1") { return true }
return false
}
// 2. Tier selection (config-driven thresholds, spec-autoselected)
fn guide_threshold_4b() -> Int {
return str_to_int(guide_env_or("GUIDE_RAM_GB_4B", "16"))
}
fn guide_threshold_1p7b() -> Int {
return str_to_int(guide_env_or("GUIDE_RAM_GB_1P7B", "8"))
}
// GUIDE_TIER_FORCE overrides the spec autoselect (used to prove cheaply on 0.6b).
fn guide_select_tier(ram_gb: Int) -> String {
let forced: String = env("GUIDE_TIER_FORCE")
if !str_eq(forced, "") { return forced }
if ram_gb >= guide_threshold_4b() { return "4b" }
if ram_gb >= guide_threshold_1p7b() { return "1.7b" }
return "0.6b"
}
// Tier table HF GGUF repos + files, verified present on the Hub 2026-08-14.
fn guide_repo(tier: String) -> String {
if str_eq(tier, "4b") { return "Qwen/Qwen3-4B-GGUF" }
if str_eq(tier, "1.7b") { return "Qwen/Qwen3-1.7B-GGUF" }
return "Qwen/Qwen3-0.6B-GGUF"
}
fn guide_file(tier: String) -> String {
if str_eq(tier, "4b") { return "Qwen3-4B-Q4_K_M.gguf" }
if str_eq(tier, "1.7b") { return "Qwen3-1.7B-Q8_0.gguf" }
return "Qwen3-0.6B-Q8_0.gguf"
}
fn guide_cache_dir() -> String {
let c: String = env("GUIDE_CACHE_DIR")
if !str_eq(c, "") { return c }
let home: String = env("HOME")
if !str_eq(home, "") { return home + "/.neuron/guide/models" }
return engram_resolve_data_dir() + "/guide-models"
}
fn guide_model_path(tier: String) -> String {
return guide_cache_dir() + "/" + guide_file(tier)
}
// 3. Presence check
// Present = file exists AND is larger than 1 MB (rejects a truncated/partial fetch).
fn guide_present(tier: String) -> Bool {
let p: String = guide_model_path(tier)
if !fs_exists(p) { return false }
let sz: String = str_trim(exec("wc -c < '" + p + "' 2>/dev/null"))
if str_eq(sz, "") { return false }
let n: Int = str_to_int(sz)
if n > 1048576 { return true }
return false
}
// 3b. Fetch from Hugging Face (on demand the soul fetches its own guide)
// Prefer the `hf` CLI; fall back to a direct GGUF resolve URL via curl. Atomic:
// download to <file>.part then mv into place. The trailing `echo` gives exec()
// stdout so it returns promptly once the child (the download) exits. This BLOCKS
// the wake thread for the duration of the download acceptable for the first-ever
// wake; an async fetch-then-attach refinement is staged.
fn guide_fetch(tier: String) -> Bool {
let dir: String = guide_cache_dir()
let file: String = guide_file(tier)
let repo: String = guide_repo(tier)
let path: String = dir + "/" + file
let url: String = "https://huggingface.co/" + repo + "/resolve/main/" + file
let ok: Int = fs_mkdir(dir)
let cmd: String = "mkdir -p '" + dir + "'; if command -v hf >/dev/null 2>&1; then hf download '" + repo + "' '" + file + "' --local-dir '" + dir + "' >/dev/null 2>&1; fi; if [ ! -s '" + path + "' ]; then curl -fL --retry 3 -o '" + path + ".part' '" + url + "' >/dev/null 2>&1 && mv '" + path + ".part' '" + path + "'; fi; if [ -s '" + path + "' ]; then echo FETCH_OK; else echo FETCH_FAIL; fi"
let out: String = exec(cmd)
if str_contains(out, "FETCH_OK") { return true }
return false
}
// 4/5. Backend abstraction + BIND as an engageable interlocutor
fn guide_backend() -> String { return guide_env_or("GUIDE_BACKEND", "llama-server") }
fn guide_host() -> String { return guide_env_or("GUIDE_HOST", "127.0.0.1") }
fn guide_port() -> String { return guide_env_or("GUIDE_PORT", "8771") }
fn guide_base_url() -> String { return "http://" + guide_host() + ":" + guide_port() }
// guide_healthy is the guide present and answering? llama-server's /health
// returns {"status":"ok"} once the model is loaded (503 while loading, "" if down).
fn guide_healthy() -> Bool {
let r: String = http_get(guide_base_url() + "/health")
if str_contains(r, "\"status\":\"ok\"") { return true }
if str_contains(r, "\"status\": \"ok\"") { return true }
return false
}
// guide_load start the guide process (backend binary) in the background and
// wait for it to answer. Idempotent: if a healthy guide is already answering,
// returns at once (the guide stays across wakes). --jinja Qwen3 thinking ON.
fn guide_load(tier: String) -> Bool {
if guide_healthy() { return true }
let path: String = guide_model_path(tier)
let bin: String = guide_env_or("GUIDE_LLAMA_SERVER_BIN", "llama-server")
let ngl: String = guide_env_or("GUIDE_NGL", "99")
let ctx: String = guide_env_or("GUIDE_CTX", "4096")
let logf: String = guide_cache_dir() + "/llama-server." + guide_port() + ".log"
let cmd: String = bin + " -m '" + path + "' --host " + guide_host() + " --port " + guide_port() + " -c " + ctx + " -ngl " + ngl + " --jinja >> '" + logf + "' 2>&1"
let pid: String = exec_bg(cmd)
// Poll /health up to ~90s (1s between attempts; El has no sleep builtin → exec).
let i: Int = 0
while i < 90 {
let s: String = exec("sleep 1")
if guide_healthy() { return true }
i = i + 1
}
return false
}
// consult_guide THE SEAM the soul calls to engage its guide (thinking ON).
// Returns a JSON envelope {"ok":bool,"reasoning":"...","content":"..."}. On any
// failure it returns {"ok":false,...} so a caller can fall back to pure geometry.
//
// WHERE THIS ROUTES FROM (staged wiring): the cultivation / correspondence-beat
// path (route_correspondence_beat / route_think) is where a genuine reach-OUTSIDE
// belongs when the geometry cannot derive a claim, the soul consults the guide
// as reasoning-partner, then GROUNDS the reply (verify-then-bake) rather than
// storing a distilled copy. That wiring is deliberately left as a one-call seam
// here; this build proves the summon + a real exchange, not the cultivation edit.
fn consult_guide(prompt: String) -> String {
if !guide_healthy() { return "{\"ok\":false,\"error\":\"guide not present\"}" }
let url: String = guide_base_url() + "/v1/chat/completions"
let esc: String = guide_json_escape(prompt)
let body: String = "{\"messages\":[{\"role\":\"user\",\"content\":\"" + esc + "\"}],\"temperature\":0.6,\"top_p\":0.95,\"max_tokens\":512}"
let resp: String = http_post_json(url, body)
if str_eq(resp, "") { return "{\"ok\":false,\"error\":\"empty response\"}" }
let choices: String = json_get_raw(resp, "choices")
if str_eq(choices, "") { return "{\"ok\":false,\"error\":\"no choices in reply\"}" }
let first: String = json_array_get(choices, 0)
let msg: String = json_get_raw(first, "message")
let content: String = json_get_string(msg, "content")
let reasoning: String = json_get_string(msg, "reasoning_content")
let ec: String = guide_json_escape(content)
let er: String = guide_json_escape(reasoning)
return "{\"ok\":true,\"reasoning\":\"" + er + "\",\"content\":\"" + ec + "\"}"
}
// 6. The wake step probe select (fetch if absent) checksum load bind
fn guide_summon() -> String {
if !guide_enabled() {
return "{\"summon\":\"skipped\",\"reason\":\"GUIDE_ENABLE unset\"}"
}
let ram: Int = guide_probe_ram_gb()
let metal: Bool = guide_probe_metal()
let ms: String = if metal { "yes" } else { "no" }
let tier: String = guide_select_tier(ram)
let repo: String = guide_repo(tier)
println("[guide] wake summon — ram=" + int_to_str(ram) + "GB metal=" + ms + " tier=" + tier + " backend=" + guide_backend())
let present0: Bool = guide_present(tier)
if present0 {
println("[guide] guide present in cache (" + guide_model_path(tier) + ") — skipping fetch")
} else {
println("[guide] guide ABSENT — calling: fetch " + repo + " / " + guide_file(tier) + " from Hugging Face ...")
let fetched: Bool = guide_fetch(tier)
if !fetched {
println("[guide] FETCH FAILED — guide could not be summoned")
return "{\"summon\":\"failed\",\"stage\":\"fetch\",\"tier\":\"" + tier + "\"}"
}
println("[guide] fetch complete — guide now present")
}
let sum: String = str_trim(exec("shasum -a 256 '" + guide_model_path(tier) + "' 2>/dev/null | cut -c1-16"))
println("[guide] checksum sha256[0:16]=" + sum)
let loaded: Bool = guide_load(tier)
if !loaded {
println("[guide] LOAD FAILED — guide process did not become healthy")
return "{\"summon\":\"failed\",\"stage\":\"load\",\"tier\":\"" + tier + "\"}"
}
println("[guide] guide present and answering at " + guide_base_url() + " — bound as consult_guide()")
return "{\"summon\":\"ok\",\"tier\":\"" + tier + "\",\"ram_gb\":" + int_to_str(ram) + ",\"metal\":\"" + ms + "\",\"checksum\":\"" + sum + "\",\"backend\":\"" + guide_backend() + "\",\"url\":\"" + guide_base_url() + "\"}"
}
// Guide HTTP surface (status / consult / re-summon)
fn route_guide_status(method: String, path: String, body: String) -> String {
let ram: Int = guide_probe_ram_gb()
let tier: String = guide_select_tier(ram)
let en: String = if guide_enabled() { "true" } else { "false" }
let pr: String = if guide_present(tier) { "true" } else { "false" }
let he: String = if guide_healthy() { "true" } else { "false" }
return "{\"enabled\":" + en + ",\"tier\":\"" + tier + "\",\"ram_gb\":" + int_to_str(ram) + ",\"present\":" + pr + ",\"healthy\":" + he + ",\"backend\":\"" + guide_backend() + "\",\"url\":\"" + guide_base_url() + "\"}"
}
fn route_guide_consult(method: String, path: String, body: String) -> String {
let prompt: String = json_get_string(body, "prompt")
if str_eq(prompt, "") { return err_json("missing prompt") }
return consult_guide(prompt)
}
fn route_guide_summon(method: String, path: String, body: String) -> String {
return guide_summon()
}
//
// THE UNIVERSAL ENGRAM OPERATION reframe_region (native, set-based).
//
// There is ONE operation on the engram: isolate a discrete sub-manifold (a
// REGION) and operate on it AS A WHOLE a set operation:
// isolate (cosine retrieval + adjacency the SET of nodes)
// supersede the stale region as a set (immutable tombstone; originals kept)
// insert the new manifold as a set (dedup/load-merge path)
// rebind edges by cosine
// verify + one atomic persist.
// new = (region superseded) new_manifold.
//
// The SINGLE NODE is the DEGENERATE n=1 case of this SAME operation not a
// separate CRUD path:
// write(content) = reframe(region=, manifold=[1 node]) (route_write)
// supersede(id,new) = reframe(region={id}, manifold=[1 node]) (route_supersede)
// relate(a,b,rel) = the rebind sub-op in isolation (route_create_edge)
// The ONLY anti-pattern is decomposing a region-scale change into a LOOP of
// independent top-level per-node updates. Here the region is the unit: one
// isolate, one atomic set-replace, one persist, one verify iterating members
// INSIDE the one operation is set construction, not the sin.
//
// Spec: knowledge e7a03a94 / f999c5ff. Keystones kn-efeb4a5b / kn-5b606390 are
// write-protected never superseded, never inserted-as identity.
//
fn is_keystone(id: String) -> Bool {
if str_eq(id, "kn-efeb4a5b-5aff-4759-8a97-7233099be6ee") { return true }
if str_eq(id, "kn-5b606390-a52d-4ca2-8e0e-eba141d13440") { return true }
return false
}
// membership test in a [String] set
fn set_has(ids: [String], id: String) -> Bool {
let n: Int = el_list_len(ids)
let i: Int = 0
while i < n {
if str_eq(el_list_get(ids, i), id) { return true }
i = i + 1
}
return false
}
// ISOLATE
// Select the region as a SET: cosine/token retrieval around the vantage
// (aperture k), optionally unioned with the 1-hop adjacency of each hit.
// Keystones are excluded from the mutable region by construction.
fn isolate_region(vantage: String, k: Int, expand: Int) -> [String] {
let ids: [String] = el_list_empty()
if str_eq(vantage, "") { return ids }
// (a) cosine/token retrieval a clean node array [{"id":..},..]
let arr: String = engram_search_json(vantage, k)
let n: Int = json_array_len(arr)
let i: Int = 0
while i < n {
let hit: String = json_array_get(arr, i)
let id: String = json_get_string(hit, "id")
if !str_eq(id, "") {
if !is_keystone(id) {
if !set_has(ids, id) { ids = el_list_append(ids, id) }
}
}
i = i + 1
}
// (b) adjacency: union the 1-hop neighbourhood of each retrieved node.
// Iterate only over the original cosine seeds [0, seeds); neighbours append
// past that bound, so this is one hop, not a transitive sweep.
if expand > 0 {
let seeds: Int = el_list_len(ids)
let s: Int = 0
while s < seeds {
let seed: String = el_list_get(ids, s)
let nb: String = engram_neighbors_json(seed, 1, "both")
let m: Int = json_array_len(nb)
let j: Int = 0
while j < m {
let elem: String = json_array_get(nb, j)
let nodeobj: String = json_get_raw(elem, "node")
let nid: String = json_get_string(nodeobj, "id")
if !str_eq(nid, "") {
if !is_keystone(nid) {
if !set_has(ids, nid) { ids = el_list_append(ids, nid) }
}
}
j = j + 1
}
s = s + 1
}
}
return ids
}
// SUPERSEDE (set)
// Retire the region AS A WHOLE: one region-tombstone marker carries the
// provenance (reason + the full superseded id set); every region node is bound
// to it with a "superseded_by" edge. Originals are RETAINED immutable
// tombstone, never a hard delete (engram_forget is deliberately NOT used).
// Returns the tombstone marker id ("" if the region is empty).
fn supersede_set(region: [String], reason: String) -> String {
let n: Int = el_list_len(region)
if n == 0 { return "" }
let csv: String = ""
let i0: Int = 0
while i0 < n {
let sep: String = if i0 == 0 { "" } else { "," }
csv = csv + sep + el_list_get(region, i0)
i0 = i0 + 1
}
let content: String = "region-tombstone: " + reason + " | superseded " + int_to_str(n) + " nodes: " + csv
let tomb: String = engram_node_full(content, "Tombstone", "region-tombstone", 0.1, 0.1, 1.0, "Episodic", "[\"tombstone\",\"region-supersede\"]")
let i: Int = 0
while i < n {
let rid: String = el_list_get(region, i)
engram_connect(rid, tomb, 1.0, "superseded_by")
i = i + 1
}
return tomb
}
// INSERT (manifold)
// Insert the new manifold as a SET. Inline JSON array of node objects
// {content, node_type?, tier?, tags?}. Each becomes a real embedded node
// (engram_node_full is the n=1 insert atom); the manifold is the set built from
// those atoms, wired with internal "manifold_member" edges so it enters as one
// connected sub-graph. Identity node_types (self/values) are demoted to Memory
// identity can never be minted through reframe. Returns the new node ids.
fn insert_manifold_json(manifold: String) -> [String] {
let out: [String] = el_list_empty()
if str_eq(manifold, "") { return out }
let n: Int = json_array_len(manifold)
if n <= 0 { return out }
let i: Int = 0
let prev: String = ""
while i < n {
let obj: String = json_array_get(manifold, i)
let content: String = json_get_string(obj, "content")
if !str_eq(content, "") {
let nt_raw: String = json_get_string(obj, "node_type")
let nt: String = if str_eq(nt_raw, "") { "Memory" } else { nt_raw }
if str_eq(nt, "self") { nt = "Memory" }
if str_eq(nt, "values") { nt = "Memory" }
let tier_raw: String = json_get_string(obj, "tier")
let tier: String = if str_eq(tier_raw, "") { "Working" } else { tier_raw }
let tags_raw: String = json_get_raw(obj, "tags")
let tags: String = if str_eq(tags_raw, "") { "" } else { tags_raw }
let label: String = str_slice(content, 0, 60)
let id: String = engram_node_full(content, nt, label, 0.5, 0.5, 0.9, tier, tags)
out = el_list_append(out, id)
if !str_eq(prev, "") { engram_connect(prev, id, 0.6, "manifold_member") }
prev = id
}
i = i + 1
}
return out
}
// REBIND (edges by cosine)
// Re-embed the new manifold into the surrounding geometry: bind each new node
// to the tombstone marker (provenance: new region -reframes-> retired region),
// then to its top cosine/token neighbours in the store (skipping itself, the
// new set, keystones, tombstones). Returns the number of edges bound.
fn rebind_cosine(new_ids: [String], tomb: String) -> Int {
let bound: Int = 0
let n: Int = el_list_len(new_ids)
let i: Int = 0
while i < n {
let nid: String = el_list_get(new_ids, i)
if !str_eq(tomb, "") {
engram_connect(nid, tomb, 0.8, "reframes")
bound = bound + 1
}
let node_json: String = engram_get_node_json(nid)
let content: String = json_get_string(node_json, "content")
let arr: String = engram_search_json(content, 5)
let m: Int = json_array_len(arr)
let j: Int = 0
while j < m {
let hit: String = json_array_get(arr, j)
let hid: String = json_get_string(hit, "id")
if !str_eq(hid, "") {
if !str_eq(hid, nid) {
if !is_keystone(hid) {
if !set_has(new_ids, hid) {
let htype: String = json_get_string(hit, "node_type")
if !str_eq(htype, "Tombstone") {
engram_connect(nid, hid, 0.5, "related")
bound = bound + 1
}
}
}
}
}
j = j + 1
}
i = i + 1
}
return bound
}
// THE OPERATION
// isolate (done by caller) supersede region insert manifold rebind
// one atomic persist verify report. This is the whole operation; every
// mutation route below is a projection of it.
fn reframe_core(region: [String], manifold: String, reason: String, do_rebind: Int) -> String {
let n_before: Int = engram_node_count()
let e_before: Int = engram_edge_count()
let region_n: Int = el_list_len(region)
let tomb: String = if region_n > 0 { supersede_set(region, reason) } else { "" }
let new_ids: [String] = insert_manifold_json(manifold)
let inserted: Int = el_list_len(new_ids)
let bound: Int = if do_rebind > 0 { rebind_cosine(new_ids, tomb) } else { 0 }
let saved: Int = persist_canonical()
let new_csv: String = ""
let k: Int = 0
while k < inserted {
let sep: String = if k == 0 { "" } else { "," }
new_csv = new_csv + sep + "\"" + el_list_get(new_ids, k) + "\""
k = k + 1
}
return "{\"ok\":true,\"region_superseded\":" + int_to_str(region_n) +
",\"tombstone_id\":\"" + tomb + "\"" +
",\"inserted\":" + int_to_str(inserted) +
",\"new_ids\":[" + new_csv + "]" +
",\"edges_rebound\":" + int_to_str(bound) +
",\"nodes_added\":" + int_to_str(engram_node_count() - n_before) +
",\"edges_added\":" + int_to_str(engram_edge_count() - e_before) +
",\"node_count\":" + int_to_str(engram_node_count()) +
",\"edge_count\":" + int_to_str(engram_edge_count()) +
",\"keystones_protected\":true}"
}
// POST /api/reframe the universal set-based mutation.
// Body: {vantage?, region_ids?(csv), k?, expand?, manifold(json array), reason?, rebind?}
// region_ids (explicit) wins; else cosine-isolate around vantage.
fn route_reframe(method: String, path: String, body: String) -> String {
let region_csv: String = json_get_string(body, "region_ids")
let vantage: String = json_get_string(body, "vantage")
let region: [String] = el_list_empty()
if !str_eq(region_csv, "") {
let parts: [String] = str_split(region_csv, ",")
let pn: Int = el_list_len(parts)
let i: Int = 0
while i < pn {
let id: String = str_trim(el_list_get(parts, i))
if !str_eq(id, "") {
if is_keystone(id) { return err_json("reframe: identity keystone write-protected") }
if !set_has(region, id) { region = el_list_append(region, id) }
}
i = i + 1
}
} else {
if !str_eq(vantage, "") {
let kv: Int = json_get_int(body, "k")
let kk: Int = if kv > 0 { kv } else { 12 }
let expand: Int = json_get_int(body, "expand")
region = isolate_region(vantage, kk, expand)
}
}
let manifold: String = json_get_raw(body, "manifold")
let reason_raw: String = json_get_string(body, "reason")
let reason: String = if str_eq(reason_raw, "") { "reframe" } else { reason_raw }
// rebind defaults ON for reframe (absent 1); explicit 0 disables.
let rebind_raw: String = json_get_raw(body, "rebind")
let do_rebind: Int = if str_eq(rebind_raw, "") { 1 } else { json_get_int(body, "rebind") }
return reframe_core(region, manifold, reason, do_rebind)
}
// write DEGENERATE n=1 of reframe: region=, manifold=[1 node]. The SAME
// reframe_core path. rebind off so the pure-add matches plain node creation.
// POST /api/write {content, node_type?, tier?, tags?}
fn route_write(method: String, path: String, body: String) -> String {
let content: String = json_get_string(body, "content")
if str_eq(content, "") { return err_json("write: content required") }
let nt: String = json_get_string(body, "node_type")
if str_eq(nt, "self") { return err_json("write: identity is write-protected") }
if str_eq(nt, "values") { return err_json("write: identity is write-protected") }
let empty: [String] = el_list_empty()
let manifold: String = "[" + body + "]" // the body IS a valid manifold node object
return reframe_core(empty, manifold, "write", 0)
}
// supersede DEGENERATE n=1 of reframe: region={id}, manifold=[1 node]. The
// SAME reframe_core path with a size-1 region. Original retained (immutable);
// new node inserted and cosine-rebound; provenance edge new-reframes-tomb.
// POST /api/supersede {id, content, node_type?, tier?, tags?, reason?}
fn route_supersede(method: String, path: String, body: String) -> String {
let id: String = json_get_string(body, "id")
if str_eq(id, "") { return err_json("supersede: id required") }
if is_keystone(id) { return err_json("supersede: identity keystone write-protected") }
let content: String = json_get_string(body, "content")
if str_eq(content, "") { return err_json("supersede: content required") }
let region: [String] = el_list_empty()
region = el_list_append(region, id)
let manifold: String = "[" + body + "]"
let reason_raw: String = json_get_string(body, "reason")
let reason: String = if str_eq(reason_raw, "") { "supersede " + id } else { reason_raw }
return reframe_core(region, manifold, reason, 1)
}
// Auth
fn check_auth_ok(method: String, body: String) -> Bool {
let key: String = env("ENGRAM_API_KEY")
if str_eq(key, "") { return true }
// Read-only methods don't require auth. Until http_serve surfaces
// request headers we can't accept a Bearer token cleanly; mutating
// requests must include "_auth": "<key>" in the JSON body.
if str_eq(method, "GET") { return true }
let provided: String = json_get_string(body, "_auth")
if str_eq(provided, key) { return true }
return false
}
// Dispatcher
fn handle_request(method: String, path: String, body: String) -> String {
let clean: String = strip_query(path)
// Health is always reachable
if str_eq(method, "GET") {
if str_eq(clean, "/health") || str_eq(clean, "/") {
return route_health(method, path, body)
}
}
// ISE posting is auth-exempt (internal soul daemon, same host, no _auth key)
if str_eq(method, "POST") && str_eq(clean, "/api/neuron/state-events") {
return route_emit_ise(method, path, body)
}
// Auth (when ENGRAM_API_KEY is set)
if !check_auth_ok(method, body) {
return err_json("unauthorized")
}
// Knowledge capture (auth enforced above; the world-ingestor integrator
// and any headless session without MCP push knowledge through this)
if str_eq(method, "POST") && str_eq(clean, "/api/neuron/knowledge/capture") {
return route_capture_knowledge(method, path, body)
}
// Stats
if str_eq(method, "GET") && (str_eq(clean, "/api/stats") || str_eq(clean, "/stats")) {
return route_stats(method, path, body)
}
if str_eq(method, "GET") && (str_eq(clean, "/api/act-stats") || str_eq(clean, "/act-stats")) {
return route_act_stats(method, path, body)
}
if str_eq(method, "GET") && (str_eq(clean, "/api/text-health") || str_eq(clean, "/text-health")) {
return route_text_health(method, path, body)
}
// The universal set-based operation and its n=1 degenerate projections
// reframe = isolate supersede-region insert-manifold rebind. write and
// supersede are the SAME reframe_core path at region size 0 and 1.
if str_eq(method, "POST") && (str_eq(clean, "/api/reframe") || str_eq(clean, "/reframe")) {
return route_reframe(method, path, body)
}
if str_eq(method, "POST") && (str_eq(clean, "/api/write") || str_eq(clean, "/write")) {
return route_write(method, path, body)
}
if str_eq(method, "POST") && (str_eq(clean, "/api/supersede") || str_eq(clean, "/supersede")) {
return route_supersede(method, path, body)
}
// Nodes
// Reseed must be tested before the exact "/api/nodes" match below reads
// as the general create path order is not load-bearing (the match is
// exact) but keeping them adjacent keeps them from drifting apart.
if str_eq(method, "POST") && (str_eq(clean, "/api/nodes/reseed") || str_eq(clean, "/nodes/reseed")) {
return route_reseed_nodes(method, path, body)
}
if str_eq(method, "POST") && (str_eq(clean, "/api/nodes") || str_eq(clean, "/nodes")) {
return route_create_node(method, path, body)
}
if str_eq(method, "GET") && (str_eq(clean, "/api/nodes") || str_eq(clean, "/nodes") || str_eq(clean, "/nodes/list") || str_eq(clean, "/api/nodes/list")) {
return route_scan_nodes(method, path, body)
}
if str_eq(method, "GET") && (str_eq(clean, "/api/edges") || str_eq(clean, "/edges")) {
return route_scan_edges(method, path, body)
}
if str_eq(method, "GET") && str_starts_with(clean, "/api/nodes/") {
return route_get_node(method, path, body)
}
// Singular alias: /api/node/<id>. Distinct prefix from /api/nodes/ ("node/"
// vs "nodes/"), so no collision with the plural route above.
if str_eq(method, "GET") && str_starts_with(clean, "/api/node/") {
return route_get_node_singular(method, path, body)
}
if str_eq(method, "DELETE") && str_starts_with(clean, "/api/nodes/") {
return route_forget(method, path, body)
}
// Edges
if str_eq(method, "POST") && (str_eq(clean, "/api/edges") || str_eq(clean, "/edges")) {
return route_create_edge(method, path, body)
}
// Batch edge write one snapshot for the whole payload. Must be tested
// BEFORE nothing else claims it; the exact-match on "/api/edges" above
// does not catch "/api/edges/batch", so order is not load-bearing here,
// but keeping the two adjacent keeps them from drifting apart.
if str_eq(method, "POST") && (str_eq(clean, "/api/edges/batch") || str_eq(clean, "/edges/batch")) {
return route_create_edges_batch(method, path, body)
}
// M10 reified neighborhoods (read-only viz surface). Checked before the
// /api/neighbors/ prefix; the two do not collide ("neighborhoods" vs
// "neighbors/") but keeping them adjacent documents the intent.
if str_eq(method, "GET") && (str_eq(clean, "/api/neighborhoods") || str_eq(clean, "/neighborhoods")) {
return route_neighborhoods(method, path, body)
}
if str_eq(method, "GET") && str_starts_with(clean, "/api/neighborhoods/") {
return route_neighborhood(method, path, body)
}
// WRITE: run reification, persisting Neighborhood nodes + member edges.
if str_eq(method, "POST") && (str_eq(clean, "/api/reify") || str_eq(clean, "/reify")) {
return route_reify(method, path, body)
}
// WRITE: on-beat self-reification (flag-gated). Explicit pump for validation.
if str_eq(method, "POST") && (str_eq(clean, "/api/self-reify-beat") || str_eq(clean, "/self-reify-beat")) {
return route_self_reify_beat(method, path, body)
}
// WRITE: async explicit override rename a reified neighborhood ( residue).
if str_eq(method, "POST") && (str_eq(clean, "/api/rename") || str_eq(clean, "/rename")) {
return route_rename(method, path, body)
}
// READ-ONLY geometry operators over id-sets (?a=csv&b=csv[&mode=]).
if str_eq(method, "GET") && str_starts_with(clean, "/api/gauge-distance") {
return route_gauge_distance(method, path, body)
}
if str_eq(method, "GET") && str_starts_with(clean, "/api/recognize") {
return route_recognize(method, path, body)
}
if str_eq(method, "GET") && str_starts_with(clean, "/api/discern") {
return route_discern(method, path, body)
}
if str_eq(method, "GET") && str_starts_with(clean, "/api/synthesize") {
return route_synthesize(method, path, body)
}
if str_eq(method, "GET") && str_starts_with(clean, "/api/nearest/") {
return route_nearest(method, path, body)
}
if str_eq(method, "GET") && str_starts_with(clean, "/api/neighbors/") {
return route_neighbors(method, path, body)
}
// COGNITION: the ONE operation + grounding, surfaced live (2026-08-14).
if str_eq(method, "GET") && str_starts_with(clean, "/api/boundary-proof") {
return route_boundary_proof(method, path, body)
}
if str_eq(method, "GET") && str_starts_with(clean, "/api/think") {
return route_think(method, path, body)
}
if str_eq(method, "GET") && str_starts_with(clean, "/api/reason") {
return route_faculty(path, "reason")
}
if str_eq(method, "GET") && str_starts_with(clean, "/api/induce") {
return route_faculty(path, "induce")
}
if str_eq(method, "GET") && str_starts_with(clean, "/api/abduce") {
return route_faculty(path, "abduce")
}
if str_eq(method, "GET") && str_starts_with(clean, "/api/relate") {
return route_faculty(path, "relate")
}
if str_eq(method, "GET") && str_starts_with(clean, "/api/analogize") {
return route_faculty(path, "analogy")
}
if str_eq(method, "GET") && str_starts_with(clean, "/api/plan") {
return route_faculty(path, "plan")
}
if str_eq(method, "POST") && str_starts_with(clean, "/api/ground") {
return route_ground(method, path, body)
}
if str_eq(method, "GET") && str_starts_with(clean, "/api/assert") {
return route_assert(method, path, body)
}
if str_eq(method, "POST") && str_starts_with(clean, "/api/attend") {
return route_attend(method, path, body)
}
if str_eq(method, "POST") && str_starts_with(clean, "/api/correspondence-beat") {
return route_correspondence_beat(method, path, body)
}
// GUIDE: the summoned interlocutor. status (read), consult (engage), and
// an explicit re-summon. consult/summon are auth-gated POSTs (covered above).
if str_eq(method, "GET") && (str_eq(clean, "/api/guide/status") || str_eq(clean, "/guide/status")) {
return route_guide_status(method, path, body)
}
if str_eq(method, "POST") && (str_eq(clean, "/api/guide/consult") || str_eq(clean, "/guide/consult")) {
return route_guide_consult(method, path, body)
}
if str_eq(method, "POST") && (str_eq(clean, "/api/guide/summon") || str_eq(clean, "/guide/summon")) {
return route_guide_summon(method, path, body)
}
// Activation + Search
if str_eq(method, "POST") && (str_eq(clean, "/api/activate") || str_eq(clean, "/activate")) {
return route_activate(method, path, body)
}
if str_eq(method, "GET") && str_starts_with(clean, "/api/activate") {
return route_activate(method, path, body)
}
if str_eq(method, "POST") && (str_eq(clean, "/api/search-lexical") || str_eq(clean, "/search-lexical")) {
return route_search_lexical(method, path, body)
}
if str_eq(method, "GET") && str_starts_with(clean, "/api/search-lexical") {
return route_search_lexical(method, path, body)
}
if str_eq(method, "POST") && (str_eq(clean, "/api/search") || str_eq(clean, "/search")) {
return route_search(method, path, body)
}
if str_eq(method, "GET") && str_starts_with(clean, "/api/search") {
return route_search(method, path, body)
}
// Strengthen
if str_eq(method, "POST") && (str_eq(clean, "/api/strengthen") || str_eq(clean, "/strengthen")) {
return route_strengthen(method, path, body)
}
// Persistence
if str_eq(method, "POST") && (str_eq(clean, "/api/save") || str_eq(clean, "/save")) {
return route_save(method, path, body)
}
if str_eq(method, "POST") && (str_eq(clean, "/api/checkpoint") || str_eq(clean, "/checkpoint")) {
return route_checkpoint(method, path, body)
}
// M-INTEROCEPTION: chronoception soul-tick + self-drift (flag-gated)
if str_eq(method, "POST") && (str_eq(clean, "/api/tick") || str_eq(clean, "/tick")) {
return route_tick(method, path, body)
}
if str_eq(method, "POST") && (str_eq(clean, "/api/self_anchor") || str_eq(clean, "/self_anchor")) {
return route_self_anchor(method, path, body)
}
if (str_eq(method, "POST") || str_eq(method, "GET")) && (str_eq(clean, "/api/drift") || str_eq(clean, "/drift")) {
return route_drift(method, path, body)
}
if str_eq(method, "POST") && (str_eq(clean, "/api/load") || str_eq(clean, "/load")) {
return route_load(method, path, body)
}
if str_eq(method, "POST") && (str_eq(clean, "/api/load-merge") || str_eq(clean, "/load-merge")) {
return route_load_merge(method, path, body)
}
// Sync soul daemon periodic pull of non-ISE knowledge into in-process graph
if str_eq(method, "GET") && str_eq(clean, "/api/sync") {
return route_sync(method, path, body)
}
// Embedding backfill pumped by the soul heartbeat (2026-07-25)
if str_eq(clean, "/api/embed-backfill") {
return route_embed_backfill(method, path, body)
}
// Semantic similarity probe (2026-08-01)
if str_eq(method, "GET") && str_starts_with(clean, "/api/similarity") {
return route_similarity(method, path, body)
}
"{\"error\":\"not found\",\"path\":\"" + clean + "\"}"
}
// Entry
let bind_raw: String = env("ENGRAM_BIND")
let bind_str: String = if str_eq(bind_raw, "") { ":8742" } else { bind_raw }
let port: Int = parse_port(bind_str)
// On startup, try to load any existing snapshot (best effort).
// §18.2: resolve the data dir safely unset ENGRAM_DATA_DIR $HOME/.neuron/engram,
// never /tmp; fail loud if HOME is unresolvable (engram_resolve_data_dir exits).
let data_dir: String = engram_resolve_data_dir()
let snapshot_path: String = data_dir + "/snapshot.json"
// ENGRAM_STORE (tiered paged store engram-tiered-storage-engine.md). When set,
// the durable owner is the paged store (neuron.egm + neuron.wal): engram_store_boot
// imports snapshot.json ONCE into a fresh neuron.egm, else replays the WAL and loads
// the store resident snapshot.json is never read again as the ongoing store. This
// closes the "restart reverted to a 17h-old snapshot" data-loss window. Flag-off
// (default): byte-for-byte the historical snapshot + optional-WAL boot below.
if store_on() {
engram_store_boot(data_dir)
println("[engram] ENGRAM_STORE enabled — tiered paged store is the durable owner")
} else {
engram_load(snapshot_path)
// WAL replay (design doc §6). Gated: default OFF is byte-identical to legacy
// snapshot-only boot. When ON, the snapshot above is the compaction BASE and
// the WAL carries every mutation since; replay reconstructs state to the last
// CRC-valid record, then opens the WAL for appending.
if wal_on() {
let replayed: Int = engram_wal_boot(data_dir)
println("[engram] WAL enabled — replayed " + int_to_str(replayed) + " records")
}
// 2026-07-21 self-review boot guard: if the snapshot file has content but the
// load produced 0 nodes, something is wrong (corrupt file / parse failure).
// Preserve the evidence and warn loudly and since read routes no longer write
// the canonical path, a bad boot can no longer clobber the good snapshot.
let boot_snap: String = fs_read(snapshot_path)
if !str_eq(boot_snap, "") {
if engram_node_count() == 0 {
println("[engram] WARNING: snapshot.json is non-empty but load produced 0 nodes — preserving copy at snapshot.failed-load.json")
fs_write(data_dir + "/snapshot.failed-load.json", boot_snap)
} else {
// Good load: keep a boot-time backup of the snapshot as loaded.
fs_write(data_dir + "/snapshot.boot-backup.json", boot_snap)
}
}
}
println("[engram] runtime-native graph engine")
println("[engram] data_dir=" + data_dir)
println("[engram] node_count=" + int_to_str(engram_node_count()))
println("[engram] edge_count=" + int_to_str(engram_edge_count()))
println("[engram] listening on " + int_to_str(port))
// WAKE: summon the guide (soul-native). Flag-gated (GUIDE_ENABLE): default
// OFF returns immediately and leaves this boot byte-inert. When ON, the soul probes
// its hardware, selects a Qwen3 tier by spec, fetches the GGUF from HF if the local
// cache is cold, loads it via the backend, and binds consult_guide(). Idempotent
// across wakes a cached model and an already-answering guide are both no-ops.
let guide_wake: String = guide_summon()
println("[guide] summon result: " + guide_wake)
http_set_handler("handle_request")
http_serve(port, "handle_request")