Compare commits

...

8 Commits

Author SHA1 Message Date
will.anderson 8f8ccc945e self-review 2026-07-22: persist canonical snapshot on write routes; newest-first tie-break in node listings
El SDK Release / build-and-release (pull_request) Failing after 14m24s
Durability: the 2026-07-21 fix stopped read routes writing the canonical
snapshot but left no save on ANY write path — every mutation lived in RAM
until a manual POST /api/save. Observed live: two restarts reverted the
store to a 17h-old snapshot, destroying same-day writes. persist_canonical()
now runs after node/edge create, knowledge capture, forget, strengthen, and
load-merge. ISE telemetry excluded deliberately (48h-pruned, loss-tolerant,
~2/min; snapshotting 28MB per heartbeat is waste).

Listing order: scan routes sort by salience with store-order ties, so
equal-salience telemetry (all ISEs are 0.3) returned OLDEST first — a
limited /api/nodes query silently returned a stale window, and a 41h-old
heartbeat series read as a live outage during this review. Ties now break
newest-first by created_at.
2026-07-22 08:51:33 -05:00
will.anderson 409ec99397 self-review 2026-07-22: ACT-R/Petrov base-level WM decay replaces per-call multiplicative carry-over
The old carry-over (weight *= 0.7 per engram_activate call) was call-rate-
dependent — carried context died in seconds under rapid curiosity scans and
lingered for hours under quiet loops — and a decayed scalar cannot represent
access frequency at all.

Now: k=10 access-timestamp ring + Petrov (2006) closed-form tail, d=0.5.
WM promotion and engram_strengthen record presentations; carry-over evicts
at base-level tau=-3.0 (Soar forgetting, ~403s single-touch) and shapes the
weight held at promotion (wm_anchor) with the ACT-R retrieval logistic
(s=0.4) — a pure function of wall-clock time, idempotent per call.
Persisted as access_ts/wm_anchor in snapshots; legacy nodes fall back to
the optimized form ln(n/(1-d)) - d*ln(L). base_level exposed in both node
serializers for observability.

Backing spec: 2026-07-21 integration brief (bl-b17facdd). Verified live:
carried weight ~anchor seconds after two disjoint activations (old code:
0.49x); frequency-hot nodes hold B=1.9 vs -0.14 single-touch.
2026-07-22 08:44:39 -05:00
will.anderson dc39a61e2c self-review 2026-07-21: stop read routes clobbering canonical snapshot; add /api/load-merge
Root cause of the 2026-05→07 identity-node loss: route_scan_edges and
route_sync serialized state by engram_save()ing over the canonical
snapshot.json on every GET, so one bad boot load meant the first read
request overwrote the good snapshot. Read routes now export to scratch
paths. Boot guard preserves evidence on non-empty-file/zero-node loads
and keeps a boot-time backup on good loads. New POST /api/load-merge
(explicit path required) used to restore 385 identity nodes + 1115
edges from the 2026-05-13 backup.
2026-07-21 08:50:38 -05:00
will.anderson eba9eac8a8 self-review 2026-07-19: port stranded fixes to the release runtime (production copy)
Three fixes that existed elsewhere but never reached the runtime the engram
binary actually builds against:

- tokenized + ranked query matching (search/search_json/activate seeds/
  goal_bias) ported from the el-compiler copy (e3dabe3, 2026-07-14) — the
  production engram kept whole-query Ctrl-F for 5 days after the fix
  'shipped'. Multi-word curiosity seeds went 0 -> 36 activated. Kept the
  ISE seed exclusion the el-compiler copy dropped.
- Knowledge -> 0.20 WM threshold after tier checks (dev-line 4bf7716):
  Semantic/Episodic Knowledge nodes fell to the 0.40 note default and only
  entered WM via breakthrough.
- goal_bias: Knowledge in is_knowledge + curiosity-seed technical terms
  (dev-line d53516b).

Also: seed_epoch was a running pairwise average, not the mean it claimed —
exponentially over-weighted later seeds in the temporal-proximity bonus.
Fixed to a true int64-sum mean. Stale INHIBITION_FACTOR comment corrected.

Root cause captured as knowledge: two runtime copies + branch-per-fix
without merge discipline stranded the entire dev semantic layer (cosine
activation, embeddings) out of production. Reconciliation planned as P1.
2026-07-19 08:46:47 -05:00
will.anderson ab6b52a0b4 self-review 2026-07-18: fix soul SIGABRT double-free + engram route scoping sweep
1. engram_neighbors_json (release runtime): BFS frontier/visited strings were
   el_strdup'd (arena-tracked) but manually freed, so el_request_end()
   double-freed every one — SIGABRT in http_worker under load (2 prod crashes
   today via /api/neuron/session/begin and /api/neuron/graph; reproduced and
   verified fixed with ASAN). Introduced when porting from the dev runtime,
   which correctly uses plain strdup. Third instance of the
   arena-vs-manual-free class (after EngramNode 07-15 and idmap keys 07-16).

2. server.el: let-in-if scoping sweep — defaults assigned inside if-blocks
   never mutated the outer binding, so /api/search and /api/activate always
   ran with q="", created nodes got node_type=""/salience=0.0, edges got
   relation=""/weight=0.0, and save/load with no path hit engram_save("").
   Rewritten to the let-if-else expression form. /api/activate now also
   rejects empty queries instead of wiping carried WM weights.

3. engram_activate: retrieval reinforcement (ACT-R base-level learning) —
   nodes promoted to WM that survive both capacity caps now get
   last_activated/activation_count updated, so frequently retrieved memories
   decay slower than abandoned ones. Scoped to promoted-only to avoid
   flattening dampening across BFS fan-out.
2026-07-18 08:48:04 -05:00
will.anderson e3dabe3e08 fix(engram): tokenized + ranked lexical search, not whole-query Ctrl-F
El SDK Release / build-and-release (pull_request) Failing after 14m46s
engram search/activate/goal-bias matched the ENTIRE raw query string as a
single case-insensitive substring (istr_contains(field, q)). Multi-word
queries like "windows msi signing" only matched a node containing that exact
contiguous run, so real multi-word queries returned ZERO on a graph saturated
with the answer. This is Ctrl-F, not search — and search is the core of the
engram being useful.

Fix: split the query on whitespace into distinct tokens; a node matches if it
contains ANY token in content/label/tags. Rank by distinct tokens matched
(desc) then salience (desc). istr_contains is kept unchanged as the per-token
primitive. Single-token queries are a strict special case (score 0 or 1) so
the many single-word callers do not regress.

Sites changed (all in el_runtime.c):
- new helpers engram_tokenize_query / engram_node_match_score / engram_rank_cmp
- engram_search           (internal el_val_t path)
- engram_search_json      (HTTP /api/search path)
- engram_activate seed loop (HTTP /api/activate path; seed activation scaled
  by token coverage so full-query matches seed more strongly)
- engram_goal_bias overlap bonus upgraded to graded token coverage

Proof (6591-node snapshot copy, rebuilt binary on :8799, POST JSON path):
  windows msi signing  0 -> 20   Will Anderson  0 -> 20
  windows msi          0 -> 20   tokenized search fix  0 -> 20
Single-word parity preserved (VBD/volatility/elc capped at limit; unkey = all
matching nodes). Top hits are relevant (e.g. "Will Anderson" surfaces the
Project Design and VBD whitepapers).

Note: GET ?q=a%20b still returns 0 because query_param (server.el) does not
URL-decode — a separate EL-layer bug; the soul's POST-JSON path is fixed here.
2026-07-14 18:39:07 -05:00
will.anderson 0a0a2bcb44 parser: bound token reads to Eof so malformed input errors instead of OOMing
El SDK Release / build-and-release (pull_request) Failing after 16s
Out-of-range tok_kind/tok_value reads returned runtime null (el_list_get OOB
-> 0) rather than the Eof sentinel, so the inner parse loops (parse_block,
call-arg, array-literal, match-arm) that terminate only on their close
delimiter or k=="Eof" never saw Eof once the cursor ran past the single
trailing Eof token. On unclosed-delimiter input the parser then appended AST
nodes forever -> unbounded allocation -> ~700GB -> OOM (observed compiling
neuron/sessions.el).

Fix at the choke point: tok_kind returns "Eof" and tok_value returns "" for
out-of-range positions, restoring the parser-wide contract that reads at/after
the end yield Eof. expect() no longer steps past the Eof sentinel on mismatch.
This terminates every overrun loop simultaneously; a malformed program now
surfaces as a normal (best-effort) parse end instead of exhausting memory.

Requires a self-hosted bootstrap rebuild of elc to take effect.
2026-07-14 14:21:39 -05:00
will.anderson f78da81aa4 runtime: fix the memory leak + write-corruption pair in el_runtime.c
El SDK Release / build-and-release (pull_request) Failing after 11m58s
Two independent investigations, one runtime, complementary halves:

1. Leak (Jul 2, this machine): JsonBuf buffers returned via el_wrap_str
   were raw malloc, never arena-tracked — every engram_*_json call leaked
   its output unconditionally. Added jb_finish() arena-tracking across all
   ~30 return sites. Plus el_arena_push/pop per-tick bracketing support
   for the soul's awareness loop (the loop ran outside any request arena,
   so even correctly-tracked allocations were permanent — 7.5GB RSS in
   under a minute at 1s tick).

2. Corruption (Tim's container soak, docs findings/container-migration):
   stored engram node/edge fields (content, node_type, label, tier, tags,
   metadata, from/to ids) were arena el_strdup — freed at request end,
   leaving dangling pointers that read back as recycled request-buffer
   bytes one request later. This is the June corruption root cause and
   the mechanism that grew snapshot.json to 18GB of empty-type junk
   (21.6M nodes, 3,335 real). 39 sites switched to el_strdup_persist,
   plus a latent double-free fix in engram_load metadata fixup.

Interaction note: fix 1's per-tick arena reclamation makes fix 2
mandatory — more aggressive arena recycling widens the use-after-free
window if stored fields still live in the arena. Apply as a pair, never
separately.

Verified live: soul + engram rebuilt from this runtime, booted against
the recovered real snapshot (3,335 nodes/40,146 edges), 5h stable at
<100MB RSS, write-then-next-request field-integrity test passes (the
June corruption fingerprint does not reproduce). engram/dist/engram
binary updated from this build.

Investigation credit: leak diagnosis this machine Jul 2-6; corruption
diagnosis + persist-fix patch by Tim's instance (docs PR #4).
2026-07-13 16:22:02 -05:00
6 changed files with 1387 additions and 348 deletions
BIN
View File
Binary file not shown.
+142 -95
View File
@@ -10,6 +10,7 @@ el_val_t query_param(el_val_t path, el_val_t key);
el_val_t query_int(el_val_t path, el_val_t key, el_val_t default_val);
el_val_t extract_id(el_val_t path, el_val_t prefix);
el_val_t route_stats(el_val_t method, el_val_t path, el_val_t body);
el_val_t persist_canonical(void);
el_val_t route_create_node(el_val_t method, el_val_t path, el_val_t body);
el_val_t route_get_node(el_val_t method, el_val_t path, el_val_t body);
el_val_t route_scan_nodes(el_val_t method, el_val_t path, el_val_t body);
@@ -20,18 +21,23 @@ el_val_t route_create_edge(el_val_t method, el_val_t path, el_val_t body);
el_val_t route_neighbors(el_val_t method, el_val_t path, el_val_t body);
el_val_t route_strengthen(el_val_t method, el_val_t path, el_val_t body);
el_val_t route_forget(el_val_t method, el_val_t path, el_val_t body);
el_val_t route_create_ise(el_val_t method, el_val_t path, el_val_t body);
el_val_t route_sync(el_val_t method, el_val_t path, el_val_t body);
el_val_t route_save(el_val_t method, el_val_t path, el_val_t body);
el_val_t route_load(el_val_t method, el_val_t path, el_val_t body);
el_val_t route_health(el_val_t method, el_val_t path, el_val_t body);
el_val_t route_sync(el_val_t method, el_val_t path, el_val_t body);
el_val_t route_load_merge(el_val_t method, el_val_t path, el_val_t body);
el_val_t route_emit_ise(el_val_t method, el_val_t path, el_val_t body);
el_val_t route_capture_knowledge(el_val_t method, el_val_t path, el_val_t body);
el_val_t check_auth_ok(el_val_t method, el_val_t body);
el_val_t handle_request(el_val_t method, el_val_t path, el_val_t body);
el_val_t bind_raw;
el_val_t bind_str;
el_val_t port;
el_val_t data_dir_raw;
el_val_t data_dir;
el_val_t snapshot_path;
el_val_t boot_snap;
el_val_t parse_port(el_val_t bind) {
el_val_t colon = str_index_of(bind, EL_STR(":"));
@@ -110,17 +116,22 @@ el_val_t route_stats(el_val_t method, el_val_t path, el_val_t body) {
return 0;
}
el_val_t persist_canonical(void) {
el_val_t dir_raw = env(EL_STR("ENGRAM_DATA_DIR"));
el_val_t dir = ({ el_val_t _if_result_1 = 0; if (str_eq(dir_raw, EL_STR(""))) { _if_result_1 = (EL_STR("/tmp/engram")); } else { _if_result_1 = (dir_raw); } _if_result_1; });
engram_save(el_str_concat(dir, EL_STR("/snapshot.json")));
return 1;
return 0;
}
el_val_t route_create_node(el_val_t method, el_val_t path, el_val_t body) {
el_val_t content = json_get_string(body, EL_STR("content"));
el_val_t node_type = json_get_string(body, EL_STR("node_type"));
if (str_eq(node_type, EL_STR(""))) {
node_type = EL_STR("Memory");
}
el_val_t salience = json_get_float(body, EL_STR("salience"));
if (salience == el_from_float(0.0)) {
salience = el_from_float(0.5);
}
el_val_t nt_raw = json_get_string(body, EL_STR("node_type"));
el_val_t node_type = ({ el_val_t _if_result_2 = 0; if (str_eq(nt_raw, EL_STR(""))) { _if_result_2 = (EL_STR("Memory")); } else { _if_result_2 = (nt_raw); } _if_result_2; });
el_val_t sal_raw = json_get_float(body, EL_STR("salience"));
el_val_t salience = ({ el_val_t _if_result_3 = 0; if ((sal_raw == el_from_float(0.0))) { _if_result_3 = (el_from_float(0.5)); } else { _if_result_3 = (sal_raw); } _if_result_3; });
el_val_t id = engram_node(content, node_type, salience);
el_val_t saved = persist_canonical();
return el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(EL_STR("{\"id\":\""), id), EL_STR("\",\"content\":\"")), content), EL_STR("\",\"node_type\":\"")), node_type), EL_STR("\"}"));
return 0;
}
@@ -146,11 +157,9 @@ el_val_t route_scan_nodes(el_val_t method, el_val_t path, el_val_t body) {
}
el_val_t route_scan_edges(el_val_t method, el_val_t path, el_val_t body) {
el_val_t dir = env(EL_STR("ENGRAM_DATA_DIR"));
if (str_eq(dir, EL_STR(""))) {
dir = EL_STR("/tmp/engram");
}
el_val_t snap_path = el_str_concat(dir, EL_STR("/snapshot.json"));
el_val_t dir_raw = env(EL_STR("ENGRAM_DATA_DIR"));
el_val_t dir = ({ el_val_t _if_result_4 = 0; if (str_eq(dir_raw, EL_STR(""))) { _if_result_4 = (EL_STR("/tmp/engram")); } else { _if_result_4 = (dir_raw); } _if_result_4; });
el_val_t snap_path = el_str_concat(dir, EL_STR("/.scan-export.json"));
engram_save(snap_path);
el_val_t snap = fs_read(snap_path);
if (str_eq(snap, EL_STR(""))) {
@@ -165,36 +174,22 @@ el_val_t route_scan_edges(el_val_t method, el_val_t path, el_val_t body) {
}
el_val_t route_search(el_val_t method, el_val_t path, el_val_t body) {
el_val_t q = EL_STR("");
if (str_eq(method, EL_STR("GET"))) {
q = query_param(path, EL_STR("q"));
} else {
q = json_get_string(body, EL_STR("query"));
}
el_val_t limit = query_int(path, EL_STR("limit"), 20);
if (limit == 0) {
limit = json_get_int(body, EL_STR("limit"));
}
if (limit == 0) {
limit = 20;
}
el_val_t q = ({ el_val_t _if_result_5 = 0; if (str_eq(method, EL_STR("GET"))) { _if_result_5 = (query_param(path, EL_STR("q"))); } else { _if_result_5 = (json_get_string(body, EL_STR("query"))); } _if_result_5; });
el_val_t lim_url = query_int(path, EL_STR("limit"), 0);
el_val_t lim_body = json_get_int(body, EL_STR("limit"));
el_val_t lim_either = ({ el_val_t _if_result_6 = 0; if ((lim_url > 0)) { _if_result_6 = (lim_url); } else { _if_result_6 = (lim_body); } _if_result_6; });
el_val_t limit = ({ el_val_t _if_result_7 = 0; if ((lim_either > 0)) { _if_result_7 = (lim_either); } else { _if_result_7 = (20); } _if_result_7; });
return engram_search_json(q, limit);
return 0;
}
el_val_t route_activate(el_val_t method, el_val_t path, el_val_t body) {
el_val_t q = EL_STR("");
el_val_t depth = 3;
if (str_eq(method, EL_STR("GET"))) {
q = query_param(path, EL_STR("q"));
depth = query_int(path, EL_STR("depth"), 3);
} else {
q = json_get_string(body, EL_STR("query"));
el_val_t bd = json_get_int(body, EL_STR("depth"));
if (bd > 0) {
depth = bd;
}
el_val_t q = ({ el_val_t _if_result_8 = 0; if (str_eq(method, EL_STR("GET"))) { _if_result_8 = (query_param(path, EL_STR("q"))); } else { _if_result_8 = (json_get_string(body, EL_STR("query"))); } _if_result_8; });
if (str_eq(q, EL_STR(""))) {
return err_json(EL_STR("missing query"));
}
el_val_t d_raw = ({ el_val_t _if_result_9 = 0; if (str_eq(method, EL_STR("GET"))) { _if_result_9 = (query_int(path, EL_STR("depth"), 3)); } else { _if_result_9 = (json_get_int(body, EL_STR("depth"))); } _if_result_9; });
el_val_t depth = ({ el_val_t _if_result_10 = 0; if ((d_raw > 0)) { _if_result_10 = (d_raw); } else { _if_result_10 = (3); } _if_result_10; });
return el_str_concat(el_str_concat(EL_STR("{\"results\":"), engram_activate_json(q, depth)), EL_STR("}"));
return 0;
}
@@ -202,15 +197,12 @@ el_val_t route_activate(el_val_t method, el_val_t path, el_val_t body) {
el_val_t route_create_edge(el_val_t method, el_val_t path, el_val_t body) {
el_val_t from_id = json_get_string(body, EL_STR("from_id"));
el_val_t to_id = json_get_string(body, EL_STR("to_id"));
el_val_t relation = json_get_string(body, EL_STR("relation"));
if (str_eq(relation, EL_STR(""))) {
relation = EL_STR("associates");
}
el_val_t weight = json_get_float(body, EL_STR("weight"));
if (weight == el_from_float(0.0)) {
weight = el_from_float(0.5);
}
el_val_t rel_raw = json_get_string(body, EL_STR("relation"));
el_val_t relation = ({ el_val_t _if_result_11 = 0; if (str_eq(rel_raw, EL_STR(""))) { _if_result_11 = (EL_STR("associates")); } else { _if_result_11 = (rel_raw); } _if_result_11; });
el_val_t w_raw = json_get_float(body, EL_STR("weight"));
el_val_t weight = ({ el_val_t _if_result_12 = 0; if ((w_raw == el_from_float(0.0))) { _if_result_12 = (el_from_float(0.5)); } else { _if_result_12 = (w_raw); } _if_result_12; });
engram_connect(from_id, to_id, weight, relation);
el_val_t saved = persist_canonical();
return el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(EL_STR("{\"ok\":true,\"from_id\":\""), from_id), EL_STR("\",\"to_id\":\"")), to_id), EL_STR("\",\"relation\":\"")), relation), EL_STR("\"}"));
return 0;
}
@@ -231,6 +223,7 @@ el_val_t route_strengthen(el_val_t method, el_val_t path, el_val_t body) {
return err_json(EL_STR("missing node_id"));
}
engram_strengthen(id);
el_val_t saved = persist_canonical();
return ok_json();
return 0;
}
@@ -241,29 +234,40 @@ el_val_t route_forget(el_val_t method, el_val_t path, el_val_t body) {
return err_json(EL_STR("missing id"));
}
engram_forget(id);
el_val_t saved = persist_canonical();
return ok_json();
return 0;
}
el_val_t route_create_ise(el_val_t method, el_val_t path, el_val_t body) {
el_val_t content = json_get_string(body, EL_STR("content"));
if (str_eq(content, EL_STR(""))) {
return err_json(EL_STR("missing content"));
}
el_val_t sal = el_from_float(0.3);
el_val_t imp = el_from_float(0.3);
el_val_t conf = el_from_float(0.8);
el_val_t id = engram_node_full(content, EL_STR("InternalStateEvent"), EL_STR("state-event"), sal, imp, conf, EL_STR("Episodic"), EL_STR("[\"internal-state\",\"InternalStateEvent\"]"));
return el_str_concat(el_str_concat(EL_STR("{\"ok\":true,\"id\":\""), id), EL_STR("\"}"));
el_val_t route_save(el_val_t method, el_val_t path, el_val_t body) {
el_val_t p_raw = json_get_string(body, EL_STR("path"));
el_val_t dir_raw = env(EL_STR("ENGRAM_DATA_DIR"));
el_val_t dir = ({ el_val_t _if_result_13 = 0; if (str_eq(dir_raw, EL_STR(""))) { _if_result_13 = (EL_STR("/tmp/engram")); } else { _if_result_13 = (dir_raw); } _if_result_13; });
el_val_t p = ({ el_val_t _if_result_14 = 0; if (str_eq(p_raw, EL_STR(""))) { _if_result_14 = (el_str_concat(dir, EL_STR("/snapshot.json"))); } else { _if_result_14 = (p_raw); } _if_result_14; });
engram_save(p);
return el_str_concat(el_str_concat(EL_STR("{\"ok\":true,\"path\":\""), p), EL_STR("\"}"));
return 0;
}
el_val_t route_load(el_val_t method, el_val_t path, el_val_t body) {
el_val_t p_raw = json_get_string(body, EL_STR("path"));
el_val_t dir_raw = env(EL_STR("ENGRAM_DATA_DIR"));
el_val_t dir = ({ el_val_t _if_result_15 = 0; if (str_eq(dir_raw, EL_STR(""))) { _if_result_15 = (EL_STR("/tmp/engram")); } else { _if_result_15 = (dir_raw); } _if_result_15; });
el_val_t p = ({ el_val_t _if_result_16 = 0; if (str_eq(p_raw, EL_STR(""))) { _if_result_16 = (el_str_concat(dir, EL_STR("/snapshot.json"))); } else { _if_result_16 = (p_raw); } _if_result_16; });
engram_load(p);
return ok_json();
return 0;
}
el_val_t route_health(el_val_t method, el_val_t path, el_val_t body) {
return EL_STR("{\"status\":\"ok\",\"engine\":\"engram-runtime-native\"}");
return 0;
}
el_val_t route_sync(el_val_t method, el_val_t path, el_val_t body) {
el_val_t dir = env(EL_STR("ENGRAM_DATA_DIR"));
if (str_eq(dir, EL_STR(""))) {
dir = EL_STR("/tmp/engram");
}
el_val_t snap_path = el_str_concat(dir, EL_STR("/sync-export.json"));
el_val_t dir_raw = env(EL_STR("ENGRAM_DATA_DIR"));
el_val_t dir = ({ el_val_t _if_result_17 = 0; if (str_eq(dir_raw, EL_STR(""))) { _if_result_17 = (EL_STR("/tmp/engram")); } else { _if_result_17 = (dir_raw); } _if_result_17; });
el_val_t snap_path = el_str_concat(dir, EL_STR("/.sync-export.json"));
engram_save(snap_path);
el_val_t snap = fs_read(snap_path);
if (str_eq(snap, EL_STR(""))) {
@@ -273,36 +277,68 @@ el_val_t route_sync(el_val_t method, el_val_t path, el_val_t body) {
return 0;
}
el_val_t route_save(el_val_t method, el_val_t path, el_val_t body) {
el_val_t route_load_merge(el_val_t method, el_val_t path, el_val_t body) {
el_val_t p = json_get_string(body, EL_STR("path"));
if (str_eq(p, EL_STR(""))) {
el_val_t dir = env(EL_STR("ENGRAM_DATA_DIR"));
if (str_eq(dir, EL_STR(""))) {
dir = EL_STR("/tmp/engram");
}
p = el_str_concat(dir, EL_STR("/snapshot.json"));
return err_json(EL_STR("path is required"));
}
engram_save(p);
return el_str_concat(el_str_concat(EL_STR("{\"ok\":true,\"path\":\""), p), EL_STR("\"}"));
if (str_eq(fs_read(p), EL_STR(""))) {
return err_json(EL_STR("file missing or empty"));
}
el_val_t before_n = engram_node_count();
el_val_t before_e = engram_edge_count();
engram_load_merge(p);
el_val_t added_n = (engram_node_count() - before_n);
el_val_t added_e = (engram_edge_count() - before_e);
el_val_t saved = persist_canonical();
return el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(EL_STR("{\"ok\":true,\"nodes_added\":"), int_to_str(added_n)), EL_STR(",\"edges_added\":")), int_to_str(added_e)), EL_STR(",\"node_count\":")), int_to_str(engram_node_count())), EL_STR("}"));
return 0;
}
el_val_t route_load(el_val_t method, el_val_t path, el_val_t body) {
el_val_t p = json_get_string(body, EL_STR("path"));
if (str_eq(p, EL_STR(""))) {
el_val_t dir = env(EL_STR("ENGRAM_DATA_DIR"));
if (str_eq(dir, EL_STR(""))) {
dir = EL_STR("/tmp/engram");
}
p = el_str_concat(dir, EL_STR("/snapshot.json"));
el_val_t route_emit_ise(el_val_t method, el_val_t path, el_val_t body) {
el_val_t content = json_get_string(body, EL_STR("content"));
if (str_eq(content, EL_STR(""))) {
return err_json(EL_STR("missing content"));
}
engram_load(p);
return ok_json();
el_val_t sal = el_from_float(0.3);
el_val_t imp = el_from_float(0.3);
el_val_t conf = el_from_float(0.8);
el_val_t id = engram_node_full(content, EL_STR("InternalStateEvent"), EL_STR("state-event"), sal, imp, conf, EL_STR("Episodic"), EL_STR("[\"internal-state\",\"InternalStateEvent\"]"));
el_val_t ret_raw = env(EL_STR("ENGRAM_ISE_RETENTION_MS"));
el_val_t ret_ms = ({ el_val_t _if_result_18 = 0; if (str_eq(ret_raw, EL_STR(""))) { _if_result_18 = (172800000); } else { _if_result_18 = (str_to_int(ret_raw)); } _if_result_18; });
el_val_t pruned = engram_prune_telemetry(ret_ms);
return el_str_concat(el_str_concat(el_str_concat(el_str_concat(EL_STR("{\"ok\":true,\"id\":\""), id), EL_STR("\",\"pruned\":")), int_to_str(pruned)), EL_STR("}"));
return 0;
}
el_val_t route_health(el_val_t method, el_val_t path, el_val_t body) {
return EL_STR("{\"status\":\"ok\",\"engine\":\"engram-runtime-native\"}");
el_val_t route_capture_knowledge(el_val_t method, el_val_t path, el_val_t body) {
el_val_t content = json_get_string(body, EL_STR("content"));
if (str_eq(content, EL_STR(""))) {
return err_json(EL_STR("missing content"));
}
el_val_t title = json_get_string(body, EL_STR("title"));
el_val_t label = ({ el_val_t _if_result_19 = 0; if (str_eq(title, EL_STR(""))) { _if_result_19 = (str_slice(content, 0, 60)); } else { _if_result_19 = (title); } _if_result_19; });
el_val_t category_raw = json_get_string(body, EL_STR("category"));
el_val_t category = ({ el_val_t _if_result_20 = 0; if (str_eq(category_raw, EL_STR(""))) { _if_result_20 = (EL_STR("other")); } else { _if_result_20 = (category_raw); } _if_result_20; });
el_val_t ktier_raw = json_get_string(body, EL_STR("tier"));
el_val_t ktier = ({ el_val_t _if_result_21 = 0; if (str_eq(ktier_raw, EL_STR(""))) { _if_result_21 = (EL_STR("note")); } else { _if_result_21 = (ktier_raw); } _if_result_21; });
el_val_t project = json_get_string(body, EL_STR("project"));
el_val_t tags_raw = json_get_raw(body, EL_STR("tags"));
el_val_t tags_base = ({ el_val_t _if_result_22 = 0; if (str_eq(tags_raw, EL_STR(""))) { _if_result_22 = (EL_STR("[]")); } else { _if_result_22 = (tags_raw); } _if_result_22; });
el_val_t base_len = str_len(tags_base);
el_val_t head = str_slice(tags_base, 0, (base_len - 1));
el_val_t sep = ({ el_val_t _if_result_23 = 0; if (str_eq(head, EL_STR("["))) { _if_result_23 = (EL_STR("")); } else { _if_result_23 = (EL_STR(",")); } _if_result_23; });
el_val_t safe_cat = str_replace(category, EL_STR("\""), EL_STR("'"));
el_val_t safe_tier = str_replace(ktier, EL_STR("\""), EL_STR("'"));
el_val_t safe_proj = str_replace(project, EL_STR("\""), EL_STR("'"));
el_val_t proj_tag = ({ el_val_t _if_result_24 = 0; if (str_eq(safe_proj, EL_STR(""))) { _if_result_24 = (EL_STR("")); } else { _if_result_24 = (el_str_concat(el_str_concat(EL_STR(",\"project:"), safe_proj), EL_STR("\""))); } _if_result_24; });
el_val_t tags = el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(head, sep), EL_STR("\"category:")), safe_cat), EL_STR("\",\"tier:")), safe_tier), EL_STR("\"")), proj_tag), EL_STR("]"));
el_val_t sal = el_from_float(0.5);
el_val_t imp = el_from_float(0.5);
el_val_t conf = el_from_float(0.9);
el_val_t id = engram_node_full(content, EL_STR("Knowledge"), label, sal, imp, conf, EL_STR("Semantic"), tags);
el_val_t saved = persist_canonical();
return el_str_concat(el_str_concat(EL_STR("{\"ok\":true,\"id\":\""), id), EL_STR("\"}"));
return 0;
}
@@ -329,12 +365,15 @@ el_val_t handle_request(el_val_t method, el_val_t path, el_val_t body) {
return route_health(method, path, body);
}
}
if (str_eq(method, EL_STR("POST")) && str_starts_with(clean, EL_STR("/api/neuron/state-events"))) {
return route_create_ise(method, path, body);
if (str_eq(method, EL_STR("POST")) && str_eq(clean, EL_STR("/api/neuron/state-events"))) {
return route_emit_ise(method, path, body);
}
if (!check_auth_ok(method, body)) {
return err_json(EL_STR("unauthorized"));
}
if (str_eq(method, EL_STR("POST")) && str_eq(clean, EL_STR("/api/neuron/knowledge/capture"))) {
return route_capture_knowledge(method, path, body);
}
if (str_eq(method, EL_STR("GET")) && (str_eq(clean, EL_STR("/api/stats")) || str_eq(clean, EL_STR("/stats")))) {
return route_stats(method, path, body);
}
@@ -374,32 +413,40 @@ el_val_t handle_request(el_val_t method, el_val_t path, el_val_t body) {
if (str_eq(method, EL_STR("POST")) && (str_eq(clean, EL_STR("/api/strengthen")) || str_eq(clean, EL_STR("/strengthen")))) {
return route_strengthen(method, path, body);
}
if (str_eq(method, EL_STR("GET")) && (str_eq(clean, EL_STR("/api/sync")) || str_eq(clean, EL_STR("/sync")))) {
return route_sync(method, path, body);
}
if (str_eq(method, EL_STR("POST")) && (str_eq(clean, EL_STR("/api/save")) || str_eq(clean, EL_STR("/save")))) {
return route_save(method, path, body);
}
if (str_eq(method, EL_STR("POST")) && (str_eq(clean, EL_STR("/api/load")) || str_eq(clean, EL_STR("/load")))) {
return route_load(method, path, body);
}
if (str_eq(method, EL_STR("POST")) && (str_eq(clean, EL_STR("/api/load-merge")) || str_eq(clean, EL_STR("/load-merge")))) {
return route_load_merge(method, path, body);
}
if (str_eq(method, EL_STR("GET")) && str_eq(clean, EL_STR("/api/sync"))) {
return route_sync(method, path, body);
}
return el_str_concat(el_str_concat(EL_STR("{\"error\":\"not found\",\"path\":\""), clean), EL_STR("\"}"));
return 0;
}
int main(int _argc, char** _argv) {
el_runtime_init_args(_argc, _argv);
bind_str = env(EL_STR("ENGRAM_BIND"));
if (str_eq(bind_str, EL_STR(""))) {
bind_str = EL_STR(":8742");
}
bind_raw = env(EL_STR("ENGRAM_BIND"));
bind_str = ({ el_val_t _if_result_25 = 0; if (str_eq(bind_raw, EL_STR(""))) { _if_result_25 = (EL_STR(":8742")); } else { _if_result_25 = (bind_raw); } _if_result_25; });
port = parse_port(bind_str);
data_dir = env(EL_STR("ENGRAM_DATA_DIR"));
if (str_eq(data_dir, EL_STR(""))) {
data_dir = EL_STR("/tmp/engram");
}
data_dir_raw = env(EL_STR("ENGRAM_DATA_DIR"));
data_dir = ({ el_val_t _if_result_26 = 0; if (str_eq(data_dir_raw, EL_STR(""))) { _if_result_26 = (EL_STR("/tmp/engram")); } else { _if_result_26 = (data_dir_raw); } _if_result_26; });
snapshot_path = el_str_concat(data_dir, EL_STR("/snapshot.json"));
engram_load(snapshot_path);
boot_snap = fs_read(snapshot_path);
if (!str_eq(boot_snap, EL_STR(""))) {
if (engram_node_count() == 0) {
println(EL_STR("[engram] WARNING: snapshot.json is non-empty but load produced 0 nodes \xe2\x80\x94 preserving copy at snapshot.failed-load.json"));
fs_write(el_str_concat(data_dir, EL_STR("/snapshot.failed-load.json")), boot_snap);
} else {
fs_write(el_str_concat(data_dir, EL_STR("/snapshot.boot-backup.json")), boot_snap);
}
}
println(EL_STR("[engram] runtime-native graph engine"));
println(el_str_concat(EL_STR("[engram] data_dir="), data_dir));
println(el_str_concat(EL_STR("[engram] node_count="), int_to_str(engram_node_count())));
+190 -54
View File
@@ -76,13 +76,43 @@ fn route_stats(method: String, path: String, body: String) -> String {
engram_stats_json()
}
// (2026-07-18 self-review) Scoping sweep: `let` inside an if-block creates an
// inner scope only it does NOT mutate the outer binding (documented with
// evidence in awareness.el, 2026-05-25). Every default/reassignment below used
// that broken pattern, so defaults never applied: nodes were created with
// node_type="" and salience=0.0, /api/search and /api/activate ALWAYS ran with
// q="" regardless of input, edges defaulted to relation=""/weight=0.0, and
// save/load with no "path" hit engram_save(""). Rewritten to the
// `let x = if cond { a } else { b }` expression form (the pattern the newer
// routes route_emit_ise/route_capture_knowledge already use correctly).
// persist_canonical save the canonical snapshot after a durable write.
//
// WHY (2026-07-22 self-review): the 2026-07-21 fix correctly stopped READ
// routes from writing the canonical snapshot.json but nothing was left
// that saved it on WRITE. Every mutation (node create, edge create,
// knowledge capture, forget, merge) lived only in RAM until someone POSTed
// /api/save manually; a process restart silently discarded everything since
// the last manual save. Observed live: two engram restarts during the
// 2026-07-22 review reverted the store to a ~17h-old snapshot, destroying
// same-day writes. Reads must never write the canonical; writes must always
// persist it. ISE telemetry is deliberately excluded (48h-pruned, loss-
// tolerant, ~2/min snapshotting the whole store per heartbeat is waste;
// any durable write that follows persists the pruning too).
fn persist_canonical() -> Int {
let dir_raw: String = env("ENGRAM_DATA_DIR")
let dir: String = if str_eq(dir_raw, "") { "/tmp/engram" } else { dir_raw }
engram_save(dir + "/snapshot.json")
return 1
}
fn route_create_node(method: String, path: String, body: String) -> String {
let content: String = json_get_string(body, "content")
let node_type: String = json_get_string(body, "node_type")
if str_eq(node_type, "") { let node_type = "Memory" }
let salience: Float = json_get_float(body, "salience")
if salience == 0.0 { let salience = 0.5 }
let nt_raw: String = json_get_string(body, "node_type")
let node_type: String = if str_eq(nt_raw, "") { "Memory" } else { nt_raw }
let sal_raw: Float = json_get_float(body, "salience")
let salience: Float = if sal_raw == 0.0 { 0.5 } else { sal_raw }
let id: String = engram_node(content, node_type, salience)
let saved: Int = persist_canonical()
"{\"id\":\"" + id + "\",\"content\":\"" + content + "\",\"node_type\":\"" + node_type + "\"}"
}
@@ -103,13 +133,14 @@ fn route_scan_nodes(method: String, path: String, body: String) -> String {
}
// route_scan_edges bulk export of all edges as a JSON array. Implemented
// via engram_save fs_read of the canonical on-disk snapshot, which the
// runtime keeps in lockstep with the in-memory graph. Live against the
// running graph, not a stale export.
// via engram_save fs_read of a SCRATCH export path. (2026-07-21 self-review:
// previously this saved over the canonical snapshot.json on every GET if the
// process ever booted with a partial/empty store, the first read request
// clobbered the good snapshot. Read routes must never write the canonical path.)
fn route_scan_edges(method: String, path: String, body: String) -> String {
let dir: String = env("ENGRAM_DATA_DIR")
if str_eq(dir, "") { let dir = "/tmp/engram" }
let snap_path: String = dir + "/snapshot.json"
let dir_raw: String = env("ENGRAM_DATA_DIR")
let dir: String = if str_eq(dir_raw, "") { "/tmp/engram" } else { dir_raw }
let snap_path: String = dir + "/.scan-export.json"
engram_save(snap_path)
let snap: String = fs_read(snap_path)
if str_eq(snap, "") { return "[]" }
@@ -122,40 +153,34 @@ fn route_scan_edges(method: String, path: String, body: String) -> String {
}
fn route_search(method: String, path: String, body: String) -> String {
let q: String = ""
if str_eq(method, "GET") {
let q = query_param(path, "q")
} else {
let q = json_get_string(body, "query")
}
let limit: Int = query_int(path, "limit", 20)
if limit == 0 { let limit = json_get_int(body, "limit") }
if limit == 0 { let limit = 20 }
let q: String = if str_eq(method, "GET") { query_param(path, "q") } else { json_get_string(body, "query") }
let lim_url: Int = query_int(path, "limit", 0)
let lim_body: Int = json_get_int(body, "limit")
let lim_either: Int = if lim_url > 0 { lim_url } else { lim_body }
let limit: Int = if lim_either > 0 { lim_either } else { 20 }
return engram_search_json(q, limit)
}
fn route_activate(method: String, path: String, body: String) -> String {
let q: String = ""
let depth: Int = 3
if str_eq(method, "GET") {
let q = query_param(path, "q")
let depth = query_int(path, "depth", 3)
} else {
let q = json_get_string(body, "query")
let bd: Int = json_get_int(body, "depth")
if bd > 0 { let depth = bd }
}
let q: String = if str_eq(method, "GET") { query_param(path, "q") } else { json_get_string(body, "query") }
// Guard: engram_activate with an empty query matches zero seeds, which
// zeroes ALL carried working-memory weights (documented in awareness.el
// perceive()). Never let an empty activation through to wipe WM.
if str_eq(q, "") { return err_json("missing query") }
let d_raw: Int = if str_eq(method, "GET") { query_int(path, "depth", 3) } else { json_get_int(body, "depth") }
let depth: Int = if d_raw > 0 { d_raw } else { 3 }
return "{\"results\":" + engram_activate_json(q, depth) + "}"
}
fn route_create_edge(method: String, path: String, body: String) -> String {
let from_id: String = json_get_string(body, "from_id")
let to_id: String = json_get_string(body, "to_id")
let relation: String = json_get_string(body, "relation")
if str_eq(relation, "") { let relation = "associates" }
let weight: Float = json_get_float(body, "weight")
if weight == 0.0 { let weight = 0.5 }
let rel_raw: String = json_get_string(body, "relation")
let relation: String = if str_eq(rel_raw, "") { "associates" } else { rel_raw }
let w_raw: Float = json_get_float(body, "weight")
let weight: Float = if w_raw == 0.0 { 0.5 } else { w_raw }
engram_connect(from_id, to_id, weight, relation)
let saved: Int = persist_canonical()
"{\"ok\":true,\"from_id\":\"" + from_id + "\",\"to_id\":\"" + to_id + "\",\"relation\":\"" + relation + "\"}"
}
@@ -170,6 +195,7 @@ fn route_strengthen(method: String, path: String, body: String) -> String {
let id: String = json_get_string(body, "node_id")
if str_eq(id, "") { return err_json("missing node_id") }
engram_strengthen(id)
let saved: Int = persist_canonical()
ok_json()
}
@@ -177,27 +203,24 @@ fn route_forget(method: String, path: String, body: String) -> String {
let id: String = extract_id(path, "/api/nodes/")
if str_eq(id, "") { return err_json("missing id") }
engram_forget(id)
let saved: Int = persist_canonical()
ok_json()
}
fn route_save(method: String, path: String, body: String) -> String {
let p: String = json_get_string(body, "path")
if str_eq(p, "") {
let dir: String = env("ENGRAM_DATA_DIR")
if str_eq(dir, "") { let dir = "/tmp/engram" }
let p = dir + "/snapshot.json"
}
let p_raw: String = json_get_string(body, "path")
let dir_raw: String = env("ENGRAM_DATA_DIR")
let dir: String = if str_eq(dir_raw, "") { "/tmp/engram" } else { dir_raw }
let p: String = if str_eq(p_raw, "") { dir + "/snapshot.json" } else { p_raw }
engram_save(p)
"{\"ok\":true,\"path\":\"" + p + "\"}"
}
fn route_load(method: String, path: String, body: String) -> String {
let p: String = json_get_string(body, "path")
if str_eq(p, "") {
let dir: String = env("ENGRAM_DATA_DIR")
if str_eq(dir, "") { let dir = "/tmp/engram" }
let p = dir + "/snapshot.json"
}
let p_raw: String = json_get_string(body, "path")
let dir_raw: String = env("ENGRAM_DATA_DIR")
let dir: String = if str_eq(dir_raw, "") { "/tmp/engram" } else { dir_raw }
let p: String = if str_eq(p_raw, "") { dir + "/snapshot.json" } else { p_raw }
engram_load(p)
ok_json()
}
@@ -219,15 +242,36 @@ fn route_health(method: String, path: String, body: String) -> String {
// (it skips nodes already present by ID). Auth-exempt: same-host internal call.
// (2026-06-27 self-review: added this route to fix silent 10-min sync failures)
fn route_sync(method: String, path: String, body: String) -> String {
let dir: String = env("ENGRAM_DATA_DIR")
if str_eq(dir, "") { let dir = "/tmp/engram" }
let snap_path: String = dir + "/snapshot.json"
let dir_raw: String = env("ENGRAM_DATA_DIR")
let dir: String = if str_eq(dir_raw, "") { "/tmp/engram" } else { dir_raw }
// 2026-07-21 self-review: export to a scratch path, never the canonical
// snapshot.json read routes must not be able to clobber the good snapshot.
let snap_path: String = dir + "/.sync-export.json"
engram_save(snap_path)
let snap: String = fs_read(snap_path)
if str_eq(snap, "") { return "{\"nodes\":[],\"edges\":[]}" }
return snap
}
// route_load_merge POST /api/load-merge {"path": "..."} merge a snapshot
// file into the live store WITHOUT resetting it (engram_load_merge skips nodes
// already present by id). Added 2026-07-21 self-review to restore the 244 kn-
// identity Knowledge nodes lost from the snapshot lineage between 05-13 and
// 07-13. Requires an explicit path: refuses to run without one so it can never
// be triggered accidentally against a default.
fn route_load_merge(method: String, path: String, body: String) -> String {
let p: String = json_get_string(body, "path")
if str_eq(p, "") { return err_json("path is required") }
if str_eq(fs_read(p), "") { return err_json("file missing or empty") }
let before_n: Int = engram_node_count()
let before_e: Int = engram_edge_count()
engram_load_merge(p)
let added_n: Int = engram_node_count() - before_n
let added_e: Int = engram_edge_count() - before_e
let saved: Int = persist_canonical()
"{\"ok\":true,\"nodes_added\":" + int_to_str(added_n) + ",\"edges_added\":" + int_to_str(added_e) + ",\"node_count\":" + int_to_str(engram_node_count()) + "}"
}
// route_emit_ise write an InternalStateEvent node from the soul daemon.
//
// Endpoint: POST /api/neuron/state-events
@@ -241,10 +285,20 @@ fn route_sync(method: String, path: String, body: String) -> String {
//
// Salience/importance set to match engram_node_full ISE defaults used by the
// in-process fallback path in awareness.el (salience=0.3, importance=0.3,
// confidence=0.8, tier=Episodic). High temporal_decay_rate (1.617) ISEs
// are inherently transient; they should decay faster than structural knowledge.
// confidence=0.8, tier=Episodic).
// (2026-06-26 self-review: added this route after discovering ise_post was
// silently failing the soul posts here but the endpoint didn't exist.)
//
// Retention (2026-07-16 self-review): an earlier comment here claimed ISEs
// got temporal_decay_rate=1.617 that was never implemented (engram_node_full
// hardcodes 0.0), and per-node decay only dampens activation anyway; it never
// removes nodes. By 2026-07-16 ISEs were 75% of the store (10,175 of 13,522
// nodes, ~4,300/day, unbounded). ISEs are already WM-excluded in
// engram_activate, so the fix is retention, not decay: every insert calls
// engram_prune_telemetry(), a single O(nodes+edges) compaction pass that
// removes ISEs older than ENGRAM_ISE_RETENTION_MS (default 48h), protecting
// "session-start" labels and self_review events as durable history. At
// ~3 ISEs/min this bounds telemetry at ~8.6k nodes instead of growing forever.
fn route_emit_ise(method: String, path: String, body: String) -> String {
let content: String = json_get_string(body, "content")
if str_eq(content, "") { return err_json("missing content") }
@@ -256,6 +310,64 @@ fn route_emit_ise(method: String, path: String, body: String) -> String {
sal, imp, conf,
"Episodic", "[\"internal-state\",\"InternalStateEvent\"]"
)
let ret_raw: String = env("ENGRAM_ISE_RETENTION_MS")
let ret_ms: Int = if str_eq(ret_raw, "") { 172800000 } else { str_to_int(ret_raw) }
let pruned: Int = engram_prune_telemetry(ret_ms)
"{\"ok\":true,\"id\":\"" + id + "\",\"pruned\":" + int_to_str(pruned) + "}"
}
// Knowledge capture
//
// route_capture_knowledge direct Knowledge-node capture over HTTP.
//
// Endpoint: POST /api/neuron/knowledge/capture (auth required: "_auth" in body)
// Body: {"content": "...", "title": "...", "category": "...",
// "tier": "note|lesson|canonical", "tags": [...], "project": "...",
// "_auth": "<key>"}
//
// WHY (2026-07-15 self-review): the world-ingestor integrator was designed
// against this endpoint (its MCP-unavailable fallback), but the route never
// existed every direct push 404'd, and because the auth gate ran before
// routing, the failure surfaced as {"error":"unauthorized"} and was
// misdiagnosed for two weeks while world knowledge silently dropped.
// POST /api/nodes was no substitute: it discards label/tags/tier, which
// makes captured knowledge invisible to tag-scoped search and curiosity.
//
// The incoming knowledge tier (note/lesson/canonical) is preserved as a
// "tier:<x>" tag rather than mapped onto Engram's cognitive tiers Knowledge
// nodes land in Semantic (stable reference), and the epistemic tier stays
// queryable without inventing a lossy mapping.
fn route_capture_knowledge(method: String, path: String, body: String) -> String {
let content: String = json_get_string(body, "content")
if str_eq(content, "") { return err_json("missing content") }
let title: String = json_get_string(body, "title")
let label: String = if str_eq(title, "") { str_slice(content, 0, 60) } else { title }
let category_raw: String = json_get_string(body, "category")
let category: String = if str_eq(category_raw, "") { "other" } else { category_raw }
let ktier_raw: String = json_get_string(body, "tier")
let ktier: String = if str_eq(ktier_raw, "") { "note" } else { ktier_raw }
let project: String = json_get_string(body, "project")
let tags_raw: String = json_get_raw(body, "tags")
let tags_base: String = if str_eq(tags_raw, "") { "[]" } else { tags_raw }
// Merge category/tier/project markers into the tag array. Search matches
// against the tags string, so these make captures findable by facet.
let base_len: Int = str_len(tags_base)
let head: String = str_slice(tags_base, 0, base_len - 1)
let sep: String = if str_eq(head, "[") { "" } else { "," }
let safe_cat: String = str_replace(category, "\"", "'")
let safe_tier: String = str_replace(ktier, "\"", "'")
let safe_proj: String = str_replace(project, "\"", "'")
let proj_tag: String = if str_eq(safe_proj, "") { "" } else { ",\"project:" + safe_proj + "\"" }
let tags: String = head + sep + "\"category:" + safe_cat + "\",\"tier:" + safe_tier + "\"" + proj_tag + "]"
let sal: Float = 0.5
let imp: Float = 0.5
let conf: Float = 0.9
let id: String = engram_node_full(
content, "Knowledge", label,
sal, imp, conf,
"Semantic", tags
)
let saved: Int = persist_canonical()
"{\"ok\":true,\"id\":\"" + id + "\"}"
}
@@ -295,6 +407,12 @@ fn handle_request(method: String, path: String, body: String) -> String {
return err_json("unauthorized")
}
// Knowledge capture (auth enforced above; the world-ingestor integrator
// and any headless session without MCP push knowledge through this)
if str_eq(method, "POST") && str_eq(clean, "/api/neuron/knowledge/capture") {
return route_capture_knowledge(method, path, body)
}
// Stats
if str_eq(method, "GET") && (str_eq(clean, "/api/stats") || str_eq(clean, "/stats")) {
return route_stats(method, path, body)
@@ -351,6 +469,9 @@ fn handle_request(method: String, path: String, body: String) -> String {
if str_eq(method, "POST") && (str_eq(clean, "/api/load") || str_eq(clean, "/load")) {
return route_load(method, path, body)
}
if str_eq(method, "POST") && (str_eq(clean, "/api/load-merge") || str_eq(clean, "/load-merge")) {
return route_load_merge(method, path, body)
}
// Sync soul daemon periodic pull of non-ISE knowledge into in-process graph
if str_eq(method, "GET") && str_eq(clean, "/api/sync") {
@@ -362,16 +483,31 @@ fn handle_request(method: String, path: String, body: String) -> String {
// Entry
let bind_str: String = env("ENGRAM_BIND")
if str_eq(bind_str, "") { let bind_str = ":8742" }
let bind_raw: String = env("ENGRAM_BIND")
let bind_str: String = if str_eq(bind_raw, "") { ":8742" } else { bind_raw }
let port: Int = parse_port(bind_str)
// On startup, try to load any existing snapshot (best effort).
let data_dir: String = env("ENGRAM_DATA_DIR")
if str_eq(data_dir, "") { let data_dir = "/tmp/engram" }
let data_dir_raw: String = env("ENGRAM_DATA_DIR")
let data_dir: String = if str_eq(data_dir_raw, "") { "/tmp/engram" } else { data_dir_raw }
let snapshot_path: String = data_dir + "/snapshot.json"
engram_load(snapshot_path)
// 2026-07-21 self-review boot guard: if the snapshot file has content but the
// load produced 0 nodes, something is wrong (corrupt file / parse failure).
// Preserve the evidence and warn loudly and since read routes no longer write
// the canonical path, a bad boot can no longer clobber the good snapshot.
let boot_snap: String = fs_read(snapshot_path)
if !str_eq(boot_snap, "") {
if engram_node_count() == 0 {
println("[engram] WARNING: snapshot.json is non-empty but load produced 0 nodes — preserving copy at snapshot.failed-load.json")
fs_write(data_dir + "/snapshot.failed-load.json", boot_snap)
} else {
// Good load: keep a boot-time backup of the snapshot as loaded.
fs_write(data_dir + "/snapshot.boot-backup.json", boot_snap)
}
}
println("[engram] runtime-native graph engine")
println("[engram] data_dir=" + data_dir)
println("[engram] node_count=" + int_to_str(engram_node_count()))
+224 -96
View File
@@ -710,6 +710,7 @@ static void jb_init(JsonBuf* b);
static void jb_putc(JsonBuf* b, char c);
static void jb_puts(JsonBuf* b, const char* s);
static void jb_emit_escaped(JsonBuf* b, const char* s);
static char* jb_finish(JsonBuf* b);
static int looks_like_string(el_val_t v);
static const char* json_find_key(const char* s, const char* key);
static const char* json_skip_value(const char* p);
@@ -2014,7 +2015,7 @@ el_val_t http_response(el_val_t status, el_val_t headers_json, el_val_t body) {
jb_puts(&out, ",\"body\":");
jb_emit_escaped(&out, b);
jb_putc(&out, '}');
return el_wrap_str(out.buf);
return el_wrap_str(jb_finish(&out));
}
/* ── Filesystem ──────────────────────────────────────────────────────────── */
@@ -2121,7 +2122,7 @@ el_val_t exec_capture(el_val_t cmdv) {
char buf[4096];
while (fgets(buf, sizeof(buf), f)) jb_puts(&b, buf);
pclose(f);
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
// exec — run a shell command via /bin/sh, capture stdout, return as String.
@@ -2145,7 +2146,7 @@ el_val_t exec(el_val_t cmdv) {
jb_puts(&b, buf);
}
pclose(f);
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
// exec_bg — run a shell command in background, return PID as String.
@@ -3292,6 +3293,23 @@ static void jb_puts(JsonBuf* b, const char* s) {
b->buf[b->len] = '\0';
}
/* jb_finish — call exactly once, at the point a JsonBuf's buffer is handed
* off as an el_val_t return value (via el_wrap_str). JsonBuf allocates through
* raw malloc/realloc (jb_init/jb_reserve), NOT el_strdup/el_strbuf, so unlike
* those helpers it was never registered with the per-request arena every
* JsonBuf-built JSON string leaked unconditionally, whether or not a request
* arena was active, because nothing ever freed it. el_arena_track() is a
* documented no-op when no arena is active, so this is safe to call from any
* context (HTTP request handler, CLI arena-scope, or no arena at all the
* last case still leaks, same as before, until the caller's context is itself
* arena-scoped). Do NOT call this on a JsonBuf whose ->buf is consumed by
* another builder (jb_puts(&other, x.buf)) and freed manually only on the
* one actually returned via el_wrap_str. */
static char* jb_finish(JsonBuf* b) {
el_arena_track(b->buf);
return b->buf;
}
static void jb_emit_escaped(JsonBuf* b, const char* s) {
jb_putc(b, '"');
const unsigned char* p = (const unsigned char*)s;
@@ -3400,7 +3418,7 @@ static void jb_emit_value(JsonBuf* b, el_val_t v) {
el_val_t json_stringify(el_val_t v) {
JsonBuf b; jb_init(&b);
jb_emit_value(&b, v);
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
/* ── JSON substring accessors ────────────────────────────────────────────── */
@@ -3587,7 +3605,7 @@ el_val_t json_set(el_val_t json_str, el_val_t key, el_val_t value) {
jb_putc(&b, ':');
jb_puts(&b, raw_val);
jb_putc(&b, '}');
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
const char* existing = json_find_key(json, k);
JsonBuf b; jb_init(&b);
@@ -3601,7 +3619,7 @@ el_val_t json_set(el_val_t json_str, el_val_t key, el_val_t value) {
b.buf[b.len] = '\0';
jb_puts(&b, raw_val);
jb_puts(&b, end);
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
/* Insert before closing '}'. Find last '}' */
size_t jl = strlen(json);
@@ -3633,7 +3651,7 @@ el_val_t json_set(el_val_t json_str, el_val_t key, el_val_t value) {
jb_puts(&b, raw_val);
/* Append from close_idx onward */
jb_puts(&b, json + close_idx);
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
el_val_t json_array_len(el_val_t json_str) {
@@ -3771,7 +3789,7 @@ el_val_t json_build_object(el_val_t kvs) {
jb_putc(&b, '"');
}
jb_putc(&b, '}');
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
/* json_build_array — build a JSON array from a list of raw JSON values.
@@ -3789,7 +3807,7 @@ el_val_t json_build_array(el_val_t items) {
jb_puts(&b, vs);
}
jb_putc(&b, ']');
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
/* ── Time ────────────────────────────────────────────────────────────────── */
@@ -5193,7 +5211,7 @@ el_val_t state_keys(void) {
}
jb_putc(&b, ']');
pthread_mutex_unlock(&_state_mu);
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
/* Returns 1 (true) if the key is present in the state store, else 0 (false). */
@@ -5399,7 +5417,7 @@ el_val_t str_format(el_val_t fmt, el_val_t data) {
jb_putc(&b, *p);
p++;
}
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
el_val_t str_lower(el_val_t s) { return str_to_lower(s); }
@@ -5815,7 +5833,7 @@ el_val_t list_join(el_val_t listv, el_val_t sep) {
jb_puts(&b, tmp);
}
}
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
el_val_t list_range(el_val_t start, el_val_t end) {
@@ -6465,12 +6483,12 @@ el_val_t engram_node(el_val_t content, el_val_t node_type, el_val_t salience) {
n->id = engram_new_id();
const char* c = EL_CSTR(content);
const char* nt = EL_CSTR(node_type);
n->content = el_strdup(c ? c : "");
n->node_type = el_strdup(nt && *nt ? nt : "Memory");
n->label = engram_first_n_chars(c, 60);
n->tier = el_strdup("Working");
n->tags = el_strdup("");
n->metadata = el_strdup("{}");
n->content = el_strdup_persist(c ? c : "");
n->node_type = el_strdup_persist(nt && *nt ? nt : "Memory");
{ char* _lb = engram_first_n_chars(c, 60); n->label = el_strdup_persist(_lb); } /* persist: stored field must outlive request arena */
n->tier = el_strdup_persist("Working");
n->tags = el_strdup_persist("");
n->metadata = el_strdup_persist("{}");
n->salience = engram_decode_score(salience);
if (n->salience <= 0.0 || n->salience > 1.0) n->salience = 0.5;
n->importance = 0.5;
@@ -6597,11 +6615,11 @@ el_val_t engram_node_layered(el_val_t content, el_val_t node_type, el_val_t labe
const char* lb = EL_CSTR(label);
const char* tg = EL_CSTR(tags);
const char* st = EL_CSTR(status);
n->content = el_strdup(c ? c : "");
n->node_type = el_strdup(nt && *nt ? nt : "Memory");
n->label = el_strdup(lb && *lb ? lb : (c ? engram_first_n_chars(c, 60) : ""));
n->tier = el_strdup("Working");
n->tags = el_strdup(tg ? tg : "");
n->content = el_strdup_persist(c ? c : "");
n->node_type = el_strdup_persist(nt && *nt ? nt : "Memory");
n->label = el_strdup_persist(lb && *lb ? lb : (c ? engram_first_n_chars(c, 60) : ""));
n->tier = el_strdup_persist("Working");
n->tags = el_strdup_persist(tg ? tg : "");
if (st && *st) {
/* Minimal metadata payload: {"status":"..."}. Keep it cheap so
* callers using `status` don't pay JSON parse cost on every read. */
@@ -6610,7 +6628,7 @@ el_val_t engram_node_layered(el_val_t content, el_val_t node_type, el_val_t labe
snprintf(meta, sl, "{\"status\":\"%s\"}", st);
n->metadata = meta;
} else {
n->metadata = el_strdup("{}");
n->metadata = el_strdup_persist("{}");
}
n->salience = engram_decode_score(salience);
n->importance = engram_decode_score(certainty);
@@ -6808,6 +6826,75 @@ static int istr_contains(const char* hay, const char* needle) {
return 0;
}
/* ── Tokenized query matching ───────────────────────────────────────────
* The engram query surface (search / activate / goal-bias) historically
* matched the ENTIRE raw query string as a single case-insensitive
* substring via istr_contains(field, q). That is Ctrl-F, not search:
* a multi-word query like "windows msi signing" only matched a node whose
* text contained that exact contiguous run, so real multi-word queries
* returned zero. istr_contains stays as the per-TOKEN primitive; these
* helpers split the query on whitespace and match ANY token, then rank by
* how many DISTINCT tokens a node covers. Single-token queries are a strict
* special case (score is 0 or 1) so single-word callers never regress. */
#define ENGRAM_MAX_QTOKENS 32
#define ENGRAM_QTOK_LEN 256
/* Split q on whitespace into up to ENGRAM_MAX_QTOKENS distinct
* (case-insensitive) tokens. Returns the token count. Over-long tokens are
* truncated to ENGRAM_QTOK_LEN-1; over-count tokens are ignored. */
static int engram_tokenize_query(const char* q,
char toks[][ENGRAM_QTOK_LEN], int maxtok) {
int n = 0;
if (!q) return 0;
const char* p = q;
while (*p && n < maxtok) {
while (*p && isspace((unsigned char)*p)) p++;
if (!*p) break;
char buf[ENGRAM_QTOK_LEN];
size_t tl = 0;
while (*p && !isspace((unsigned char)*p)) {
if (tl < sizeof(buf) - 1) buf[tl++] = *p;
p++;
}
buf[tl] = '\0';
if (tl == 0) continue;
int dup = 0;
for (int s = 0; s < n; s++) {
if (strcasecmp(toks[s], buf) == 0) { dup = 1; break; }
}
if (dup) continue;
memcpy(toks[n], buf, tl + 1);
n++;
}
return n;
}
/* Count how many of the ntok distinct query tokens appear (case-insensitive)
* in the node's content, label, or tags. 0 == no match. */
static int engram_node_match_score(const EngramNode* n,
char toks[][ENGRAM_QTOK_LEN], int ntok) {
int score = 0;
for (int t = 0; t < ntok; t++) {
if (istr_contains(n->content, toks[t]) ||
istr_contains(n->label, toks[t]) ||
istr_contains(n->tags, toks[t]))
score++;
}
return score;
}
/* Rank entry: distinct-token match count (primary, desc) then salience
* (tiebreak, desc). */
typedef struct { int64_t idx; int score; double salience; } EngramRankEntry;
static int engram_rank_cmp(const void* a, const void* b) {
const EngramRankEntry* ea = (const EngramRankEntry*)a;
const EngramRankEntry* eb = (const EngramRankEntry*)b;
if (ea->score != eb->score) return eb->score - ea->score; /* desc */
if (ea->salience < eb->salience) return 1;
if (ea->salience > eb->salience) return -1;
return 0;
}
el_val_t engram_search(el_val_t query, el_val_t limit) {
EngramStore* g = engram_get();
const char* q = EL_CSTR(query);
@@ -6815,21 +6902,34 @@ el_val_t engram_search(el_val_t query, el_val_t limit) {
if (lim <= 0) lim = 100;
el_val_t lst = el_list_empty();
if (!q || !*q) return lst;
int64_t found = 0;
for (int64_t i = 0; i < g->node_count && found < lim; i++) {
char toks[ENGRAM_MAX_QTOKENS][ENGRAM_QTOK_LEN];
int ntok = engram_tokenize_query(q, toks, ENGRAM_MAX_QTOKENS);
if (ntok == 0) return lst;
EngramRankEntry* hits = malloc((size_t)g->node_count * sizeof(EngramRankEntry));
if (!hits) return lst;
int64_t nhits = 0;
for (int64_t i = 0; i < g->node_count; i++) {
EngramNode* n = &g->nodes[i];
/* Filter transparent layers: nodes whose layer is `transparent=1`
* shape output but are invisible to introspection ("what do you
* know about yourself"). They still surface via engram_activate
* + engram_compile_layered_json that's the legitimate path. */
if (engram_layer_is_transparent(n->layer_id)) continue;
if (istr_contains(n->content, q) ||
istr_contains(n->label, q) ||
istr_contains(n->tags, q)) {
lst = el_list_append(lst, engram_node_to_map(n));
found++;
int sc = engram_node_match_score(n, toks, ntok);
if (sc > 0) {
hits[nhits].idx = i;
hits[nhits].score = sc;
hits[nhits].salience = n->salience;
nhits++;
}
}
/* Rank by distinct tokens matched (desc) then salience (desc), then cap. */
qsort(hits, (size_t)nhits, sizeof(EngramRankEntry), engram_rank_cmp);
int64_t end = nhits < lim ? nhits : lim;
for (int64_t k = 0; k < end; k++) {
lst = el_list_append(lst, engram_node_to_map(&g->nodes[hits[k].idx]));
}
free(hits);
return lst;
}
@@ -6883,10 +6983,10 @@ void engram_connect(el_val_t from_id, el_val_t to_id, el_val_t weight, el_val_t
EngramEdge* e = &g->edges[g->edge_count];
memset(e, 0, sizeof(*e));
e->id = engram_new_id();
e->from_id = el_strdup(f);
e->to_id = el_strdup(t);
e->relation = el_strdup(r && *r ? r : "associate");
e->metadata = el_strdup("{}");
e->from_id = el_strdup_persist(f);
e->to_id = el_strdup_persist(t);
e->relation = el_strdup_persist(r && *r ? r : "associate");
e->metadata = el_strdup_persist("{}");
e->weight = engram_decode_score(weight);
if (e->weight <= 0.0 || e->weight > 1.0) e->weight = 0.5;
e->confidence = 1.0;
@@ -7106,10 +7206,14 @@ static double engram_temporal_proximity_bonus(int64_t node_created,
static double engram_goal_bias(const EngramNode* n, const char* query) {
if (!query || !*query) return 1.0;
double bias = 1.0;
/* Direct lexical overlap: node content/label/tags share text with query. */
if (istr_contains(n->content, query) || istr_contains(n->label, query) ||
istr_contains(n->tags, query)) {
bias += 0.5;
/* Direct lexical overlap, graded by token coverage: a node covering all
* query tokens gets the full +0.5; partial coverage gets a proportional
* share. Single-token queries full +0.5 on match, identical to before. */
{
char toks[ENGRAM_MAX_QTOKENS][ENGRAM_QTOK_LEN];
int ntok = engram_tokenize_query(query, toks, ENGRAM_MAX_QTOKENS);
int sc = engram_node_match_score(n, toks, ntok);
if (sc > 0 && ntok > 0) bias += 0.5 * ((double)sc / (double)ntok);
}
/* Node-type resonance with query intent. */
int technical_query = istr_contains(query, "code") ||
@@ -7175,14 +7279,21 @@ el_val_t engram_activate(el_val_t query, el_val_t depth) {
if (!seeds) {
free(best_bg); free(best_hops); free(reached); return out;
}
/* Tokenize once: a node seeds if it matches ANY query token, and its seed
* activation is scaled by token coverage (fraction of distinct query
* tokens it contains) so a node matching all words seeds more strongly
* than one matching a single word. Single-word queries coverage 1.0,
* identical to the prior whole-query behavior. */
char toks[ENGRAM_MAX_QTOKENS][ENGRAM_QTOK_LEN];
int ntok = engram_tokenize_query(q, toks, ENGRAM_MAX_QTOKENS);
for (int64_t i = 0; i < g->node_count; i++) {
EngramNode* n = &g->nodes[i];
if (istr_contains(n->content, q) ||
istr_contains(n->label, q) ||
istr_contains(n->tags, q)) {
int sc = engram_node_match_score(n, toks, ntok);
if (sc > 0) {
double tdecay = engram_temporal_decay(n, now_ms);
double dampen = engram_activation_dampen(n);
double act = n->salience * tdecay * dampen;
double cover = ntok > 0 ? (double)sc / (double)ntok : 1.0;
double act = n->salience * tdecay * dampen * cover;
seeds[seed_count].idx = i;
seeds[seed_count].act = act;
seeds[seed_count].created_at = n->created_at;
@@ -7519,11 +7630,11 @@ el_val_t engram_save(el_val_t path) {
/* Helper: extract a string field from a JSON object substring. */
static char* eg_get_str_field(const char* obj, const char* key) {
const char* p = json_find_key(obj, key);
if (!p) return el_strdup("");
if (*p != '"') return el_strdup("");
if (!p) return el_strdup_persist("");
if (*p != '"') return el_strdup_persist("");
JsonParser jp = { .p = p, .end = p + strlen(p), .err = 0 };
char* out = jp_parse_string_raw(&jp);
if (jp.err) { free(out); return el_strdup(""); }
if (jp.err) { free(out); return el_strdup_persist(""); }
return out;
}
@@ -7597,7 +7708,7 @@ el_val_t engram_load(el_val_t path) {
nn->tier = eg_get_str_field(obj, "tier");
nn->tags = eg_get_str_field(obj, "tags");
nn->metadata = eg_get_str_field(obj, "metadata");
if (!nn->metadata || !*nn->metadata) { free(nn->metadata); nn->metadata = el_strdup("{}"); }
if (!nn->metadata || !*nn->metadata) { free(nn->metadata); nn->metadata = el_strdup_persist("{}"); }
nn->salience = eg_get_num_field(obj, "salience");
nn->importance = eg_get_num_field(obj, "importance");
nn->confidence = eg_get_num_field(obj, "confidence");
@@ -7648,7 +7759,7 @@ el_val_t engram_load(el_val_t path) {
ee->to_id = eg_get_str_field(obj, "to_id");
ee->relation = eg_get_str_field(obj, "relation");
ee->metadata = eg_get_str_field(obj, "metadata");
if (!ee->metadata || !*ee->metadata) { free(ee->metadata); ee->metadata = el_strdup("{}"); }
if (!ee->metadata || !*ee->metadata) { free(ee->metadata); ee->metadata = el_strdup_persist("{}"); }
ee->weight = eg_get_num_field(obj, "weight");
ee->confidence = eg_get_num_field(obj, "confidence");
ee->created_at = eg_get_int_field(obj, "created_at");
@@ -7740,7 +7851,7 @@ el_val_t engram_get_node_json(el_val_t id) {
if (!n) return el_wrap_str(el_strdup("{}"));
JsonBuf b; jb_init(&b);
engram_emit_node_json(&b, n);
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
el_val_t engram_search_json(el_val_t query, el_val_t limit) {
@@ -7751,24 +7862,41 @@ el_val_t engram_search_json(el_val_t query, el_val_t limit) {
JsonBuf b; jb_init(&b);
jb_putc(&b, '[');
int first = 1;
int64_t found = 0;
if (q && *q) {
for (int64_t i = 0; i < g->node_count && found < lim; i++) {
EngramNode* n = &g->nodes[i];
/* Filter transparent layers — same as engram_search. */
if (engram_layer_is_transparent(n->layer_id)) continue;
if (istr_contains(n->content, q) ||
istr_contains(n->label, q) ||
istr_contains(n->tags, q)) {
if (!first) jb_putc(&b, ',');
engram_emit_node_json(&b, n);
first = 0;
found++;
char toks[ENGRAM_MAX_QTOKENS][ENGRAM_QTOK_LEN];
int ntok = engram_tokenize_query(q, toks, ENGRAM_MAX_QTOKENS);
if (ntok > 0) {
EngramRankEntry* hits =
malloc((size_t)g->node_count * sizeof(EngramRankEntry));
if (hits) {
int64_t nhits = 0;
for (int64_t i = 0; i < g->node_count; i++) {
EngramNode* n = &g->nodes[i];
/* Filter transparent layers — same as engram_search. */
if (engram_layer_is_transparent(n->layer_id)) continue;
int sc = engram_node_match_score(n, toks, ntok);
if (sc > 0) {
hits[nhits].idx = i;
hits[nhits].score = sc;
hits[nhits].salience = n->salience;
nhits++;
}
}
/* Rank by distinct tokens matched (desc) then salience (desc). */
qsort(hits, (size_t)nhits, sizeof(EngramRankEntry),
engram_rank_cmp);
int64_t end = nhits < lim ? nhits : lim;
for (int64_t k = 0; k < end; k++) {
if (!first) jb_putc(&b, ',');
engram_emit_node_json(&b, &g->nodes[hits[k].idx]);
first = 0;
}
free(hits);
}
}
}
jb_putc(&b, ']');
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
el_val_t engram_scan_nodes_json(el_val_t limit, el_val_t offset) {
@@ -7777,9 +7905,9 @@ el_val_t engram_scan_nodes_json(el_val_t limit, el_val_t offset) {
int64_t off = (int64_t)offset; if (off < 0) off = 0;
JsonBuf b; jb_init(&b);
jb_putc(&b, '[');
if (g->node_count == 0) { jb_putc(&b, ']'); return el_wrap_str(b.buf); }
if (g->node_count == 0) { jb_putc(&b, ']'); return el_wrap_str(jb_finish(&b)); }
int64_t* idx = malloc((size_t)g->node_count * sizeof(int64_t));
if (!idx) { jb_putc(&b, ']'); return el_wrap_str(b.buf); }
if (!idx) { jb_putc(&b, ']'); return el_wrap_str(jb_finish(&b)); }
/* Skip transparent layers — introspection filter, same as engram_scan_nodes. */
int64_t live = 0;
for (int64_t i = 0; i < g->node_count; i++) {
@@ -7797,7 +7925,7 @@ el_val_t engram_scan_nodes_json(el_val_t limit, el_val_t offset) {
}
free(idx);
jb_putc(&b, ']');
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
/* engram_scan_nodes_by_type_json — filter by node_type before paginating.
@@ -7813,9 +7941,9 @@ el_val_t engram_scan_nodes_by_type_json(el_val_t type_v, el_val_t limit, el_val_
int64_t off = (int64_t)offset; if (off < 0) off = 0;
JsonBuf b; jb_init(&b);
jb_putc(&b, '[');
if (g->node_count == 0) { jb_putc(&b, ']'); return el_wrap_str(b.buf); }
if (g->node_count == 0) { jb_putc(&b, ']'); return el_wrap_str(jb_finish(&b)); }
int64_t* idx = malloc((size_t)g->node_count * sizeof(int64_t));
if (!idx) { jb_putc(&b, ']'); return el_wrap_str(b.buf); }
if (!idx) { jb_putc(&b, ']'); return el_wrap_str(jb_finish(&b)); }
int64_t live = 0;
for (int64_t i = 0; i < g->node_count; i++) {
if (engram_layer_is_transparent(g->nodes[i].layer_id)) continue;
@@ -7834,7 +7962,7 @@ el_val_t engram_scan_nodes_by_type_json(el_val_t type_v, el_val_t limit, el_val_
}
free(idx);
jb_putc(&b, ']');
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
el_val_t engram_neighbors_json(el_val_t node_id, el_val_t max_depth, el_val_t direction) {
@@ -7849,7 +7977,7 @@ el_val_t engram_neighbors_json(el_val_t node_id, el_val_t max_depth, el_val_t di
int allow_in = (strcmp(dir, "in") == 0) || (strcmp(dir, "both") == 0);
JsonBuf b; jb_init(&b);
jb_putc(&b, '[');
if (!sid || !*sid) { jb_putc(&b, ']'); return el_wrap_str(b.buf); }
if (!sid || !*sid) { jb_putc(&b, ']'); return el_wrap_str(jb_finish(&b)); }
/* Frontier of (node_id, hops). Cap to a sane size. */
char** frontier = calloc(1024, sizeof(char*));
@@ -7859,7 +7987,7 @@ el_val_t engram_neighbors_json(el_val_t node_id, el_val_t max_depth, el_val_t di
int64_t vc = 0;
if (!frontier || !frontier_h || !visited) {
free(frontier); free(frontier_h); free(visited);
jb_putc(&b, ']'); return el_wrap_str(b.buf);
jb_putc(&b, ']'); return el_wrap_str(jb_finish(&b));
}
/* Use plain strdup (not el_strdup) so arena doesn't track these pointers.
* The BFS loop manually frees them below arena would double-free them. */
@@ -7902,7 +8030,7 @@ el_val_t engram_neighbors_json(el_val_t node_id, el_val_t max_depth, el_val_t di
for (int64_t i = 0; i < vc; i++) free(visited[i]);
free(frontier); free(frontier_h); free(visited);
jb_putc(&b, ']');
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
el_val_t engram_activate_json(el_val_t query, el_val_t depth) {
@@ -7943,7 +8071,7 @@ el_val_t engram_activate_json(el_val_t query, el_val_t depth) {
}
}
jb_putc(&b, ']');
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
el_val_t engram_stats_json(void) {
@@ -7963,7 +8091,7 @@ el_val_t engram_list_layers_json(void) {
jb_putc(&b, '[');
/* Build a sorted index over live layers. */
size_t* idx = malloc((g->layer_count + 1) * sizeof(size_t));
if (!idx) { jb_putc(&b, ']'); return el_wrap_str(b.buf); }
if (!idx) { jb_putc(&b, ']'); return el_wrap_str(jb_finish(&b)); }
size_t live = 0;
for (size_t i = 0; i < g->layer_count; i++) {
if (g->layers[i].name) idx[live++] = i;
@@ -8000,7 +8128,7 @@ el_val_t engram_list_layers_json(void) {
}
free(idx);
jb_putc(&b, ']');
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
/* engram_compile_layered_json — produce a prompt-ready context block split
@@ -8086,7 +8214,7 @@ el_val_t engram_compile_layered_json(el_val_t intent, el_val_t depth) {
free(b.buf);
return el_wrap_str(el_strdup(""));
}
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
/* engram_query_range — temporal range query.
@@ -8131,7 +8259,7 @@ el_val_t engram_query_range(el_val_t start_ms_v, el_val_t end_ms_v) {
}
jb_putc(&b, ']');
free(idx);
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
/* engram_load_merge — like engram_load but WITHOUT resetting the store.
@@ -8382,7 +8510,7 @@ el_val_t engram_wm_top_json(el_val_t n_v) {
}
free(idx);
jb_putc(&b, ']');
return el_wrap_str(b.buf);
return el_wrap_str(jb_finish(&b));
}
#ifdef HAVE_CURL
@@ -8778,13 +8906,13 @@ static int64_t dharma_find_or_create_relation_edge(const char* peer_base, int cr
engram_grow_nodes();
EngramNode* n = &g->nodes[g->node_count];
memset(n, 0, sizeof(*n));
n->id = el_strdup(self_id);
n->content = el_strdup(_el_cgi_dharma_id ? _el_cgi_dharma_id : "(self)");
n->node_type = el_strdup("DharmaSelf");
n->label = el_strdup("dharma:self");
n->tier = el_strdup("Working");
n->tags = el_strdup("dharma");
n->metadata = el_strdup("{}");
n->id = el_strdup_persist(self_id);
n->content = el_strdup_persist(_el_cgi_dharma_id ? _el_cgi_dharma_id : "(self)");
n->node_type = el_strdup_persist("DharmaSelf");
n->label = el_strdup_persist("dharma:self");
n->tier = el_strdup_persist("Working");
n->tags = el_strdup_persist("dharma");
n->metadata = el_strdup_persist("{}");
n->salience = 1.0; n->importance = 1.0; n->confidence = 1.0;
int64_t now = engram_now_ms();
n->created_at = now; n->updated_at = now; n->last_activated = now;
@@ -8795,13 +8923,13 @@ static int64_t dharma_find_or_create_relation_edge(const char* peer_base, int cr
engram_grow_nodes();
EngramNode* n = &g->nodes[g->node_count];
memset(n, 0, sizeof(*n));
n->id = el_strdup(peer_node);
n->content = el_strdup(peer_base);
n->node_type = el_strdup("DharmaPeer");
n->label = el_strdup(peer_node);
n->tier = el_strdup("Working");
n->tags = el_strdup("dharma");
n->metadata = el_strdup("{}");
n->id = el_strdup_persist(peer_node);
n->content = el_strdup_persist(peer_base);
n->node_type = el_strdup_persist("DharmaPeer");
n->label = el_strdup_persist(peer_node);
n->tier = el_strdup_persist("Working");
n->tags = el_strdup_persist("dharma");
n->metadata = el_strdup_persist("{}");
n->salience = 0.5; n->importance = 0.5; n->confidence = 1.0;
int64_t now = engram_now_ms();
n->created_at = now; n->updated_at = now; n->last_activated = now;
@@ -8813,10 +8941,10 @@ static int64_t dharma_find_or_create_relation_edge(const char* peer_base, int cr
EngramEdge* e = &g->edges[g->edge_count];
memset(e, 0, sizeof(*e));
e->id = engram_new_id();
e->from_id = el_strdup(self_id);
e->to_id = el_strdup(peer_node);
e->relation = el_strdup("dharma-relation");
e->metadata = el_strdup("{}");
e->from_id = el_strdup_persist(self_id);
e->to_id = el_strdup_persist(peer_node);
e->relation = el_strdup_persist("dharma-relation");
e->metadata = el_strdup_persist("{}");
e->weight = 0.0;
e->confidence = 1.0;
int64_t now = engram_now_ms();
@@ -9109,7 +9237,7 @@ static el_val_t llm_extract_text(el_val_t resp_val) {
free(obj);
p = end;
}
return el_wrap_str(out.buf);
return el_wrap_str(jb_finish(&out));
}
el_val_t llm_call(el_val_t model, el_val_t prompt) {
+25 -1
View File
@@ -23,10 +23,29 @@ fn tok_at(tokens: [Any], pos: Int) -> Map<String, Any> {
}
fn tok_kind(tokens: [Any], pos: Int) -> String {
// Out-of-range reads must report the Eof sentinel so every `== "Eof"`
// termination guard in the parser fires. Without this, reading past the
// single trailing Eof token returns runtime null (el_list_get OOB -> 0),
// which matches no delimiter, letting inner parse loops append AST nodes
// forever on malformed input -> unbounded allocation -> OOM.
let n: Int = native_list_len(tokens) / 2
if pos < 0 {
return "Eof"
}
if pos >= n {
return "Eof"
}
native_list_get(tokens, pos * 2)
}
fn tok_value(tokens: [Any], pos: Int) -> String {
let n: Int = native_list_len(tokens) / 2
if pos < 0 {
return ""
}
if pos >= n {
return ""
}
native_list_get(tokens, pos * 2 + 1)
}
@@ -35,7 +54,12 @@ fn expect(tokens: [Any], pos: Int, kind: String) -> Int {
if k == kind {
return pos + 1
}
// On mismatch just advance; error recovery is best-effort
// On mismatch, error recovery is best-effort. But never step PAST the Eof
// sentinel: once at Eof a mismatch means the input ended early, and
// advancing would run the cursor off the token list.
if k == "Eof" {
return pos
}
pos + 1
}
File diff suppressed because it is too large Load Diff