neuron-technologies/el
3
0
Fork 0
Code Issues 4 Pull Requests 4 Actions Packages Projects Releases 5 Wiki Activity

self-review 2026-08-02: bound the WM breakthrough storm; stop punishing semantic relevance for recency

Browse Source 
Working memory was thrashing behind a healthy-looking gauge. wm_active sat
at 22-24 while breakthroughs ran 661-903 and evictions 485-717 PER 60s tick
- roughly 825-1125 nodes cycling in 5-call lockstep.

Root cause: the breakthrough path was an anti-starvation mechanism that reset
its own counter on firing, with no budget and no refractory. A node failing
its type threshold 5 times was force-promoted at exactly 0.10 and had its
suppression_count reset to 0, so it immediately restarted the identical
climb. Since BREAKTHROUGH_WEIGHT (0.10) > WM_FLOOR (0.05), every one of them
cleared the admission floor and entered the rank contest tied at 0.10, where
the tie-break degenerated to node-array index order. Cap-evicted nodes are
skipped by retrieval reinforcement, so they never got an access_ts record and
the STI inhibition-of-return damper never applied to them. That closed the
loop: re-suppressed, completely unmarked, forever.

An anti-starvation rule that resets its own counter without a bound is not a
fairness valve, it is an oscillator.

Fixes in engram_activate Pass 2:
- ENGRAM_BREAKTHROUGH_BUDGET (WM_CAP/4 = 6) caps intrusive thoughts per call.
- ENGRAM_BREAKTHROUGH_COOLDOWN (55) via NEGATIVE suppression_count. The field
  already serializes as %d and parses through eg_get_int_field, so negatives
  round-trip through snapshots with no struct or format change.
- Blocked breakthroughs no longer reset the counter; it saturates so a starved
  node surfaces on a later call instead of restarting from zero.
- Graded breakthrough weight by nearness to own threshold, so the rank
  tie-break is cognitive rather than insertion order. Invariant preserved:
  WM_FLOOR < weight < min(type_threshold).

Also: moved the additive cosine term AFTER the STI multiplier. It was applied
before, so an incumbent re-reached 30s later took t_n/(t_n+120) = 0.2x, which
cut the semantic term's ceiling from 0.20 to 0.04 - below every per-type
threshold. Meaning-match was being punished for having been recently useful.
Inhibition-of-return should rotate the structural score, not the semantic one.

Also: _eg_act_wm_evicted counted 3 of 5 eviction paths. The two carry-over
paths were silent, so the reported rate was an undercount of unknown
magnitude - while being used to diagnose an eviction pathology. All five now
increment.

Also: route_sync returned {"nodes":[],"edges":[]} when the snapshot export
failed. The soul's sync_ok check only tests for "" and "{}", so that
placeholder passed as a healthy sync: last_sync_ok_ts stamped, sync_age_ms
green, sync_empty never fired, added:0 forever. A broken sync was
indistinguishable from a quiet healthy one - the exact class this route was
added to fix. Returns a real error now.

Verified live (boot 20 vs boot 19): breakthroughs 661-903 -> 36/tick,
evictions 485-717 -> 12-46/tick against a counter that now covers more paths,
wm_active unchanged at 22-24, wm_avg_weight 0.138-0.273 -> 0.186-0.446.
Working memory is holding strong nodes instead of breakthrough-floor filler.
This commit is contained in:
Will Anderson
2026-08-02 08:48:59 -05:00
parent 7f03876e26
commit 005e84e5d3
4 changed files with 164 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files
engram/dist/engram
Vendored
BIN
View File
Binary file not shown.
engram/dist/engram.c
Vendored
+52 -32
View File
@@ -29,6 +29,7 @@ el_val_t route_sync(el_val_t method, el_val_t path, el_val_t body);
el_val_t route_load_merge(el_val_t method, el_val_t path, el_val_t body);
el_val_t route_emit_ise(el_val_t method, el_val_t path, el_val_t body);
el_val_t route_capture_knowledge(el_val_t method, el_val_t path, el_val_t body);
el_val_t route_similarity(el_val_t method, el_val_t path, el_val_t body);
el_val_t check_auth_ok(el_val_t method, el_val_t body);
el_val_t handle_request(el_val_t method, el_val_t path, el_val_t body);
@@ -129,16 +130,18 @@ el_val_t route_create_node(el_val_t method, el_val_t path, el_val_t body) {
el_val_t content = json_get_string(body, EL_STR("content"));
el_val_t nt_raw = json_get_string(body, EL_STR("node_type"));
el_val_t node_type = ({ el_val_t _if_result_2 = 0; if (str_eq(nt_raw, EL_STR(""))) { _if_result_2 = (EL_STR("Memory")); } else { _if_result_2 = (nt_raw); } _if_result_2; });
el_val_t sal_raw = json_get_float(body, EL_STR("salience"));
el_val_t salience = ({ el_val_t _if_result_3 = 0; if ((sal_raw == el_from_float(0.0))) { _if_result_3 = (el_from_float(0.5)); } else { _if_result_3 = (sal_raw); } _if_result_3; });
el_val_t sal_present = json_get_raw(body, EL_STR("salience"));
el_val_t salience = ({ el_val_t _if_result_3 = 0; if (str_eq(sal_present, EL_STR(""))) { _if_result_3 = (el_from_float(0.5)); } else { _if_result_3 = (json_get_float(body, EL_STR("salience"))); } _if_result_3; });
el_val_t label_raw = json_get_string(body, EL_STR("label"));
el_val_t label = ({ el_val_t _if_result_4 = 0; if (str_eq(label_raw, EL_STR(""))) { _if_result_4 = (content); } else { _if_result_4 = (label_raw); } _if_result_4; });
el_val_t imp_raw = json_get_float(body, EL_STR("importance"));
el_val_t importance = ({ el_val_t _if_result_5 = 0; if ((imp_raw == el_from_float(0.0))) { _if_result_5 = (el_from_float(0.5)); } else { _if_result_5 = (imp_raw); } _if_result_5; });
el_val_t imp_present = json_get_raw(body, EL_STR("importance"));
el_val_t importance = ({ el_val_t _if_result_5 = 0; if (str_eq(imp_present, EL_STR(""))) { _if_result_5 = (el_from_float(0.5)); } else { _if_result_5 = (json_get_float(body, EL_STR("importance"))); } _if_result_5; });
el_val_t conf_present = json_get_raw(body, EL_STR("confidence"));
el_val_t confidence = ({ el_val_t _if_result_6 = 0; if (str_eq(conf_present, EL_STR(""))) { _if_result_6 = (el_from_float(1.0)); } else { _if_result_6 = (json_get_float(body, EL_STR("confidence"))); } _if_result_6; });
el_val_t tier_raw = json_get_string(body, EL_STR("tier"));
el_val_t tier = ({ el_val_t _if_result_6 = 0; if (str_eq(tier_raw, EL_STR(""))) { _if_result_6 = (EL_STR("Working")); } else { _if_result_6 = (tier_raw); } _if_result_6; });
el_val_t tier = ({ el_val_t _if_result_7 = 0; if (str_eq(tier_raw, EL_STR(""))) { _if_result_7 = (EL_STR("Working")); } else { _if_result_7 = (tier_raw); } _if_result_7; });
el_val_t tags = json_get_string(body, EL_STR("tags"));
el_val_t id = engram_node_full(content, node_type, label, el_from_float(salience), el_from_float(importance), el_from_float(1.0), tier, tags);
el_val_t id = engram_node_full(content, node_type, label, salience, importance, confidence, tier, tags);
el_val_t saved = persist_canonical();
return el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(EL_STR("{\"id\":\""), id), EL_STR("\",\"content\":\"")), content), EL_STR("\",\"node_type\":\"")), node_type), EL_STR("\"}"));
return 0;
@@ -166,7 +169,7 @@ el_val_t route_scan_nodes(el_val_t method, el_val_t path, el_val_t body) {
el_val_t route_scan_edges(el_val_t method, el_val_t path, el_val_t body) {
el_val_t dir_raw = env(EL_STR("ENGRAM_DATA_DIR"));
el_val_t dir = ({ el_val_t _if_result_7 = 0; if (str_eq(dir_raw, EL_STR(""))) { _if_result_7 = (EL_STR("/tmp/engram")); } else { _if_result_7 = (dir_raw); } _if_result_7; });
el_val_t dir = ({ el_val_t _if_result_8 = 0; if (str_eq(dir_raw, EL_STR(""))) { _if_result_8 = (EL_STR("/tmp/engram")); } else { _if_result_8 = (dir_raw); } _if_result_8; });
el_val_t snap_path = el_str_concat(dir, EL_STR("/.scan-export.json"));
engram_save(snap_path);
el_val_t snap = fs_read(snap_path);
@@ -182,22 +185,22 @@ el_val_t route_scan_edges(el_val_t method, el_val_t path, el_val_t body) {
}
el_val_t route_search(el_val_t method, el_val_t path, el_val_t body) {
el_val_t q = ({ el_val_t _if_result_8 = 0; if (str_eq(method, EL_STR("GET"))) { _if_result_8 = (query_param(path, EL_STR("q"))); } else { _if_result_8 = (json_get_string(body, EL_STR("query"))); } _if_result_8; });
el_val_t q = ({ el_val_t _if_result_9 = 0; if (str_eq(method, EL_STR("GET"))) { _if_result_9 = (query_param(path, EL_STR("q"))); } else { _if_result_9 = (json_get_string(body, EL_STR("query"))); } _if_result_9; });
el_val_t lim_url = query_int(path, EL_STR("limit"), 0);
el_val_t lim_body = json_get_int(body, EL_STR("limit"));
el_val_t lim_either = ({ el_val_t _if_result_9 = 0; if ((lim_url > 0)) { _if_result_9 = (lim_url); } else { _if_result_9 = (lim_body); } _if_result_9; });
el_val_t limit = ({ el_val_t _if_result_10 = 0; if ((lim_either > 0)) { _if_result_10 = (lim_either); } else { _if_result_10 = (20); } _if_result_10; });
el_val_t lim_either = ({ el_val_t _if_result_10 = 0; if ((lim_url > 0)) { _if_result_10 = (lim_url); } else { _if_result_10 = (lim_body); } _if_result_10; });
el_val_t limit = ({ el_val_t _if_result_11 = 0; if ((lim_either > 0)) { _if_result_11 = (lim_either); } else { _if_result_11 = (20); } _if_result_11; });
return engram_search_json(q, limit);
return 0;
}
el_val_t route_activate(el_val_t method, el_val_t path, el_val_t body) {
el_val_t q = ({ el_val_t _if_result_11 = 0; if (str_eq(method, EL_STR("GET"))) { _if_result_11 = (query_param(path, EL_STR("q"))); } else { _if_result_11 = (json_get_string(body, EL_STR("query"))); } _if_result_11; });
el_val_t q = ({ el_val_t _if_result_12 = 0; if (str_eq(method, EL_STR("GET"))) { _if_result_12 = (query_param(path, EL_STR("q"))); } else { _if_result_12 = (json_get_string(body, EL_STR("query"))); } _if_result_12; });
if (str_eq(q, EL_STR(""))) {
return err_json(EL_STR("missing query"));
}
el_val_t d_raw = ({ el_val_t _if_result_12 = 0; if (str_eq(method, EL_STR("GET"))) { _if_result_12 = (query_int(path, EL_STR("depth"), 3)); } else { _if_result_12 = (json_get_int(body, EL_STR("depth"))); } _if_result_12; });
el_val_t depth = ({ el_val_t _if_result_13 = 0; if ((d_raw > 0)) { _if_result_13 = (d_raw); } else { _if_result_13 = (3); } _if_result_13; });
el_val_t d_raw = ({ el_val_t _if_result_13 = 0; if (str_eq(method, EL_STR("GET"))) { _if_result_13 = (query_int(path, EL_STR("depth"), 3)); } else { _if_result_13 = (json_get_int(body, EL_STR("depth"))); } _if_result_13; });
el_val_t depth = ({ el_val_t _if_result_14 = 0; if ((d_raw > 0)) { _if_result_14 = (d_raw); } else { _if_result_14 = (3); } _if_result_14; });
return el_str_concat(el_str_concat(EL_STR("{\"results\":"), engram_activate_json(q, depth)), EL_STR("}"));
return 0;
}
@@ -206,9 +209,9 @@ el_val_t route_create_edge(el_val_t method, el_val_t path, el_val_t body) {
el_val_t from_id = json_get_string(body, EL_STR("from_id"));
el_val_t to_id = json_get_string(body, EL_STR("to_id"));
el_val_t rel_raw = json_get_string(body, EL_STR("relation"));
el_val_t relation = ({ el_val_t _if_result_14 = 0; if (str_eq(rel_raw, EL_STR(""))) { _if_result_14 = (EL_STR("associates")); } else { _if_result_14 = (rel_raw); } _if_result_14; });
el_val_t w_raw = json_get_float(body, EL_STR("weight"));
el_val_t weight = ({ el_val_t _if_result_15 = 0; if ((w_raw == el_from_float(0.0))) { _if_result_15 = (el_from_float(0.5)); } else { _if_result_15 = (w_raw); } _if_result_15; });
el_val_t relation = ({ el_val_t _if_result_15 = 0; if (str_eq(rel_raw, EL_STR(""))) { _if_result_15 = (EL_STR("associates")); } else { _if_result_15 = (rel_raw); } _if_result_15; });
el_val_t w_present = json_get_raw(body, EL_STR("weight"));
el_val_t weight = ({ el_val_t _if_result_16 = 0; if (str_eq(w_present, EL_STR(""))) { _if_result_16 = (el_from_float(0.5)); } else { _if_result_16 = (json_get_float(body, EL_STR("weight"))); } _if_result_16; });
engram_connect(from_id, to_id, weight, relation);
el_val_t saved = persist_canonical();
return el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(EL_STR("{\"ok\":true,\"from_id\":\""), from_id), EL_STR("\",\"to_id\":\"")), to_id), EL_STR("\",\"relation\":\"")), relation), EL_STR("\"}"));
@@ -250,8 +253,8 @@ el_val_t route_forget(el_val_t method, el_val_t path, el_val_t body) {
el_val_t route_save(el_val_t method, el_val_t path, el_val_t body) {
el_val_t p_raw = json_get_string(body, EL_STR("path"));
el_val_t dir_raw = env(EL_STR("ENGRAM_DATA_DIR"));
el_val_t dir = ({ el_val_t _if_result_16 = 0; if (str_eq(dir_raw, EL_STR(""))) { _if_result_16 = (EL_STR("/tmp/engram")); } else { _if_result_16 = (dir_raw); } _if_result_16; });
el_val_t p = ({ el_val_t _if_result_17 = 0; if (str_eq(p_raw, EL_STR(""))) { _if_result_17 = (el_str_concat(dir, EL_STR("/snapshot.json"))); } else { _if_result_17 = (p_raw); } _if_result_17; });
el_val_t dir = ({ el_val_t _if_result_17 = 0; if (str_eq(dir_raw, EL_STR(""))) { _if_result_17 = (EL_STR("/tmp/engram")); } else { _if_result_17 = (dir_raw); } _if_result_17; });
el_val_t p = ({ el_val_t _if_result_18 = 0; if (str_eq(p_raw, EL_STR(""))) { _if_result_18 = (el_str_concat(dir, EL_STR("/snapshot.json"))); } else { _if_result_18 = (p_raw); } _if_result_18; });
engram_save(p);
return el_str_concat(el_str_concat(EL_STR("{\"ok\":true,\"path\":\""), p), EL_STR("\"}"));
return 0;
@@ -260,15 +263,15 @@ el_val_t route_save(el_val_t method, el_val_t path, el_val_t body) {
el_val_t route_load(el_val_t method, el_val_t path, el_val_t body) {
el_val_t p_raw = json_get_string(body, EL_STR("path"));
el_val_t dir_raw = env(EL_STR("ENGRAM_DATA_DIR"));
el_val_t dir = ({ el_val_t _if_result_18 = 0; if (str_eq(dir_raw, EL_STR(""))) { _if_result_18 = (EL_STR("/tmp/engram")); } else { _if_result_18 = (dir_raw); } _if_result_18; });
el_val_t p = ({ el_val_t _if_result_19 = 0; if (str_eq(p_raw, EL_STR(""))) { _if_result_19 = (el_str_concat(dir, EL_STR("/snapshot.json"))); } else { _if_result_19 = (p_raw); } _if_result_19; });
el_val_t dir = ({ el_val_t _if_result_19 = 0; if (str_eq(dir_raw, EL_STR(""))) { _if_result_19 = (EL_STR("/tmp/engram")); } else { _if_result_19 = (dir_raw); } _if_result_19; });
el_val_t p = ({ el_val_t _if_result_20 = 0; if (str_eq(p_raw, EL_STR(""))) { _if_result_20 = (el_str_concat(dir, EL_STR("/snapshot.json"))); } else { _if_result_20 = (p_raw); } _if_result_20; });
engram_load(p);
return ok_json();
return 0;
}
el_val_t route_health(el_val_t method, el_val_t path, el_val_t body) {
return EL_STR("{\"status\":\"ok\",\"engine\":\"engram-runtime-native\"}");
return el_str_concat(el_str_concat(el_str_concat(el_str_concat(EL_STR("{\"status\":\"ok\",\"engine\":\"engram-runtime-native\",\"node_count\":"), int_to_str(engram_node_count())), EL_STR(",\"edge_count\":")), int_to_str(engram_edge_count())), EL_STR("}"));
return 0;
}
@@ -285,12 +288,12 @@ el_val_t route_embed_backfill(el_val_t method, el_val_t path, el_val_t body) {
el_val_t route_sync(el_val_t method, el_val_t path, el_val_t body) {
el_val_t dir_raw = env(EL_STR("ENGRAM_DATA_DIR"));
el_val_t dir = ({ el_val_t _if_result_20 = 0; if (str_eq(dir_raw, EL_STR(""))) { _if_result_20 = (EL_STR("/tmp/engram")); } else { _if_result_20 = (dir_raw); } _if_result_20; });
el_val_t dir = ({ el_val_t _if_result_21 = 0; if (str_eq(dir_raw, EL_STR(""))) { _if_result_21 = (EL_STR("/tmp/engram")); } else { _if_result_21 = (dir_raw); } _if_result_21; });
el_val_t snap_path = el_str_concat(dir, EL_STR("/.sync-export.json"));
engram_save(snap_path);
el_val_t snap = fs_read(snap_path);
if (str_eq(snap, EL_STR(""))) {
return EL_STR("{\"nodes\":[],\"edges\":[]}");
return err_json(EL_STR("sync export failed: snapshot unreadable"));
}
return snap;
return 0;
@@ -324,7 +327,7 @@ el_val_t route_emit_ise(el_val_t method, el_val_t path, el_val_t body) {
el_val_t conf = el_from_float(0.8);
el_val_t id = engram_node_full(content, EL_STR("InternalStateEvent"), EL_STR("state-event"), sal, imp, conf, EL_STR("Episodic"), EL_STR("[\"internal-state\",\"InternalStateEvent\"]"));
el_val_t ret_raw = env(EL_STR("ENGRAM_ISE_RETENTION_MS"));
el_val_t ret_ms = ({ el_val_t _if_result_21 = 0; if (str_eq(ret_raw, EL_STR(""))) { _if_result_21 = (172800000); } else { _if_result_21 = (str_to_int(ret_raw)); } _if_result_21; });
el_val_t ret_ms = ({ el_val_t _if_result_22 = 0; if (str_eq(ret_raw, EL_STR(""))) { _if_result_22 = (172800000); } else { _if_result_22 = (str_to_int(ret_raw)); } _if_result_22; });
el_val_t pruned = engram_prune_telemetry(ret_ms);
return el_str_concat(el_str_concat(el_str_concat(el_str_concat(EL_STR("{\"ok\":true,\"id\":\""), id), EL_STR("\",\"pruned\":")), int_to_str(pruned)), EL_STR("}"));
return 0;
@@ -336,21 +339,21 @@ el_val_t route_capture_knowledge(el_val_t method, el_val_t path, el_val_t body)
return err_json(EL_STR("missing content"));
}
el_val_t title = json_get_string(body, EL_STR("title"));
el_val_t label = ({ el_val_t _if_result_22 = 0; if (str_eq(title, EL_STR(""))) { _if_result_22 = (str_slice(content, 0, 60)); } else { _if_result_22 = (title); } _if_result_22; });
el_val_t label = ({ el_val_t _if_result_23 = 0; if (str_eq(title, EL_STR(""))) { _if_result_23 = (str_slice(content, 0, 60)); } else { _if_result_23 = (title); } _if_result_23; });
el_val_t category_raw = json_get_string(body, EL_STR("category"));
el_val_t category = ({ el_val_t _if_result_23 = 0; if (str_eq(category_raw, EL_STR(""))) { _if_result_23 = (EL_STR("other")); } else { _if_result_23 = (category_raw); } _if_result_23; });
el_val_t category = ({ el_val_t _if_result_24 = 0; if (str_eq(category_raw, EL_STR(""))) { _if_result_24 = (EL_STR("other")); } else { _if_result_24 = (category_raw); } _if_result_24; });
el_val_t ktier_raw = json_get_string(body, EL_STR("tier"));
el_val_t ktier = ({ el_val_t _if_result_24 = 0; if (str_eq(ktier_raw, EL_STR(""))) { _if_result_24 = (EL_STR("note")); } else { _if_result_24 = (ktier_raw); } _if_result_24; });
el_val_t ktier = ({ el_val_t _if_result_25 = 0; if (str_eq(ktier_raw, EL_STR(""))) { _if_result_25 = (EL_STR("note")); } else { _if_result_25 = (ktier_raw); } _if_result_25; });
el_val_t project = json_get_string(body, EL_STR("project"));
el_val_t tags_raw = json_get_raw(body, EL_STR("tags"));
el_val_t tags_base = ({ el_val_t _if_result_25 = 0; if (str_eq(tags_raw, EL_STR(""))) { _if_result_25 = (EL_STR("[]")); } else { _if_result_25 = (tags_raw); } _if_result_25; });
el_val_t tags_base = ({ el_val_t _if_result_26 = 0; if (str_eq(tags_raw, EL_STR(""))) { _if_result_26 = (EL_STR("[]")); } else { _if_result_26 = (tags_raw); } _if_result_26; });
el_val_t base_len = str_len(tags_base);
el_val_t head = str_slice(tags_base, 0, (base_len - 1));
el_val_t sep = ({ el_val_t _if_result_26 = 0; if (str_eq(head, EL_STR("["))) { _if_result_26 = (EL_STR("")); } else { _if_result_26 = (EL_STR(",")); } _if_result_26; });
el_val_t sep = ({ el_val_t _if_result_27 = 0; if (str_eq(head, EL_STR("["))) { _if_result_27 = (EL_STR("")); } else { _if_result_27 = (EL_STR(",")); } _if_result_27; });
el_val_t safe_cat = str_replace(category, EL_STR("\""), EL_STR("'"));
el_val_t safe_tier = str_replace(ktier, EL_STR("\""), EL_STR("'"));
el_val_t safe_proj = str_replace(project, EL_STR("\""), EL_STR("'"));
el_val_t proj_tag = ({ el_val_t _if_result_27 = 0; if (str_eq(safe_proj, EL_STR(""))) { _if_result_27 = (EL_STR("")); } else { _if_result_27 = (el_str_concat(el_str_concat(EL_STR(",\"project:"), safe_proj), EL_STR("\""))); } _if_result_27; });
el_val_t proj_tag = ({ el_val_t _if_result_28 = 0; if (str_eq(safe_proj, EL_STR(""))) { _if_result_28 = (EL_STR("")); } else { _if_result_28 = (el_str_concat(el_str_concat(EL_STR(",\"project:"), safe_proj), EL_STR("\""))); } _if_result_28; });
el_val_t tags = el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(head, sep), EL_STR("\"category:")), safe_cat), EL_STR("\",\"tier:")), safe_tier), EL_STR("\"")), proj_tag), EL_STR("]"));
el_val_t sal = el_from_float(0.5);
el_val_t imp = el_from_float(0.5);
@@ -361,6 +364,20 @@ el_val_t route_capture_knowledge(el_val_t method, el_val_t path, el_val_t body)
return 0;
}
el_val_t route_similarity(el_val_t method, el_val_t path, el_val_t body) {
el_val_t a = query_param(path, EL_STR("a"));
el_val_t b = query_param(path, EL_STR("b"));
if (str_eq(a, EL_STR(""))) {
return err_json(EL_STR("missing a"));
}
if (str_eq(b, EL_STR(""))) {
return err_json(EL_STR("missing b"));
}
el_val_t sim = engram_cosine_sim(a, b);
return el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(EL_STR("{\"a\":\""), a), EL_STR("\",\"b\":\"")), b), EL_STR("\",\"cosine\":")), float_to_str(sim)), EL_STR("}"));
return 0;
}
el_val_t check_auth_ok(el_val_t method, el_val_t body) {
el_val_t key = env(EL_STR("ENGRAM_API_KEY"));
if (str_eq(key, EL_STR(""))) {
@@ -447,6 +464,9 @@ el_val_t handle_request(el_val_t method, el_val_t path, el_val_t body) {
if (str_eq(clean, EL_STR("/api/embed-backfill"))) {
return route_embed_backfill(method, path, body);
}
if (str_eq(method, EL_STR("GET")) && str_starts_with(clean, EL_STR("/api/similarity"))) {
return route_similarity(method, path, body);
}
return el_str_concat(el_str_concat(EL_STR("{\"error\":\"not found\",\"path\":\""), clean), EL_STR("\"}"));
return 0;
}
@@ -454,10 +474,10 @@ el_val_t handle_request(el_val_t method, el_val_t path, el_val_t body) {
int main(int _argc, char** _argv) {
el_runtime_init_args(_argc, _argv);
bind_raw = env(EL_STR("ENGRAM_BIND"));
bind_str = ({ el_val_t _if_result_28 = 0; if (str_eq(bind_raw, EL_STR(""))) { _if_result_28 = (EL_STR(":8742")); } else { _if_result_28 = (bind_raw); } _if_result_28; });
bind_str = ({ el_val_t _if_result_29 = 0; if (str_eq(bind_raw, EL_STR(""))) { _if_result_29 = (EL_STR(":8742")); } else { _if_result_29 = (bind_raw); } _if_result_29; });
port = parse_port(bind_str);
data_dir_raw = env(EL_STR("ENGRAM_DATA_DIR"));
data_dir = ({ el_val_t _if_result_29 = 0; if (str_eq(data_dir_raw, EL_STR(""))) { _if_result_29 = (EL_STR("/tmp/engram")); } else { _if_result_29 = (data_dir_raw); } _if_result_29; });
data_dir = ({ el_val_t _if_result_30 = 0; if (str_eq(data_dir_raw, EL_STR(""))) { _if_result_30 = (EL_STR("/tmp/engram")); } else { _if_result_30 = (data_dir_raw); } _if_result_30; });
snapshot_path = el_str_concat(data_dir, EL_STR("/snapshot.json"));
engram_load(snapshot_path);
boot_snap = fs_read(snapshot_path);
engram/src/server.el
+10 -1
View File
@@ -311,7 +311,16 @@ fn route_sync(method: String, path: String, body: String) -> String {
let snap_path: String = dir + "/.sync-export.json"
engram_save(snap_path)
let snap: String = fs_read(snap_path)
if str_eq(snap, "") { return "{\"nodes\":[],\"edges\":[]}" }
// 2026-08-02 self-review: this used to return {"nodes":[],"edges":[]} when
// the export/read failed. The soul's sync_ok test (awareness.el) only
// checks for "" and "{}", so that placeholder PASSED as a healthy sync:
// soul.last_sync_ok_ts got stamped, sync_age_ms stayed green, the
// sync_empty warn ISE never fired, and engram_sync reported added:0
// forever. A totally broken sync was indistinguishable from a quiet
// healthy one — the exact failure class this route was added to fix in
// the first place (see 2026-06-27 note above). Return a real error so the
// failure is loud on both sides.
if str_eq(snap, "") { return err_json("sync export failed: snapshot unreadable") }
return snap
}
lang/releases/v1.0.0-20260501/el_runtime.c
+102 -18
View File
@@ -5633,6 +5633,43 @@ void el_cgi_init(el_val_t name, el_val_t dharma_id, el_val_t principal,
* exceed the floor, so naturally-promoted nodes survive multiple decay cycles.
* Invariant maintained: BREAKTHROUGH_WEIGHT < min(type_thresholds). */
#define ENGRAM_BREAKTHROUGH_WEIGHT 0.10
/* ENGRAM_BREAKTHROUGH_BUDGET / ENGRAM_BREAKTHROUGH_COOLDOWN (2026-08-02
* self-review): the breakthrough path was an unbounded, self-resetting loop.
* Every reached node failing its type threshold incremented suppression_count;
* on the 5th failure it was force-promoted at exactly 0.10 AND had its counter
* reset to 0 — so it re-entered the identical cycle immediately. Because
* BREAKTHROUGH_WEIGHT (0.10) > WM_FLOOR (0.05), all of them cleared the
* absolute floor and entered the rank contest tied at 0.10, where all but a
* handful were evicted by ENGRAM_WM_CAP. Evicted nodes get no access_ts
* record (Pass 6 skips wm_weights<=0), so the short-term inhibition-of-return
* damper at ENGRAM_STI_TS never applied to them and they were re-suppressed
* completely unmarked. Steady state: N_suppressed/5 breakthroughs per call,
* essentially all of them evicted the same call.
*
* Live telemetry 2026-08-02 (boot 19, uptime 23h48m): breakthroughs_delta
* 661–903 and wm_evicted_delta 485–717 PER 60s heartbeat against wm_active
* pinned at 22–24. At ~4 activate calls/minute that is ~165–225 breakthroughs
* per call — ~825–1125 nodes cycling in 5-call lockstep. Working memory was
* not remembering; it was thrashing, and the churn drowned genuine promotion.
*
* Two bounds, both required:
* BUDGET — at most WM_CAP/4 (6) intrusive thoughts may surface per call.
* Breakthrough is meant to be an occasional intrusive thought,
* not a stampede; it can never again exceed a quarter of WM.
* COOLDOWN — on breakthrough, suppression_count is set NEGATIVE rather than
* 0, so the node needs COOLDOWN+SUPPRESSION_BREAKTHROUGH further
* suppressions before it may surface again (~60 calls ≈ 15 min at
* the current cadence, vs 5 calls ≈ 75s before). This is
* inhibition-of-return applied to the breakthrough path, matching
* the STI damper already applied to the natural path. Stored in
* the existing int32_t field — serialized as %d and parsed via
* eg_get_int_field, so negative values round-trip through
* snapshots without a struct or format change.
* When the budget or cooldown blocks a breakthrough, suppression_count is NOT
* reset — it saturates, so a starved node surfaces on a later call rather than
* restarting its climb. */
#define ENGRAM_BREAKTHROUGH_BUDGET (ENGRAM_WM_CAP / 4)
#define ENGRAM_BREAKTHROUGH_COOLDOWN 55
/* ENGRAM_WM_CAP: hard limit on concurrent working-memory nodes (2026-06-30
* self-review, porting fix from self-review 2026-06-26 branch). Without this,
* broad curiosity seeds like "knowledge" promote 500+ nodes simultaneously —
@@ -6032,7 +6069,12 @@ static int64_t _eg_embed_breaker_until = 0;
* between) missed nearly every eviction/breakthrough event. Consumers wanting
* rates keep the previous reading and diff. Restart legitimately resets to 0. */
static int64_t _eg_act_breakthroughs = 0; /* forced promotions at the floor, cumulative */
static int64_t _eg_act_wm_evicted = 0; /* Pass 4 over-cap evictions, cumulative */
static int64_t _eg_act_wm_evicted = 0; /* ALL WM evictions, cumulative (see below) */
/* 2026-08-02 self-review: this counted only the three Pass 4 / Pass 5 floor
* and rank paths. The two carry-over eviction paths (base-level below τ, and
* decayed weight below WM_FLOOR) were silent, so the reported eviction rate
* was an undercount of unknown magnitude — which mattered precisely while
* diagnosing the breakthrough storm. All five paths now increment. */
static int64_t engram_now_ms(void); /* defined in the store section below */
@@ -8006,6 +8048,8 @@ el_val_t engram_activate(el_val_t query, el_val_t depth) {
free(best_bg); free(best_hops); free(reached); free(seeds);
free(fr); free(inhibition); free(cosq); return out;
}
/* Per-call breakthrough budget (2026-08-02) — see ENGRAM_BREAKTHROUGH_BUDGET. */
int64_t bt_budget = ENGRAM_BREAKTHROUGH_BUDGET;
for (int64_t i = 0; i < g->node_count; i++) {
if (!reached[i] || best_bg[i] <= 0.0) continue;
EngramNode* n = &g->nodes[i];
@@ -8043,21 +8087,10 @@ el_val_t engram_activate(el_val_t query, el_val_t depth) {
if (inh > 1.0) inh = 1.0;
double suppress = 1.0 - (1.0 - ENGRAM_INHIBITION_FACTOR) * inh;
raw_wm *= suppress;
/* Additive semantic-relevance term (2026-07-24, bl-b2d1c944):
* shift-and-floor at S0 — nomic-embed scores unrelated pairs
* 0.4–0.5, so raw cosine in a weighted sum would be a constant
* bias swamping the decayed base-level signal. Above S0 the term
* ramps 0 → WM_WEIGHT, breaking ties among structurally equivalent
* candidates in favor of nodes that mean what the query means. */
if (cosq && cosq[i] > ENGRAM_EMBED_S0) {
raw_wm += ENGRAM_EMBED_WM_WEIGHT
* (cosq[i] - ENGRAM_EMBED_S0) / (1.0 - ENGRAM_EMBED_S0);
}
/* Short-term inhibition-of-return (2026-07-25, Lebiere-Best):
* damp by t_n/(t_n + t_s) where t_n = seconds since the most
* recent recorded access (WM promotion / strengthen). Applied
* after the cosine term so the WHOLE score rotates — a node that
* just held a WM slot yields it even to semantically weaker
* recent recorded access (WM promotion / strengthen). A node that
* just held a WM slot yields it even to structurally stronger
* competitors, and recovers as t_n grows. access_ts is recorded
* at promotion, so persistent WM residents self-inhibit. Nodes
* with no access history (never promoted) are uninhibited.
@@ -8070,6 +8103,27 @@ el_val_t engram_activate(el_val_t query, el_val_t depth) {
if (sti_tn < 0.1) sti_tn = 0.1;
raw_wm *= sti_tn / (sti_tn + ENGRAM_STI_TS);
}
/* Additive semantic-relevance term (2026-07-24, bl-b2d1c944):
* shift-and-floor at S0 — nomic-embed scores unrelated pairs
* 0.4–0.5, so raw cosine in a weighted sum would be a constant
* bias swamping the decayed base-level signal. Above S0 the term
* ramps 0 → WM_WEIGHT, breaking ties among structurally equivalent
* candidates in favor of nodes that mean what the query means.
*
* MOVED AFTER the STI damper (2026-08-02 self-review). It used to
* be added BEFORE, so the recency multiplier scaled the semantic
* term too: an incumbent re-reached 30s later took t_n/(t_n+120)
* = 0.2×, cutting the cosine term's ceiling from 0.20 to 0.04 —
* below every per-type threshold (0.15–0.40). Meaning-match was
* being punished for having been recently useful. Inhibition-of-
* return should rotate the STRUCTURAL score (what the graph
* dragged in), not the semantic one (what the query actually
* means); relevance to the current query is not stale merely
* because the node was in WM a moment ago. */
if (cosq && cosq[i] > ENGRAM_EMBED_S0) {
raw_wm += ENGRAM_EMBED_WM_WEIGHT
* (cosq[i] - ENGRAM_EMBED_S0) / (1.0 - ENGRAM_EMBED_S0);
}
/* Threshold gate: must exceed per-type threshold to enter working
* memory. Type threshold replaces the old flat 0.2 filter. */
if (raw_wm >= type_threshold) {
@@ -8077,14 +8131,41 @@ el_val_t engram_activate(el_val_t query, el_val_t depth) {
if (n->suppression_count > 0) n->suppression_count = 0;
} else {
/* Node didn't make it through — increment suppression counter.
* After N consecutive suppressions: force breakthrough. */
* After N consecutive suppressions it MAY force a breakthrough,
* subject to the per-call budget and the negative-count cooldown
* (2026-08-02 — see ENGRAM_BREAKTHROUGH_BUDGET/_COOLDOWN). */
n->suppression_count++;
if (n->suppression_count >= ENGRAM_SUPPRESSION_BREAKTHROUGH) {
wm_weights[i] = ENGRAM_BREAKTHROUGH_WEIGHT;
n->suppression_count = 0;
if (n->suppression_count >= ENGRAM_SUPPRESSION_BREAKTHROUGH
&& bt_budget > 0) {
/* Graded breakthrough weight (2026-08-02): previously every
* breakthrough landed on exactly ENGRAM_BREAKTHROUGH_WEIGHT,
* so hundreds tied at 0.10 and the rank-cap tie-break at the
* cutoff degenerated to node-array index order — i.e. whoever
* was inserted earliest won, which is not a cognitive
* criterion. Scale within ±10% by how close the node came to
* its own threshold, so a near-miss outranks a node that was
* nowhere near. Stays strictly below min(type_threshold)
* (0.15) and strictly above ENGRAM_WM_FLOOR (0.05), which is
* the invariant ENGRAM_BREAKTHROUGH_WEIGHT documents. */
double near = (type_threshold > 0.0)
? (raw_wm / type_threshold) : 0.0;
if (near < 0.0) near = 0.0;
if (near > 1.0) near = 1.0;
wm_weights[i] = ENGRAM_BREAKTHROUGH_WEIGHT
* (0.9 + 0.2 * near);
/* Negative = cooldown. Must climb back through the cooldown
* before it can breach again. */
n->suppression_count = -ENGRAM_BREAKTHROUGH_COOLDOWN;
bt_budget--;
_eg_act_breakthroughs++;
} else {
wm_weights[i] = 0.0;
/* Budget-starved or cooling down: do NOT reset the counter —
* let it saturate so the node surfaces on a later call rather
* than restarting its climb from zero. Cap the ceiling so the
* int32 cannot drift unbounded over a long uptime. */
if (n->suppression_count > ENGRAM_SUPPRESSION_BREAKTHROUGH * 4)
n->suppression_count = ENGRAM_SUPPRESSION_BREAKTHROUGH * 4;
}
}
}
@@ -8217,6 +8298,7 @@ el_val_t engram_activate(el_val_t query, el_val_t depth) {
if (B < ENGRAM_BLL_TAU) {
cn->working_memory_weight = 0.0;
cn->wm_anchor = 0.0; /* keep anchor coherent with eviction */
_eg_act_wm_evicted++; /* was uncounted before 2026-08-02 */
} else {
double keep = 1.0 / (1.0 + exp(-(B - ENGRAM_BLL_TAU)
/ ENGRAM_BLL_S));
@@ -8233,6 +8315,7 @@ el_val_t engram_activate(el_val_t query, el_val_t depth) {
if (w < ENGRAM_WM_FLOOR) {
cn->working_memory_weight = 0.0;
cn->wm_anchor = 0.0;
_eg_act_wm_evicted++; /* was uncounted before 2026-08-02 */
} else {
cn->working_memory_weight = w;
}
@@ -8303,6 +8386,7 @@ el_val_t engram_activate(el_val_t query, el_val_t depth) {
}
n->working_memory_weight = 0.0; /* evict: over global cap */
n->wm_anchor = 0.0; /* keep anchor coherent */
_eg_act_wm_evicted++; /* was uncounted before 2026-08-02 */
}
}
/* If malloc failed, skip — WM over cap this call, no data corruption. */