From 9f1db8278c4588fa6d0c71dcf731a84ba4dc5e1e Mon Sep 17 00:00:00 2001 From: Will Anderson Date: Thu, 6 Aug 2026 08:44:27 -0500 Subject: [PATCH] self-review 2026-08-06: eligibility traces for Hebbian co-activation; dedup WM globally MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Hebbian consolidation was inert. Census over the live graph (41,213 edges, 13,091 nodes, 23h44m uptime): strongest association hebb=0.000799 against a 0.15 consolidation threshold, and zero hebbian-associate edges ever formed. Since the awareness loop calls engram_connect nowhere, this was the only path by which the graph could grow its own structure — every edge was authored or imported, none learned. The defect was the event, not the rate. hebb is an EWMA whose fixed point is P(event); raising ETA changes convergence speed, never the plateau. The event was "both endpoints in WM in the same activate call" — demanded exact simultaneity from a working memory that inhibition-of-return, breakthrough rotation and the 24-slot global cap are all engineered to keep turning over (~142 evictions/60s). The three mechanisms that make WM healthy are the ones that made this measurement empty. Replaced with three-factor eligibility traces (Sutton & Barto ch.7; Gerstner et al. 2018; PLOS Comp Biol 2018 differential Hebbian learning): a node entering WM sets a trace to 1.0, the trace decays exponentially in wall-clock time (TC=300s, chosen against the measured ~31s scan cadence), and the increment becomes ETA·trace(a)·trace(b). Strict generalization — co-resident pairs read 1.0 on both ends and get exactly ETA, bit-identical to before. warm×warm is deliberately not paired: eligibility must gate on something happening now. Homeostatic ENGRAM_HEBB_NODE_BUDGET still bounds per-node mass. Measured over a 60-call soak: hebb_max 0.0008 -> 0.0060, climbing at ~0.87 ETA/call against an all-time ceiling of 0.0008 before. hebb_mass 0.011 -> 0.019, no runaway. Projected consolidation of a genuinely recurring pair: ~1,730 calls, ~14h at autonomous cadence. links still 0 — that is expected and is what tomorrow's review must check. Also: Pass 3½ deduplicates this call's WM candidates, but the persisted WM population is a union of fresh promotions and carry-over residents, and Pass 3½ never sees the second set. Confirmed live: two byte-identical copies of one 3,193-char document both holding slots (0.289 / 0.271). Added global redundancy suppression in Pass 5 before the cap count. Post-fix census: 24 residents, 24 distinct contents, 0 wasted slots. New gauges: hebb_warm (eligible-but-not-co-resident population), dup_wm_global. --- lang/releases/v1.0.0-20260501/el_runtime.c | 306 +++++++++++++++++++-- 1 file changed, 278 insertions(+), 28 deletions(-) diff --git a/lang/releases/v1.0.0-20260501/el_runtime.c b/lang/releases/v1.0.0-20260501/el_runtime.c index 8ef3407..aea3a9c 100644 --- a/lang/releases/v1.0.0-20260501/el_runtime.c +++ b/lang/releases/v1.0.0-20260501/el_runtime.c @@ -5929,6 +5929,19 @@ typedef struct EngramNode { * engram_prune_telemetry; shift-copies move the pointer intact. */ float* emb; int32_t emb_dim; + /* Hebbian eligibility trace (2026-08-06 self-review, ENGRAM_HEBB_TRACE_*). + * hebb_elig: trace amplitude, set to 1.0 whenever the node holds a WM slot. + * hebb_elig_ts: wall-clock ms at which that happened; the trace is read as + * hebb_elig·exp(−Δt/TC) rather than stored decayed, so it is a pure + * function of time and independent of how often activate is called. + * + * DELIBERATELY NOT SERIALIZED. A trace is a behavioral-timescale quantity + * (minutes); persisting it across a restart would resurrect coincidences + * from an arbitrarily distant past as if they had just happened. Zero via + * the calloc at store init and the memset in engram_grow_nodes, so a cold + * boot simply has no eligible pairs until WM starts turning over. */ + double hebb_elig; + int64_t hebb_elig_ts; } EngramNode; /* Record an access (ACT-R "presentation") into the base-level ring buffer. */ @@ -6158,6 +6171,71 @@ static void engram_bll_parse_access(EngramNode* nn, const char* s) { * the potentiation rule to gain any real influence. They are tagged * precisely so every self-formed association stays auditable and the whole * set is removable with one query if this turns out to be wrong. */ +/* ── Eligibility traces: why co-activation had to stop meaning "same call" ──── + * (2026-08-06 self-review. Measured, not theorized.) + * + * CENSUS. Across the live graph — 41,213 edges, 13,091 nodes, 23h44m uptime, + * boot 23 — the strongest Hebbian association in the entire store measured + * hebb = 0.000799, and `hebbian-associate` edges formed since the mechanism + * shipped: ZERO. 0.000799 / ENGRAM_HEBB_ETA ≈ 8. The best pair of nodes in the + * graph had co-activated eight times, net of decay, ever. ENGRAM_HEBB_LINK_MIN + * is 0.15 — 1875× further away. The effective propagation bonus the mechanism + * was delivering was GAIN·hebb = 0.5 × 0.0008 = +0.04%. + * + * Since the awareness loop calls engram_connect nowhere, Hebbian consolidation + * is the ONLY path by which this system grows its own structure. It was inert. + * Every edge in the graph was authored or imported; none was learned. + * + * WHY RAISING ETA IS THE WRONG FIX. hebb is an EWMA: h ← DECAY·h + ETA·[event], + * with ETA = 1 − DECAY. Its fixed point is P(event) — the learning rate sets how + * fast it converges there and has NO effect on where it converges. If the + * measured ceiling is 0.0008, then P(event) ≈ 0.0008, and ETA could be raised a + * thousandfold without moving the plateau a single decimal. The defect is not + * the rate. It is the event. + * + * WHAT THE EVENT WAS. "Both endpoints hold a WM slot in the same activate call." + * With ENGRAM_WM_CAP = 24 against a 13k-node store, and measured turnover of + * ~142 evictions per 60s heartbeat, two related nodes essentially never occupy + * the same 24 twice. The rule demanded exact simultaneity from a working memory + * engineered — by ENGRAM_STI_TS inhibition-of-return, by breakthrough rotation, + * by the global cap — to never let anything sit still. The three mechanisms + * that make WM healthy are precisely the ones that made this measurement empty. + * + * THE FIX (three-factor / TD(λ) eligibility traces; Sutton & Barto ch. 7, + * Gerstner et al. 2018 on neoHebbian eligibility, PLOS Comp Biol 2018 on + * differential Hebbian learning with leaky-integrator traces). Coincidence + * detection at behavioral timescales does not require simultaneity. A synapse + * that fires sets a decaying flag; potentiation occurs if the partner fires + * while the flag is still up. So: every node entering WM sets a trace to 1.0, + * the trace decays exponentially in WALL-CLOCK time, and the potentiation + * increment becomes ETA · trace(a) · trace(b) instead of ETA · [both in WM now]. + * + * This is a strict generalization, which is what makes it safe to ship: when + * both endpoints are in WM at this instant, both traces read exactly 1.0 and + * the increment is exactly ETA — bit-identical to the previous rule. The change + * is purely additive on near-coincidences the old rule discarded outright. + * + * TC = 300s. Chosen against measured cadence, not taste: curiosity scans run + * every ~31s, so 300s spans ~10 activation cycles (one cycle back reads 0.90, + * three back 0.73, ten back 0.36). Long enough to bridge WM rotation; short + * enough that two nodes surfacing an hour apart (exp(−12) ≈ 6e-6) are correctly + * treated as unrelated. It sits deliberately between ENGRAM_STI_TS (120s, the + * rotation it must survive) and ENGRAM_CARRY_TC (3600s, conversational span). + * + * TRACE_MIN snaps sub-threshold traces to zero so the warm set stays small and + * a node cannot linger as a faint associate of everything for hours. + * + * The homeostatic ENGRAM_HEBB_NODE_BUDGET cap is what keeps this from running + * away: more pairs now potentiate per call, and per-node associative mass is + * still hard-bounded and scaled down proportionally. The safety net predates + * this change and is exactly why loosening the event is not reckless. */ +#define ENGRAM_HEBB_TRACE_TC 300.0 +#define ENGRAM_HEBB_TRACE_MIN 0.05 +/* Bound on non-WM warm nodes considered for candidate pairing in one call. + * At 24 slots × ~10 cycles per trace window the warm set tops out near 240, but + * most are re-promoted incumbents already in WM; 96 is non-binding in practice + * and caps the pairing loop at 24×96 slot probes. */ +#define ENGRAM_HEBB_WARM_MAX 96 #define ENGRAM_HEBB_CAND_SLOTS 8192 #define ENGRAM_HEBB_LINK_MIN 0.15 #define ENGRAM_HEBB_LINK_PER_CALL 2 @@ -6175,6 +6253,14 @@ static void engram_bll_parse_access(EngramNode* nn, const char* s) { typedef struct { char* a; char* b; double score; } EgHebbCand; static EgHebbCand _eg_hebb_cand[ENGRAM_HEBB_CAND_SLOTS]; static int64_t _eg_hebb_links_formed = 0; +/* Observability for the eligibility-trace rule (2026-08-06). hebb_warm is the + * size of the last call's warm set — nodes eligible but not co-resident, i.e. + * exactly the population the old simultaneity rule discarded. If this reads 0 + * forever the traces are not arming and the change bought nothing; if it reads + * healthy while hebb_max stays flat, the bottleneck is somewhere else. That + * distinction is the whole reason yesterday's diagnosis took a census instead + * of a guess. */ +static int _eg_act_hebb_warm = 0; static float* _eg_ctx_c = NULL; static int32_t _eg_ctx_dim = 0; @@ -6204,6 +6290,13 @@ static int64_t _eg_act_wm_evicted = 0; /* ALL WM evictions, cumulative (see b * candidate's content. Both cumulative for the process lifetime. */ static int64_t _eg_act_dup_seeds = 0; static int64_t _eg_act_dup_wm = 0; +/* Redundant WM residents evicted by the GLOBAL pass (2026-08-06). Counted + * separately from _eg_act_dup_wm on purpose: dup_wm measures duplicates caught + * among this call's candidates, dup_wm_global measures duplicates that reached + * the persisted WM population through the carry-over path, which is the leak + * Pass 3½ structurally could not see. Merging them would hide whether the new + * pass is doing anything. */ +static int64_t _eg_act_dup_wm_global = 0; /* 2026-08-02 self-review: this counted only the three Pass 4 / Pass 5 floor * and rank paths. The two carry-over eviction paths (base-level below τ, and * decayed weight below WM_FLOOR) were silent, so the reported eviction rate @@ -7802,6 +7895,18 @@ static void eg_wm_carry_over(EngramNode* cn, int64_t now_ms, int64_t* evict_ctr) * simply loses this round. That is a lossy table by design — consolidation * should favor associations that recur, and a pair that keeps losing a * collision is by definition not recurring often enough to matter. */ +/* eg_hebb_trace — the node's eligibility trace right now, in [0,1]. + * Stored as (amplitude, timestamp) and decayed on read, so the value is a pure + * function of wall-clock time: idempotent no matter how often activate runs. + * Snapped to 0 below ENGRAM_HEBB_TRACE_MIN. See ENGRAM_HEBB_TRACE_TC. */ +static double eg_hebb_trace(const EngramNode* n, int64_t now_ms) { + if (n->hebb_elig <= 0.0 || n->hebb_elig_ts <= 0) return 0.0; + double dt = (double)(now_ms - n->hebb_elig_ts) / 1000.0; + if (dt < 0.0) dt = 0.0; /* clock skew ⇒ treat as fresh */ + double t = n->hebb_elig * exp(-dt / ENGRAM_HEBB_TRACE_TC); + return (t < ENGRAM_HEBB_TRACE_MIN) ? 0.0 : t; +} + static int eg_hebb_slot(const char* a, const char* b) { if (!a || !b) return -1; const char* lo = (strcmp(a, b) <= 0) ? a : b; @@ -7821,6 +7926,33 @@ static void eg_hebb_slot_clear(EgHebbCand* c) { c->a = NULL; c->b = NULL; c->score = 0.0; } +/* eg_hebb_cand_bump — reinforce the (a,b) candidate association by `inc`. + * Extracted 2026-08-06 so the same collision policy serves both the co-resident + * pairs and the eligibility-trace pairs; two copies of this logic would have + * drifted. Collision policy is unchanged: claim a free slot, reinforce our own, + * evict an incumbent only once it has decayed to nothing, otherwise lose the + * round. `inc` is graded by the partner's trace, so an incumbent is never + * displaced by a challenger carrying less weight than one full co-activation. */ +static void eg_hebb_cand_bump(const char* a, const char* b, double inc) { + if (!a || !b || inc <= 0.0) return; + int s = eg_hebb_slot(a, b); + if (s < 0) return; + EgHebbCand* c = &_eg_hebb_cand[s]; + if (!c->a) { /* free slot: claim */ + c->a = el_strdup_persist(a); + c->b = el_strdup_persist(b); + c->score = inc; + } else if (eg_hebb_slot_holds(c, a, b)) { + c->score += inc; /* ours: reinforce */ + } else if (c->score <= ENGRAM_HEBB_ETA) { + eg_hebb_slot_clear(c); /* dead incumbent: take the slot */ + c->a = el_strdup_persist(a); + c->b = el_strdup_persist(b); + c->score = inc; + } + /* else: live incumbent keeps the slot this round. */ +} + /* Does any edge already connect these two nodes, in either direction? * Linear over the edge array, but called at most ENGRAM_HEBB_LINK_PER_CALL * times per activation and only for pairs that already cleared the @@ -8829,6 +8961,80 @@ el_val_t engram_activate(el_val_t query, el_val_t depth) { _eg_act_wm_evicted++; } } + /* ── Global redundancy suppression (2026-08-06 self-review) ────────── + * Pass 3½ deduplicates THIS CALL'S candidates. But the WM population + * that actually exists after persist is a UNION of two sets: nodes + * promoted this call, and nodes carried over from earlier calls by + * eg_wm_carry_over. Pass 3½ never sees the second set, so the dedup + * guarantee it advertises does not hold for the thing it is a + * guarantee about. + * + * OBSERVED, not inferred. A live WM census caught two byte-identical + * Knowledge nodes — f93d5f90 and b578d6a7, the same 3,193-character + * "# Memory Integration" document under the same node_type — both + * holding slots at 0.289 and 0.271, having arrived by the two + * different routes. Pass 3½ had run and had correctly passed, because + * only one of the two was a candidate that call. + * + * The cost is small and constant: 1–2 of 24 slots, ~4–8% of working + * memory, indefinitely. It is worth fixing anyway, because the failure + * is silent and self-reinforcing — a duplicate that holds a slot gets + * reinforced for holding it, and (as of this session) now also earns + * Hebbian eligibility, so redundancy would start teaching the graph + * that a document is associated with itself. + * + * Runs BEFORE the cap count below, so slots freed here are reclaimed + * by distinct content in the same pass rather than left empty. Same + * identity test and same highest-weight-survives rule as Pass 3½. */ + { + int64_t gn = 0; + for (int64_t i = 0; i < g->node_count; i++) + if (g->nodes[i].working_memory_weight > 0.0) gn++; + if (gn > 1) { + EgDupCand* gd = malloc((size_t)gn * sizeof(EgDupCand)); + if (gd) { + int64_t gi = 0; + for (int64_t i = 0; i < g->node_count; i++) { + if (g->nodes[i].working_memory_weight > 0.0) { + gd[gi].w = g->nodes[i].working_memory_weight; + gd[gi].idx = i; + gi++; + } + } + qsort(gd, (size_t)gn, sizeof(EgDupCand), eg_dupcand_cmp_desc); + int64_t gkeep[ENGRAM_WM_CAP]; + uint64_t gkey[ENGRAM_WM_CAP]; + int gnk = 0; + double gcap_w = 0.0; + for (int64_t z = 0; z < gn; z++) { + if (gnk >= ENGRAM_WM_CAP && gd[z].w < gcap_w) break; + int64_t i = gd[z].idx; + EngramNode* n = &g->nodes[i]; + uint64_t key = eg_content_key(n); + int dup = 0; + for (int s = 0; s < gnk; s++) { + if (eg_same_content(n, &g->nodes[gkeep[s]], key, gkey[s])) { + dup = 1; break; + } + } + if (dup) { + n->working_memory_weight = 0.0; + n->wm_anchor = 0.0; + _eg_act_wm_evicted++; + _eg_act_dup_wm_global++; + continue; + } + if (gnk < ENGRAM_WM_CAP) { + gkeep[gnk] = i; gkey[gnk] = key; gnk++; + if (gnk == ENGRAM_WM_CAP) gcap_w = gd[z].w; + } + } + free(gd); + } + /* malloc failure: skip — duplicates may share slots this call, + * which is the pre-2026-08-06 behavior. No corruption. */ + } + } int64_t global_wm_count = 0; for (int64_t i = 0; i < g->node_count; i++) { if (g->nodes[i].working_memory_weight > 0.0) global_wm_count++; @@ -8918,19 +9124,40 @@ el_val_t engram_activate(el_val_t query, el_val_t depth) { wm_n++; } } + /* Step 0 (2026-08-06): refresh the eligibility trace of everything + * currently in WM. Done BEFORE the edge pass so a pair that is + * co-resident right now reads trace 1.0 on both ends and receives + * exactly ENGRAM_HEBB_ETA — identical to the pre-trace rule. */ + for (int64_t i = 0; i < g->node_count; i++) { + if (!in_wm[i]) continue; + g->nodes[i].hebb_elig = 1.0; + g->nodes[i].hebb_elig_ts = now_ms; + } /* Steps 1+2: decay all, potentiate co-active. Fused into one O(E) * pass. Edges whose endpoints resolve to nothing still decay — - * a dangling edge should not hold learned strength forever. */ + * a dangling edge should not hold learned strength forever. + * + * Co-activation is now graded by the product of the endpoints' + * eligibility traces rather than gated on same-call WM residency; + * see the ENGRAM_HEBB_TRACE_* block for the census that forced + * this. The old `wm_n > 1` guard is gone because a single node + * entering WM can now legitimately potentiate against a partner + * that was in WM moments ago — that asymmetric case is precisely + * the signal the simultaneity rule was throwing away. When no + * trace is warm, co is 0 and the pass degenerates to pure decay. */ for (int64_t ei = 0; ei < g->edge_count; ei++) { EngramEdge* e = &g->edges[ei]; double h = e->hebb * ENGRAM_HEBB_DECAY; - if (wm_n > 1 && !e->inhibitory) { + if (!e->inhibitory) { int64_t a = engram_idmap_get(g, e->from_id); int64_t b = engram_idmap_get(g, e->to_id); - if (a >= 0 && a < g->node_count && b >= 0 && b < g->node_count - && in_wm[a] && in_wm[b]) { - h += ENGRAM_HEBB_ETA; - e->last_fired = now_ms; /* first real writer outside dharma_strengthen */ + if (a >= 0 && a < g->node_count && b >= 0 && b < g->node_count) { + double co = eg_hebb_trace(&g->nodes[a], now_ms) + * eg_hebb_trace(&g->nodes[b], now_ms); + if (co > 0.0) { + h += ENGRAM_HEBB_ETA * co; + e->last_fired = now_ms; /* first real writer outside dharma_strengthen */ + } } } e->hebb = (h < ENGRAM_HEBB_MIN) ? 0.0 : h; @@ -8993,28 +9220,48 @@ el_val_t engram_activate(el_val_t query, el_val_t depth) { for (int64_t i = 0; i < g->node_count && wm_k < ENGRAM_WM_CAP; i++) { if (in_wm[i]) wm_idx[wm_k++] = i; } - /* Reinforce every co-active pair's candidate score. */ + /* Warm set (2026-08-06): nodes NOT in WM right now but whose + * eligibility trace is still up. These are the partners the + * simultaneity rule could never see. Bounded by + * ENGRAM_HEBB_WARM_MAX; the scan is O(node_count), which is + * an order of magnitude cheaper than the O(edge_count) pass + * already running above it. */ + int64_t warm_idx[ENGRAM_HEBB_WARM_MAX]; + double warm_t[ENGRAM_HEBB_WARM_MAX]; + int warm_k = 0; + for (int64_t i = 0; i < g->node_count + && warm_k < ENGRAM_HEBB_WARM_MAX; i++) { + if (in_wm[i]) continue; + double t = eg_hebb_trace(&g->nodes[i], now_ms); + if (t <= 0.0) continue; + warm_idx[warm_k] = i; + warm_t[warm_k] = t; + warm_k++; + } + _eg_act_hebb_warm = warm_k; + (void)wm_n; /* superseded as a gate by the trace product */ + /* Reinforce candidate scores. Two families, one rule: + * WM × WM — both traces are 1.0 ⇒ increment ETA, exactly + * the pre-2026-08-06 behavior. + * WM × warm — increment ETA·trace, so a partner that left + * working memory one cycle ago still earns most + * of the credit and one that left ten cycles ago + * earns a third of it. + * warm × warm is deliberately NOT paired: with nothing currently + * active there is no event to be eligible FOR, and pairing decayed + * residue against decayed residue would manufacture associations + * out of two absences. Eligibility gates on something happening + * now — that is the whole content of the three-factor rule. */ for (int x = 0; x < wm_k; x++) { for (int y = x + 1; y < wm_k; y++) { - const char* ia = g->nodes[wm_idx[x]].id; - const char* ib = g->nodes[wm_idx[y]].id; - if (!ia || !ib) continue; - int s = eg_hebb_slot(ia, ib); - if (s < 0) continue; - EgHebbCand* c = &_eg_hebb_cand[s]; - if (!c->a) { /* free slot: claim */ - c->a = el_strdup_persist(ia); - c->b = el_strdup_persist(ib); - c->score = ENGRAM_HEBB_ETA; - } else if (eg_hebb_slot_holds(c, ia, ib)) { - c->score += ENGRAM_HEBB_ETA; /* ours: reinforce */ - } else if (c->score <= ENGRAM_HEBB_ETA) { - eg_hebb_slot_clear(c); /* dead incumbent */ - c->a = el_strdup_persist(ia); - c->b = el_strdup_persist(ib); - c->score = ENGRAM_HEBB_ETA; - } - /* else: live incumbent keeps the slot this round. */ + eg_hebb_cand_bump(g->nodes[wm_idx[x]].id, + g->nodes[wm_idx[y]].id, + ENGRAM_HEBB_ETA); + } + for (int w = 0; w < warm_k; w++) { + eg_hebb_cand_bump(g->nodes[wm_idx[x]].id, + g->nodes[warm_idx[w]].id, + ENGRAM_HEBB_ETA * warm_t[w]); } } /* Consolidate the strongest qualifying candidates into real @@ -10222,14 +10469,17 @@ el_val_t engram_act_stats_json(void) { "\"ctx_cos\":%.3f," "\"hebb_edges\":%lld,\"hebb_max\":%.4f,\"hebb_mass\":%.3f," "\"hebb_cands\":%d,\"hebb_cand_max\":%.4f,\"hebb_links\":%lld," - "\"dup_seeds\":%lld,\"dup_wm\":%lld}", + "\"hebb_warm\":%d," + "\"dup_seeds\":%lld,\"dup_wm\":%lld,\"dup_wm_global\":%lld}", (long long)_eg_act_wm_evicted, (long long)_eg_act_breakthroughs, breaker_open, _eg_embed_consec_fail, _eg_act_ctx_cos, (long long)hebb_edges, hebb_max, hebb_mass, hebb_cands, hebb_cand_max, (long long)_eg_hebb_links_formed, - (long long)_eg_act_dup_seeds, (long long)_eg_act_dup_wm); + _eg_act_hebb_warm, + (long long)_eg_act_dup_seeds, (long long)_eg_act_dup_wm, + (long long)_eg_act_dup_wm_global); return el_wrap_str(el_strdup(buf)); }