engram tiered storage M3.5: persist activation field updates (pre-flip gate)

Flag-on checkpoint now full-walks the resident graph: store_put_node (WM weight,
activation_count, last_activated, wm_anchor) + store_put_edge (hebb, last_fired)
for every node/edge, then engram_checkpoint. Uses store_put_edge (idempotent
upsert) not store_hebb_batch, because activation FORMS new hebbian-associate edges
that bypass the create hook and delta-only hebb_batch can't create them. Store-on
boot now applies the same WM-halving + floor + cap transforms as engram_load.

This is the hebb-survives-restart fix. Gate: reboot from neuron.egm with
snapshot.json deleted -> edge hebb + activation_count survive unchanged, WM weight
survives with identical boot transform; negative control proves persist is
load-bearing (hebb->0 without it). M1 33/33 + M2 36/36 + M3 parity PASS, ASan/UBSan
clean, flag-off untouched. Engine unchanged (boundary held).
This commit is contained in:
2026-08-11 23:37:52 -05:00
parent a72145b44e
commit 9a0266cbf9
4 changed files with 351 additions and 4 deletions
+55 -3
View File
@@ -7339,8 +7339,10 @@ static void eg_store_put_edge(const EngramEdge* e) {
}
/* Resident-load callbacks: StoreNode/StoreEdge → a fresh EngramNode/EngramEdge
* appended to the in-RAM graph. Mirrors engram_load's field set (minus the
* boot-time WM laundering the store already holds the authoritative weights). */
* appended to the in-RAM graph. Mirrors engram_load's field set. The boot-time
* WM laundering (halve + floor + global cap) that engram_load applies is done
* once, after all nodes are loaded, in engram_store_boot see the block there
* so the store-on boot behaves byte-identically to the JSON path (M3.5 parity). */
static void eg_load_node_cb(const StoreNode* sn, void* ctx) {
EngramStore* g = (EngramStore*)ctx;
engram_grow_nodes();
@@ -7427,6 +7429,10 @@ static void eg_reset_resident(EngramStore* g) {
engram_adj_free(g);
}
/* Defined later with engram_load; declared here for the boot-time WM laundering
* that keeps the store-on boot byte-identical to the JSON path (M3.5 parity). */
static void eg_enforce_wm_cap_on_load(EngramStore* g);
/* engram_store_boot(data_dir) — open (import-once or WAL-replay) the durable
* paged store and load it whole into RAM (Phase 1). No-op / returns 0 when the
* flag is off. Returns 1 on success. Idempotent (a second call is a no-op). */
@@ -7446,14 +7452,60 @@ el_val_t engram_store_boot(el_val_t data_dir) {
for (size_t i = 0; i < ln; i++) eg_load_layer_cb(g, &ls[i]);
store_layers_free(ls, ln);
}
/* Boot-time WM laundering — MUST match engram_load exactly (M3.5 parity).
* The JSON load path halves every persisted working_memory_weight on boot
* (stale pinned weights decay out over successive restarts; genuine WM state
* keeps continuity) and floors sub-ENGRAM_WM_FLOOR residue to zero, then
* enforces the global WM cap. The store now persists post-activation WM
* weights, so the store-on boot must apply the identical transform or the two
* persistence paths would diverge on the very first restart. */
for (int64_t i = 0; i < g->node_count; i++) {
g->nodes[i].working_memory_weight *= 0.5;
if (g->nodes[i].working_memory_weight < ENGRAM_WM_FLOOR)
g->nodes[i].working_memory_weight = 0.0;
}
eg_enforce_wm_cap_on_load(g);
g->adj_dirty = 1;
return (el_val_t)1;
}
/* engram_store_checkpoint() — flush dirty pages + advance the checkpoint LSN.
/* engram_store_checkpoint() — M3.5 PRE-FLIP GATE.
*
* Spreading activation mutates fields IN PLACE on the resident graph edge
* `hebb`/`last_fired` (potentiation + homeostatic scaling), node
* `activation_count`/`last_activated`/`working_memory_weight`/`wm_anchor`
* (reinforcement + WM caps) and also FORMS brand-new `hebbian-associate`
* edges. M3 only mirrored node/edge *creates* and *forgets*; none of those
* in-place mutations or activation-formed edges reached the paged store, so
* learned associations were lost on every restart. This is the fix that makes
* "learned edges survive a restart" true, and it gates the live cutover.
*
* On the soul's save/checkpoint path we push the resident graph's current field
* state through the store's WAL-logged API, then checkpoint:
* - store_put_node(n) for every node persists WM weight, activation_count,
* last_activated, wm_anchor, the base-level access ring, etc.
* - store_put_edge(e) for every edge persists hebb + last_fired AND creates
* any activation-formed edges. (store_hebb_batch is deliberately NOT used: it
* is a delta-only op that skips edge ids not already resident in the store
* see apply_hebb_batch so it cannot persist the newly-formed hebbian edges
* that are the whole point of this milestone. store_put_edge is the idempotent
* upsert that subsumes the hebb delta.)
* then engram_checkpoint flushes dirty pages + advances the checkpoint LSN. Every
* push is a WAL record, so a graceful restart restores them.
*
* Approach: a FULL WALK of the resident graph (not a dirty-set). At Phase-1
* ~64 MB resident this is a cheap linear pass on an already-in-RAM array, run at
* the soul's save cadence (seconds-to-minutes), and it is trivially complete
* no mutation site can be missed and no separate new-edge tracking is needed. An
* inter-checkpoint crash loses only the most-recent unsaved learning, exactly the
* same durability envelope as today's JSON-snapshot cadence.
*
* The storeJSON export path stays engram_save (the JSON is an export artifact). */
el_val_t engram_store_checkpoint(void) {
if (!engram_store_enabled() || !g_engram_store) return (el_val_t)0;
EngramStore* g = engram_get();
for (int64_t i = 0; i < g->node_count; i++) eg_store_put_node(&g->nodes[i]);
for (int64_t i = 0; i < g->edge_count; i++) eg_store_put_edge(&g->edges[i]);
return (el_val_t)(int64_t)(engram_checkpoint(g_engram_store) == 0 ? 1 : 0);
}