engram tiered storage M3: wire store behind ENGRAM_STORE (default off) + .egm rename

Caller-side shim in el_runtime.c maps EngramNode/Edge <-> StoreNode/Edge; engine
keeps zero soul deps (libengram boundary, design §10). Flag off = today's JSON
path byte-for-byte (proven: no neuron.egm created, graph identical). Flag on =
engram_open (import snapshot.json once into neuron.egm, else WAL-replay) +
resident load; node/edge create + forget dual-write via guarded hooks. Files
renamed engram.store->neuron.egm, engram.wal->neuron.wal.

Gate: M3 parity PASS (graph on==off byte-exact modulo ordering; snapshot round-trip;
reboot-from-egm with snapshot.json deleted; activation set+sequence identical;
ASan/UBSan clean). M1 33/33 + M2 36/36 green post-rename.

Known gap (pre-flip): in-place hebb/WM/activation_count updates during activation
are not yet persisted to the store (create/connect/forget are). Must close before
live flip so learned edges survive restart.
This commit is contained in:
2026-08-11 23:21:21 -05:00
parent 8affb1d6e0
commit a72145b44e
6 changed files with 588 additions and 12 deletions
+13 -2
View File
@@ -134,7 +134,7 @@ uint64_t store_page_count(const EngramPagedStore* s);
/* ── M2: WAL + checkpoint + crash recovery + one-time legacy import ─────────────
*
* The durable engram is `engram.store` (paged) fronted by `engram.wal`
* The durable engram is `neuron.egm` (paged) fronted by `neuron.wal`
* (append-only). A mutation is durable once its WAL record is fsync'd
* (group-commit). Pages are held write-back in RAM (no-steal) and flushed to the
* store only at a checkpoint, so the store file on disk always reflects a
@@ -158,7 +158,7 @@ typedef struct StoreLayer {
int tombstoned;
} StoreLayer;
/* Boot the durable engram in `data_dir` (holds engram.store + engram.wal). If the
/* Boot the durable engram in `data_dir` (holds neuron.egm + neuron.wal). If the
* store is absent but a legacy snapshot.json exists, it is imported ONCE into a
* fresh store; thereafter the store is authoritative and JSON is never read again.
* On open, the WAL is replayed to recover any post-checkpoint mutations. */
@@ -194,6 +194,17 @@ int store_supersede(EngramPagedStore* s, const char* old_id, const char* new_id
/* Forget (GC): tombstone id at the store (hard-free deferred to compaction). */
int store_forget(EngramPagedStore* s, const char* id);
/* ── M3: full live enumeration (for the CALLER's resident load + JSON export) ──
* Walk the whole store and invoke `cb` once per DISTINCT live node/edge with a
* borrowed view (the engine frees it after cb returns — the callback must copy
* anything it keeps). De-duplicated by id (canonical latest-live per id, matching
* point-read semantics). Returns the count emitted, or <0 on error. The engine
* hands out StoreNode/StoreEdge only — it never sees a soul struct (design §10). */
typedef void (*StoreNodeScanCb)(const StoreNode* n, void* ctx);
typedef void (*StoreEdgeScanCb)(const StoreEdge* e, void* ctx);
int store_scan_nodes(EngramPagedStore* s, StoreNodeScanCb cb, void* ctx);
int store_scan_edges(EngramPagedStore* s, StoreEdgeScanCb cb, void* ctx);
/* Introspection / test hooks. */
uint64_t engram_wal_next_lsn(const EngramPagedStore* s);
uint64_t engram_last_checkpoint_lsn(const EngramPagedStore* s);