bigmerge
777ccc02f0
El SDK CI - dev / build-and-test (pull_request) Failing after 10m45s
Checkpointing pushes the ENTIRE resident graph through store_put_node and
store_put_edge (see engram_store_checkpoint). Nodes were cheap: a durable-hash
compare skipped unchanged records with zero page I/O. Edges had no barrier at
all — struct comment at PgCache.barrier_on even says "node durable-hash
barrier" — so every edge was rewritten on every checkpoint, and each rewrite
runs the idempotency probe max_page_lsn_for_id -> btree lookup -> page_read.
Edges outnumber nodes ~3:1 here (37,663 vs 13,436), so routine checkpointing
degenerated into a FULL-STORE WALK in id order: random page access across the
whole 2 GiB store, repeated, overwhelmingly to rediscover nothing had changed.
LRU is worst-case under exactly that pattern — it evicts the page it is about
to want — so once the page cache was smaller than the store, the walk collapsed
into thrashing: 100% CPU, flat RSS, no forward progress, port never bound.
That took the live engram down twice on 2026-08-15.
The walk is the defect. Sizing the cache to survive it treats the symptom.
Changes:
- dh_edge_hash(): edge counterpart of dh_node_hash, with a kind discriminator
byte so an edge can never collide with a node of the same id in the shared
map. created_at/updated_at/last_fired are excluded deliberately: last_fired
is touched by activation without changing what the edge IS, and folding it
in would defeat the barrier on precisely the hot edges that most need it.
- store_put_edge(): barrier check + dh_set on success, mirroring
store_put_node exactly.
- store_scan_edges(): seed the barrier map from on-disk truth at load, so the
FIRST post-boot checkpoint already skips unchanged edges. store_scan_nodes
already did this and its comment says why; edges were simply never done.
Verified: with the exact configuration that killed production
(ENGRAM_POOL_FRAMES=65536 -> 1 GiB cache against a 2 GiB store), the engram now
boots clean and serves — LISTENING, 13,436 nodes / 37,663 edges, embeddings
complete, 0.0% CPU, RSS 1.14 GiB (cache resting at its budget rather than
thrashing against it). Same small cache, same store, no walk.