Durability: the 2026-07-21 fix stopped read routes writing the canonical
snapshot but left no save on ANY write path — every mutation lived in RAM
until a manual POST /api/save. Observed live: two restarts reverted the
store to a 17h-old snapshot, destroying same-day writes. persist_canonical()
now runs after node/edge create, knowledge capture, forget, strengthen, and
load-merge. ISE telemetry excluded deliberately (48h-pruned, loss-tolerant,
~2/min; snapshotting 28MB per heartbeat is waste).
Listing order: scan routes sort by salience with store-order ties, so
equal-salience telemetry (all ISEs are 0.3) returned OLDEST first — a
limited /api/nodes query silently returned a stale window, and a 41h-old
heartbeat series read as a live outage during this review. Ties now break
newest-first by created_at.
The old carry-over (weight *= 0.7 per engram_activate call) was call-rate-
dependent — carried context died in seconds under rapid curiosity scans and
lingered for hours under quiet loops — and a decayed scalar cannot represent
access frequency at all.
Now: k=10 access-timestamp ring + Petrov (2006) closed-form tail, d=0.5.
WM promotion and engram_strengthen record presentations; carry-over evicts
at base-level tau=-3.0 (Soar forgetting, ~403s single-touch) and shapes the
weight held at promotion (wm_anchor) with the ACT-R retrieval logistic
(s=0.4) — a pure function of wall-clock time, idempotent per call.
Persisted as access_ts/wm_anchor in snapshots; legacy nodes fall back to
the optimized form ln(n/(1-d)) - d*ln(L). base_level exposed in both node
serializers for observability.
Backing spec: 2026-07-21 integration brief (bl-b17facdd). Verified live:
carried weight ~anchor seconds after two disjoint activations (old code:
0.49x); frequency-hot nodes hold B=1.9 vs -0.14 single-touch.
Root cause of the 2026-05→07 identity-node loss: route_scan_edges and
route_sync serialized state by engram_save()ing over the canonical
snapshot.json on every GET, so one bad boot load meant the first read
request overwrote the good snapshot. Read routes now export to scratch
paths. Boot guard preserves evidence on non-empty-file/zero-node loads
and keeps a boot-time backup on good loads. New POST /api/load-merge
(explicit path required) used to restore 385 identity nodes + 1115
edges from the 2026-05-13 backup.
Three fixes that existed elsewhere but never reached the runtime the engram
binary actually builds against:
- tokenized + ranked query matching (search/search_json/activate seeds/
goal_bias) ported from the el-compiler copy (e3dabe3, 2026-07-14) — the
production engram kept whole-query Ctrl-F for 5 days after the fix
'shipped'. Multi-word curiosity seeds went 0 -> 36 activated. Kept the
ISE seed exclusion the el-compiler copy dropped.
- Knowledge -> 0.20 WM threshold after tier checks (dev-line 4bf7716):
Semantic/Episodic Knowledge nodes fell to the 0.40 note default and only
entered WM via breakthrough.
- goal_bias: Knowledge in is_knowledge + curiosity-seed technical terms
(dev-line d53516b).
Also: seed_epoch was a running pairwise average, not the mean it claimed —
exponentially over-weighted later seeds in the temporal-proximity bonus.
Fixed to a true int64-sum mean. Stale INHIBITION_FACTOR comment corrected.
Root cause captured as knowledge: two runtime copies + branch-per-fix
without merge discipline stranded the entire dev semantic layer (cosine
activation, embeddings) out of production. Reconciliation planned as P1.
1. engram_neighbors_json (release runtime): BFS frontier/visited strings were
el_strdup'd (arena-tracked) but manually freed, so el_request_end()
double-freed every one — SIGABRT in http_worker under load (2 prod crashes
today via /api/neuron/session/begin and /api/neuron/graph; reproduced and
verified fixed with ASAN). Introduced when porting from the dev runtime,
which correctly uses plain strdup. Third instance of the
arena-vs-manual-free class (after EngramNode 07-15 and idmap keys 07-16).
2. server.el: let-in-if scoping sweep — defaults assigned inside if-blocks
never mutated the outer binding, so /api/search and /api/activate always
ran with q="", created nodes got node_type=""/salience=0.0, edges got
relation=""/weight=0.0, and save/load with no path hit engram_save("").
Rewritten to the let-if-else expression form. /api/activate now also
rejects empty queries instead of wiping carried WM weights.
3. engram_activate: retrieval reinforcement (ACT-R base-level learning) —
nodes promoted to WM that survive both capacity caps now get
last_activated/activation_count updated, so frequently retrieved memories
decay slower than abandoned ones. Scoped to promoted-only to avoid
flattening dampening across BFS fan-out.
Two independent investigations, one runtime, complementary halves:
1. Leak (Jul 2, this machine): JsonBuf buffers returned via el_wrap_str
were raw malloc, never arena-tracked — every engram_*_json call leaked
its output unconditionally. Added jb_finish() arena-tracking across all
~30 return sites. Plus el_arena_push/pop per-tick bracketing support
for the soul's awareness loop (the loop ran outside any request arena,
so even correctly-tracked allocations were permanent — 7.5GB RSS in
under a minute at 1s tick).
2. Corruption (Tim's container soak, docs findings/container-migration):
stored engram node/edge fields (content, node_type, label, tier, tags,
metadata, from/to ids) were arena el_strdup — freed at request end,
leaving dangling pointers that read back as recycled request-buffer
bytes one request later. This is the June corruption root cause and
the mechanism that grew snapshot.json to 18GB of empty-type junk
(21.6M nodes, 3,335 real). 39 sites switched to el_strdup_persist,
plus a latent double-free fix in engram_load metadata fixup.
Interaction note: fix 1's per-tick arena reclamation makes fix 2
mandatory — more aggressive arena recycling widens the use-after-free
window if stored fields still live in the arena. Apply as a pair, never
separately.
Verified live: soul + engram rebuilt from this runtime, booted against
the recovered real snapshot (3,335 nodes/40,146 edges), 5h stable at
<100MB RSS, write-then-next-request field-integrity test passes (the
June corruption fingerprint does not reproduce). engram/dist/engram
binary updated from this build.
Investigation credit: leak diagnosis this machine Jul 2-6; corruption
diagnosis + persist-fix patch by Tim's instance (docs PR #4).
Runtime now includes engram_load_merge — soul daemon awareness.el calls
this function during its periodic sync refresh cycle. Binary rebuilt from
server.el (unchanged source) + updated el_runtime.c.
Port critical WM fixes from self-review 2026-06-26 branch (f7bd99a) that were
never merged to HEAD. Running binary had these fixes; source did not — rebuild
would have silently regressed all three improvements.
1. ENGRAM_BREAKTHROUGH_WEIGHT 0.25→0.10
With 0.25, naturally-promoted nodes (threshold ≥0.15) decayed below the
breakthrough floor within one activation call and lost their WM slot to
fresh breakthrough candidates. All 524/525 WM nodes were at floor = useless.
Invariant: BREAKTHROUGH_WEIGHT < min(type_thresholds = 0.15 Canonical).
2. ENGRAM_WM_CAP=24 with Pass 4 (per-call) + Pass 5 (global) enforcement
Without cap, broad curiosity seeds promote 500+ nodes simultaneously.
wm_avg_weight collapses, goal-bias differentiation is lost. Verified:
"knowledge" query now promotes exactly 24 nodes (was 525). Cowan (2001)
cognitive basis: WM capacity ~4 chunks; 24 allows rich multi-topic context.
3. ISE exclusion from WM (Pass 2 guard)
InternalStateEvent JSON content ("knowledge", "memory", etc.) triggered
lexical seeding → suppression accumulation → breakthrough at floor. ISEs
are observability-only and must never surface in context compilation.
suppression_count cleared so ISEs never build toward breakthrough.
4. route_create_ise importance fix (0.5→0.3)
Corrects mismatch between HTTP route and awareness.el in-process fallback.
Also adds body comment clarifying auth-exempt rationale.
SYNAPSE (arXiv 2601.02744) validates WM cap design and ISE exclusion principle.
Next priority: cosine similarity seeding to complement lexical BFS.
el_strdup tracks pointers in the arena. The BFS arrays in
engram_neighbors_json are manually freed — using el_strdup caused a
double-free when the arena was later popped. Changed to plain strdup
for those allocations.
engram/dist/engram.c rebuilt from engram/src/server.el with current
elc (minor codegen diff: parenthesisation and _argc/_argv rename).
Dharma's EngramDB client calls /nodes/list to retrieve all nodes.
Add this as an alias for the existing /nodes (and /api/nodes) route
so downstream clients don't need to be updated when the API drifts.
Also update dist/engram.c to match server.el.
When the query string includes node_type, we route to the new
engram_scan_nodes_by_type_json builtin instead of the unfiltered
scan. Existing callers without the param get identical behaviour.
Smoke-tested live on the neuron engram (3,200+ nodes):
?node_type=Knowledge → all Knowledge
?node_type=BacklogItem → all BacklogItem
?node_type=Imprint → 1 Imprint (only one cultivated so far)
?node_type=DoesNotExist → []
Engram is now a thin HTTP face over the El runtime's in-process graph
store. The C runtime owns the data; engram_*_json builtins serialize
results directly. There is no SQL, no SQLite, no db layer, no state
machine — the runtime IS the database.
src/server.el (348 lines, replacing 5797 lines across 15 legacy files):
GET /health
GET /api/stats
POST /api/nodes (auth required)
GET /api/nodes
GET /api/nodes/:id
DELETE /api/nodes/:id (auth required)
POST /api/edges (auth required)
GET /api/neighbors/:id
POST /api/activate
GET /api/activate
POST /api/search
GET /api/search
POST /api/strengthen (auth required)
POST /api/save (auth required)
POST /api/load (auth required)
Auth: ENGRAM_API_KEY in env. GET routes pass through (read-only).
Mutating routes require {"_auth": "<key>"} in the JSON body until
http_serve surfaces request headers and we can switch to Bearer.
Persistence: engram_save / engram_load via JSON snapshot at
$ENGRAM_DATA_DIR/snapshot.json. Loaded best-effort on startup.
Build: dist/platform/elc src/server.el > dist/engram.c
cc -std=c11 -O2 -I <runtime> -lcurl -lpthread -o dist/engram
dist/engram.c <runtime>/el_runtime.c
Live: native binary at dist/engram (113 KB), running under
~/Library/LaunchAgents/ai.neuron.engram.plist on :8742. Verified:
GET /api/stats returns counts; POST /api/nodes with auth creates
node with UUID; GET /api/search returns full node JSON; spreading
activation returns hop-decayed strengths (0.8 × edge × decay per
hop) with epistemic confidence filtering.
Legacy (5797 lines of SQLite-era src) sealed at
~/Archives/engram-src-legacy-20260430.tar.gz and removed from disk.
Memory is not stored and retrieved — it is activated and propagated.
Implements the spreading activation model with salience decay, typed edges,
four memory tiers, and flat cosine vector search over a sled embedded store.
will.anderson
Will Anderson