el/engram/README.md

Engram

A local-first memory substrate for accumulating intelligence.

An engram is the physical trace of a memory in the brain — the actual encoded substrate, not an abstraction above it. That's what this is.

Doc status (2026-08-16). Everything from "Implementation" down was rewritten against the code. The previous revision documented a Rust engram-core crate backed by sled, with a Cargo.toml, a crates/ tree, examples/basic.rs, and a EngramDb API. None of that exists. Measured: engram/ contains src/server.el, spec/, test/, dist/, manifest.el — zero .rs files, no Cargo.toml, no crates/, and sled appears nowhere in the tree outside two Old-English/Old-High-German vocabulary entries in elp/. The engine is C, in lang/runtime/engram_*.{c,h}; the server is El, in engram/src/server.el.

---

Why existing databases are wrong for this use case

Relational databases store rows and retrieve them by predicate. Key-value stores retrieve by exact key. Vector databases retrieve by geometric proximity. All of them share the same fundamental model: you store data in, you query it out. Storage and retrieval are separate systems.

The brain doesn't work this way.

When you remember something, you don't query your hippocampus. You activate a memory trace and the pattern propagates. Long-term potentiation — the strengthening of synaptic connections through co-activation — is simultaneously the storage mechanism and the retrieval mechanism. The structure that holds the memory is the same structure that surfaces it.

No existing database models this. Engram does.

---

The Spreading Activation Model

Engram retrieval works through spreading activation:

1. Seeds — you name one or more nodes you know are relevant (e.g. the current task, recent context, a concept you're reasoning about)
2. Query embedding — you provide a semantic vector representing the direction of your current thought
3. Propagation — activation flows outward from seeds through weighted edges, attenuating multiplicatively per hop
4. Pruning — paths weaker than a threshold are cut (the attention filter)
5. Return — the top-N nodes by activation strength

This is not a query. It is a pattern completion.

Activation conducts through well-grounded relations because weight is groundedness — see "Grounding is the weight" below. Nothing filters the traversal for grounded evidence; it falls out of spreading.

---

The Four Memory Tiers

Tier	Analogy	Contents
Working	Prefrontal working memory	K most recently activated nodes — hot, fast
Episodic	Hippocampus	Time-ordered events and experiences
Semantic	Neocortex	Concept graph — long-term structural knowledge
Procedural	Cerebellum / basal ganglia	Patterns, workflows, habits

Tier is a string field on the node (StoreNode.tier, engram_store.h), defaulting to "Working" on creation (el_runtime.c:8514, 8734).

---

Salience — Forgetting as Adaptation

Salience decays from three signals — importance (set at creation, stable), recency, and a log-compressed activation frequency. Base-level learning keeps a ring buffer of the last STORE_BLL_K (= 10) access timestamps per node (engram_store.h:29).

Forgetting in Engram is not a bug. It is adaptive pruning. Unreinforced memories stop competing for attention without being deleted.

Immutability. Nothing is mutated and nothing is hard-deleted: writes are additive, corrections are supersessions, removals are tombstones. The predecessor is always present, which is what makes supersession an audit trail rather than an edit log.

---

Implementation

Part	Language	Where
storage engine, graph, activation, geometry, cognition	C11	lang/runtime/engram_{store,geometry,reason,cognition,verify,vindex}.{c,h}
HTTP server + routes	El	engram/src/server.el (2043 lines)
build artifact	generated C	engram/dist/engram.c
tests	shell + C	engram/test/

On-disk format (engram_store.h): a paged store — superblock plus mirror, slotted 16 KiB pages, self-describing TLV records, overflow chains, and two B+-tree indexes (primary id → loc, adjacency from_id/to_id → edge locs) over a free-listed page file. Magic ENGST01, format version 1. The TLV scheme means new fields never force a migration.

---

The vector index is published, not guarded

Vector search is an HNSW (Hierarchical Navigable Small World) index — lang/runtime/engram_vindex.{c,h}. The previous revision of this README claimed a "flat cosine scan… until retrieval quality at scale demands" HNSW. That is no longer true, and the reason it changed matters more than the fact.

eg_vindex_sync used to exist: a function that repaired the index from read paths. All three of its callers were reads (engram_activate, eg_knn_for_node — whose own header comment said "No writes." — and engram_geo_reify_run_json), and it mutated five process-global statics. Reads mutated because index maintenance had never been given an owner on the write side.

It is now split (el_runtime.c:10121, 10137, 10151, 10161):

• eg_vindex_maintain — the sole mutator. Takes the boundary exclusively; never runs beside a reader.
• eg_vindex_view — returns a const VIndex* with the boundary held for read. N readers project concurrently; none can mutate. Paired with eg_vindex_view_release on every path including error returns.
• eg_vindex_note_embedded — the write-side owner. Index membership belongs to the event "an embedding became present on this ordinal," not to node append: a node without an embedding cannot be in a vector index at all. One O(log n) insert, no O(node_count) presence scan.

Two things carry the discipline, and neither is a review habit:

• const is the capability. The per-search visited / visit_epoch scratch left struct VIndex and went back into the call frame where it belonged — it was one traversal's local, hoisted into the struct as an allocation optimisation, never derived geometry. Once it was gone, vindex_search could take a const VIndex*, so a read path physically cannot call vindex_insert, and it is a compile error rather than a comment. The capability type was already in the language; it is spelled const.
• Publication, not ownership. HNSW insert is not an append: vindex_insert rewires the NeighList links of already-existing elements and reallocs elems[]. The store's append-only property does not transfer to an index derived from it, which is why purity alone was insufficient and a view/maintain boundary was required.

Measured (engram/test/run_vindex_concurrency_tests.sh, 2026-08-16):

half	before	after
single — 3000 vectors, 1 thread, ASan+UBSan	clean	clean
readers — 4 readers, no writer, TSan	race at engram_vindex.c:195	clean
unsynchronized — writer+reader, bare index, TSan	race	race, expected and permanent — the proof the boundary must exist
published — owner + 4 readers through the boundary, TSan	(did not exist)	clean, all 3000 inserts landed

recall@10 = 0.9365 at ef_search=128 (gate ≥ 0.90); the determinism test still yields byte-identical results across two independent builds.

Not yet done. The resident RAM graph (g->nodes / g->edges) is a separate instance of the same defect and has not received this treatment — it is realloc'd in place, so a reader holding EngramNode* n = &g->nodes[i] across a concurrent append holds a dangling pointer. Until it gets the same publication boundary, the fb32d15 request guard stays. Full argument: ../lang/spec/runtime-ownership.md.

---

Cognition

The cognition surface is live over lang/runtime/engram_cognition.{c,h}, routed in engram/src/server.el.

route	method	what it is
/api/think	GET	the read: a warped traversal-read of the seed region, returning a gradient (direction + spread + calibrated confidence), never a point
/api/reason /api/induce /api/abduce /api/relate /api/analogize /api/plan	GET	named faculties — see the correction below
/api/ground	POST	grounding between a claim and evidence
/api/assert	GET	the honesty floor, queried at assertion time only
/api/attend	POST	salience as a relation (salient-to), grounded-for-whom
/api/correspondence-beat	POST	one calibration beat against outcome

Anchor the read, or every faculty returns the same null

engram_think_json passed NULL as the anchor. NULL is not "no opinion" — engram_think re-origins at anchor ? anchor : region->centroid, and the centroid is the one point where the gradient is zero by construction: r = x − centroid = 0, so every axis projection is 0 and direction takes the at-rest branch.

Measured consequence: every faculty — reason, abduce, induce, plan, analogize — returned an identical null result differing only in its label:

{"direction":[0,0,...],"spread":0,"magnitude":1,"confidence":0.5}

magnitude: 1 is membership evaluated at the centroid; spread: 0 is its distance to itself; confidence: 0.5 is the stance fallback. The geometry was never the problem — /api/drift computed real values (centroid_sep 0.104, core_disp 0.045) over the very same 87 members. Fixed in #141/#142: the read anchors at the first resolvable embedded seed, copied not borrowed (g->nodes is realloc'd in place on append). Gradients now vary by seed.

The learned stance is resumed, not discarded

engram_think_json also built a neutral stance every call — all axis_gain 1.0, bias_dir NULL, reliability 0.5 — and never loaded the one the correspondence-beat had been persisting under stance-<faculty>-<hub>. Every beat's calibration was written and then thrown away on the next read.

Fixed in #146: think resumes the same id the beat writes, so learning compounds across beats and cold boot, and the response now carries stance_resumed so an informed confidence: 0.5 is distinguishable from an uninformed one. On a calibrated region, confidence went 0.5 → 0.930726.

Signal can enter as geometry

Until 2026-08-16 no El ingest path could carry a vector: nodes took text and geometry was derived from that text. Text was the mandatory entry medium, so any non-text modality had to be described in prose first — and the geometry being reasoned over was the geometry of the description, not of the signal. #141/#144 ended that. See ../lang/spec/language.md §20 for the Geometry type, realizers, and transduce.

---

Corrections — read these before extending the cognition surface

Authority: lang/spec/correspondence-and-censorship.md (design branch design/correspondence-and-censorship, PR #149) and lang/spec/runtime-ownership.md. Do not re-derive them; several earlier versions were wrong and each correction was argued down.

Grounding is not a subsystem. It is the weight.

Grounding is an attribute of the edge, and it is the hebbian weight. One quantity, not two fields. A relation that keeps holding up strengthens; one that stops corresponding decays — that is not analogous to grounding, it is grounding.

Consequences:

• There is no grounding subsystem to build. The graph already is the grounding structure.
• grounded-by as a relation type should not exist. It models grounding as a relation between nodes when it is a property of a relation. Minting an edge is the error, not merely which endpoints it chose.
• Grounding is never computed on demand. An operation may read the grounding of a path; computing-and-writing a score makes reads write, which is exactly the eg_vindex_sync defect one level up.
• Traversal is already grounded inference. Nothing needs filtering.
• Decision provenance is the path, not a log. A log records the action; the path records the meaning under which it was taken.

Known wrong shape, in the code today. COG_GROUNDED_BY_RELATION "grounded-by" (lang/runtime/engram_cognition.h:158) and cog_ground_edge (engram_cognition.c:249) still exist and still mint an edge. #147 fixed ground's honesty — it now grounds the node asked about rather than the region hub, reports claim_region/evidence_region separately, and refuses three shapes of circular support (same-region, claim-region-is-evidence, evidence-region-is-claim) instead of returning a confident 1.0. That corrected a scalar rather than deleting the operation. Deletion is sequenced, not done.

Faculties are operations, not parameters

• reason changes the estimate — a read.
• induce changes the parameters — the correspondence-beat, which already exists and measurably works.
• abduce changes the structure — a write, which the current GeoGradient signature cannot express.

Known wrong shape, in the code today. engram/src/server.el:1870–1886 routes six faculties into one call with a string argument — route_faculty(path, "reason"), ("induce"), ("abduce"), ("relate"), ("analogy"), ("plan"). Underneath, engram_cognition.h:8–11 states the theory explicitly: "the named faculties … are human LABELS on regions of think's steering space: each faculty == { think + a named stance }." The faculty name enters engram_think only through the stance, and cog_stance_init stores it while nothing reads it — so before #146 all five were byte-identical (el_runtime.c:14352–14359). A write cannot be a parameter of a read; abduce in particular is not expressible this way.

Wonder is the boundary; curiosity is wonder crystallized

Wonder is where structure ends — where activation spreads and finds thin or absent geometry. Any structure at all has an edge, necessarily, the moment it exists. It is not a manifest of open-question nodes to maintain, and a "wonder-manifest manager" materializes a property as a stored artifact — the same disease as a grounding subsystem, or a self stored as a document.

There are about six wonders, they are the same for everyone, and they never close: What is this? / Why? / Who am I? / Am I alone? / What should I do? / What happens when it ends? "Why" is the first and the only one; the others are it asked of particular things. Each already lives somewhere in the substrate — "why" is grounding, because the weight is the answer to why.

Curiosity is not a second object. Wonder and curiosity are one thing at two phases: wonder is the field (unbounded, objectless, invariant); curiosity is the precipitate — the same wonder localized, having taken definite form against particular material at a nucleation site. This is why curiosity can be satisfied and wonder cannot. It is also why abduction needs no trigger and no threshold: a structurally_unanticipated observation is a nucleation site.

co_registration is deprecated — the disagreement belongs on the edge

GeoDescriptor.co_registration — corr(hebb strength, semantic proximity) over internal edges — has always been computed, always persisted, and never read. It is also the wrong shape: whether use and meaning agree is a property of each edge, and a correlation averages that per-edge property into one scalar per region. A region holding one violently disagreeing edge beside one violently agreeing edge reports ≈ 0 — the disagreements cancel, and the summary destroys exactly what it was built to reveal.

Measured: 375 live reified neighbourhoods — 340 positive, 31 at zero, 4 negative. Read as a count of things to be curious about, that says "four." Read correctly, four disagreements were lopsided enough to survive averaging and the 31 zeros are where opposing sites cancelled.

The replacement is per-edge (engram_geometry.h:43–47, engram_geometry.c:454–473):

discord = z(semantic proximity) − z(association strength)

standardized within the region from accumulators the aggregate loop already gathered — no second statistic, no constant, no threshold. discord > 0: near in meaning yet unlinked by use. discord < 0: linked by use yet far in meaning. Both are surprising, and |discord| is the nucleation strength.

Do not scan for nucleation sites. Once the signal was a per-region number the only way to find sites was to enumerate regions, which is why surfacing curiosity looked like a search problem. Nothing in a mind scans its neighbourhoods to find what is surprising — the surprise captures attention. With the disagreement on the edge there is nothing to scan.

co_registration is deprecated rather than deleted only because it is embedded in the persisted GEO1 blob; removing it is a format migration and must not ride along. Nothing new may read it.

Consolidation is ambient, not scheduled

A brain has no cron job. Boredom is not an absence and not leftover capacity — low activation is aversive and the system self-activates. There is one activation process with two seed sources: external (a request) and internal (a curiosity). Spreading is bounded; it settles; then it needs a new seed. Nothing waits on capacity, nothing polls, nothing checks a clock, and there is no dreamer thread.

The presence of a ticker is the diagnostic. Every StartInterval, every Hour/Minute, and every POST-to-beat marks a place where an intrinsic rhythm was replaced by an external clock.

Consolidation currently has ten implementations (measured 2026-08-16). Three of them are POST beats on this server — /api/tick (server.el:1947), /api/correspondence-beat (1897), /api/self-reify-beat (1836) — and a POST beat puts a supervisor back in: something outside decides when Neuron consolidates. soul.el's continuous in-process loop is the one fragment with the correct shape; the rest fold into it. Full table in lang/spec/correspondence-and-censorship.md §7.

Immutability already refuses what a guard would refuse

In an immutable substrate, any mechanism that refuses a write is either redundant with immutability, or an epistemic constraint misfiled as a protective one.

This resolves keystone_write_blocked (CogStance.keystone, engram_cognition.h:83) rather than replacing it. "Keystone" means load-bearing, not precious: the self anchor is the reference frame every other stance calibrates against, and a reference fitted to its own readings reports perfect correspondence forever while drift becomes undetectable from inside. The real requirement is non-circularity of the reference frame, and that is satisfied temporally — the frame updates while activation is internally seeded, not while it is being used to act. Independence is when, not what. Corruption requires mutation, and the engram does not mutate; recoverability, governance, evidence quality, and rate all fall out of the substrate. Authorization is the only residue, and it is bounded: an unauthorized writer can propose, never erase.

---

Design Decisions

Why multiplicative activation? Because memory is conjunctive. A path requires all of its links to be strong to carry signal. Addition would let many weak associations accumulate into false relevance.

Why salience decay? Because not everything that was once important remains important. A memory system that never forgets is one that can never focus.

Why supersede instead of update? Because provenance is the point. The old edge never leaves and the values frame does not fit to outcomes, so a decision cannot be made to look justified after the fact. It makes an otherwise impossible distinction available: wrong then, or wrong since.

Why publication instead of locking? Because what does not mutate needs no ownership discipline. The question "who is permitted to mutate the shared thing?" presupposes a shared mutable thing; for the store there isn't one, and for the index derived from it the answer is a publication boundary, not a capability ABI.

---

Specs

• ../lang/spec/runtime-ownership.md — ownership, the capability ABI that was dissolved, and the vector-index publication boundary
• ../lang/spec/correspondence-and-censorship.md — grounding, wonder, curiosity, dreaming (lands with PR #149)
• spec/cognitive-architecture.design.md — the original one-operation design. Superseded in part — see its header
• spec/architecture-hardening.design.md, spec/engram-el.md, spec/at-rest-encryption.md, spec/engram-db-tooling-design.md