Extends engram_act_stats_json with five monotonic counters for the raw incoming
signals the mind receives — aff_activations (spreading activations run),
aff_queries (activate_json entries), aff_node_creates, aff_ise_ingests (ISE
nodes), aff_edge_creates. Module-scope statics incremented at the entry points,
emitted via the existing act-stats mechanism and rotated with it — MEASURED, and
never accreted as memory nodes. Process-lifetime totals, reset on restart like
the other _eg_act_* gauges. Additive JSON fields (backward-compatible); no graph
behavior changes. act-stats buffer grown 896->1088 to hold them.
MEASURED on a copy: after 5 node creates (2 ISE), 2 edge creates, then 4+3
queries — counters read exactly node_creates=5, ise_ingests=2, edge_creates=2,
queries=activations=4 then 7; create counters unchanged by queries; queries
strictly monotonic across readings. ASan+UBSan clean.
The felt passage of time is the cooling of the activation field, not a tick
count and not an elapsed-seconds readout. engram_age_field(delta_ms) cools the
field (working_memory_weight + background_activation) by the caller's MEASURED
wall-clock delta with a pure exponential exp(-dt/TC) — no per-call floor — so it
is exactly scale-invariant: N ticks summing to the same elapsed time produce the
same total cooling. It returns the cooling MAGNITUDE (1-exp(-dt/TC), a bounded
[0,1) drift signal), never elapsed seconds.
Reboot = anesthesia: a global last-tick wall-clock stamp is persisted to a
sidecar (chrono_last_tick) in the data dir. engram_age_field_catchup() reads it
on boot, applies ONE cooling for the whole unconscious gap, refreshes the stamp,
and reports the magnitude — timestamps are bookkeeping to COMPUTE the drift,
never the felt signal. TC env-tunable via ENGRAM_CHRONO_TC (default 3600s).
All inert unless ENGRAM_CHRONOCEPTION is set → OFF path byte-identical.
MEASURED on a copy (throwaway HOME, TC=3600s):
- Cooling scales with dt, matching 1-exp(-dt/TC) to 1e-6: dt=600s->0.1535,
1800s->0.3935, 3600s->0.6321, 7200s->0.8647.
- Scale-invariance EXACT: age(dt) once vs age(dt/N) N times gives identical
field sum (|delta|=0.0) for N=2, 10, 100.
- Reboot catch-up over a 1h gap cooled the field 0.60->0.4415 in one shot,
magnitude 0.6321, bounded in [0,1).
- Flag OFF: age & catchup return 0, field untouched (sum 1.2).
ASan+UBSan clean.
The co-activation accrual already works (hebb is an EWMA over co-firing); what
was missing is the promotion layer from design §9 that turns accrual into
durable structure. This adds it on top, entirely behind ENGRAM_CONSOLIDATION so
the OFF path is byte-identical to trunk.
CONNECTION threshold ("connection IS consolidation"): when a strongly-firing
InternalStateEvent is created (salience >= ENGRAM_CONSOL_CONN_MIN, default 0.6),
wire hebbian-associate edges from it to the top-K working-memory nodes active at
that instant (ENGRAM_CONSOL_WM_TOPK, default 5), provenance-tagged
"consolidated-from-ISE" and dedup-guarded. A sub-threshold ISE forms nothing (a
shower thought) and drifts out at the existing 48h prune.
PERMANENCE threshold (rare): engram_consolidate_permanence(node) marks a node
whose rehearsed ACT-R base-level clears ENGRAM_CONSOL_PERM_MIN durable via a
reversible metadata marker; engram_prune_telemetry then exempts it (gated, so no
trunk node is ever affected). Idempotent, no double-promote.
Thresholds are env-tunable for A/B without a rebuild.
MEASURED on a copy (throwaway HOME):
- Headline accrual curve (flag OFF, pure trunk) over N co-activations of a wired
pair — hebb_max: N=1 -> 1e-4 (=ETA), N=100 -> 0.010, N=1625 -> 0.150
(LINK_MIN, where an unwired pair consolidates), N=3000 -> 0.259; tracks the
analytic EWMA 1-0.9999^N within measurement noise (co-activation P~1).
- Strong ISE wired 2 edges to exactly the wm_top nodes (hebb-a, hebb-b); weak
ISE formed 0; edges carry the reversible provenance marker.
- Promoted node survived the 48h prune (2->1 nodes); ephemeral ISE swept.
- Flag OFF: ISE creation added 0 edges, permanence returned 0 (no-op).
ASan+UBSan clean.
Read routes (GET /api/embeddings, /api/graph/dump) need node embedding
vectors, but every consumer emit path deliberately drops the ~5.7KB emb
vector (include_emb=0) to stay under MCP token limits. Rather than perturb
that shared path, add a dedicated additive builtin that pages nodes WITH
their dense vector, emitting id/node_type/label/created_at/emb_dim plus emb
as a JSON array whose length equals emb_dim (so a consumer can verify the
vector round-trips). Un-embedded nodes emit emb_dim:0 / emb:[]. Same
salience-sorted, transparent-layer-skipped, bounded pagination as
engram_scan_nodes_json; default page 256.
Purely additive: engram_emit_node_json and the default include_emb=0 are
untouched, so every existing endpoint is byte-identical to trunk (verified:
scan_nodes_json still carries no emb). The HTTP route wiring in server.el is
DEFERRED to cutover per the elc-drift blocker (regenerating engram/dist
diverges ~285 lines with no source change); the builtin is exercised
directly by the pure-C gate instead.
Measured on a copy: len(emb)==emb_dim for all nodes, pagination disjoint,
existing path unchanged; full 256-node x 768-dim page = 16.7 ms / 1.18 MB.
ASan+UBSan clean.
Reification, not a cache. Densely co-wired relational neighborhoods are crystallized
into DURABLE first-class store records that survive restart, load on boot, and evolve
via supersede+provenance -- so the geometry-priming hot path READS persisted structure
instead of computing a per-query descriptor (the M9 3.2x/13x latency blocker).
engram_geometry.{h,c}:
- engram_geo_reify_store(): detect hub-anchored neighborhoods on the hebb-weighted
graph (greedy non-redundant cover), compute each centered descriptor ONCE against
the true store-wide mean, persist as node_type="Neighborhood" (raw centroid in emb,
membership+scalars+axis-extents in a compact GEO1 metadata schema) + member edges,
superseding any prior same-hub record. The mean is persisted once as "GeoMeanFrame".
- resident loaded form (index_new/add/finalize/lookup): parses the durable records at
boot (never recomputes geometry); O(seeds) membership lookup, miss -> centroid-nearest.
- descriptor: skip the Jacobi eigensolve when top_axes==0; reject structural records as
members (id-convention + node_type guards) so re-reify/ad-hoc stay clean.
el_runtime.c:
- boot routes Neighborhood/GeoMeanFrame records OUT of the activation graph into the
reify index, and skips member edges from adjacency -> ENGRAM_GEOMETRY_PRIMING OFF is
byte-identical to M8/M9 (verified across 15 queries).
- priming hot path reads the persisted membership; ENGRAM_GEO_PRIMING_NOCACHE=1 keeps
the M9 per-query descriptor for ad-hoc geometries / A/B control.
A/B on a COPY (128 neighborhoods): priming ON is now FLAT latency (1.06x median / 1.03x
p90 vs OFF) where the M9 per-query path is 3.30x/10.5x. Reified records provably inert
when OFF. Restart survival + supersede verified. Build 0 warnings (my code); ASan/UBSan
clean on module and full server hot path. Recall quality re-eval against the TRUE mean
still shows no reliable gain (mean coherence -0.017), so geometry-priming STAYS default-OFF
-- but the latency blocker is removed and the durable structure now exists. See
docs/architecture/design/engram-m10-reification.md.
Wire the centered relational-neighborhood geometry (engram_geometry.c) into
activation seed selection behind a reversible env flag that defaults OFF. Flag
unset => byte-identical to M8 (verified: identical result id sequence + order
across 15 queries vs the M8 baseline binary). When set, composes with M8's ANN
candidate generation: damp-only seed reweight by centered membership
(disambiguation) + sub-threshold neighborhood priming (warm floor below the WM
gate, capped, ISE-skipped). Safe because the BFS keeps the max, so priming only
raises a floor and can never cap a legitimate activation.
Descriptor + global mean run over the paged store; the resident-array vindex is
bridged with a vids[] map. Tunables via env (SEED_LO/PRIME_SCALE/PRIME_MAX).
Default-OFF binary is behavior-neutral and safe to deploy. Enabling the flag is
currently NO-GO on cost/benefit: 3.2x median / 13x p90 latency for no reliable
coherence/disambiguation gain (see perf profile). Not a defect - WM cap holds,
no crash, ASan/UBSan clean.
Persistent process-lifetime HNSW index (engram_vindex) over resident node
embeddings, node_id == resident g->nodes[] index. Built lazily on first
activation, grown incrementally as appended nodes get embedded, rebuilt on
emb-dim change or resident-array shrink.
Seed selection queries the ANN for the nearest embedded nodes to the effective
query vector, replacing the O(K*N) exact argmax scan as the DISCOVERY step.
Each ANN candidate is admitted through the identical gate the exact scan uses
(exact cosine >= SEED_MIN via cosq, reached/dup skips, content dedup, same
decay/dampen shaping), so scoring/dynamics are unchanged. The exact argmax
scan is preserved verbatim and tops up any unfilled seed slot, and runs in
full when the index is empty/too-small/unavailable -> pre-M8 behaviour exactly.
cosq (O(N*dim) cosine fill) is intentionally retained: it still feeds the
propagation qgate and the Pass-2 WM term (activation dynamics, out of M8
scope). ANN accelerates SELECTION only.
Spreading activation rebuilt the entire per-node adjacency index
(engram_adj_rebuild, O(E)) lazily before every BFS whenever any edge/node
was added — so a curiosity-loop query that touched a small frontier still
paid to rebuild the whole edge set. This makes the index incrementally
maintained behind ENGRAM_STORE: single node/edge creates APPEND to the live
adjacency in amortized O(1) instead of marking it dirty, so a query only
pays for the frontier it touches (one initial O(E) build, then O(1)/edge).
Approach (b), not (a): the store's from/to adjacency B-tree was rejected
because with the store on the whole graph is already resident and activation
reads in-RAM edges, whose hebb/weight only sync to the store at checkpoint
cadence — reading StoreEdge copies would use stale weights and break
byte-identical parity. The incremental in-RAM index reads the exact same
g->edges[ei] the scan path does, so activation is identical by construction.
Correctness: edges are only ever appended, so incremental append reproduces
the rebuild's ascending-edge-index ordering exactly (same skip rule for null
endpoints). Any index-invalidating mutation (forget/prune/clear) still frees
the index + sets adj_dirty=1, falling back to a full rebuild. Flag-off is
untouched: the mutation hooks just set adj_dirty=1 as before — proven
byte-identical.
engram_store.c is NOT modified (avoids the M5 compaction collision).
Tests (plain gcc, ASan/UBSan clean): engram/test/{test_m7_traversal.c,
run_m7_traversal.sh}. Parity gate proves flag-on incremental == flag-on
forced-full-rebuild == flag-off scan, byte-identical on a mutating query
sequence (activated set, weights, ordering, hops, WM promotion). Perf on a
13k-node / 43k-edge graph over 120 (add-edge + activate) iterations:
adjacency edge-touches 5,167,260 -> 43,001 (120x fewer), full rebuilds
120 -> 1, adjacency-maintenance wall-time 0.74s -> 0.006s (~121x). Prior
gates green: M1 store (33), M2 (36), M3 parity, M3.5, M4 bufpool (37).
Resolve 3 conflicts:
- lang/el-compiler/runtime/el_runtime.c: keep deletion (deprecated runtime fork;
single-source-of-truth is lang/runtime/, enforced by scripts/check-single-runtime.sh).
- lang/releases/v1.0.0-20260501/el_runtime.h: keep deletion (releases/ is a generated
artifact folder, not a source path; a release is a git tag, not a folder).
- lang/runtime/el_platform_win.h: union of dev's Windows port (#80: setsockopt optval
wrapper + curl-less libcurl stubs) and our fsync(->_commit) shim needed by engram_store WAL.
Nothing in dev's build consumes the deprecated fork or releases/ folder.
- Wire paged store into the ENGRAM SERVICE (server.el, the authoritative durable
owner): boot->engram_store_boot, persist_canonical->engram_store_checkpoint,
gated by ENGRAM_STORE.
- elc (lang/elc.c + src/parser.el + codegen.el + elc-combined.el): OOB guard in
tok_kind/tok_value + parse_block progress backstop — fixes the pre-existing
unbounded-memory fold hang on sessions.el.
- engram_prune_telemetry mirrors ISE prune to the store (store_forget) so store
live-count tracks resident and stale telemetry stays bounded.
- Deployed live 2026-08-12: engram :8742 on neuron.egm+WAL, count reconciled 11552.
Flag-on checkpoint now full-walks the resident graph: store_put_node (WM weight,
activation_count, last_activated, wm_anchor) + store_put_edge (hebb, last_fired)
for every node/edge, then engram_checkpoint. Uses store_put_edge (idempotent
upsert) not store_hebb_batch, because activation FORMS new hebbian-associate edges
that bypass the create hook and delta-only hebb_batch can't create them. Store-on
boot now applies the same WM-halving + floor + cap transforms as engram_load.
This is the hebb-survives-restart fix. Gate: reboot from neuron.egm with
snapshot.json deleted -> edge hebb + activation_count survive unchanged, WM weight
survives with identical boot transform; negative control proves persist is
load-bearing (hebb->0 without it). M1 33/33 + M2 36/36 + M3 parity PASS, ASan/UBSan
clean, flag-off untouched. Engine unchanged (boundary held).
Caller-side shim in el_runtime.c maps EngramNode/Edge <-> StoreNode/Edge; engine
keeps zero soul deps (libengram boundary, design §10). Flag off = today's JSON
path byte-for-byte (proven: no neuron.egm created, graph identical). Flag on =
engram_open (import snapshot.json once into neuron.egm, else WAL-replay) +
resident load; node/edge create + forget dual-write via guarded hooks. Files
renamed engram.store->neuron.egm, engram.wal->neuron.wal.
Gate: M3 parity PASS (graph on==off byte-exact modulo ordering; snapshot round-trip;
reboot-from-egm with snapshot.json deleted; activation set+sequence identical;
ASan/UBSan clean). M1 33/33 + M2 36/36 green post-rename.
Known gap (pre-flip): in-place hebb/WM/activation_count updates during activation
are not yet persisted to the store (create/connect/forget are). Must close before
live flip so learned edges survive restart.
Establish lang/runtime/ as the ONE canonical el runtime (from the active
runtime that carries hebb/emb persistence + the new WAL); repoint the el CI
publish, engram build, elb default, and in-repo build scripts to it; delete
the el-compiler/runtime + lang/releases/ forks; add scripts/check-single-runtime.sh
drift guard.
Fixes a live prod bug: the el CI published el-runtime-c/-h from the LAGGING
el-compiler fork (0 hebb refs), so the shipped soul never persisted Hebbian
edge weights — learned co-activation was wiped on every restart. Publishing
from canonical ships the stranded 'learning that cannot outlive the process'
fix.
WAL storage engine + integrity fixes (DELETE->tombstone + store-layer
protection, safe data-dir default) ride in behind ENGRAM_WAL (default off =
byte-identical to today). Verified: engram elb per-module build clean, WAL
gate 66/66, native smoke ok, drift-guard green.
will.anderson