neuron/docs/architecture/design/perf/engram-m8-profile.md

Engram Tiered Storage — M8 Performance Profile (milestone-0 sample)

Milestone: M8 (ANN wired into engram_activate seed selection). Trunk = worktree /tmp/engram-tiered-wt, branch engram-tiered-storage, HEAD 1507614. Date: 2026-08-12. Author: first full-binary build + profile of the tiered trunk.

This is sample zero of an accumulating per-milestone profile series (see the BUILD-LEDGER "keep performance telemetry CONTINUOUSLY" decision). Compare future milestones (M9, M10, …) against these numbers.

Methodology (read this before trusting a number)

• On a COPY, never live. All runtime measurements used a read-only copy of the live store booted on a non-live port (:8798) with a throwaway $HOME. The live engram service (:8742, ~/.neuron/engram) was never touched.
• Two store substrates were used:
  1. Live-egm copy (neuron.egm 458 MB + neuron.wal 44 MB, copied read-only) — this substrate crashes both the M8 binary and the live binary on boot (see Integration Findings). Unusable for runtime measurement.
  2. Clean import — a dir seeded with only snapshot.json (65 MB, stable 06:10), which the binary imported into a fresh 59.9 MB neuron.egm. All healthy runtime numbers below are from this substrate (real graph content, healthy store).
• Build machine: Apple Silicon (arm64), macOS. Native cc -O2 compile; fold done in a memory-capped (--memory=3g --memory-swap=3g, no swap) linux/amd64 container running elc-linux-amd64.
• Hardware/thermals uncontrolled; single run per metric unless noted. Treat as order-of-magnitude, not benchmark-grade, except the module benchmark (Test suite).

Build

Metric	Value
Fold input	engram/src/server.el (44,273 B El, no imports)
Fold output	engram.c (30,855 B, 645 lines C), ELC_EXIT=0, 0 fold warnings
Fold time (pure elc)	sub-second (server.el is small, importless)
Fold container wall	~38 s (dominated by one-time apt-get install libcurl4 in the throwaway container; the elc invocation itself is <1 s)
Compile	cc -O2 -DHAVE_CURL engram.c el_runtime.c engram_store.c engram_vindex.c -lssl -lcrypto -lcurl -lpthread -lm
Compile time	1.83 s wall
Compile warnings	3, all -Wparentheses-equality in the folded engram.c (El if-expr codegen emits if ((x == 0))); cosmetic. el_runtime.c / engram_store.c / engram_vindex.c compiled 0 warnings — notably none around the M8 deferred free(e_eff) or the vindex integration.
Binary	482,008 B (471 KB) Mach-O arm64 executable
ANN linkage verified	nm: vindex_search, vindex_build_from_store, eg_vindex_sync, _eg_vindex, store_scan_nodes, engram_store_boot, engram_activate all present

Boot & footprint (clean-import substrate)

Metric	Value
Boot from healthy neuron.egm	~2 s to listening
Boot from snapshot.json (one-time import + fresh egm)	~9 s
node_count	13,036 (matches ledger import-dedup: 13,038 snapshot − 2 dup-id entries)
edge_count / layer_count	43,402 / 5
embedded_count	4,190
Fresh egm size	59.9 MB (vs the live egm's bloated 458 MB — see Findings)
RSS after boot	126.2 MB (whole 13k-node/43k-edge/4,190-emb graph resident + fresh egm)

Activation latency — q="bullshit" (clean-import substrate)

50 sequential GET /api/activate?q=bullshit&limit=10&depth=3:

Metric	Value
p50	34.35 ms
p95	35.26 ms
min / max	33.50 ms / 9,886 ms
Sample	n=50

• The max = 9.9 s is the first call only — a cold query-embedding fetch (eg_embed_fetch → Ollama nomic-embed-text, cold model load). All subsequent calls hit the single-slot query-embedding cache (_eg_qcache) → 34 ms steady state.
• This 34 ms is the lexical/spread path, NOT the ANN seed path. The store's 4,190 embeddings were generated by the live neuron's native embedding model; the harness's nomic-embed-text query vectors are a different vector space, so no candidate cleared ENGRAM_EMBED_SEED_MIN=0.60 (act-stats: dup_seeds:0, ctx_cos:-2.000 sentinel) → results empty, ANN discovery produced no admitted seeds. Environmental (embedding provenance), not an M8 defect. The ANN wiring still executed (embed fetch succeeded, embed_breaker_open:0; eg_vindex_sync + seed block + free(e_eff) all ran) without crashing on the real store.

KEY M8 METRIC — ANN vs O(n) seed selection (authoritative)

Because the HTTP path can't exercise ANN seeding without embedding-space parity, the authoritative ANN-vs-exact-scan numbers come from the module benchmark (engram/test/run_vindex_tests.sh, PASS 1, optimised), which measures the exact vindex_search code the M8 wiring calls, at full size:

N (768-dim vectors)	Brute-force (O(n))	ANN (HNSW)	Speedup
5,000	3.475 ms/query	0.353 ms/query	9.8×
20,000	13.809 ms/query	0.698 ms/query	19.8×

• recall@10 = 0.9365 at ef_search=128 (gate ≥0.90 — PASS). Lower ef trades recall for latency: ef=64→0.844, ef=32→0.750, ef=10→0.625.
• Determinism: two independent seeded builds give byte-identical query results.
• vindex_build_from_store over a real engram_store: inserts exactly the embedded nodes, top-1 resolves to the correct node id at ~0 distance.
• HNSW build cost (single-threaded, note for boot/index-build budgeting): 5,000 vectors ≈ 15–35 s, 20,000 vectors ≈ 75 s. In-process the index is built lazily on first activation (eg_vindex_sync) and grown incrementally; the M8 seed block only fires once vindex_size ≥ ENGRAM_EMBED_SEED_K. At the real store's 4,190 embedded nodes this is a one-time few-second first-activation cost — worth watching as the embedded set grows (a future milestone may want to build the index at boot or persist it via vindex_save/vindex_load).

Seed-set parity note (why there is no runtime A/B toggle)

M8 has no ANN on/off env flag by design (ENGRAM_EMBED_SEED_K is a compile-time constant). The exact O(n) cosine scan is preserved verbatim and "tops up" any seed slot the ANN leaves unfilled; every ANN candidate is admitted through the identical cosine/dedup/threshold gate the exact scan uses. So ANN changes only which nodes are discovered and how fast, never the final seed set — parity is structural, not A/B-tested via flag. Recovered-behaviour-when-index-absent is the pre-M8 exact scan.

Integration Findings (first full build of the tiered trunk)

1. CRITICAL / pre-existing (NOT M8): btree_insert stack-buffer-overflow on opening the live 458 MB neuron.egm. SIGABRT (__stack_chk_fail) via btree_insert ← btree_put ← edge_place/apply_edge_put ← engram_open ← engram_store_boot. Reproduces on the M8 binary and the deployed live binary — the live engram :8742 was crash-looping (18 crash reports 16:18→17:44 on this date; service refusing connections). A clean import into a fresh 59.9 MB egm does not crash (43k edges load fine), so the trigger is the specific pathological live store: 458 MB (8× the healthy 59.9 MB) from a day of churn/tombstones with no M5 compaction + a 44 MB un-checkpointed WAL replayed on open. The bug is in engram_store.c (the edge B-tree / WAL-redo path), upstream of everything M8 touched (M8 lives in el_runtime.c::engram_activate). Fix required before any re-cutover; btree_insert must bound-check regardless of on-disk content.
2. M8 deferred free(e_eff) (the flagged memory-management concern): compiled warning-free, and the wired path executed end-to-end over HTTP on the real store (with a real query embedding) with no crash / no new crash report — no double-free or use-after-free observed. Freed on all early-return paths and exactly once post-seed-selection.
3. Write durability + retrieval-fix dedup: create → checkpoint → clean SIGTERM → restart → node found by id and by search (node_count 13,036→13,037 preserved).

Caveats

• All on a copy; healthy-substrate numbers are from a re-imported store, not the live paged store (which is currently un-bootable — Finding 1).
• Single-run metrics; no thermal control.
• Embedding-space mismatch prevented a real semantic q=bullshit activation in this harness; the ANN speedup number is the module benchmark, which is the correct gauge.