#116 shipped the ggml strategy at 0.9933 id-recall against the CPU oracle
while the hand-rolled Metal kernel it replaced scored 0.9997 — a ~150x worse
error margin. That was not an inherent property of ggml. It was a usage bug in
this file, and this commit fixes it.
ggml-metal has two F32xF32 matmul kernels and picks between them purely on
ne11, the number of B rows, which for us is the query-batch size:
ne11 <= 8 -> kernel_mul_mv_ext_f32_f32_* / kernel_mul_mv_f32_f32_*,
templated <float, float> — genuine F32.
ne11 > 8 -> kernel_mul_mm_f32_f32, templated
<half, half4x4, simdgroup_half8x8, half, half2x4, ...> —
BOTH operands narrowed to F16, despite F32 tensors on both
sides.
The old code issued one ggml_mul_mat with ne11 = nq (300 in the benchmark),
landing squarely on the F16 path. The file's own header comment asserted the
opposite ("computes in F32 on the Metal backend"); that claim was wrong and is
replaced with the measurement.
Fix: emit ceil(nq/8) mul_mats over ne11<=8 ggml_view_2d slices of one query
tensor, all expanded into ONE graph and one ggml_backend_graph_compute, so the
node matrix is still uploaded and shared exactly once. EL_GGML_MULMAT_CHUNK
overrides the 8; setting it >= nq reproduces the old behaviour exactly, which
is also how the before/after below was measured in a single binary.
Measured, real store snapshot, 13415 live embedded nodes, dim=768, 300 real
queries, vs the CPU double-accumulated oracle (vindex_bench, offline copy of
the store — no live service touched):
id-recall same-rank |Δdist| max mean
old (ne11=300) 0.9933 6.80e-05 1.43e-05
new (ne11<=8) 0.9987 4.77e-07 9.30e-08
hand-rolled 0.9997 3.58e-07 7.55e-08
~145x better max error, ~154x better mean — now the same order of magnitude as
the hand-rolled kernel rather than 150x off it.
The cost is real and is documented rather than buried. Median of 15 reps of
the whole batch_multi() call, three runs: 13.2-14.4ms unchunked, 19.9-20.2ms
chunked, 17.7-18.0ms hand-rolled. Correctness costs ~+6.7ms per 300-query
batch and leaves ggml ~12% behind the hand-rolled kernel instead of ~35%
ahead. It cannot be recovered inside ggml: an fp32 matmul on Metal must
re-stream the node matrix once per <=8 queries, and ggml's Metal backend ships
no fp32 TILED matmul, so "fast" and "fp32" are genuinely exclusive there.
Two things that did NOT work, recorded so nobody retries them:
- ggml_mul_mat_set_prec(t, GGML_PREC_F32) does nothing here. Error was
bit-identical with and without it (1.038e-05 either way) — ggml-metal has
no F32-accumulating mul_mm kernel to switch to. ne11 is the only lever.
- The ACCEL/BLAS device looked excellent in an isolated compute-only probe
(3.4-4.0ms, mean |Δdot| 1.5e-08) but is dominated on BOTH axes end-to-end
(0.191 ms/query at 0.9973 recall vs 0.125-0.142 at 0.9987), because the
probe was not competing for the same CPU cores the real call path is. It
stays reachable via EL_GGML_DEVICE as a no-Metal fallback, labelled as
measured-and-rejected, not as a recommendation.
Also corrected: the ~7.8s "cold start" blamed on this file is not this file
re-initialising per call — init was already cached. It is Apple's shader cache
missing on ggml's embedded metallib (~650 kernels), keyed on the library and
shared across processes: the first load on a machine reports
"loaded in 7.670 sec", the next run of a *different* binary reports 0.009 sec.
Once per machine per ggml version, not once per process, and not ours to fix.
Warm ggml init is 44-53ms vs 36-117ms for the hand-rolled strategy.
Loading only libggml-metal.so instead of every plugin in the directory is kept
for tidiness, and explicitly documented as NOT a speedup: 44.7-52.4ms against
46.9-58.9ms, the same number inside noise.
The -2.0 sentinel contract is unchanged and re-verified at batch sizes that
straddle the chunk boundary (1,7,8,9,16,17,33), plus NULL rows, dim
mismatches, zero-norm rows, and an all-invalid population. Notably the old
ne11=300 path fails that same check at a 2e-6 cosine tolerance with 2299
mismatches, which is an independent confirmation of the defect.
Stop hand-rolling GPU kernels for batch cosine similarity — use ggml (the
MIT-licensed compute library underneath llama.cpp, installed standalone via
Homebrew) as the preferred backend, without ripping out PR #114's
carefully-verified hand-rolled Metal shader.
Structure: one stable public adapter (eg_cosine_batch.h, zero #ifdef at call
sites) backed by three selectable concrete Strategies behind an internal
vtable (eg_cosine_batch_strategy.h) chosen by a Factory (eg_cosine_batch.c):
- eg_cosine_batch_strategy_ggml.c — NEW. ggml + dynamically-loaded Metal
backend plugin (ggml_backend_load_all_from_path
+ ggml_mul_mat for the batched dot
product), gather/scatter around the
-2.0 sentinel contract.
- eg_cosine_batch_strategy_metal_hand.m — PR #114's original hand-rolled
Metal shader bridge, preserved
almost verbatim, now one strategy
among several rather than the only
option. eg_cosine_batch.metal kept
byte-identical to the original.
- eg_cosine_batch_strategy_cpu.c — universal always-false fallback
(direct descendant of PR #114's
eg_metal_cosine_stub.c).
Selection: EL_COSINE_BATCH_STRATEGY=ggml|metal|cpu|auto (default: ggml first,
then hand-rolled Metal, then CPU — first available wins), plus back-compat
EL_METAL_COSINE=0 to disable every GPU-backed strategy. build_vindex_bench.sh
compiles all three strategies on Darwin, CPU-fallback-only elsewhere.
vindex_bench.c now reports BRUTE-GGML and BRUTE-METAL side by side against
the same CPU oracle, on the same dataset, in one run (real numbers vs. real
store snapshot in the PR body).
Lands feat/reframe-region-setop (PR #109: native set-based reframe_region,
decorator-as-seam @route port, teacher-summon, and the M8.1 activate-latency
work — lazy-memoized cosq via eg_cosq_at + engram_vindex HNSW-accelerated
seed discovery + vindex_harvest_from_store/vindex_bench oracle) onto dev's
actual current HEAD, plus engram-tiered-storage's still-unique test suite.
RECONCILING #109 WITH engram-tiered-storage (M4-M10 HNSW/geometry/reason/
verify work): not a two-way merge. engram_vindex.c's HNSW core (search_layer/
select_neighbors/prune_links/insert) is BYTE-IDENTICAL between the two
branches; #109's copy is a strict superset (adds vindex_harvest_from_store,
used by vindex_bench.c's brute-force-vs-HNSW oracle). engram_reason.c and
engram_verify.c are also byte-identical. #109's own branch point already
carried engram-tiered-storage's M4-M10 lineage forward, so there was nothing
left to merge into #109 for those files. The one thing engram-tiered-storage
had that #109's tree dropped: its full test suite (test_vindex.c,
test_geometry.c, test_reason.c, test_verify.c, test_m7_traversal.c, the
interoception P0-P5 tests, bufpool/compaction tests, and their run_*.sh
harnesses) — ported over here unchanged.
WHY THIS NEEDED HAND RECONCILIATION, NOT A MECHANICAL MERGE: #109's branch
forked from dev on 2026-08-14 15:40 (before restructure-adjacent history
diverged the file's merge-base for `git merge` — it presented as an add/add
conflict). A straight two-dot diff (dev tip -> PR tip) applied cleanly, but
it silently reverted THREE dev fixes landed on 2026-08-14/15, after the
branch point, that the PR's diff had no way to know about:
1. qgate rescale (2026-08-14 self-review): PR's lazy eg_cosq_at rewrite of
the query-aware propagation gate dropped the shift-and-floor rescale
about ENGRAM_EMBED_S0 (measured: unrelated-pair median 0.562->raw gate
0.67, i.e. "a small tax, not a gate"). Restored the rescale, wrapped
around the lazy accessor -- the PR's actual improvement (WHEN cosq[oi]
is computed) is orthogonal to WHAT it gates on and both are kept.
2. Eviction cause decomposition (2026-08-14 self-review): dev decomposes
wm_evicted into evict_floor/evict_cap/evict_bll so WM churn is
diagnosable (identity: evicted == floor+cap+bll+dup_wm+dup_wm_global).
PR's tree predates this and dropped all three counters + their JSON
stats fields. Restored declarations, all 4 direct increment sites, the
eg_wm_carry_over bll increment, and the act-stats JSON fields --
alongside (not instead of) the PR's own P4 afferent / API-reshape
counters already in that same struct/JSON.
3. Hebbian link-formation selection (2026-08-15 self-review, TODAY): dev
selects the STRONGEST qualifying candidate for consolidation each call;
PR's tree predates this and reverted to hash-slot order (arbitrary wrt
association strength) for edge formation -- the one path that writes
PERMANENT structure. Restored the strongest-candidate while-loop,
keeping the PR's own genuine improvement at that site
(engram_adj_on_edge_added incremental-index append instead of a bare
adj_dirty=1 full-rebuild flag).
engram/src/server.el's 3-way conflicts (autoconnect_on/ise_offgraph_on env
flags, /api/nodes connected-count in responses) were pure additive: dev's
side was empty, PR's side added the feature. Took PR's side whole.
VERIFIED (nsbx sandbox only, live :8742/:7770 never touched):
- cc -std=c11 -O2, clean link against the real engram/src/server.el via
elc, zero errors.
- vindex_bench (built standalone, read-only harvest) against the real
production store clone (13,671 embedded nodes, 768-dim nomic-embed-text):
recall@10 = 1.0000 at ef 64/128/200; HNSW search 0.28-0.79ms/query vs
2.03ms/query brute-force oracle (2.6x-7.2x). HNSW build itself: 46.5s
for the full 13,671-node set -- see the flagged risk below.
- Booted the reconciled binary in an isolated nsbx sandbox (:8905, cloned
snapshot of the live store, 13,424 nodes / 37,656 edges) and called
/api/activate for real: first call after boot 41.5s (pays the one-time
HNSW build inline -- matches the standalone bench), second/third calls
356ms/605ms, no crash, correct results, act-stats JSON (including the
restored evict_floor/cap/bll fields) reads correctly.
KNOWN RISK TO FLAG BEFORE ANY LIVE CUTOVER (not fixed here; out of scope for
this dev-only land per instructions not to touch :8742/:7770): eg_vindex_sync
builds the HNSW index synchronously, inline, on the first engram_activate()
call after every process start (or index invalidation). On the real node
count that is a ~46s blocking stall on a single-threaded server -- the first
request after every restart (or its concurrent siblings) waits the full
build. Recommend a background/incremental build (or a bounded per-call build
budget) before this ever reaches the live daemon. See PR description / final
report for the fuller writeup.
Adds an O(1) "seen" bitmap so lazily-embedded older nodes get picked up
incrementally instead of only on a full rebuild (embed-gap #20).
Replaces engram_activate's O(N*D) cosine prescan with a lazy-memoized
cosine cache (eg_cosq_at), proven bit-identical to the old path.
Extracts a clean vindex_harvest_from_store primitive (read-only vector
harvest, careful malloc/ownership/error-path handling) reused by both
index-build and the new vindex_bench.c — a read-only proof harness
comparing brute-force vs HNSW recall/latency on both the real store and
synthetic data.
.nsbx-env intentionally excluded — local sandbox config (ports, paths,
dev-only placeholder key), not checked in.
Cancellation-token control channel checked at every step boundary lets a
coordinator PAUSE/RESUME/REDIRECT/KILL a running worker mid-task instead of
waiting for the whole (possibly wrong) plan to finish. Bounded purviews
mean no half-committed state to unwind on interrupt. Includes a proof
harness (proof.el, run.sh) comparing a broken non-interruptible worker
against the new one under identical kill/redirect/pause timing.
Distinct from the already-preserved swarm-ccr orchestrator (fan-out/
converge dispatch): this is single-worker interruptibility, a
complementary mechanism, not a duplicate.
Will waived diff review -> build it for real. Add engram_boundary_beat() to the
runtime (afferent counter++ + engram_chrono_tick + engram_strengthen(self-anchor)
+ dharma_emit) and two act-stats counters (aff_boundary_ops, dharma_emits).
codegen cg_fn injects ONE engram_boundary_beat(op) at the entry of every
@manager/@accessor fn (fn_has_decorator, so it fires under @route @manager too) —
a decorated op self-reports with ZERO hand-written instrumentation. Rebuilt elc
self-host + the cognition engram in the worktree; ran it as the clone daemon on
:8900. Proof (/api/boundary-proof, @manager, empty body, 5x): aff_boundary_ops
0->5, dharma_emits 0->5, self activation_count 1510->1513, chrono stamp advanced.
Brought in feat/cognitive-architecture engram runtime+server for the build.
strengthen = activation bump (not content/edge write) -> identity protection
intact. Live :8742 untouched; no push, no cutover.
Resolve 3 conflicts:
- lang/el-compiler/runtime/el_runtime.c: keep deletion (deprecated runtime fork;
single-source-of-truth is lang/runtime/, enforced by scripts/check-single-runtime.sh).
- lang/releases/v1.0.0-20260501/el_runtime.h: keep deletion (releases/ is a generated
artifact folder, not a source path; a release is a git tag, not a folder).
- lang/runtime/el_platform_win.h: union of dev's Windows port (#80: setsockopt optval
wrapper + curl-less libcurl stubs) and our fsync(->_commit) shim needed by engram_store WAL.
Nothing in dev's build consumes the deprecated fork or releases/ folder.
- Wire paged store into the ENGRAM SERVICE (server.el, the authoritative durable
owner): boot->engram_store_boot, persist_canonical->engram_store_checkpoint,
gated by ENGRAM_STORE.
- elc (lang/elc.c + src/parser.el + codegen.el + elc-combined.el): OOB guard in
tok_kind/tok_value + parse_block progress backstop — fixes the pre-existing
unbounded-memory fold hang on sessions.el.
- engram_prune_telemetry mirrors ISE prune to the store (store_forget) so store
live-count tracks resident and stale telemetry stays bounded.
- Deployed live 2026-08-12: engram :8742 on neuron.egm+WAL, count reconciled 11552.
Flag-on checkpoint now full-walks the resident graph: store_put_node (WM weight,
activation_count, last_activated, wm_anchor) + store_put_edge (hebb, last_fired)
for every node/edge, then engram_checkpoint. Uses store_put_edge (idempotent
upsert) not store_hebb_batch, because activation FORMS new hebbian-associate edges
that bypass the create hook and delta-only hebb_batch can't create them. Store-on
boot now applies the same WM-halving + floor + cap transforms as engram_load.
This is the hebb-survives-restart fix. Gate: reboot from neuron.egm with
snapshot.json deleted -> edge hebb + activation_count survive unchanged, WM weight
survives with identical boot transform; negative control proves persist is
load-bearing (hebb->0 without it). M1 33/33 + M2 36/36 + M3 parity PASS, ASan/UBSan
clean, flag-off untouched. Engine unchanged (boundary held).
Caller-side shim in el_runtime.c maps EngramNode/Edge <-> StoreNode/Edge; engine
keeps zero soul deps (libengram boundary, design §10). Flag off = today's JSON
path byte-for-byte (proven: no neuron.egm created, graph identical). Flag on =
engram_open (import snapshot.json once into neuron.egm, else WAL-replay) +
resident load; node/edge create + forget dual-write via guarded hooks. Files
renamed engram.store->neuron.egm, engram.wal->neuron.wal.
Gate: M3 parity PASS (graph on==off byte-exact modulo ordering; snapshot round-trip;
reboot-from-egm with snapshot.json deleted; activation set+sequence identical;
ASan/UBSan clean). M1 33/33 + M2 36/36 green post-rename.
Known gap (pre-flip): in-place hebb/WM/activation_count updates during activation
are not yet persisted to the store (create/connect/forget are). Must close before
live flip so learned edges survive restart.
Write-back no-steal buffer pool makes the fsync'd WAL load-bearing (M1 was
write-through). Logical WAL with record-granularity page-LSN redo idempotency.
Checkpoint = flush dirty pages, fsync store, advance last_checkpoint_lsn,
reclaim WAL prefix. One-time snapshot.json import only when store absent;
JSON never read as the ongoing store thereafter.
Gates: 33/33 M1 (no regression) + 36/36 M2 — replay parity, torn-tail fuzz
(every byte offset), checkpoint-crash at all 5 phases, torn-page+WAL redo,
legacy-import parity, hebb-survives-crash.
Establish lang/runtime/ as the ONE canonical el runtime (from the active
runtime that carries hebb/emb persistence + the new WAL); repoint the el CI
publish, engram build, elb default, and in-repo build scripts to it; delete
the el-compiler/runtime + lang/releases/ forks; add scripts/check-single-runtime.sh
drift guard.
Fixes a live prod bug: the el CI published el-runtime-c/-h from the LAGGING
el-compiler fork (0 hebb refs), so the shipped soul never persisted Hebbian
edge weights — learned co-activation was wiped on every restart. Publishing
from canonical ships the stranded 'learning that cannot outlive the process'
fix.
WAL storage engine + integrity fixes (DELETE->tombstone + store-layer
protection, safe data-dir default) ride in behind ENGRAM_WAL (default off =
byte-identical to today). Verified: engram elb per-module build clean, WAL
gate 66/66, native smoke ok, drift-guard green.
handle_api_consolidate writes a "SessionSummary" node, but engram_valid_node_type
omitted it — so once this validation ships, every consolidate() would be silently
REJECTED at the engram boundary. Add SessionSummary to the allowlist.
Found in Will's PR review of neuron #1 / el #52.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The wrapper signature was stale and didn't match the C primitive
__engram_node_full(content, node_type, label, salience, importance, confidence, tier, tags).
Because el_val_t is an untyped machine word, the compiler coerced caller args to the
wrong declared param types and forwarded them BY POSITION — so tier received an int,
importance/confidence received strings, label received a float, etc. (~100 corrupt nodes).
- Correct the wrapper to match the C contract 1:1 (no coercion, no reorder).
- Add engram_valid_node_type / engram_valid_tier allowlists; engram_node and
engram_node_full now reject invalid values with __println + return "" (fail loud,
no silent malformed write).
See neuron repo: HANDOFF-engram-write-corruption.md for the full write-up + deploy runbook.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
bigmerge
will.anderson