d9653b1a221f5cd43eaaebd76418185c62df2d21
12 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Tim Lingo
|
b9ef66cae9 |
fix(engram): the semantic leg was deleting 234 memory records per boot
The accumulated retrieval stack (iterations 1-9) put the claim-24 semantic
leg and the claim-10 associative leg on engram_search_json — the function
~40 internal .el call sites already used as a KEYED read. Seven of those
sites delete every record that comes back ("prune all existing X nodes,
keep exactly one"): memory.el:176, sessions.el:250/268/444/523,
soul.el:359.
mem_boot_count_inc() calls engram_search_json("soul:boot_count", 50) and
engram_forget()s all 50 results. With a lexical leg that returned 1 record.
With a semantic leg it returns 50 — the 49 nearest neighbours of the STRING
"soul:boot_count" — and the soul deletes them.
MEASURED on the harness corpus, isolated, read-only, zero writes from any
caller: 234 node records destroyed in a single boot. The deletion list is
the soul's own lookup result list, in rank order. Casualties include 6
Knowledge nodes, a layer-1 "CORE IDENTITY - GENESIS, LINEAGE" Memory, the
value node kn-58874a74, and the gold answers to 8 of the 75 gold-set
queries. After the fix: 1 deletion, which is the one the code intends.
THE BOUNDARY, from Will. Claim 24 authorises the vector index "to respond
to EMBEDDING SEARCH QUERIES by returning the node records whose embedding
vectors have the highest cosine similarity to a query vector". A keyed
state read is not an embedding search query; it is the identifier-keyed
retrieval of claim 23 ("node records are stored under a key encoding the
node identifier"). One function served both, so a nearest neighbour of
"soul:boot_count" was treated as a boot counter.
So: engram_search_json returns to its lexical contract, and the legs move
to engram_recall_json, which is what /api/neuron/recall reaches — the route
the MCP wrapper, the app, and this harness all call. Retrieval quality on
that route is unchanged by construction.
MEASURED, 75-query extended gold set, embedded corpus, vs the iteration-9
baseline: +3 / -0 (q15, q28, q60), p=0.2500, hit@5 53.8 -> 58.5%, latency
1.02x, every regression guard held, nonsense 10/10. Net +3 against a floor
of 6 is NOT-SHOWN and I am not calling it an improvement. The deliverable
is the defect.
Diagnostics kept, env-gated (EG_DIAG / EG_DIAG_ID), zero cost when unset:
node/embedding census at load, per-query leg dump, and a FORGET log — the
last is the regression detector for exactly this class of bug.
LIMIT, stated: handle_api_search_knowledge still uses the lexical function.
It is a retrieval surface and arguably wants the legs, but nothing in this
harness measures it, so I did not change unmeasured behaviour.
|
||
|
Tim Lingo
|
9717a4eeaf |
measure: claim 24 unflooring is +4 (NOT-SHOWN); asymmetric embedding prefixes are -5 (discarded)
Measured on the 75-query extended gold set (iteration 8's held-out extension) against the certified stack baseline results-stack-ext.json, on the embedded corpus. Three runs of the candidate, zero drift. A. CLAIM 24 WITHOUT THE THRESHOLD - net +4, NOT-SHOWN, kept in the tree. fixed : q14, q25 (in-sample paraphrase), q43, q52, q63, q67 (held-out) broken : q15 (paraphrase), q28 (associative) 8 discordant, McNemar exact p = 0.2891, floor is 6. heldout_paraphrase 16.7% -> 30.0%, paraphrase 61.5% -> 69.2%. Every regression guard held: exact_rare 6/6, phrase 7/7, nonsense 10/10, superseded 2/3. Latency FLAT: p50 641 -> 632 ms. The in-sample half (+q14 +q25 -q15 -q28 = 0) was already on record in iteration 7's cmp-nogate.json, so only the held-out +4 is new. B. ASYMMETRIC TASK PREFIXES ON THE EMBEDDER - net -5, REVERTED in this commit. Rationale was sound and the prediction was wrong, which is why it was worth measuring: nomic-embed-text is an asymmetric retrieval encoder and this file embedded query and document bare on both sides. Prefixing does exactly what the model card implies for the far-away cases - it rescued q42 (gold at GLOBAL COSINE RANK 25,564) and q39 - but it re-ranks the whole space and broke more than it fixed: fixed : q24, q39, q42 broken : q18, q19, q22, q31, q43, q44, q52, q63 heldout_paraphrase 30.0% -> 23.3%, paraphrase 69.2% -> 53.8%. The corpus and the reproducer are kept (embed-corpus-prefixed.py, snapshot-pre-repair-20260806-embedded-prefixed.json) so nobody re-runs it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Tim Lingo
|
9790d9342d |
feat(engram): claim 24 without a threshold, and an embedding substrate that knows query from document
Two changes to the ONE leg that generalises. Iteration 8 measured that out of sample the semantic leg contributes 100% of the stack's gain and the graph leg contributes nothing, so this is where the remaining headroom is. 1. THE 0.60 FLOOR IS A PER-QUERY LOTTERY, AND CLAIM 24 HAS NO THRESHOLD IN IT. 06-claims.md l.148: "respond to embedding search queries by returning the node records whose embedding vectors have the HIGHEST COSINE SIMILARITY to a query vector, independently of the spreading activation traversal." A ranking. ENGRAM_EMBED_SEED_MIN is defined at el_runtime.c l.6094 as the HippoRAG seed-JOIN threshold and l.6102 admits the read-path leg merely "reuses" it. Measured on the 30 held-out paraphrases: the query's own top-1 cosine ranges 0.564-0.680, so the constant keeps a rank-1 answer for one query and discards a rank-1 answer for the next. Six golds sit at global cosine rank 1-2 scoring 0.564-0.589 - discarded by nothing but the constant. What holds the nonsense controls is the corpus-vocabulary gate (nhits == 0), not this floor. Cosine clamped to [0,1] per 05-detailed-description l.69. 2. THE VECTORS THEMSELVES ANSWER THE WRONG QUESTION. EL_EMBED_MODEL defaults to nomic-embed-text, an ASYMMETRIC retrieval encoder trained with task prefixes. Embedding query and document bare - as this file did on both sides - measures topical similarity rather than answer-hood. eg_embed_fetch now takes the task prefix: EL_EMBED_QUERY_PREFIX on the three query call sites, EL_EMBED_DOC_PREFIX on the two backfill sites. Restores no claim, and says so: Will specifies only "computed by an embedding model over the node's content" (l.17), so the model is his and its correct use is ours. It is the substrate under claim 24 - the index is only as good as the vectors in it. Reproducer for the derived corpus: tools/retrieval-eval/embed-corpus-prefixed.py. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Tim Lingo
|
4eb4c9e287 |
feat(engram): word-start match primitive + corpus-vocabulary gate on recall
The retrieval match test is a raw substring scan, so a query token matches anywhere INSIDE a corpus word: "throom" matches "bathroom". Measured over the 38-query gold set on this corpus that is not a rare accident - q28's lexical leg is 36,954 records of which only 13 contain a query token at a word start (99.96% mid-word noise), six further queries carry ~20,500 mid-word-only records each, and the nonsense control q35 returns 7 records ALL of which match only mid-word. istr_contains_wordstart() anchors a token to a word start (preceding char not alphanumeric) while still matching suffixes, so "value" still hits "values". That empties the lexical leg for gibberish, and the nhits==0 corpus-vocabulary gate (iteration 6's mechanism, feat/claim24-unfloored-semantic) then makes the whole query decline rather than let the semantic leg answer it. Measured vs feat/bm25-lexical-leg on the embedded corpus, 2 runs each, 0 queries of run-to-run drift on both sides: net +1 (nonsense:q35), 0 losses, McNemar p=1.0 -> NOT-SHOWN (floor is 6) nonsense clean 2/3 -> 3/3; exact_rare 100%, phrase 100%, paraphrase 61.5%, associative 66.7%, superseded 2/3 all UNCHANGED latency p50 1184 -> 543 ms (0.46x) Iteration 6 called q35 "a DEFECTIVE CONTROL ... cannot be cleaned without breaking the lexical leg". It can: the defect was the match primitive, and cleaning it cost nothing. Also committed: results-wsclaim24.json + cmp-nogate.json, a measured negative for bundling the claim-24 unfloored semantic leg on top (gains q14/q25, breaks q15/q28/q33/q34, net -2) - it independently reproduces iteration 6's q15/q28 losses and shows unflooring REQUIRES the vocabulary gate. Reproducers: legs.py (leg-level replica, reproduces baseline hit@5 exactly on all 38 queries), policy2.py, ceiling.py, wb2.py. |
||
|
Tim Lingo
|
9c39084e60 |
feat(engram): BM25-shaped lexical leg + addressability guard on the read path
engram_search_json ranked its lexical leg by raw distinct-token coverage with salience as tiebreak: a token in 30,000 nodes counted the same as a token in 1, and a 1.3 MB record matched nearly every query token by surface area alone. Score it BM25-shaped instead - Lucene-form IDF and length normalisation over the corpus mean - with per-token document frequency accumulated in the SAME corpus pass that finds the hits (no extra scan, no extra round-trip). Also refuse to return records whose identifier is not printable ASCII. This corpus carries 1,032 such records (453 by the printable test) from a save-side corruption; they occupy 125 of 303 returned slots on main. Claims 12, 23 and 27 all key on the node identifier, so such a record is unfetchable by any caller. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Tim Lingo
|
6f3a048f36 |
feat(engram): semantically seed the graph leg (Will's HippoRAG pass, SEED_K=8)
engram_assoc_leg previously took its seeds only from the top-3 LEXICAL hits. For a paraphrase query the lexical hits are noise by construction, so the walk never reached the neighbourhood that holds the answer. This adds the seeding pass Will documents at el_runtime.c l.6082 — "Semantic seeding (HippoRAG pattern, use similarity twice): the query is embedded, the top-K nodes by cosine join the seed set" — using his own ENGRAM_EMBED_SEED_K (8). Similarity is now used twice, coherently: cosine picks where to STAND in the graph, the structural-relation walk decides what is REACHABLE, and cosine orders what was reached (iteration 2's finding, unchanged). The seed list is deliberately NOT floored at ENGRAM_EMBED_SEED_MIN. Measured over all 38 gold queries: true paraphrase targets score cosine 0.46-0.66 and the three nonsense controls' own nearest neighbours score 0.55/0.60/0.62 — the distributions OVERLAP, so no absolute cosine floor separates signal from gibberish. The gate that works is reachability: gibberish's nearest neighbours carry no structural edge, so its graph leg is empty and the controls hold. The raw top-K is selected inside the existing scoring pass, so the cosine is computed exactly once per node: no extra corpus pass, no extra embed round-trip, latency flat (p50 1220 -> 1227 ms, 1.01x). Measured vs the certified baseline feat/hybrid-semantic-recall, embedded corpus, 2 runs each, zero run-to-run drift on both sides: hit@5 51.4% -> 68.6% MRR@10 0.387 -> 0.461 paraphrase 38.5% -> 61.5% associative 0% -> 66.7% exact_rare 100% held, nonsense 2/3 held, superseded 2/3 held phrase 85.7% -> 71.4% (q11, the known rank-5 rotation tax) net +6 queries (7 fixed / 1 broken), McNemar p=0.0703 |
||
|
Tim Lingo
|
059ce02003 |
feat(engram): an associative leg on the recall path (claim 10 typed relations)
The recall route had no way to reach a node that shares no token and no embedding neighbourhood with the query. The design reserves that case for the graph, and nothing on the read path consulted an edge. This adds a third ranked leg beside the lexical and semantic ones: expand the top 3 lexical hits along STRUCTURAL relations only (claim 10 — identity, contains, superseded_by, references, ...), two hops, both directions, pruned at the same 0.02 firing threshold engram_activate uses; order what was reached by query similarity. Merged by strict rotation, never by score blending. Not PR #135. That wired recall wholesale to engram_activate and lost 57 points of phrase accuracy. The failure there was RANK, not reach — a 2-hop associate at strength 0.06 cannot outrank thousands of 1-hop neighbours of strong lexical seeds. Here the lexical leg is untouched and the associative list is empty for most queries, because a node whose only edges are `tagged` and `related` expands to nothing. MEASURED, hybrid-semantic baseline -> this, 38-query gold set, embedded corpus: associative 0.0% -> 66.7% (first non-zero ever recorded on that category) hit@5 51.4% -> 62.9% exact_rare, phrase, paraphrase, nonsense, superseded: all unchanged latency p50 1.01x 4 queries moved, all gains, 0 losses, McNemar p=0.125 deterministic: two runs of the same binary differ on 0 of 38 rows VERDICT: NOT-SHOWN. The harness needs 6 queries to clear p<0.05 and the whole associative category is only 6 queries, so even 4/6 fixed cannot reach the floor. The mechanism is confirmed to work; the gold set cannot certify it. |
||
|
Neuron
|
635453b936 |
feat(engram): rank-interleave the semantic leg into recall; embed the corpus
Replaces the score-fusion first cut with rank fusion, which is what the data called for. nomic's cosine scale is compressed (true matches 0.55-0.70, unrelated pairs 0.35-0.50), so an additive blend of cosine onto token-coverage is dominated by whichever leg has the wider spread. Alternation is invariant to both scales: L1, S1, L2, S2, ... deduped, capped at limit Lexical ranking is left byte-identical; the semantic ranking is computed beside it and admitted only above ENGRAM_EMBED_SEED_MIN (0.60) — Will's existing seed floor, no new tuning constant. That floor is what keeps the nonsense controls clean: a query with no real match must not be answered with its neighbours. embed-corpus.py / merge-corpus.py produce the derived corpus the semantic leg needs (76,986 vectors, nomic-embed-text, 0 failures, 11 min). Zero of 78,791 nodes carried an embedding before this; the field round-tripped through the snapshot but nothing ever wrote it. MEASURED, 38-query gold set, paired against the SAME derived corpus so the comparison isolates the code change: hit@5 34.3% -> 51.4% paraphrase 0.0% -> 38.5% MRR@10 0.294 -> 0.387 superseded 1/3 -> 2/3 outranks recall@10 33.3% -> 50.5% latency p50 1146 -> 1220ms (1.06x) exact_rare 100% -> 100% phrase 85.7% -> 85.7% nonsense 2/3 -> 2/3 6 queries fixed, 0 broken, McNemar exact p=0.0312, 0 drift across repeats. Regression guards all held. Contrast PR #135, which swapped the read path to spreading activation wholesale: phrase 85.7 -> 28.6, latency 2.81x. Correct mechanism, wrong substrate. The substrate is now present. Restores engram claim 24 (previously 0% honoured). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Neuron
|
315b2eff00 |
feat(engram): fuse cosine similarity into the recall read path (claim 24)
engram_search_json — the function /api/neuron/recall actually reaches — ranked only by distinct-token match count, so the embedding field on every node record was inert. Add the semantic leg as a UNION beside the lexical one, not a replacement for it: fused = (distinct_tokens_matched / query_tokens) + 0.90 * sem sem = clamp01((cos(q,n) - 0.60) / (1 - 0.60)) ; 0 when not comparable Holding the semantic weight strictly below 1.0 means a node matching every query token can never be displaced by semantics alone — the regression guard that PR #135 lacked when it swapped the read path to spreading activation and took phrase recall from 85.7% to 28.6%. No query embedding (embedder down, circuit breaker open) => sem == 0 for all nodes => fused == sc/ntok, a monotone map of the old integer score, so the ordering degrades to the historical behaviour exactly. Restores engram claim 24: 'maintain a vector similarity index over the semantic embedding vectors of all stored node records, and ... respond to embedding search queries by returning the node records whose embedding vectors have the highest cosine similarity to a query vector, independently of the spreading activation traversal.' Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Tim Lingo
|
98ccbd4704 |
fix(engine): a client that leaves must not kill the daemon, and a long round must say it started
Round 9.1, spec §3 D + ADR 0006 items 2 and 4. Two small changes, both proven
by measurement, both E2E-verified locally against a rebuilt brain.
D1 — SIGPIPE/EPIPE survival (vendor/el-runtime el_runtime.c).
Root cause, at the layer that owns it: the whole HTTP server lives in the C
runtime; .el has no socket primitive. http_send_all() called send() with flags
0 and nothing anywhere in the runtime set a SIGPIPE disposition, so the default
disposition — terminate the process — applied. When a handler finished after
its client had gone (Tim's VM: reply at 116.9 s, client cancelled at 25.0 s),
the second of the four sends that write one reply raised SIGPIPE and the daemon
died: `exited due to SIGPIPE ... ran for 361177ms`, launchd respawn 4 ms later,
every other in-flight session's work lost, user never told.
Fix: SIGPIPE -> SIG_IGN at runtime init and at each http_serve* entry, plus
per-connection SO_NOSIGPIPE / MSG_NOSIGNAL so the guard survives an embedder
resetting dispositions. http_send_all now retries EINTR and preserves errno;
http_send_response classifies it once — a departure is logged as routine
("client left before the reply was written ... reply discarded") and ANY other
errno is logged as a real "send failed: <strerror>". Spec §5.3: the routine
case must not mask a genuine write fault, and it does not.
Proof (scratch HOME + free port, 3 disconnects mid-reply):
round-9 shipped brain 4402179554… — DIED, exit 141 (128+13 = SIGPIPE), round 1
round-9 sources rebuilt with this exact recipe — DIED, exit 141, round 1
this build — SURVIVED 3/3, /health 200 after, still serving the full graph,
three honest "client left" lines in the log naming Broken pipe / Connection
reset by peer.
D2 — the round-start marker (chat.el, agentic_loop).
The ledger only ever appended AFTER a round returned, so a healthy first leg
produced zero progress by construction; since server-side web_search moved
inside the outbound call that leg is 60-120 s of silence, which is how a 25 s
client watchdog came to kill a healthy mission. One entry,
{"i":N,"t":"","tool":"__working__"}, written to the existing
run_progress_<session_id> ledger BEFORE each round's outbound call — the wire
shape ChatView.kt:1148 has handled as a life signal since 2026-07-13 and never
received. No new key, no new route, no new lifecycle: a strict subset of WS3
item 3. WS3's run registry is untouched and stays Will's.
Proof (live Anthropic key, real research mission, scratch HOME + free port):
round-9 baseline — ledger EMPTY for the whole 59.7 s leg
this build — {"i":0,"t":"","tool":"__working__"} visible at 18.6 s of a
70.0 s leg; both builds returned correct ~4.9 KB answers
Regression: prompt-matrix gate 32/32 on this build (round-9 baseline also 32/32
under the same recipe, so the score is not a build artifact). Soul contract
gate PASS — 27/27 routes, immutability clean. neuron#111 miscompile guard: 0
sites in the generated amalgam this binary was compiled from.
NOT included, deliberately: the regenerated dist/soul.c. CI compiles that file,
so production stays exposed until it is regenerated — the same open ask as
neuron#111 / ui#209. The regen recipe is now known and recorded; landing it is
Will's call, per BUILD-HYGIENE.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
 will.anderson
|
7f3d6ed8cd |
ci: update vendored el-runtime to complete v1.0.0-20260501
The runtime vendored alongside the CI pin was the Jul-21 snapshot, which predates two builtins the reconciled ship-soul now calls: - http_delete_json (boot-counter HTTP write-back, awareness/memory self-review) - engram_act_stats_json (heartbeat activation observability) Compiling dist/soul.c against the stale runtime fails with implicit-declaration errors. Vendor the current release runtime (identical to the one the soul was gate-verified against: verify-soul-contract PASS, genesis boots clean, full safety-contact) so the CI Linux soul is byte-for-byte the verified soul. |
||
|
will.anderson
|
eed6487114 |
ci: pin soul build to vendored release runtime v1.0.0-20260501
The soul build downloaded el-runtime-c 'latest' from Artifact Registry. The merged ship-soul calls engram_prune_telemetry, which the latest published runtime no longer defines, so an unpinned build fails to link — the failure mode that let a broken/handlerless soul reach prod. Vendor the release runtime v1.0.0-20260501 (el_runtime.c/.h) into the repo and compile the soul against it. This is the exact runtime the merged soul was verified against (verify-soul-contract GATE PASS, genesis boot survives, full safety-contact response), making the build reproducible and independent of a moving AR 'latest'. The verify-soul-contract.sh HARD-BLOCK gate already runs before Publish (from the CI-hardening arc on main), so a destructive or stale soul can never publish/deploy again. |