4eb4c9e287cdc5818afb4229acb0040e9a4fc5a9
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Tim Lingo
|
4eb4c9e287 |
feat(engram): word-start match primitive + corpus-vocabulary gate on recall
The retrieval match test is a raw substring scan, so a query token matches anywhere INSIDE a corpus word: "throom" matches "bathroom". Measured over the 38-query gold set on this corpus that is not a rare accident - q28's lexical leg is 36,954 records of which only 13 contain a query token at a word start (99.96% mid-word noise), six further queries carry ~20,500 mid-word-only records each, and the nonsense control q35 returns 7 records ALL of which match only mid-word. istr_contains_wordstart() anchors a token to a word start (preceding char not alphanumeric) while still matching suffixes, so "value" still hits "values". That empties the lexical leg for gibberish, and the nhits==0 corpus-vocabulary gate (iteration 6's mechanism, feat/claim24-unfloored-semantic) then makes the whole query decline rather than let the semantic leg answer it. Measured vs feat/bm25-lexical-leg on the embedded corpus, 2 runs each, 0 queries of run-to-run drift on both sides: net +1 (nonsense:q35), 0 losses, McNemar p=1.0 -> NOT-SHOWN (floor is 6) nonsense clean 2/3 -> 3/3; exact_rare 100%, phrase 100%, paraphrase 61.5%, associative 66.7%, superseded 2/3 all UNCHANGED latency p50 1184 -> 543 ms (0.46x) Iteration 6 called q35 "a DEFECTIVE CONTROL ... cannot be cleaned without breaking the lexical leg". It can: the defect was the match primitive, and cleaning it cost nothing. Also committed: results-wsclaim24.json + cmp-nogate.json, a measured negative for bundling the claim-24 unfloored semantic leg on top (gains q14/q25, breaks q15/q28/q33/q34, net -2) - it independently reproduces iteration 6's q15/q28 losses and shows unflooring REQUIRES the vocabulary gate. Reproducers: legs.py (leg-level replica, reproduces baseline hit@5 exactly on all 38 queries), policy2.py, ceiling.py, wb2.py. |
||
|
Tim Lingo
|
9c39084e60 |
feat(engram): BM25-shaped lexical leg + addressability guard on the read path
engram_search_json ranked its lexical leg by raw distinct-token coverage with salience as tiebreak: a token in 30,000 nodes counted the same as a token in 1, and a 1.3 MB record matched nearly every query token by surface area alone. Score it BM25-shaped instead - Lucene-form IDF and length normalisation over the corpus mean - with per-token document frequency accumulated in the SAME corpus pass that finds the hits (no extra scan, no extra round-trip). Also refuse to return records whose identifier is not printable ASCII. This corpus carries 1,032 such records (453 by the printable test) from a save-side corruption; they occupy 125 of 303 returned slots on main. Claims 12, 23 and 27 all key on the node identifier, so such a record is unfetchable by any caller. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Tim Lingo
|
6f3a048f36 |
feat(engram): semantically seed the graph leg (Will's HippoRAG pass, SEED_K=8)
engram_assoc_leg previously took its seeds only from the top-3 LEXICAL hits. For a paraphrase query the lexical hits are noise by construction, so the walk never reached the neighbourhood that holds the answer. This adds the seeding pass Will documents at el_runtime.c l.6082 — "Semantic seeding (HippoRAG pattern, use similarity twice): the query is embedded, the top-K nodes by cosine join the seed set" — using his own ENGRAM_EMBED_SEED_K (8). Similarity is now used twice, coherently: cosine picks where to STAND in the graph, the structural-relation walk decides what is REACHABLE, and cosine orders what was reached (iteration 2's finding, unchanged). The seed list is deliberately NOT floored at ENGRAM_EMBED_SEED_MIN. Measured over all 38 gold queries: true paraphrase targets score cosine 0.46-0.66 and the three nonsense controls' own nearest neighbours score 0.55/0.60/0.62 — the distributions OVERLAP, so no absolute cosine floor separates signal from gibberish. The gate that works is reachability: gibberish's nearest neighbours carry no structural edge, so its graph leg is empty and the controls hold. The raw top-K is selected inside the existing scoring pass, so the cosine is computed exactly once per node: no extra corpus pass, no extra embed round-trip, latency flat (p50 1220 -> 1227 ms, 1.01x). Measured vs the certified baseline feat/hybrid-semantic-recall, embedded corpus, 2 runs each, zero run-to-run drift on both sides: hit@5 51.4% -> 68.6% MRR@10 0.387 -> 0.461 paraphrase 38.5% -> 61.5% associative 0% -> 66.7% exact_rare 100% held, nonsense 2/3 held, superseded 2/3 held phrase 85.7% -> 71.4% (q11, the known rank-5 rotation tax) net +6 queries (7 fixed / 1 broken), McNemar p=0.0703 |
||
|
Tim Lingo
|
059ce02003 |
feat(engram): an associative leg on the recall path (claim 10 typed relations)
The recall route had no way to reach a node that shares no token and no embedding neighbourhood with the query. The design reserves that case for the graph, and nothing on the read path consulted an edge. This adds a third ranked leg beside the lexical and semantic ones: expand the top 3 lexical hits along STRUCTURAL relations only (claim 10 — identity, contains, superseded_by, references, ...), two hops, both directions, pruned at the same 0.02 firing threshold engram_activate uses; order what was reached by query similarity. Merged by strict rotation, never by score blending. Not PR #135. That wired recall wholesale to engram_activate and lost 57 points of phrase accuracy. The failure there was RANK, not reach — a 2-hop associate at strength 0.06 cannot outrank thousands of 1-hop neighbours of strong lexical seeds. Here the lexical leg is untouched and the associative list is empty for most queries, because a node whose only edges are `tagged` and `related` expands to nothing. MEASURED, hybrid-semantic baseline -> this, 38-query gold set, embedded corpus: associative 0.0% -> 66.7% (first non-zero ever recorded on that category) hit@5 51.4% -> 62.9% exact_rare, phrase, paraphrase, nonsense, superseded: all unchanged latency p50 1.01x 4 queries moved, all gains, 0 losses, McNemar p=0.125 deterministic: two runs of the same binary differ on 0 of 38 rows VERDICT: NOT-SHOWN. The harness needs 6 queries to clear p<0.05 and the whole associative category is only 6 queries, so even 4/6 fixed cannot reach the floor. The mechanism is confirmed to work; the gold set cannot certify it. |
||
|
Neuron
|
635453b936 |
feat(engram): rank-interleave the semantic leg into recall; embed the corpus
Replaces the score-fusion first cut with rank fusion, which is what the data called for. nomic's cosine scale is compressed (true matches 0.55-0.70, unrelated pairs 0.35-0.50), so an additive blend of cosine onto token-coverage is dominated by whichever leg has the wider spread. Alternation is invariant to both scales: L1, S1, L2, S2, ... deduped, capped at limit Lexical ranking is left byte-identical; the semantic ranking is computed beside it and admitted only above ENGRAM_EMBED_SEED_MIN (0.60) — Will's existing seed floor, no new tuning constant. That floor is what keeps the nonsense controls clean: a query with no real match must not be answered with its neighbours. embed-corpus.py / merge-corpus.py produce the derived corpus the semantic leg needs (76,986 vectors, nomic-embed-text, 0 failures, 11 min). Zero of 78,791 nodes carried an embedding before this; the field round-tripped through the snapshot but nothing ever wrote it. MEASURED, 38-query gold set, paired against the SAME derived corpus so the comparison isolates the code change: hit@5 34.3% -> 51.4% paraphrase 0.0% -> 38.5% MRR@10 0.294 -> 0.387 superseded 1/3 -> 2/3 outranks recall@10 33.3% -> 50.5% latency p50 1146 -> 1220ms (1.06x) exact_rare 100% -> 100% phrase 85.7% -> 85.7% nonsense 2/3 -> 2/3 6 queries fixed, 0 broken, McNemar exact p=0.0312, 0 drift across repeats. Regression guards all held. Contrast PR #135, which swapped the read path to spreading activation wholesale: phrase 85.7 -> 28.6, latency 2.81x. Correct mechanism, wrong substrate. The substrate is now present. Restores engram claim 24 (previously 0% honoured). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Neuron
|
315b2eff00 |
feat(engram): fuse cosine similarity into the recall read path (claim 24)
engram_search_json — the function /api/neuron/recall actually reaches — ranked only by distinct-token match count, so the embedding field on every node record was inert. Add the semantic leg as a UNION beside the lexical one, not a replacement for it: fused = (distinct_tokens_matched / query_tokens) + 0.90 * sem sem = clamp01((cos(q,n) - 0.60) / (1 - 0.60)) ; 0 when not comparable Holding the semantic weight strictly below 1.0 means a node matching every query token can never be displaced by semantics alone — the regression guard that PR #135 lacked when it swapped the read path to spreading activation and took phrase recall from 85.7% to 28.6%. No query embedding (embedder down, circuit breaker open) => sem == 0 for all nodes => fused == sc/ntok, a monotone map of the old integer score, so the ordering degrades to the historical behaviour exactly. Restores engram claim 24: 'maintain a vector similarity index over the semantic embedding vectors of all stored node records, and ... respond to embedding search queries by returning the node records whose embedding vectors have the highest cosine similarity to a query vector, independently of the spreading activation traversal.' Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Tim Lingo
|
98ccbd4704 |
fix(engine): a client that leaves must not kill the daemon, and a long round must say it started
Round 9.1, spec §3 D + ADR 0006 items 2 and 4. Two small changes, both proven
by measurement, both E2E-verified locally against a rebuilt brain.
D1 — SIGPIPE/EPIPE survival (vendor/el-runtime el_runtime.c).
Root cause, at the layer that owns it: the whole HTTP server lives in the C
runtime; .el has no socket primitive. http_send_all() called send() with flags
0 and nothing anywhere in the runtime set a SIGPIPE disposition, so the default
disposition — terminate the process — applied. When a handler finished after
its client had gone (Tim's VM: reply at 116.9 s, client cancelled at 25.0 s),
the second of the four sends that write one reply raised SIGPIPE and the daemon
died: `exited due to SIGPIPE ... ran for 361177ms`, launchd respawn 4 ms later,
every other in-flight session's work lost, user never told.
Fix: SIGPIPE -> SIG_IGN at runtime init and at each http_serve* entry, plus
per-connection SO_NOSIGPIPE / MSG_NOSIGNAL so the guard survives an embedder
resetting dispositions. http_send_all now retries EINTR and preserves errno;
http_send_response classifies it once — a departure is logged as routine
("client left before the reply was written ... reply discarded") and ANY other
errno is logged as a real "send failed: <strerror>". Spec §5.3: the routine
case must not mask a genuine write fault, and it does not.
Proof (scratch HOME + free port, 3 disconnects mid-reply):
round-9 shipped brain 4402179554… — DIED, exit 141 (128+13 = SIGPIPE), round 1
round-9 sources rebuilt with this exact recipe — DIED, exit 141, round 1
this build — SURVIVED 3/3, /health 200 after, still serving the full graph,
three honest "client left" lines in the log naming Broken pipe / Connection
reset by peer.
D2 — the round-start marker (chat.el, agentic_loop).
The ledger only ever appended AFTER a round returned, so a healthy first leg
produced zero progress by construction; since server-side web_search moved
inside the outbound call that leg is 60-120 s of silence, which is how a 25 s
client watchdog came to kill a healthy mission. One entry,
{"i":N,"t":"","tool":"__working__"}, written to the existing
run_progress_<session_id> ledger BEFORE each round's outbound call — the wire
shape ChatView.kt:1148 has handled as a life signal since 2026-07-13 and never
received. No new key, no new route, no new lifecycle: a strict subset of WS3
item 3. WS3's run registry is untouched and stays Will's.
Proof (live Anthropic key, real research mission, scratch HOME + free port):
round-9 baseline — ledger EMPTY for the whole 59.7 s leg
this build — {"i":0,"t":"","tool":"__working__"} visible at 18.6 s of a
70.0 s leg; both builds returned correct ~4.9 KB answers
Regression: prompt-matrix gate 32/32 on this build (round-9 baseline also 32/32
under the same recipe, so the score is not a build artifact). Soul contract
gate PASS — 27/27 routes, immutability clean. neuron#111 miscompile guard: 0
sites in the generated amalgam this binary was compiled from.
NOT included, deliberately: the regenerated dist/soul.c. CI compiles that file,
so production stays exposed until it is regenerated — the same open ask as
neuron#111 / ui#209. The regen recipe is now known and recorded; landing it is
Will's call, per BUILD-HYGIENE.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
 will.anderson
|
7f3d6ed8cd |
ci: update vendored el-runtime to complete v1.0.0-20260501
The runtime vendored alongside the CI pin was the Jul-21 snapshot, which predates two builtins the reconciled ship-soul now calls: - http_delete_json (boot-counter HTTP write-back, awareness/memory self-review) - engram_act_stats_json (heartbeat activation observability) Compiling dist/soul.c against the stale runtime fails with implicit-declaration errors. Vendor the current release runtime (identical to the one the soul was gate-verified against: verify-soul-contract PASS, genesis boots clean, full safety-contact) so the CI Linux soul is byte-for-byte the verified soul. |
||
|
will.anderson
|
eed6487114 |
ci: pin soul build to vendored release runtime v1.0.0-20260501
The soul build downloaded el-runtime-c 'latest' from Artifact Registry. The merged ship-soul calls engram_prune_telemetry, which the latest published runtime no longer defines, so an unpinned build fails to link — the failure mode that let a broken/handlerless soul reach prod. Vendor the release runtime v1.0.0-20260501 (el_runtime.c/.h) into the repo and compile the soul against it. This is the exact runtime the merged soul was verified against (verify-soul-contract GATE PASS, genesis boot survives, full safety-contact response), making the build reproducible and independent of a moving AR 'latest'. The verify-soul-contract.sh HARD-BLOCK gate already runs before Publish (from the CI-hardening arc on main), so a destructive or stale soul can never publish/deploy again. |