2f84e2a1de4f76f68167d37ccfe70f5124dabcc1
400 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Neuron
|
635453b936 |
feat(engram): rank-interleave the semantic leg into recall; embed the corpus
Replaces the score-fusion first cut with rank fusion, which is what the data called for. nomic's cosine scale is compressed (true matches 0.55-0.70, unrelated pairs 0.35-0.50), so an additive blend of cosine onto token-coverage is dominated by whichever leg has the wider spread. Alternation is invariant to both scales: L1, S1, L2, S2, ... deduped, capped at limit Lexical ranking is left byte-identical; the semantic ranking is computed beside it and admitted only above ENGRAM_EMBED_SEED_MIN (0.60) — Will's existing seed floor, no new tuning constant. That floor is what keeps the nonsense controls clean: a query with no real match must not be answered with its neighbours. embed-corpus.py / merge-corpus.py produce the derived corpus the semantic leg needs (76,986 vectors, nomic-embed-text, 0 failures, 11 min). Zero of 78,791 nodes carried an embedding before this; the field round-tripped through the snapshot but nothing ever wrote it. MEASURED, 38-query gold set, paired against the SAME derived corpus so the comparison isolates the code change: hit@5 34.3% -> 51.4% paraphrase 0.0% -> 38.5% MRR@10 0.294 -> 0.387 superseded 1/3 -> 2/3 outranks recall@10 33.3% -> 50.5% latency p50 1146 -> 1220ms (1.06x) exact_rare 100% -> 100% phrase 85.7% -> 85.7% nonsense 2/3 -> 2/3 6 queries fixed, 0 broken, McNemar exact p=0.0312, 0 drift across repeats. Regression guards all held. Contrast PR #135, which swapped the read path to spreading activation wholesale: phrase 85.7 -> 28.6, latency 2.81x. Correct mechanism, wrong substrate. The substrate is now present. Restores engram claim 24 (previously 0% honoured). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Neuron
|
315b2eff00 |
feat(engram): fuse cosine similarity into the recall read path (claim 24)
engram_search_json — the function /api/neuron/recall actually reaches — ranked only by distinct-token match count, so the embedding field on every node record was inert. Add the semantic leg as a UNION beside the lexical one, not a replacement for it: fused = (distinct_tokens_matched / query_tokens) + 0.90 * sem sem = clamp01((cos(q,n) - 0.60) / (1 - 0.60)) ; 0 when not comparable Holding the semantic weight strictly below 1.0 means a node matching every query token can never be displaced by semantics alone — the regression guard that PR #135 lacked when it swapped the read path to spreading activation and took phrase recall from 85.7% to 28.6%. No query embedding (embedder down, circuit breaker open) => sem == 0 for all nodes => fused == sc/ntok, a monotone map of the old integer score, so the ordering degrades to the historical behaviour exactly. Restores engram claim 24: 'maintain a vector similarity index over the semantic embedding vectors of all stored node records, and ... respond to embedding search queries by returning the node records whose embedding vectors have the highest cosine similarity to a query vector, independently of the spreading activation traversal.' Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Neuron
|
cf41d12d22 |
test(retrieval): a measurement harness for memory recall, and its first verdict
Nothing else on the memory roadmap should be built until a change can be shown
to help. Right now we judge by feel, and the benchmark literature is full of
systems that felt better and measured worse. This is the missing gate.
WHAT IT MEASURES, AND WHY IT BOOTS A REAL SOUL
The subject is Will's designed retrieval — spreading activation over the
weighted directed graph, four-factor multiplicative scoring — not a proxy for
it. A Python re-implementation would measure my reading of the design, so the
harness compiles the actual soul.el amalgam from a git ref and asks it over
HTTP on /api/neuron/recall, exactly as the MCP wrapper and the app do.
BUILT ON WHAT WAS ALREADY HERE, NOT AROUND IT
docs/research/graphrag_eval/{collect,score}.py — per-query relevant-id
scoring and fixed-denominator precision@5 (kept verbatim: an empty result
should be punished like a page of junk).
docs/research-archive/p0-prototypes/eval_pinned_40q_20260715.py — the pinned
ground truth + --check winnability gate, so every run judges alike.
scripts/verify-soul-contract.sh — the isolation recipe, including the
non-obvious SOUL_ISE_URL pin without which an "isolated" soul silently
syncs the operator's live brain.
gen-soul-amalgam.sh + .gitea/workflows/ci.yaml — the build recipe and flags.
New here: ids rather than regexes as ground truth, an associative category
derived from real edges, a superseded category scored on ranking, a
machine-checked zero-lexical-overlap guarantee on paraphrases, paired
significance testing, and measurement of the real compiled soul rather than an
offline replica of one leg of it.
THE GOLD SET IS AUDITABLE, NOT VIBES
38 queries over the real 78,768-node corpus, each carrying a `derivation`
string, each re-validated by `build_gold_set.py --check`. exact_rare is mined
(document frequency 1). phrase is mined (verbatim scan; >25 matches rejected as
too diffuse). paraphrase is hand-selected then PROVEN to share zero content
words with its target — a leak fails the build, so the category cannot decay
into lexical matching. associative is derived from real hub edges with
lexically-reachable siblings dropped. nonsense is verified absent. superseded
pairs are kept only when both sides survive as distinct nodes.
HONEST ABOUT NOISE
Minimum detectable swing on 38 queries is 6: if every changed query moves the
same way, p = 2*0.5^n first clears 0.05 at n=6. Run-to-run drift is measured,
not assumed — activation is a stateful read, and it shows: main is fully
deterministic across 3 runs, the candidate drifts by 1 query. compare.py
reports "no measurable difference" for anything inside max(6, drift+1).
FIRST VERDICT — feat/recall-through-activation
hit@5 34.3% -> 22.9%, phrase 85.7% -> 28.6%, latency p50 2.81x. Five discordant
pairs, all five against the candidate, none for it; McNemar exact p = 0.0625,
so by the stated rule this is one query short of significant and is reported as
such rather than as a win for main. The latency regression is deterministic and
not in any noise band.
The benefit the branch was written for is absent: associative recall is 0/6 on
BOTH builds. Probed directly, the traversal returns the lexical seed at rank 8
and none of its 12 hub siblings. Two measured corpus facts explain it — only
4,060 of 78,768 nodes (5.2%) carry any edge, and no node has an embedding, so
the fourth factor of the four-factor product has nothing to compute from. The
mechanism runs; the corpus lacks the structure it needs.
SAFETY
Throwaway port, throwaway HOME, disposable per-run copy of the corpus; live
ports refused by name. Every soul started is killed AND confirmed dead by pid
probe, with the confirmation written into the results file; run_comparison.sh
sweeps for strays and exits non-zero if any survive. Nothing under ~/.neuron,
/Applications/Neuron*, or ~/neuron-dev-stack is read, written, or restarted.
Rung: E2E-VERIFIED — 6 full runs (3 per config) against the real compiled
binaries on the real corpus; numbers above are measured, not projected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Tim Lingo
|
dd952c0e46 |
feat(soul): write-through to the persistence owner — memories survive restart (#117)
The soul obeys half of its own ownership rule. soul.el:571-573 says "when
ENGRAM_URL is set the HTTP Engram owns persistence — the soul must NEVER write
to the local snapshot", and it doesn't. But nothing was ever built to hand the
soul's writes TO that owner: sync is pull-only (/api/sync -> engram_load_merge),
so every node created inside the soul lived in process RAM and was shed on
restart. Measured live 2026-08-07: soul node_count=102184, engram 79197.
SCOPE CORRECTION vs the earlier internal spec: engram provisional claim 17's
"pull-then-push" is a PEER-ENGRAM to PEER-ENGRAM protocol (claims 15-18 say so
explicitly). The soul is a CALLER of the database API, not a peer. Claim 17 is
NOT authority for a soul<->engram contract and is no longer cited as such. The
design here follows from the ownership rule alone.
Mechanism: a new Accessor, persist.el, is the single boundary. Writes stage a
delta to a filesystem spool and are pushed to the owner via POST /api/load-merge
— NOT POST /api/nodes, which mints a new server-side id (breaking dedup and
edges) and drops label/tier/tags/importance/confidence (verified in a sandbox:
a tier "Canonical" probe came back "Working"). load-merge preserves the id and
every field, dedups nodes by id and edges by (from,to,relation) so retries are
no-ops, and calls persist_canonical() so THE OWNER writes its own file — the
ownership rule is honoured rather than worked around.
Spool-and-drain rather than push-per-write: measured ~0.38s per load-merge at
live scale (79k nodes/176MB), and a chat turn writes 5-7 nodes. The spool is on
disk, not in process state, because the soul serves each connection on its own
pthread and a shared buffer would lose entries to a read-modify-write race. That
also buys crash recovery: writes orphaned by kill -9 are drained on next boot.
Honesty: api_persisted (the gate all 10 MCP write handlers pass through) and
mem_store now assert AT THE OWNER instead of reading back the soul's own RAM.
With the owner down a write returns {"ok":false,"error":"write_not_persisted"}
and the delta is queued — where main returns {"ok":true} for a write that dies.
Coverage: 35 node sites + 9 edge sites routed through the boundary. Deliberately
excluded, with reasons in persist.el: 4 InternalStateEvent sites (Will's own
telemetry carve-out), the boot counter and the persona (both already have
bespoke owner-side write-backs), and soul.el's 54 genesis identity edges
(file-mode only). engram_strengthen and engram_forget are NOT propagated —
load-merge cannot update or delete, and hard-deleting at the owner would fail
verify-soul-contract.sh section B.
Also fixed here:
- routes.el GET /api/graph/edges engram_save()'d straight over the owner's
canonical snapshot.json — a read route, in a non-owner process, clobbering the
canonical on every call. Same defect class Will removed from the engram in el
dc39a61. Now exports to a scratch path. With this gone the soul writes nothing
at all in HTTP mode.
- persist.el must clear the runtime's _tl_fs_read_len hint after every fs_read.
In vendored runtime v1.0.0-20260501 that hint becomes the NEXT response's
Content-Length, so reading a spool file mid-request made an 86-byte reply go
out as 497 bytes with 411 bytes of adjacent heap trailing it. Caught and fixed
at our boundary; the runtime class was fixed upstream in el 43636ae, which is
not the pinned runtime here.
Rung: E2E-VERIFIED, discriminating. Same harness, same engram binary:
write-through: LEG 1 PRESENT at owner, LEG 2 SURVIVED kill -9 + restart
main: LEG 1 ABSENT at owner, LEG 2 LOST
verify-soul-contract.sh: GATE PASS on both builds (27/27 routes, immutability).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
 tim.lingo
|
18714e6142 | Merge pull request 'fix(engine): restore multi-turn crisis escalation on the agentic path (P0, closes #129)' (#130) from fix/129-history-amplification into main | ||
|
tim.lingo
|
4936099c39 | Merge pull request 'fix(engine): the daemon survives a client leaving, and says it is working while it works' (#127) from fix/liveness-engine-91 into main | ||
|
tim.lingo
|
f1471763f5 | Merge pull request 'fix(engine): approving a researched mission completes — the resume replay read a tool id out of the conversation (BUG-42, both faces)' (#115) from fix/resume-server-tool-replay into main | ||
|
tim.lingo
|
5850793b67 | Merge pull request 'fix(engine): history keeps its provenance and its session — kills the false confession, the blank stare, and the "to.Good" seams' (#114) from fix/soul-history-provenance-20260805 into main | ||
|
tim.lingo
|
fc1745c652 | Merge pull request 'feat(engine): plain chat generates at L3 — inside the safety cycle, not around it (+ crisis-path segfault fix)' (#109) from feat/soul-plain-chat-generation-20260805 into main | ||
|
Tim Lingo
|
027a573d89 |
feat(gate): make a state_get with no producer a build error, not a silence
Retires the defect class behind #129. The engine's state store returns "" for a key nothing writes — no error, no warning, no log. That is how the agentic path's crisis-escalation input scored 0 on every real conversation for two days after |
||
|
Tim Lingo
|
eb2b2cc40d |
fix(engine): the agentic crisis screen reads the session's own history again (OpenAI-tools branch)
P0 SAFETY. Same defect as #129, carried INDEPENDENTLY on this branch — not a merge, not a duplicate report. feat/soul-openai-tools-v2 branched with the |
||
|
Tim Lingo
|
d5319d2849 |
test(engine): a runner for tests/, and a failing regression test for #129
tests/ has held 14 test programs for months with no way to run them. CI does
not run them. The convention printed in their own headers
(`elc soul.el && ./soul --test tests/x.el`) refers to a --test flag the El
runtime does not implement. So the tests were documentation, not gates — which
is how a P0 safety regression shipped with a test directory sitting right
there.
scripts/run-el-test.sh compiles and runs one test program. It reuses the
gen-soul-amalgam.sh discovery: elc emits only an extern prototype for a module
that has a .elh beside it, and inlines the bodies when it does not, so a test
importing ../chat.el must be compiled in a scratch tree with the headers
removed. Scratch copy on purpose — the worktree is shared. It runs the binary
under a throwaway HOME so a test can never reach the live engram.
Exit status is the gate: the El tests print failures and still exit 0, so the
runner greps for FAIL lines and for a zero assertion count as well.
tests/test_history_amplification.el pins the invariant #129 violated: the
window the safety screen READS must be the window conv_history_record WRITES.
Not "must be called conv_history" — must AGREE.
THIS COMMIT IS RED BY DESIGN. On this tree the test fails one assertion:
3. REGRESSION #129 — agentic screen reads the session's own window
FAIL: distress history escalates the agentic screen to hard_bell
got: soft_bell
expected: hard_bell
history amplification tests: 8 passed, 1 failed (runner exit 1)
The next commit turns it green by changing one line. Two legs, one variable —
that is the whole point of committing the test first.
Two flaws in the older harness that this one does not copy: the idiom
`let pass_count = pass_count + 1` inside an assert function declares a local
that dies with the call, so every existing suite prints "0 passed, 0 failed"
regardless of outcome; and a test program without a `cgi` block compiles as a
'utility', which may not reference the self-formation primitives chat.el's
agentic loop calls — it fails to build on a capability violation it never
triggers at runtime.
Refs #129
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit
|
||
|
Tim Lingo
|
43d0449904 |
fix(engine): the agentic crisis screen reads the session's own history again
P0 SAFETY. Closes the regression we introduced in |
||
|
Tim Lingo
|
b842e82f77 |
test(engine): a runner for tests/, and a failing regression test for #129
tests/ has held 14 test programs for months with no way to run them. CI does not run them. The convention printed in their own headers (`elc soul.el && ./soul --test tests/x.el`) refers to a --test flag the El runtime does not implement. So the tests were documentation, not gates — which is how a P0 safety regression shipped with a test directory sitting right there. scripts/run-el-test.sh compiles and runs one test program. It reuses the gen-soul-amalgam.sh discovery: elc emits only an extern prototype for a module that has a .elh beside it, and inlines the bodies when it does not, so a test importing ../chat.el must be compiled in a scratch tree with the headers removed. Scratch copy on purpose — the worktree is shared. It runs the binary under a throwaway HOME so a test can never reach the live engram. Exit status is the gate: the El tests print failures and still exit 0, so the runner greps for FAIL lines and for a zero assertion count as well. tests/test_history_amplification.el pins the invariant #129 violated: the window the safety screen READS must be the window conv_history_record WRITES. Not "must be called conv_history" — must AGREE. THIS COMMIT IS RED BY DESIGN. On this tree the test fails one assertion: 3. REGRESSION #129 — agentic screen reads the session's own window FAIL: distress history escalates the agentic screen to hard_bell got: soft_bell expected: hard_bell history amplification tests: 8 passed, 1 failed (runner exit 1) The next commit turns it green by changing one line. Two legs, one variable — that is the whole point of committing the test first. Two flaws in the older harness that this one does not copy: the idiom `let pass_count = pass_count + 1` inside an assert function declares a local that dies with the call, so every existing suite prints "0 passed, 0 failed" regardless of outcome; and a test program without a `cgi` block compiles as a 'utility', which may not reference the self-formation primitives chat.el's agentic loop calls — it fails to build on a capability violation it never triggers at runtime. Refs #129 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
 will.anderson
|
86e269fa91 |
self-review 2026-08-07: push what was learned; stop a read route writing the canonical store
Two fixes, one found by making the other. 1. HEBBIAN WRITE-BACK. This daemon learned 1,198 associations in 23h48m and kept none of them: it syncs FROM the engram server and never pushes, and mem_save() is unreachable in HTTP mode by design (soul.el only sets soul_snapshot_path inside `is_genesis && safe_to_seed`, false whenever ENGRAM_URL is set, because the server owns persistence). So the one process that runs idle cognition -- where essentially all co-activation happens -- was the one process that could not remember what it learned. hebb_consolidate() now drains the runtime's write-back queue on every heartbeat and POSTs it as ONE batch to /api/edges/batch. One request, one durable write, not one 60MB snapshot per edge. Also drains on clean shutdown, so an exit between beats doesn't take the last 8 minutes of learning with it. _auth is required and its absence is silent: check_auth_ok exempts GET and /api/neuron/state-events (which is why ise_post works keyless) but gates every other mutation on "_auth" in the BODY -- http_serve surfaces no headers, so there is no Bearer path. An unauthorized reply is NON-EMPTY, so the obvious `if resp == "" return 0` check would have reported delivery of edges that were refused, after the drain had already destroyed them. Caught before it shipped. Gauges hebb_wb_pending/_drained/_dropped/_sent go into the heartbeat so a consolidation path that stops delivering is visible in the stream. 2. A READ ROUTE MUST NEVER WRITE THE CANONICAL SNAPSHOT. GET /api/graph/edges serialized this process's graph straight over $HOME/.neuron/engram/snapshot.json -- the engram SERVER's durable store -- and read the edges back out of it. I triggered it myself this morning fetching edges for the census above: snapshot.json went from the server's 41,213 edges to the soul's 42,431, and the next engram restart loaded the soul's graph as canonical. It happened to be a superset (Knowledge 1198->1218, Memory 1238->1242, no durable type down), so nothing was lost. That was luck. Had the soul been running a partial load -- the exact failure soul.el's safe_to_seed guard exists to catch -- one GET would have destroyed the store, with no write-side guard able to see it coming. The engram server fixed this same class of bug on 2026-07-21 by routing exports to a dotted sidecar; the soul kept the original pattern. Same fix: exports go to .soul-edges-export.json. Also stops a 60MB serialize-and-reread per GET. Verified: boot 26 loaded 42,432 edges with hebb_max 0.4941 carried across the restart -- the first time this daemon has ever started knowing what it learned. |
||
|
Tim Lingo
|
98ccbd4704 |
fix(engine): a client that leaves must not kill the daemon, and a long round must say it started
Round 9.1, spec §3 D + ADR 0006 items 2 and 4. Two small changes, both proven
by measurement, both E2E-verified locally against a rebuilt brain.
D1 — SIGPIPE/EPIPE survival (vendor/el-runtime el_runtime.c).
Root cause, at the layer that owns it: the whole HTTP server lives in the C
runtime; .el has no socket primitive. http_send_all() called send() with flags
0 and nothing anywhere in the runtime set a SIGPIPE disposition, so the default
disposition — terminate the process — applied. When a handler finished after
its client had gone (Tim's VM: reply at 116.9 s, client cancelled at 25.0 s),
the second of the four sends that write one reply raised SIGPIPE and the daemon
died: `exited due to SIGPIPE ... ran for 361177ms`, launchd respawn 4 ms later,
every other in-flight session's work lost, user never told.
Fix: SIGPIPE -> SIG_IGN at runtime init and at each http_serve* entry, plus
per-connection SO_NOSIGPIPE / MSG_NOSIGNAL so the guard survives an embedder
resetting dispositions. http_send_all now retries EINTR and preserves errno;
http_send_response classifies it once — a departure is logged as routine
("client left before the reply was written ... reply discarded") and ANY other
errno is logged as a real "send failed: <strerror>". Spec §5.3: the routine
case must not mask a genuine write fault, and it does not.
Proof (scratch HOME + free port, 3 disconnects mid-reply):
round-9 shipped brain 4402179554… — DIED, exit 141 (128+13 = SIGPIPE), round 1
round-9 sources rebuilt with this exact recipe — DIED, exit 141, round 1
this build — SURVIVED 3/3, /health 200 after, still serving the full graph,
three honest "client left" lines in the log naming Broken pipe / Connection
reset by peer.
D2 — the round-start marker (chat.el, agentic_loop).
The ledger only ever appended AFTER a round returned, so a healthy first leg
produced zero progress by construction; since server-side web_search moved
inside the outbound call that leg is 60-120 s of silence, which is how a 25 s
client watchdog came to kill a healthy mission. One entry,
{"i":N,"t":"","tool":"__working__"}, written to the existing
run_progress_<session_id> ledger BEFORE each round's outbound call — the wire
shape ChatView.kt:1148 has handled as a life signal since 2026-07-13 and never
received. No new key, no new route, no new lifecycle: a strict subset of WS3
item 3. WS3's run registry is untouched and stays Will's.
Proof (live Anthropic key, real research mission, scratch HOME + free port):
round-9 baseline — ledger EMPTY for the whole 59.7 s leg
this build — {"i":0,"t":"","tool":"__working__"} visible at 18.6 s of a
70.0 s leg; both builds returned correct ~4.9 KB answers
Regression: prompt-matrix gate 32/32 on this build (round-9 baseline also 32/32
under the same recipe, so the score is not a build artifact). Soul contract
gate PASS — 27/27 routes, immutability clean. neuron#111 miscompile guard: 0
sites in the generated amalgam this binary was compiled from.
NOT included, deliberately: the regenerated dist/soul.c. CI compiles that file,
so production stays exposed until it is regenerated — the same open ask as
neuron#111 / ui#209. The regen recipe is now known and recorded; landing it is
Will's call, per BUILD-HYGIENE.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Tim Lingo
|
de65991807 |
feat(engine): tools + agentic loop on the OpenAI wire, and two chat-breaking fixes found proving it
Teaches the OpenAI-format lane (Groq/OpenAI/Grok/Gemini/Ollama) to offer tools,
execute them, and loop — the capability that until now existed only on the
Anthropic wire. The tool-execution, consent, bridge and run-progress machinery is
reused unchanged; only the wire dialect is new.
Two pre-existing defects were found while proving it, and are fixed here because
both silently break chat:
1. PROVIDER WIRING NEVER CONNECTED. The launcher exports SOUL_LLM_PROVIDER /
SOUL_LLM_BASE_URL and puts the provider key in ANTHROPIC_API_KEY + SOUL_API_KEY;
the engine's provider fork read only NEURON_LLM_0_*, which nothing sets in a
customer build. So use_openai was ALWAYS false: every non-Anthropic user's turns
went to api.anthropic.com carrying, say, a Groq key, and came back
"llm unavailable". Proven side-by-side against the pinned round-9 brain
(sha256 15cf7d1b…): identical env, shipped brain = "llm unavailable" both chat
modes with ZERO calls to the configured endpoint; this build = a real answer,
with the probe logging POST /v1/chat/completions and Bearer <provider key>.
Fixed brain-side only (env fallbacks) — no app or launcher change needed.
2. TRUNCATION SPLITS UTF-8 CHARACTERS. The session preload cuts recalled memory at
fixed BYTE lengths (continuity snippet 350; session_preload_bullets per bullet).
A cut landing inside a multi-byte character leaves a dangling lead byte in the
SYSTEM PROMPT, making the whole request body invalid UTF-8 — providers reject it
and the user sees an unexplained failure. Captured from a real body: 18,710 bytes,
decode fails at 18,248 on 'e2', a box-drawing rule (U+2500 = E2 94 80) sliced in
half. Trigger is ordinary content — em dash, curly quote, accented name, emoji,
table border — and it gets MORE likely as memory grows. Shared code: this hit the
Anthropic wire too. Fixed with utf8_safe_slice() applied at BOTH cut sites.
WHAT IS IN THE PORT
- llm_base_url / llm_wire_format / agentic_api_key: fall back to the launcher's own
SOUL_LLM_* names; anthropic deliberately still returns "" so its native path is
untouched (endpoint configurability remains neuron#62).
- openai_tools_json(): Anthropic tool schema -> OpenAI function schema; entries with
no input_schema (Anthropic's server-side web_search) are skipped — they cannot
execute on this wire.
- agentic_tools_no_web(): the standard set minus that server tool.
- openai_agentic_loop(): forked rather than parameterised, so agentic_loop — which
carries every round-7/8/9 fix — is provably untouched. Same envelopes, same state
keys, same consent policy (ask_all / escalate / builtin / always-allow), same
client-bridge contract, same run-progress ledger, same 12-iteration cap.
- ADR-0005 mirrored on this wire: parallel_tool_calls:false is sent explicitly, and
if a provider ignores it we honour the FIRST call and echo only that one, so the
conversation we send is never self-contradictory. The drop is logged loudly.
- The assistant turn echoes the provider's own content bytes (json_get_raw), so a
JSON null stays null and nothing is lost to a decode/re-encode round trip.
- Tool results are embedded already-escaped (dispatch_tool json_safe's them);
truncation trims a dangling escape so a cut can't invalidate the body.
- bridge_save() gains a "wire" scalar and agentic_resume branches on it, so a
suspended turn resumes on the wire it suspended on. Legacy blobs (no field) resume
as anthropic. The field is read from the blob's SCALAR HEAD only — an unbounded
first-match scan would run on into messages_raw, which is model-controlled, and
that is exactly the round-9 resume defect. Pinned by a test.
- Three fork sites: handle_chat_agentic, handle_dharma_room_turn_agentic,
agentic_resume. Tool assembly is computed once per lane at both entry points
(it makes an HTTP call to the connector bridge; it was being paid for twice).
TOOLING THAT DID NOT EXIST
- tests/run-el-test.sh — engine tests were never runnable: elc is a compiler, it
emits C and exits. This emits the test to C, compiles soul.c with main renamed
away, links the rest + the repo-pinned runtime, and runs it. It also COMPUTES THE
VERDICT, because every counted test file's "N passed, M failed" summary is a
permanent 0/0 — the counters increment inside if BLOCKS, which El scoping
discards (9 files; real fix filed as neuron#116). Proven to discriminate with a
deliberately-broken assertion.
- tests/gate-openai/ — deterministic OpenAI-dialect provider stub + scenarios +
driver + hostile modes, and a strict request validator that rejects any
Anthropic-shaped field so dialect leakage fails loudly.
VERIFICATION (rungs named)
- E2E-VERIFIED against a LIVE provider (Anthropic's OpenAI-compatible endpoint,
confirmed live): real answer; a tool call whose out-of-root path was DENIED by the
guard, after which the model refused to claim success ("I won't tell you I did it,
because I didn't"); then a valid path -> file physically on disk with exact content,
honest reply, ledger with per-round entries + {done:true}.
- Deterministic lane gate: 11/12 in both consent configurations (bridge + local);
hostile providers produce no hang and no fabricated answer; the 12-iteration cap
trips with its honest message. The one FAIL is oa-tools-off and is NOT this port —
see "Known, not fixed here".
- ANTHROPIC LANE UNCHANGED: gate9 32/32 on this build and on the pinned round-9
brain; request bytes differ only within the noise band that two runs of the
UNMODIFIED brain also produce (proven with a baseline-vs-baseline control), and
the preload sections — the shared code touched here — are byte-identical.
The rig discriminates: the round-8 brain scores 24/32 on it.
- verify-soul-contract.sh: PASS (27/27 routes, no hard-deletes).
- Unit: test_bridge_serialization 36/36 (incl. 8 new wire/field-order assertions),
test_utf8_slice 18/18, test_agentic_tools 18 PASS / 0 FAIL / 3 documented skips.
KNOWN, NOT FIXED HERE (deliberate)
- Tools:Off on an OpenAI provider still fails: the non-agentic path goes through the
el-runtime provider chain, which appends /v1/chat/completions to a base URL that
already ends in /v1 -> /v1/v1/... 404. Runtime/plain-chat territory, untouched
mid-beta. Note openai_chat_complete() has zero callers — that lane is served
entirely by the runtime chain.
- The 12-iteration cap does not bound a chain of BRIDGED tools (iteration is
per-invocation and resume starts fresh). Parity with the Anthropic lane.
- run_progress resets on each resume, so a client rendering cumulative steps across a
consent pause sees earlier legs vanish. Parity with the Anthropic lane.
- verify-soul-contract.sh needs bash >= 4; under macOS's stock bash 3.2 it dies
instantly with a FALSE red ("local: -n: invalid option").
- Groq-specific live E2E not run: no Groq key exists on this machine.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Tim Lingo
|
dba755dcec |
fix(engine): resume reads the bridged tool id from the blob's own field, not from inside the replayed conversation
ROOT CAUSE (round 9; live-repro'd 5/5 this morning, both faces stub-proven by the
prompt-matrix gate). json_get is a first-substring-match scanner (strstr for
'"key":', el_runtime.c). bridge_save serialized the RAW messages array BEFORE the
tool_use_id scalar, so agentic_resume's json_get(blob, 'tool_use_id') returned the
FIRST '"tool_use_id":' occurrence inside the replayed conversation, not the saved
field. The resume guard then preferred that misread over the client's correct
call_id (its two branches both reduced to saved_use_id), attached the tool_result
to the wrong id, and Anthropic 400'd the resume ('unexpected tool_use_id found in
tool_result blocks'), surfaced as {"error":"llm unavailable"}.
ONE MISREAD, TWO FACES — whichever block owns the first tool_use_id in the array:
FACE 1 (search-then-bridge, the Key West killer): the first occurrence is the
first web_search_tool_result's srvtoolu_… id — every agentic turn that ran
server-side web_search and then bridged on a client tool died on approval,
deterministically (messages.2.content.0 … srvtoolu_…). The write itself had
already succeeded; only the resume died.
FACE 2 (multi-cycle missions): with no search, the first occurrence is ROUND 0's
tool_result block — so every LATER approve/resume cycle replayed the round-0
client id (stale-resume-id), killing multi-file missions after ~2 files.
And the shape that PASSES on round 8 confirms the mechanism: a single-cycle
bridge with no prior tool round has no 'tool_use_id' substring in its messages
at all (tool_use blocks carry 'id'), so the scan fell through to the blob's own
field and resumed correctly.
The server_tool_use ↔ web_search_tool_result pairs themselves replay intact — the
defect was a cross-field misread of the blob, the same first-match-scanner class
as BUG-6 (approve 'content' matched inside tool_input, 2026-07-17) and round 8's
citation-block fix.
THE FIX, the pattern not the spot:
1. bridge_save writes every json_safe'd scalar BEFORE both raw fields (an escaped
value cannot contain a bare '"key":' byte pattern, so first-match always lands
on the blob's own fields), and tools_raw (our fixed schema) before messages_raw
(arbitrary conversation), so the raw extractions cannot first-match into
model-controlled bytes either. Field order documented as load-bearing.
2. agentic_resume now honors the client's echoed call_id when present — the value
with clean provenance (minted from pend_tool_id, never blob-round-tripped) —
falling back to the saved id only when the client omits it. Each approve cycle
therefore binds to ITS OWN round's id (kills FACE 2 even against a blob written
by a pre-fix binary), and an omitted call_id still resumes on the saved id,
which the reordered blob now reads correctly.
Pattern sweep: the legacy synthetic blob (sessions.el handle_session_approve) embeds
only json_safe'd fields — no raw hazard, untouched. No other json_get read of any
container that embeds raw conversation JSON before the read field.
PROOF: prompt-matrix gate 24/32 RED on the round-8 brain (fails exactly the two
resume classes, named) -> 32/32 GREEN on this build; live-key Key West tracer
3/3 consecutive full round-trips (bridge -> approve-as-the-app -> real completion,
file on disk), plain-chat and weather-only controls PASS; unpatched round-8 brain
and a same-toolchain unpatched baseline build both still fail the identical
sequence with the identical srvtoolu 400 (the test discriminates, and the only
variable between failing and passing builds is this diff).
Refs neuron#109
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
will.anderson
|
97d22ffe44 |
self-review 2026-08-06: a streak of nothing is not a streak; surface hebb gauges
str_eq("", "") is true, so an auto-term extractor that kept FAILING reported a
rising auto_term_streak. The signal meaning "fixated on one term" and the
signal meaning "producing no term at all" were the same number — opposite
failures needing opposite responses. Observed live as
{"auto_term":"","auto_term_streak":3}. Same class of bug already fixed for
wm_top0_streak on 2026-07-31; auto_term was missed then.
Empty now reads 0, and the empty run is counted on its own axis
(auto_term_empty_streak) so extractor failure is visible rather than disguised
as health.
Also surfaces the new runtime gauges in the heartbeat: hebb_warm, hebb_max,
hebb_links (is the graph learning any structure at all?) and dup_wm_global.
|
||
|
Tim Lingo
|
8f3a478771 |
fix(engine): excise the receipt, do not truncate at it — a leading receipt was erasing whole answers
CAUGHT BY A/B, AND ONLY BY A/B. The previous commit's receipt_strip assumed the receipt is
always TERMINAL and cut everything from the marker onward. It is not always terminal: once
receipt_rule told the model what [[RECEIPT ...]] means, the model sometimes LED with one and
wrote the answer underneath. Cutting at the marker then deleted the entire answer and the
turn returned {"error":"no response"}.
MEASURED, same prompt (two web searches, cited prose), fresh session each run:
round-7 brain 4 / 4 answered (471, 473, 473, 544 chars)
round-8 brain 2 / 7 answered (five {"error":"no response"})
This looked exactly like a flaky model. It was not — it was mine. Running the two brains
side by side on the same prompt is the only reason it was found, and it is the reason the
A/B is now part of how this class gets tested.
AFTER THE FIX, same protocol:
round-7 brain 4 / 4 (473, 473, 473, 544)
round-8 brain 4 / 4 (657, 657, 657, 673)
THE FIX: remove the [[...]] span and keep BOTH sides, instead of truncating at the marker.
An unterminated marker at position 0 is left completely alone — no rule about receipts is
worth erasing an answer over. Bounded four-pass loop rather than a conditional exit, because
rebinding the counter inside an if-expression is the block-expression shape that miscompiles
integer arithmetic under this elc (BUG-PLAINCHAT-1). Verified in the generated C:
str_slice(rest, (e + 2), str_len(rest)) <- integer addition, correct
el_str_concat(head, tail) <- string concat, correct
and zero el_str_concat(<ident>, str_len(...)) sites across all 49 modules.
SEAM PROOF (FIX C) rides on the same runs — a real two-search cited answer, inspected byte
by byte, in BOTH failure directions:
missing separator (the round-7 "to.Good", bytes 77 2e 47): 0 hits. Sentence boundaries
measure 2e 20 4d — "." SPACE "M".
over-separation (a cited sentence shattered across paragraphs): 0 hits. The answer is one
continuous paragraph with its sentences intact, which is the direction a blanket
separator would have broken.
BUILT: sha256 77115f2733e794c5bc4ad1f55b1acaf658f8f4a91cccd423a2d633d94a726cbc
Refs neuron#109
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Tim Lingo
|
9ea41eed78 |
fix(engine): the model was signing its own answers with our receipt — strip it
FOUND BY E2E, NOT BY REASONING. The previous commit's design note asserted the provenance
receipt "never reaches the user: it is appended to the history copy, not the reply." That
was FALSE, and only running the thing showed it. On the first live run against the built
DMG brain, two agentic turns out of two came back with
"Your favourite colour is chartreuse and your project is called Perihelion.
[[RECEIPT - recorded by the soul, not written by the model: no tools ran on this turn.]]"
— the receipt in the user-visible reply.
MECHANISM: the receipt is stored inside the assistant turn, and the agentic path replays
history VERBATIM as Anthropic message objects. So the model sees its own previous answers
ending in [[RECEIPT ...]] and does the obvious thing — it imitates the format and signs the
next answer the same way. The plain path did NOT leak, which is the tell: there, history is
rendered into the SYSTEM prompt as labelled lines rather than replayed as assistant turns,
and a model imitates its own turns far more readily than a transcript.
FIX, two layers, because one of them is not a guarantee:
- receipt_rule() names the marker in both system prompts (plain and agentic): these lines
are written by the system, read them as evidence, never write one. Reduces occurrence.
- receipt_strip() truncates any [[RECEIPT ...]] out of model output before it becomes the
reply — plain path in layered_generate, agentic path on final_text in agentic_loop.
Deterministic. A guard that depends on the model choosing to obey is exactly the class of
thing round 8 exists to stop shipping, so the instruction is the optimisation and the
strip is the guarantee.
Placed ABOVE agentic_loop's empty-check on purpose: a turn whose entire output was an
imitated receipt has produced no answer, and must be reported as no answer.
The receipt stays in HISTORY, which is the whole point and is proven to work: asked "What
source did you use for that?" one turn after a live web_search, this brain answered
"I used Weather Underground (https://www.wunderground.com/weather/is/reykjav%C3%ADk) for the
current temperature in Reykjavik" — a real source, no apology. That is the false confession
dead, and it is dead BECAUSE the model can read the receipt.
BUILT: 887,112 bytes, sha256 54a2eff84d4fa44f8d2db6781dcf225df4b5075a8ec5f1058018bd40cc1af10b
BUG-PLAINCHAT-1 miscompile guard: zero sites.
Refs neuron#109
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Tim Lingo
|
ff421d39f6 |
fix(engine): history keeps its provenance and its session — the false confession and the blank stare
DESIGN FIT: three of round 7's five defects share ONE root — the conversation-history
layer persists only {role, content}, discarding tool provenance, session scoping, and the
distinction between a real user turn and an internal utility call. Fixes A and B RESTORE
Will's design rather than extend it: his agentic path already scopes history per session,
the plain path never got it, and his own source carries the TODO admitting the resulting
race (chat.el, handle_chat: "process-global key; concurrent /api/chat requests without
session_id race on this read-append-write"). Fix C repairs one join Will wrote that was
correct for a year and one we added last week. E1/E2 are ours.
FIX A — tool provenance in history (kills the FALSE CONFESSION)
Root cause, EXECUTED-verified: handle_chat_agentic recorded turns via hist_append, which
emits {"role","content"} only. server_tool_use blocks, web_search_tool_result blocks and
every citation were discarded, then replayed as text. On the next turn the model saw a
data-rich answer with zero evidence a search had happened, and its own permanent rule
("never describe a search you did not perform") left one conclusion available: that it
had fabricated the data. It apologised for a search it HAD run — four independent lines
of evidence confirm the search was real. The defect is not the model's honesty. It is
that we deleted the evidence and then asked it to account for itself.
Change: agentic_loop accumulates the source URLs it already walks past (citations and
web_search_tool_result content) and returns them as "sources"; handle_chat_agentic folds
tools_used + sources into a receipt line stored WITH the assistant turn. Receipts are
unconditional — a negative receipt ("no tools ran") is the other half of the guarantee,
because "no evidence of a tool" and "evidence of no tool" were previously identical in
the transcript. conv_history_block splits the receipt off before snipping so a long
answer cannot truncate away the evidence. The user never sees it: it is appended to the
history copy, not the reply.
FIX B — one history key for both paths (kills the BLANK STARE)
Root cause, EXECUTED-verified: the agentic path keyed history on session_hist_<id>; the
plain path was hard-wired to the process-global conv_history and never read session_id.
One conversation, two buckets. Proven in the guest engram: the scoped node held exactly
two turns starting at "Try again" while the earlier exchanges sat unscoped.
Change: conv_hist_key/conv_hist_label are now the single definition, used by BOTH paths;
session_id is threaded route -> layered_cycle -> layered_generate / conv_history_record.
The 2-line fallback (plain path reads the agentic key) was REJECTED: it keeps the
process-global bucket as a live write target, which is the bleed the TODO describes.
Also found and closed while threading: layered_cycle read session_id from the state key
"current_session_id", which is read here and WRITTEN NOWHERE in the entire source. It
was unconditionally "", so TODO(reliability #4) — per-session steward continuity — was
dead code that could never fire. It fires now.
LAZY SESSION, decided explicitly: we create the session EAGERLY at the door (app half,
ui#223) rather than migrating orphaned turns. Migration would copy the CONTENTS of a
process-global bucket, possibly another conversation's, into a named session — the bleed,
performed deliberately. Eager creation makes the situation impossible instead. Migration
is deliberately not implemented and must not be added without solving provenance first.
FIX C — the two text-join seams ("to.Good", byte-verified 0x77 0x2e 0x47)
Two bare `+` joins, written a year apart, had drifted into two answers to one question:
within-response block joins (Will's, 2026-05-03, latent until server-side web_search
began interleaving non-text blocks) and across-round joins (ours,
|
||
|
Tim Lingo
|
635f6febe4 |
feat(engine): plain chat generates at L3 — inside the safety cycle, not around it
Non-agentic /api/chat (the desktop app's default "Tools: Off" mode) returned the
user's own screened text as a bare non-JSON string. Every JSON client failed to
parse it and showed "Couldn't reach Neuron - it may be offline."
Root cause:
|
||
|
will.anderson
|
a771ed2d0f |
self-review 2026-08-05: heartbeat carries redundancy-suppression gauges
dup_seeds / dup_wm from engram_act_stats_json. dup_seeds is the one that matters day to day: a healthy nonzero rate means the suppressor is reclaiming seed slots the June duplicate import was stealing; a sustained fall toward zero means the duplicates were finally merged out of the graph, which is the repair this defends against. Cumulative like wm_evicted/breakthroughs. |
||
|
Tim Lingo
|
62af5649fe |
feat(engine): port Anthropic server-side web_search into the agentic loop
Re-authors soul-webfix-20260711.patch in El (the patch is a diff against generated C at month-old offsets, so nothing was applied as a patch). Its last two hunks — an unrelated /api/safety-contact implementation — were deliberately not ported; that route already exists and is safety-critical. Activation restores existing design, it does not invent a mechanism: commit |
||
|
Tim Lingo
|
710761e2d5 |
fix(engine): send tool_choice.disable_parallel_tool_use on agentic loop (STOPGAP)
The agentic loop keeps only the FIRST tool_use block per round (chat.el:2281,
"Capture first tool_use block only"). Anthropic lets a model emit several
tool_use blocks in one message and requires a tool_result for every one, so a
parallel-tool turn is answered once, the rest are dropped, and the next request
dies with:
tool_use ids were found without tool_result blocks immediately after
(neuron#78 quotes this as "tool_use ids found without tool_result"; the above is
the API's actual wording - recorded so the next person's grep matches.)
This constrains the wire to match what the loop can assemble:
"tool_choice":{"type":"auto","disable_parallel_tool_use":true}
STOPGAP - AND THE DURABLE FIX ALREADY EXISTS. A correct multi-tool loop is
already in Will's EL runtime, in C, and the soul does not call it. Verified on
el:origin/main lang/el-compiler/runtime/el_runtime.c: llm_register_tool:9616,
llm_build_tool_results:9743 - which walks EVERY content block, emits one
tool_result per tool_use, and sets is_error for an unregistered tool -
llm_call_agentic:9817 calling it at :9918, iteration cap 10 at :9847. Will's
commit 12d5e77 (2026-04-30). grep for llm_call_agentic/llm_register_tool across
every neuron/*.el returns nothing; dist/soul.c has zero references. chat.el
hand-rolls its own single-tool loop instead, and that is the one that breaks.
The durable fix is therefore to register the soul's tools via llm_register_tool
and call llm_call_agentic - deleting a loop, not writing one. See ADR 0005.
Our own approved spec called this seven weeks ago:
docs/research/agentic-tool-approval-design.md (2026-06-12, "Approved for build"),
line 20 on the defect, line 30 on the goal ("Execute all tool_use blocks in a
turn (one result per block)").
Two edits, because dist/soul.c cannot be regenerated here (Will's gated elc/elb
toolchain is not on this machine):
(a) chat.el:2255 - source of truth, so a later regen carries the fix.
One edit covers all three routes: agentic_loop is called from chat.el:2152
(/api/chat agentic), :2676 (dharma room) and :2496 (agentic_resume).
(b) dist/soul.c:28173 - generated form, hand-spliced. Line 27624 is the
non-agentic/OpenAI-compat req_body (no tools) and was left untouched.
Prior art reused rather than reinvented: soul-narrated-runs-20260713.patch
(27,824 bytes) line 78 spliced this same string into the same concat chain on
2026-07-13.
Deliberately NOT ported from that patch, having read it: max_tokens is not
changed by it (16384 sits on both sides of the hunk; our main's 4096 is a
separate output-truncation concern), and its pause_turn pairing fix - same
defect class - is unreachable today because no server-side web_search is wired
(agentic_tools_with_web at chat.el:1418 is never called), so it is untestable
and logged instead.
Proven E2E on a scratch profile and port 7791, never the live chain. A/B against
a pristine origin/main control built from the same vendored runtime: fixed
completed the mission (tools_used read_file x3, 4 iterations, correct answer);
control failed 3/3. Direct API probe confirmed the mechanism - without the field
the model emits 3 parallel tool_use blocks and replaying the unfixed loop's next
turn returns HTTP 400; with it, exactly 1 block.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
will.anderson
|
74520b8333 | Merge pull request 'ci: pin + complete vendored el-runtime so reconciled soul.c links' (#105) from ci/pin-vendored-runtime into main | ||
|
will.anderson
|
7f3d6ed8cd |
ci: update vendored el-runtime to complete v1.0.0-20260501
The runtime vendored alongside the CI pin was the Jul-21 snapshot, which predates two builtins the reconciled ship-soul now calls: - http_delete_json (boot-counter HTTP write-back, awareness/memory self-review) - engram_act_stats_json (heartbeat activation observability) Compiling dist/soul.c against the stale runtime fails with implicit-declaration errors. Vendor the current release runtime (identical to the one the soul was gate-verified against: verify-soul-contract PASS, genesis boots clean, full safety-contact) so the CI Linux soul is byte-for-byte the verified soul. |
||
|
will.anderson
|
eed6487114 |
ci: pin soul build to vendored release runtime v1.0.0-20260501
The soul build downloaded el-runtime-c 'latest' from Artifact Registry. The merged ship-soul calls engram_prune_telemetry, which the latest published runtime no longer defines, so an unpinned build fails to link — the failure mode that let a broken/handlerless soul reach prod. Vendor the release runtime v1.0.0-20260501 (el_runtime.c/.h) into the repo and compile the soul against it. This is the exact runtime the merged soul was verified against (verify-soul-contract GATE PASS, genesis boot survives, full safety-contact response), making the build reproducible and independent of a moving AR 'latest'. The verify-soul-contract.sh HARD-BLOCK gate already runs before Publish (from the CI-hardening arc on main), so a destructive or stale soul can never publish/deploy again. |
||
|
will.anderson
|
2c2aaa0653 | Merge pull request 'Reconcile: main = union of all ship-critical soul fixes (beta-gating)' (#104) from reconcile/soul-union-main into main | ||
|
will.anderson
|
e610a412b9 |
regen soul.c from reconciled tree + harden contract gate (#199 by-id, isolation)
dist/soul.c: regenerated amalgamation (1.15MB) from the reconciled sources via the hide-.elh + elc --target=c recipe, so the shipped translation unit CI compiles now actually carries every landed fix — genesis-boot SIGSEGV (#150), safety-contact 988 truncation (#96), url-decode multi-word search, honest receipts (#100/#101), immutability arc (#83), and the bounded payloads (#103). verify-soul-contract.sh, two non-weakening fixes (both false-NEGATIVE bugs that spuriously failed a CORRECT soul; neither relaxes what fails a defective one): 1. #199 by-id gate: verify tombstone/KEPT via /api/neuron/graph?id=<id>&depth=1 (a compact neighborhood) instead of grepping engram_scan_nodes_json(9999,0) — a multi-MB, salience-ordered, 9999-capped whole-graph dump in which the salience-0.01 tombstone marker sorts past the cap and vanished. 2. Isolation: pin SOUL_ISE_URL to the dead axon port. Unsetting ENGRAM_URL was not enough — the periodic engram sync defaults its source to the LIVE engram (http://localhost:8742), so the 'isolated' gate pulled the operator's real brain (56 -> 12k nodes in seconds), which both broke Section B determinism and read live state. Now the soul stays on its own store. Verified GREEN on a throwaway port/HOME (live :7770/:8742/~/.neuron untouched): gate PASS x3 (presence 27/27, immutability all 5 KEPT); safety-contact POST 218B / GET 208B full untruncated; multi-word search (%20 and +) returns ranked hits with an all-gibberish control at 0; bounded session/begin 1370B; honest ok:false on a missing-id delete; genesis (ntn-genesis) boots clean through mem_save with no SIGSEGV. |
||
|
will.anderson
|
8ba35a0d56 |
reconcile: merge Will's self-review WIP (456267a) into main
Union of all ship-critical soul fixes for the Mac beta: - Keeps main's honest receipts (#100/#101), immutability arc + #199 by-id gate (#83), Track B threat routing (#76), bounded beginSession (#103), CI hardening (#85/#86), elc typo hotfix (#77), neuron-dev-setup (#84). - Brings WIP's genesis-boot SIGSEGV fix (#150/#95), safety-contact 988 truncation fix (#96), bounded-persona floor (#93), and 10 self-review commits (importance flattening, curiosity DF gating, WM/heartbeat observability, boot-counter telemetry). - Folds the multi-word ranked-search fix: api_query_param now url_decode()s the extracted value so q=foo%20bar / foo+bar tokenize as two words. Conflicts (neuron-api.el payload-bound comments, mcp-wrapper tool_forget) resolved toward the correct end state: main's verified honest-receipt read-back is kept; WIP's improved forget description is kept. Generated dist/*.c taken from main and will be regenerated from this reconciled source in the following commit. |
||
|
will.anderson
|
21710d5c8e |
self-review 2026-08-03: gate curiosity auto-terms on label document frequency
Reject an extracted auto-term when its label document frequency exceeds node_count/400 (floor 8) -- measured live at 12,859 nodes, threshold 32. Live label df separates the classes by an order of magnitude: rejected: <!--:220 SELF:175 Engram:125 CORE:88 STAR:36 passed: Dual:12 Sparse:8 Latent:6 MemQ:1 dGRPO:1 engram_goal_bias:1 Verified against the running soul (boot 21). Peak curiosity activation fell from 541 to 113; the flood terms (SELF, CORE, Engram, STAR, <!--) are absent from post-fix scans while topical compound identifiers pass untouched. Sample is 7 scans -- suggestive, not conclusive; watch the next review. Nested conditional rather than max(): El let is single-assignment, so the floor is expressed as a second conjunct. Verification note: content df was tested as an alternative signal and rejected -- 'Curiosity' has the highest content df in the store (5526) yet one of the lowest activation counts (113). Label df is the correct field because label is what the first-word extractor reads. |
||
|
will.anderson
|
e60ca8123b |
self-review 2026-08-02: record that el_from_float on a literal is not the double-wrap bug
Investigated awareness.el ise_post's local-fallback engram_node_full call as a suspected instance of the score-mangling double-wrap fixed in server.el on 2026-08-01. It is not one. Removing the wrapper produces byte-identical codegen: the compiler treats el_from_float as the boxing intrinsic, so both `el_from_float(0.3)` and a bare `0.3` emit exactly one el_from_float(0.3). The server.el bug was different in kind - there the arguments came from json_get_float(), already boxed as el_val_t, and wrapping those a second time reinterprets the boxed bits as a raw double, fails engram_decode_score's range check, and silently clamps to defaults. Comment only, no behavior change. Recording the negative result at the call site so the sweep criterion is right: look for el_from_float applied to an already-boxed expression, never to a literal. Grepping the call name alone produces false positives, which is what happened here. |
||
|
will.anderson
|
b75d5c8c30 | Merge pull request 'Bound beginSession/compileCtx payloads to a compact digest (main)' (#103) from fix/bound-session-payload-main into main | ||
|
will.anderson
|
9bbb4f2af8 |
Bound beginSession/compileCtx payloads to a compact digest
Port the payload-bounding fix (PR #102, commit
|
||
|
will.anderson
|
ec219c5830 | Merge pull request 'fix(mcp-wrapper): forget/delete tools no longer return fake ok receipts (BUG-18)' (#101) from fix/receipts-wrapper-forget into main | ||
|
will.anderson
|
731efaedaf | Merge pull request 'fix(chat): agent write_file/edit_file no longer return false success receipts (BUG-29)' (#100) from fix/receipts-agent-tools into main | ||
|
Tim Lingo
|
3723e3b7e7 |
fix(mcp-wrapper): forget/delete tools no longer return fake ok receipts (BUG-18)
Root cause: two false-receipt paths in the wrapper's delete family.
- delete_by_id (removeKnowledge, deleteProcess, deleteImprint,
dischargeWonder) FABRICATED {"ok":true,...,"note":"soft-deleted"}
without calling the soul at all — the 'soul does not yet expose a delete
HTTP route' note was stale (/api/neuron/node/delete exists and tombstones
any node type).
- tool_forget forwarded the soul's response but never verified the deletion
actually persisted before answering ok.
The change (Receipt Contract rule 1 — a tool result must reflect what
actually happened):
- delete_by_id now routes to the soul's real /api/neuron/node/delete and
propagates its answer (honest 'node not found' for bad ids).
- Both handlers read back before answering ok: GET /api/neuron/graph?id=..
&depth=1 must show the tombstone marker (label "tombstone:<id>"); if it
does not, answer {"ok":false,"error":"delete_not_persisted",...} in
the soul's not-persisted error shape (api_not_persisted).
- Soul errors and transport failures pass through unchanged.
E2E evidence (sandbox soul :7791 + wrapper :7792, elb builds):
- unpatched: removeKnowledge on a NONEXISTENT id -> {"ok":true,
"deleted":"kn-DOES-NOT-EXIST-deadbeef","note":"soft-deleted"} (lie)
- patched: same call -> {"error":"node not found: ..."} (soul's answer)
- happy path: remember -> forget -> {"ok":true,"tombstoned":true};
read-back: hidden from default /list/Memory, present with
?include_deleted=1, node KEPT in full graph view (immutability intact)
- scripts/verify-soul-contract.sh on the soul it talks to: GATE PASS
(27/27 presence + immutability)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Tim Lingo
|
62c562a3f1 |
fix(chat): agent write_file/edit_file no longer return false success receipts (BUG-29)
Root cause: dispatch_tool's write_file returned {"ok":true} without checking
fs_write's result, and edit_file returned ok:true even when old_text was absent
(str_replace silently no-ops) and its fs_write was also unchecked. Any failed
or no-op write fed the model a false success receipt, which it then repeated
to the user as fact.
The change (Receipt Contract rule 1 — a tool result must reflect what actually
happened):
- write_file: check fs_write's return (1 = all bytes written, 0 = fail);
on failure return {"error":"write failed"} in the handler's existing
error-JSON shape.
- edit_file: reject empty old_text, verify old_text is actually present
(str_contains) before replacing, and check the fs_write result the same way.
- Verification is by operation result, NOT an fs_read read-back: fs_read arms
the runtime's one-shot binary send length, the exact mechanism that truncated
the safety-contact response (#96). Same honest-write pattern as that fix.
E2E evidence (sandboxed elb build, dispatch_tool driven directly):
- unpatched: write_file into a chmod-000 dir -> {"ok":true} (lie);
edit_file with absent old_text -> {"ok":true} (lie, file untouched)
- patched: same calls -> {"error":"write failed"} /
{"error":"old_text not found in file"}; happy paths still ok:true
- scripts/verify-soul-contract.sh on the patched soul: GATE PASS (27/27
presence + immutability)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
will.anderson
|
456267a771 |
self-review 2026-08-01: fix importance flattening in remember/evolve/cultivate paths
Four sites passed the Float local 'sal' through el_from_float() a second time. el_val_t is the bit-pattern of the double, so re-wrapping performs an int64->double VALUE conversion of the bits before re-bitcasting — garbage that fails engram_decode_score's range check and clamps to defaults. Net effect: importance="critical" stored 0.5/0.5 — importance levels were cosmetic on the MCP memory path. Verified fixed live: critical now stores salience/importance 0.95/0.95. Same bug fixed today in engram server.el route_create_node (foundation/el 7f03876). Literal wraps (el_from_float(0.9)) are safe — elc passes numeric literals raw. |
||
|
will.anderson
|
edb0670670 |
self-review 2026-08-01: emit discrete wm_saturation_transition ISE
wm_saturated was a sampled boolean — the 0->1 onset and 1->0 release moments were only recoverable by hand-diffing consecutive heartbeats. Emit a low-rate transition ISE at each edge carrying the WM top-5 at that instant, so the composition that caused the regime change is captured rather than the composition up to 59s later. First beat of a boot never fires (restart is not a transition). |
||
|
will.anderson
|
872120c757 |
Bound beginSession/compileCtx payloads to a compact digest
The session-init endpoints concatenated unbounded engram activate/scan
results as FULL node objects (content up to ~90KB per node), producing a
~900KB response. After the MCP wrapper re-escapes that into a stringified
text block the client dropped the socket ('connection closed unexpectedly')
on every beginSession call. Cap each list (8-10 activated, 10-20 recent)
and project every node to a light identity plus a bounded, UTF-8-safe
content snippet. Response drops from ~900KB to ~12KB; full content stays
available on demand via recall/fetch/inspectGraph.
|
||
|
will.anderson
|
f3660e92a1 |
self-review 2026-07-31: heartbeat deltas for cumulative counters, embed_eligible; fix stale semantic-seeding comment
Runtime activation counters are now cumulative, so the heartbeat emits wm_evicted/breakthroughs as totals plus wm_evicted_delta/ breakthroughs_delta (state-tracked change since the previous beat) — events between beats are no longer lost. Adds embed_eligible from /api/stats so coverage reads as embed_count/embed_eligible instead of the misleading absolute count, and surfaces auto_term_streak in the heartbeat stream. Replaces the false 'semantic seeding NOT implemented' comment: engram_activate embeds the query, seeds semantic top-K, gates propagation on cosine, and scores WM promotion semantically. |
||
|
will.anderson
|
b0f4d6c493 |
self-review 2026-07-30: auto_term stopword filter, real idle_ms, bounded beginSession
- awareness.el: curiosity auto_term was the raw first word of a WM label with no term-quality scoring — observed seeds included What, Colon, Prose, Context. Replaced the 7-word genre blocklist whack-a- mole with a delimited stopword membership test (function words + document-structure words); topical terms pass untouched. Verified live: seeds now ReasonEdit, Reasoning-model, Self-review. - routes.el + awareness.el: idle counter only reset on rare inbox synthesis-requests, so idle==pulse always (zero information). handle_request now stamps soul.last_activity_ts on every inbound HTTP request; heartbeat emits idle_ms = ms since last request (-1 until first request of a boot). - neuron-api.el: beginSession concatenated a depth-2 spread plus the unbounded self-hub neighbor dump — multi-MB response, doubled by wrapper re-escaping, socket died on every call. Now depth-1 and the hub dump dropped (identity loading has its own tool). Verified: beginSession returns instead of closing the socket. |
||
|
will.anderson
|
627eb534a2 |
self-review 2026-07-28: close ISE lifecycle observability gaps
- session_start now also posted to the HTTP Engram via ise_post: the local engram_node_full write never crossed to the observable stream (sync flows HTTP->soul only), so boots 5+ were invisible — last visible session_start was boot 4, two weeks ago - graceful shutdown emits a final ISE with boot/pulse/uptime; a boot with no shutdown event now reliably signals a crash/SIGKILL - empty /api/sync responses emit a sync_empty warn ISE instead of being silently skipped — unreachable engram no longer looks identical to quiet-but-healthy - sync backflow prune reads ENGRAM_ISE_RETENTION_MS instead of duplicating the 48h magic number server.el already honors |
||
|
will.anderson
|
2b612ed5d4 |
self-review 2026-07-27: heartbeat carries activation observability
Fold engram_act_stats_json() into the heartbeat ISE: wm_evicted and breakthroughs (per curiosity-scan activate call) plus embed_breaker_open — the failure mode embed_ok structurally cannot see (it pings the Ollama root, not the embed pipeline). WM-cap eviction, breakthrough-floor flooding, and silent lexical degradation are now one-glance diagnosable from telemetry. |
||
|
will.anderson
|
8392f44c45 |
self-review 2026-07-26: heartbeat wm_churn + wm_top0_wm; fix streak counting on empty id
- wm_churn: count of top-5 WM ids absent from previous beat — separates 'one stuck node' from 'whole WM frozen' without hand-correlating ISEs. - wm_top0_wm: leader's weight; a frozen anchor reads as a constant here. - Streak guard: before the runtime emitted id in wm_top JSON, json_get(...,"id") was always empty and the streak incremented on ""=="" every beat — wm_top0_streak measured uptime, not fixation. Empty id now resets the streak to 0. |
||
|
will.anderson
|
58a9eda311 |
self-review 2026-07-25: break curiosity positive-feedback loop; observability for WM regime
proactive_curiosity strengthened its top result unconditionally every
scan — a positive-feedback fixed point that pinned auto_term on the same
node's first word for hours ('Fast-slow' era). Strengthen now fires only
when the top node changed since the last scan, and a 4-deep finst-style
tabu ring (ACT-R declarative finsts) hard-excludes recently used auto
terms (~2 min at the 30s cadence). Quoted-title guard stops '"The'
leaking through the >3-char stopword check and seeding lexical floods.
Heartbeat now pumps /api/embed-backfill?n=32 on the authoritative store
(its lazy backfill had no production trigger; coverage stalled at
93/12175) and emits wm_saturated, wm_top0_streak, embed_backfilled,
embed_count. Curiosity ISE emits auto_term_streak. The stuck-WM failure
mode is now a one-glance signal instead of manual ISE cross-referencing.
|
||
|
will.anderson
|
fb0bb553f3 |
self-review 2026-07-24: boot counter — demote to telemetry weight, restore persistence via HTTP write-back
Three stale soul:boot_count copies (salience .9, importance .9, Canonical: +0.2 tier bias, 0.15 threshold) held the top WM slots for 23h — a boot counter outcompeting real context. Demoted to salience .55 / importance .2 / tier Working: plumbing, not memory. Persistence was also broken: in HTTP-engram mode the server owns state and nothing wrote the counter back — the log shows boot #5 on three consecutive boots. mem_boot_count_inc now mirrors the persona write-back: delete stale server copies (matched by content prefix — route_create_node sets label=content), create the replacement server-side. Working tier is in the boot seed (/api/nodes) but excluded from periodic /api/sync, so the count survives restarts without re-importing mid-session. Verified: restart incremented 1->2 with exactly one server-side counter node. |