fix(engine): history keeps its provenance and its session — kills the false confession, the blank stare, and the "to.Good" seams #114
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: neuron-technologies/neuron#114
Reference in New Issue
Block a user
Delete Branch "fix/soul-history-provenance-20260805"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Builds on #109 (
feat/soul-plain-chat-generation-20260805) and targets that branch, not main.Design fit — one root, three symptoms
Three of round 7's five defects share a single cause: the conversation-history layer persists only
{role, content}, discarding tool provenance, session scoping, and the difference between a real user turn and an internal utility call.Fixes A and B restore Will's design rather than extend it. His agentic path already scopes history per session; the plain path never got it, and his own source carries the TODO admitting the resulting race (
handle_chat: "process-global key; concurrent /api/chat requests without session_id race on this read-append-write"). Fix C repairs one join Will wrote that was correct for a year plus one we added last week.FIX A — tool provenance in history (kills the FALSE CONFESSION)
Root cause, EXECUTED-verified.
handle_chat_agenticrecorded turns viahist_append, which emits{"role","content"}only.server_tool_useblocks,web_search_tool_resultblocks and every citation were discarded, then replayed as text. On the next turn the model saw a data-rich answer with zero evidence a tool ran, and its own permanent rule ("never describe a search you did not perform") left one conclusion available: that it had invented the data. It apologised for a search it had genuinely run.The defect is not the model's honesty. It is that we deleted the evidence and then asked it to account for itself.
Change.
agentic_loopaccumulates the source URLs it already walks past (citations andweb_search_tool_resultcontent) and returns them assources;handle_chat_agenticfoldstools_used+sourcesinto a receipt line stored with the assistant turn. Receipts are unconditional — a negative receipt ("no tools ran") is the other half of the guarantee, because "no evidence of a tool" and "evidence of no tool" were previously identical in the transcript.Proven live, same session, one turn apart:
A real source. No apology.
Stopgap, and said so in the code. The real answer is your Receipt Contract (#78): structured receipts on the wire that a client renders and a model cannot confuse with prose. This is the step toward it.
FIX B — one history key for both paths (kills the BLANK STARE)
Root cause, EXECUTED-verified. Agentic keyed on
session_hist_<id>; plain was hard-wired to the process-globalconv_historyand never readsession_id. One conversation, two buckets. Proven in the guest engram: the scoped node held exactly two turns starting at "Try again" while the earlier exchanges sat unscoped.Change.
conv_hist_key/conv_hist_labelare now the single definition used by both paths;session_idis threaded route →layered_cycle→layered_generate/conv_history_record. The 2-line fallback (plain path reads the agentic key) was rejected: it keeps the process-global bucket as a live write target, which is exactly the bleed your TODO describes.Found while threading:
layered_cyclereadsession_idfrom the state keycurrent_session_id, which is read there and written nowhere in the entire source. It was unconditionally"", soTODO(reliability #4)— per-session steward continuity — was dead code that could never fire. It fires now.Lazy session, decided explicitly. Sessions are created eagerly at the door (app half, ui#223) rather than migrating orphaned turns. Migration would copy the contents of a process-global bucket — possibly another conversation's — into a named session: the bleed, performed deliberately. Eager creation makes the situation impossible instead. Documented in the code as not-to-be-added-later without solving provenance first.
Proven live: Tools OFF → "my favourite colour is chartreuse and my project is Perihelion"; Tools ON → "Try again. What did I say?" → both repeated back.
FIX C — the two text-join seams ("to.Good")
Byte-verified in a shipped reply:
0x77 0x2e 0x47. Two bare+joins written a year apart had drifted into two answers to one question — within-response block joins (yours, 2026-05-03, latent until server-sideweb_searchbegan interleaving non-text blocks) and across-round joins (ours,62af564).One named rule,
text_join_sep, at both sites. Deliberately not a blanket separator: a cited answer splits mid-sentence ("The current temperature is " + "86°F" + ", with "), so separating every text block shatters every sourced sentence. The rule takes the one bit that distinguishes the cases — whether a non-text block intervened. Hoisting it also makes the fix verifiable in the shipped binary, which an inline+is not.Proven both directions on a real two-search cited answer: missing-separator hits 0 (boundaries measure
2e 20 4d, "." SPACE "M"), over-separation hits 0 (one continuous paragraph, sentences intact).FIX E1 / E2
"model":"unknown"rows in usage.jsonl).is_utility_requestreads an explicitutilityflag, with__title__/__insight__prefixes as a fallback for older clients. Proven: after a utility title turn, "what was the last thing I asked, word for word?" returns the user's actual message.OPERATOR IDENTITYblock (env USER/HOME, closing "This is a hard rule") was prepended to every system prompt including chat mode, where no filesystem is reachable. It is why a fresh conversation opened with "You're test, on your machine at /Users/test". Hoisted tooperator_identity_block()and gated on!chat_mode. Unchanged wherever a tool can actually be reached.Two defects this PR introduced and then fixed — both found by running it
I am flagging these because the commits are in the history and the second one is instructive.
[[RECEIPT ...]]and signed its next answer the same way — visible to the user, on 2 of 2 turns. Fixed withreceipt_rule()(instruction) plusreceipt_strip()(deterministic guarantee).receipt_striperased whole answers. It assumed the receipt is always terminal and truncated at the marker. When the model led with a receipt, that deleted the answer:{"error":"no response"}. Measured A/B, same prompt, fresh session each run: round-7 brain 4/4 answered; that build 2/7. It looked exactly like a flaky model. Now excises the[[...]]span and keeps both sides; an unterminated marker at position 0 is left alone entirely. After: 4/4, matching round 7.The A/B against the previous brain is the only reason #2 was caught, and it is now part of how this class gets tested.
How to test
Sandbox: throwaway
HOME, throwaway engram, dead axon/ISE,NEURON_PORTset explicitly on a verified-free port. (~/neuron_soul_smokebuild.shstill exportsSOUL_PORT, which the soul ignores — it readsNEURON_PORT, default 7770, the live port. Still unfixed, flagged on #108 and #109.)test_layered_cycle35/35 ·test_layer_contract56/59 ·test_safety51/1 ·test_bell_safety54/2 ·test_stewardship35/3 ·test_agentic_tools18/0 ·test_sessions26/1 ·test_bridge_serialization24/0 — byte-identical to baseline, all 7 failures pre-existing and documented.Coverage caveat, stated plainly: those suites do not reach the
soul.el/routes.eledits —test_layered_cyclereplicates the chain inline asrun_layered_cycle()becauselayered_cycleis not exported fromsoul.elh, and the harness excludessoul.c/routes.c. "Identical" means no collateral damage to the modules they cover; the live-route runs above are what actually exercise the threading.Build
elb+ccfrom.elsource (not the staledist/soul.c), repo-pinned runtimevendor/el-runtime/v1.0.0-20260501. As-built77115f27…. Zeroel_str_concat(<ident>, str_len(...))sites across all 49 modules (the BUG-PLAINCHAT-1 miscompile guard).web_search_20250305anddisable_parallel_tool_useboth still present — #108's web search and the ADR-0005 stopgap intact.dist/elp-c-decls.his updated: it is force-included by the documented link recipe and carried the old C arities. That is the build-support header, not generated engine logic.chat.elh/soul.elhare committed because a first-pass build against the old signatures fails (measured); the other regenerated headers are reverted as unrelated churn.Yours to decide
dist/soul.cregeneration — deliberately not done here; the localelcis dated 2026-06-05 and CI compilesdist/soul.c, so none of this reaches production until you regenerate.🤖 Generated with Claude Code
DESIGN FIT: three of round 7's five defects share ONE root — the conversation-history layer persists only {role, content}, discarding tool provenance, session scoping, and the distinction between a real user turn and an internal utility call. Fixes A and B RESTORE Will's design rather than extend it: his agentic path already scopes history per session, the plain path never got it, and his own source carries the TODO admitting the resulting race (chat.el, handle_chat: "process-global key; concurrent /api/chat requests without session_id race on this read-append-write"). Fix C repairs one join Will wrote that was correct for a year and one we added last week. E1/E2 are ours. FIX A — tool provenance in history (kills the FALSE CONFESSION) Root cause, EXECUTED-verified: handle_chat_agentic recorded turns via hist_append, which emits {"role","content"} only. server_tool_use blocks, web_search_tool_result blocks and every citation were discarded, then replayed as text. On the next turn the model saw a data-rich answer with zero evidence a search had happened, and its own permanent rule ("never describe a search you did not perform") left one conclusion available: that it had fabricated the data. It apologised for a search it HAD run — four independent lines of evidence confirm the search was real. The defect is not the model's honesty. It is that we deleted the evidence and then asked it to account for itself. Change: agentic_loop accumulates the source URLs it already walks past (citations and web_search_tool_result content) and returns them as "sources"; handle_chat_agentic folds tools_used + sources into a receipt line stored WITH the assistant turn. Receipts are unconditional — a negative receipt ("no tools ran") is the other half of the guarantee, because "no evidence of a tool" and "evidence of no tool" were previously identical in the transcript. conv_history_block splits the receipt off before snipping so a long answer cannot truncate away the evidence. The user never sees it: it is appended to the history copy, not the reply. FIX B — one history key for both paths (kills the BLANK STARE) Root cause, EXECUTED-verified: the agentic path keyed history on session_hist_<id>; the plain path was hard-wired to the process-global conv_history and never read session_id. One conversation, two buckets. Proven in the guest engram: the scoped node held exactly two turns starting at "Try again" while the earlier exchanges sat unscoped. Change: conv_hist_key/conv_hist_label are now the single definition, used by BOTH paths; session_id is threaded route -> layered_cycle -> layered_generate / conv_history_record. The 2-line fallback (plain path reads the agentic key) was REJECTED: it keeps the process-global bucket as a live write target, which is the bleed the TODO describes. Also found and closed while threading: layered_cycle read session_id from the state key "current_session_id", which is read here and WRITTEN NOWHERE in the entire source. It was unconditionally "", so TODO(reliability #4) — per-session steward continuity — was dead code that could never fire. It fires now. LAZY SESSION, decided explicitly: we create the session EAGERLY at the door (app half, ui#223) rather than migrating orphaned turns. Migration would copy the CONTENTS of a process-global bucket, possibly another conversation's, into a named session — the bleed, performed deliberately. Eager creation makes the situation impossible instead. Migration is deliberately not implemented and must not be added without solving provenance first. FIX C — the two text-join seams ("to.Good", byte-verified 0x77 0x2e 0x47) Two bare `+` joins, written a year apart, had drifted into two answers to one question: within-response block joins (Will's, 2026-05-03, latent until server-side web_search began interleaving non-text blocks) and across-round joins (ours,62af564). Change: one named rule, text_join_sep, at both sites. NOT a blanket separator — a cited answer splits MID-SENTENCE ("The current temperature is " + "86°F" + ", with "), so a blanket separator shatters every sourced sentence. The rule takes the one bit that distinguishes the cases: whether a NON-TEXT block intervened. Hoisting it also makes the fix verifiable in the shipped binary, which an inline `+` is not. FIX E1 — utility generations stay out of the transcript Title generation ("Write a 3-6 word title...") and insight passes ran down the same plain door as a real message and were recorded as if the user had typed them; the same calls are the "model":"unknown" rows in usage.jsonl. is_utility_request reads an explicit utility flag from the app, with the __title__/__insight__ id prefixes as a fallback for older clients. Answered normally, never recorded. FIX E2 — OPERATOR IDENTITY is scoped to tool-capable turns The block (env USER/HOME, closing "This is a hard rule") was prepended to EVERY system prompt including chat mode. On a Tools:Off turn there is no filesystem in reach, so it governed nothing and merely supplied the loudest fact in the prompt — which is why the model opened a fresh conversation with "You're test, on your machine at /Users/test". Hoisted to operator_identity_block() and gated on !chat_mode. Unchanged wherever a file or command tool can actually be reached. ALSO: agentic_loop's per-session history persist had a second hand-rolled copy of conv_history_persist with a different label expression, different salience scores and different tags for the same node. Since both now derive the label from conv_hist_label and engram_node_full upserts by label, two score policies were writing one node. Collapsed to one writer. BUILD NOTE: dist/elp-c-decls.h is force-included by the documented link recipe and carried the OLD C arities, so it is updated here. This is the build-support header, NOT the stale generated dist/soul.c — no dist/*.c was read or edited; all engine changes are .el source. chat.elh/soul.elh are committed because a first-pass build against the old signatures FAILS (measured); the other regenerated headers are reverted as unrelated churn. BUILT: 887,000 bytes, sha256 d632b061ad75269d6adeb52578d030eaf49e895d91289d7f946b19c08450d728 Zero el_str_concat(<int>, str_len(...)) sites (the BUG-PLAINCHAT-1 miscompile guard). web_search_20250305 and disable_parallel_tool_use both still present — PR #108's web search and the ADR-0005 stopgap are intact. Refs neuron#109 (builds on it), neuron#78 (Receipt Contract — the real fix A is a stopgap for) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>FOUND BY E2E, NOT BY REASONING. The previous commit's design note asserted the provenance receipt "never reaches the user: it is appended to the history copy, not the reply." That was FALSE, and only running the thing showed it. On the first live run against the built DMG brain, two agentic turns out of two came back with "Your favourite colour is chartreuse and your project is called Perihelion. [[RECEIPT - recorded by the soul, not written by the model: no tools ran on this turn.]]" — the receipt in the user-visible reply. MECHANISM: the receipt is stored inside the assistant turn, and the agentic path replays history VERBATIM as Anthropic message objects. So the model sees its own previous answers ending in [[RECEIPT ...]] and does the obvious thing — it imitates the format and signs the next answer the same way. The plain path did NOT leak, which is the tell: there, history is rendered into the SYSTEM prompt as labelled lines rather than replayed as assistant turns, and a model imitates its own turns far more readily than a transcript. FIX, two layers, because one of them is not a guarantee: - receipt_rule() names the marker in both system prompts (plain and agentic): these lines are written by the system, read them as evidence, never write one. Reduces occurrence. - receipt_strip() truncates any [[RECEIPT ...]] out of model output before it becomes the reply — plain path in layered_generate, agentic path on final_text in agentic_loop. Deterministic. A guard that depends on the model choosing to obey is exactly the class of thing round 8 exists to stop shipping, so the instruction is the optimisation and the strip is the guarantee. Placed ABOVE agentic_loop's empty-check on purpose: a turn whose entire output was an imitated receipt has produced no answer, and must be reported as no answer. The receipt stays in HISTORY, which is the whole point and is proven to work: asked "What source did you use for that?" one turn after a live web_search, this brain answered "I used Weather Underground (https://www.wunderground.com/weather/is/reykjav%C3%ADk) for the current temperature in Reykjavik" — a real source, no apology. That is the false confession dead, and it is dead BECAUSE the model can read the receipt. BUILT: 887,112 bytes, sha256 54a2eff84d4fa44f8d2db6781dcf225df4b5075a8ec5f1058018bd40cc1af10b BUG-PLAINCHAT-1 miscompile guard: zero sites. Refs neuron#109 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>CAUGHT BY A/B, AND ONLY BY A/B. The previous commit's receipt_strip assumed the receipt is always TERMINAL and cut everything from the marker onward. It is not always terminal: once receipt_rule told the model what [[RECEIPT ...]] means, the model sometimes LED with one and wrote the answer underneath. Cutting at the marker then deleted the entire answer and the turn returned {"error":"no response"}. MEASURED, same prompt (two web searches, cited prose), fresh session each run: round-7 brain 4 / 4 answered (471, 473, 473, 544 chars) round-8 brain 2 / 7 answered (five {"error":"no response"}) This looked exactly like a flaky model. It was not — it was mine. Running the two brains side by side on the same prompt is the only reason it was found, and it is the reason the A/B is now part of how this class gets tested. AFTER THE FIX, same protocol: round-7 brain 4 / 4 (473, 473, 473, 544) round-8 brain 4 / 4 (657, 657, 657, 673) THE FIX: remove the [[...]] span and keep BOTH sides, instead of truncating at the marker. An unterminated marker at position 0 is left completely alone — no rule about receipts is worth erasing an answer over. Bounded four-pass loop rather than a conditional exit, because rebinding the counter inside an if-expression is the block-expression shape that miscompiles integer arithmetic under this elc (BUG-PLAINCHAT-1). Verified in the generated C: str_slice(rest, (e + 2), str_len(rest)) <- integer addition, correct el_str_concat(head, tail) <- string concat, correct and zero el_str_concat(<ident>, str_len(...)) sites across all 49 modules. SEAM PROOF (FIX C) rides on the same runs — a real two-search cited answer, inspected byte by byte, in BOTH failure directions: missing separator (the round-7 "to.Good", bytes 77 2e 47): 0 hits. Sentence boundaries measure 2e 20 4d — "." SPACE "M". over-separation (a cited sentence shattered across paragraphs): 0 hits. The answer is one continuous paragraph with its sentences intact, which is the direction a blanket separator would have broken. BUILT: sha256 77115f2733e794c5bc4ad1f55b1acaf658f8f4a91cccd423a2d633d94a726cbc Refs neuron#109 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>