fix(engine): history keeps its provenance and its session — kills the false confession, the blank stare, and the "to.Good" seams #114
Merged
tim.lingo
merged 3 commits from 2026-08-07 15:53:36 +00:00
fix/soul-history-provenance-20260805 into main
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Tim Lingo
|
8f3a478771 |
fix(engine): excise the receipt, do not truncate at it — a leading receipt was erasing whole answers
CAUGHT BY A/B, AND ONLY BY A/B. The previous commit's receipt_strip assumed the receipt is
always TERMINAL and cut everything from the marker onward. It is not always terminal: once
receipt_rule told the model what [[RECEIPT ...]] means, the model sometimes LED with one and
wrote the answer underneath. Cutting at the marker then deleted the entire answer and the
turn returned {"error":"no response"}.
MEASURED, same prompt (two web searches, cited prose), fresh session each run:
round-7 brain 4 / 4 answered (471, 473, 473, 544 chars)
round-8 brain 2 / 7 answered (five {"error":"no response"})
This looked exactly like a flaky model. It was not — it was mine. Running the two brains
side by side on the same prompt is the only reason it was found, and it is the reason the
A/B is now part of how this class gets tested.
AFTER THE FIX, same protocol:
round-7 brain 4 / 4 (473, 473, 473, 544)
round-8 brain 4 / 4 (657, 657, 657, 673)
THE FIX: remove the [[...]] span and keep BOTH sides, instead of truncating at the marker.
An unterminated marker at position 0 is left completely alone — no rule about receipts is
worth erasing an answer over. Bounded four-pass loop rather than a conditional exit, because
rebinding the counter inside an if-expression is the block-expression shape that miscompiles
integer arithmetic under this elc (BUG-PLAINCHAT-1). Verified in the generated C:
str_slice(rest, (e + 2), str_len(rest)) <- integer addition, correct
el_str_concat(head, tail) <- string concat, correct
and zero el_str_concat(<ident>, str_len(...)) sites across all 49 modules.
SEAM PROOF (FIX C) rides on the same runs — a real two-search cited answer, inspected byte
by byte, in BOTH failure directions:
missing separator (the round-7 "to.Good", bytes 77 2e 47): 0 hits. Sentence boundaries
measure 2e 20 4d — "." SPACE "M".
over-separation (a cited sentence shattered across paragraphs): 0 hits. The answer is one
continuous paragraph with its sentences intact, which is the direction a blanket
separator would have broken.
BUILT: sha256 77115f2733e794c5bc4ad1f55b1acaf658f8f4a91cccd423a2d633d94a726cbc
Refs neuron#109
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Tim Lingo
|
9ea41eed78 |
fix(engine): the model was signing its own answers with our receipt — strip it
FOUND BY E2E, NOT BY REASONING. The previous commit's design note asserted the provenance
receipt "never reaches the user: it is appended to the history copy, not the reply." That
was FALSE, and only running the thing showed it. On the first live run against the built
DMG brain, two agentic turns out of two came back with
"Your favourite colour is chartreuse and your project is called Perihelion.
[[RECEIPT - recorded by the soul, not written by the model: no tools ran on this turn.]]"
— the receipt in the user-visible reply.
MECHANISM: the receipt is stored inside the assistant turn, and the agentic path replays
history VERBATIM as Anthropic message objects. So the model sees its own previous answers
ending in [[RECEIPT ...]] and does the obvious thing — it imitates the format and signs the
next answer the same way. The plain path did NOT leak, which is the tell: there, history is
rendered into the SYSTEM prompt as labelled lines rather than replayed as assistant turns,
and a model imitates its own turns far more readily than a transcript.
FIX, two layers, because one of them is not a guarantee:
- receipt_rule() names the marker in both system prompts (plain and agentic): these lines
are written by the system, read them as evidence, never write one. Reduces occurrence.
- receipt_strip() truncates any [[RECEIPT ...]] out of model output before it becomes the
reply — plain path in layered_generate, agentic path on final_text in agentic_loop.
Deterministic. A guard that depends on the model choosing to obey is exactly the class of
thing round 8 exists to stop shipping, so the instruction is the optimisation and the
strip is the guarantee.
Placed ABOVE agentic_loop's empty-check on purpose: a turn whose entire output was an
imitated receipt has produced no answer, and must be reported as no answer.
The receipt stays in HISTORY, which is the whole point and is proven to work: asked "What
source did you use for that?" one turn after a live web_search, this brain answered
"I used Weather Underground (https://www.wunderground.com/weather/is/reykjav%C3%ADk) for the
current temperature in Reykjavik" — a real source, no apology. That is the false confession
dead, and it is dead BECAUSE the model can read the receipt.
BUILT: 887,112 bytes, sha256 54a2eff84d4fa44f8d2db6781dcf225df4b5075a8ec5f1058018bd40cc1af10b
BUG-PLAINCHAT-1 miscompile guard: zero sites.
Refs neuron#109
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Tim Lingo
|
ff421d39f6 |
fix(engine): history keeps its provenance and its session — the false confession and the blank stare
DESIGN FIT: three of round 7's five defects share ONE root — the conversation-history
layer persists only {role, content}, discarding tool provenance, session scoping, and the
distinction between a real user turn and an internal utility call. Fixes A and B RESTORE
Will's design rather than extend it: his agentic path already scopes history per session,
the plain path never got it, and his own source carries the TODO admitting the resulting
race (chat.el, handle_chat: "process-global key; concurrent /api/chat requests without
session_id race on this read-append-write"). Fix C repairs one join Will wrote that was
correct for a year and one we added last week. E1/E2 are ours.
FIX A — tool provenance in history (kills the FALSE CONFESSION)
Root cause, EXECUTED-verified: handle_chat_agentic recorded turns via hist_append, which
emits {"role","content"} only. server_tool_use blocks, web_search_tool_result blocks and
every citation were discarded, then replayed as text. On the next turn the model saw a
data-rich answer with zero evidence a search had happened, and its own permanent rule
("never describe a search you did not perform") left one conclusion available: that it
had fabricated the data. It apologised for a search it HAD run — four independent lines
of evidence confirm the search was real. The defect is not the model's honesty. It is
that we deleted the evidence and then asked it to account for itself.
Change: agentic_loop accumulates the source URLs it already walks past (citations and
web_search_tool_result content) and returns them as "sources"; handle_chat_agentic folds
tools_used + sources into a receipt line stored WITH the assistant turn. Receipts are
unconditional — a negative receipt ("no tools ran") is the other half of the guarantee,
because "no evidence of a tool" and "evidence of no tool" were previously identical in
the transcript. conv_history_block splits the receipt off before snipping so a long
answer cannot truncate away the evidence. The user never sees it: it is appended to the
history copy, not the reply.
FIX B — one history key for both paths (kills the BLANK STARE)
Root cause, EXECUTED-verified: the agentic path keyed history on session_hist_<id>; the
plain path was hard-wired to the process-global conv_history and never read session_id.
One conversation, two buckets. Proven in the guest engram: the scoped node held exactly
two turns starting at "Try again" while the earlier exchanges sat unscoped.
Change: conv_hist_key/conv_hist_label are now the single definition, used by BOTH paths;
session_id is threaded route -> layered_cycle -> layered_generate / conv_history_record.
The 2-line fallback (plain path reads the agentic key) was REJECTED: it keeps the
process-global bucket as a live write target, which is the bleed the TODO describes.
Also found and closed while threading: layered_cycle read session_id from the state key
"current_session_id", which is read here and WRITTEN NOWHERE in the entire source. It
was unconditionally "", so TODO(reliability #4) — per-session steward continuity — was
dead code that could never fire. It fires now.
LAZY SESSION, decided explicitly: we create the session EAGERLY at the door (app half,
ui#223) rather than migrating orphaned turns. Migration would copy the CONTENTS of a
process-global bucket, possibly another conversation's, into a named session — the bleed,
performed deliberately. Eager creation makes the situation impossible instead. Migration
is deliberately not implemented and must not be added without solving provenance first.
FIX C — the two text-join seams ("to.Good", byte-verified 0x77 0x2e 0x47)
Two bare `+` joins, written a year apart, had drifted into two answers to one question:
within-response block joins (Will's, 2026-05-03, latent until server-side web_search
began interleaving non-text blocks) and across-round joins (ours,
|