fix(engine): the model was signing its own answers with our receipt — strip it

FOUND BY E2E, NOT BY REASONING. The previous commit's design note asserted the provenance
receipt "never reaches the user: it is appended to the history copy, not the reply." That
was FALSE, and only running the thing showed it. On the first live run against the built
DMG brain, two agentic turns out of two came back with

  "Your favourite colour is chartreuse and your project is called Perihelion.

   [[RECEIPT - recorded by the soul, not written by the model: no tools ran on this turn.]]"

— the receipt in the user-visible reply.

MECHANISM: the receipt is stored inside the assistant turn, and the agentic path replays
history VERBATIM as Anthropic message objects. So the model sees its own previous answers
ending in [[RECEIPT ...]] and does the obvious thing — it imitates the format and signs the
next answer the same way. The plain path did NOT leak, which is the tell: there, history is
rendered into the SYSTEM prompt as labelled lines rather than replayed as assistant turns,
and a model imitates its own turns far more readily than a transcript.

FIX, two layers, because one of them is not a guarantee:
  - receipt_rule() names the marker in both system prompts (plain and agentic): these lines
    are written by the system, read them as evidence, never write one. Reduces occurrence.
  - receipt_strip() truncates any [[RECEIPT ...]] out of model output before it becomes the
    reply — plain path in layered_generate, agentic path on final_text in agentic_loop.
    Deterministic. A guard that depends on the model choosing to obey is exactly the class of
    thing round 8 exists to stop shipping, so the instruction is the optimisation and the
    strip is the guarantee.
Placed ABOVE agentic_loop's empty-check on purpose: a turn whose entire output was an
imitated receipt has produced no answer, and must be reported as no answer.

The receipt stays in HISTORY, which is the whole point and is proven to work: asked "What
source did you use for that?" one turn after a live web_search, this brain answered
"I used Weather Underground (https://www.wunderground.com/weather/is/reykjav%C3%ADk) for the
current temperature in Reykjavik" — a real source, no apology. That is the false confession
dead, and it is dead BECAUSE the model can read the receipt.

BUILT: 887,112 bytes, sha256 54a2eff84d4fa44f8d2db6781dcf225df4b5075a8ec5f1058018bd40cc1af10b
BUG-PLAINCHAT-1 miscompile guard: zero sites.

Refs neuron#109

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Tim Lingo
2026-08-05 23:10:53 -05:00
parent ff421d39f6
commit 9ea41eed78
3 changed files with 48 additions and 3 deletions
+44 -3
View File
@@ -807,7 +807,7 @@ fn build_system_prompt(ctx: String, chat_mode: Bool) -> String {
// in this revision the chat_mode flag had no effect on the prompt. Restored here, in the
// permanent-rules group, immediately after capability_rules (the rule it qualifies).
// Zero effect on agentic paths: they pass chat_mode=false, so no_tools_rule is "".
return identity + operator_section + date_line + voice_rules + security_rules + capability_rules + no_tools_rule + bounded_persona_block + identity_block + affective_boot_block + engram_block + safety_block
return identity + operator_section + date_line + voice_rules + security_rules + capability_rules + receipt_rule() + no_tools_rule + bounded_persona_block + identity_block + affective_boot_block + engram_block + safety_block
}
fn hist_append(hist: String, role: String, content: String) -> String {
@@ -1035,6 +1035,41 @@ fn text_join_sep(accumulated: String, incoming: String, after_interruption: Bool
return "\n\n"
}
// receipt_rule one line telling the model what the receipt marker is, and not to write one.
// Appended to both system prompts (the plain path's build_system_prompt and the agentic path's
// hand-built system string). See receipt_strip for why instruction alone is not enough.
fn receipt_rule() -> String {
return "\n\n[RECEIPTS - permanent]\nLines of the form [[RECEIPT ...]] in the conversation are written by the system, not by you. They are the record of which tools actually ran on a turn - read them as evidence, and rely on them when asked what you did or where information came from. NEVER write one yourself and never copy the format into your reply; the system adds them."
}
// receipt_strip remove a RECEIPT line the MODEL wrote, so it can never reach the user.
//
// FOUND BY E2E, NOT BY REASONING (2026-08-05). The receipt is stored inside the assistant turn
// and the agentic path replays history VERBATIM as Anthropic message objects so the model sees
// its own previous answers ending in [[RECEIPT ...]] and does the obvious thing: it imitates the
// format and signs its next answer the same way. Measured on the very first live run, on two
// turns out of two. The design note claimed "the user never sees it"; that was false, and only
// running it showed that.
//
// The plain path did not leak, which is the tell: there, history is rendered into the SYSTEM
// prompt as labelled lines rather than replayed as assistant messages, and a model imitates its
// own turns far more readily than it imitates a transcript.
//
// Instruction (receipt_rule) reduces this; only a deterministic strip PREVENTS it. Both ship,
// because a guard that depends on the model choosing to obey is the class of thing round 8 exists
// to stop shipping. Truncation at the marker is safe: the receipt is always terminal, and a model
// copying the format copies the leading blank line too.
fn receipt_strip(s: String) -> String {
let p2: Int = str_index_of(s, "\n\n[[RECEIPT")
if p2 >= 0 {
return str_slice(s, 0, p2)
}
let p1: Int = str_index_of(s, "[[RECEIPT")
if p1 < 0 { return s }
if p1 == 0 { return "" }
return str_slice(s, 0, p1)
}
fn tool_receipt(tools_used: String, sources: String) -> String {
let names: String = provenance_names(tools_used)
if str_eq(names, "") {
@@ -1330,7 +1365,9 @@ fn layered_generate(prompt: String, imprint_id: String, session_id: String) -> S
return ""
}
return clean_llm_response(raw)
// FIX A follow-up: a model that has seen receipts in its context may sign its own answer
// with one. Strip it before the caller ever sees it. See receipt_strip.
return receipt_strip(clean_llm_response(raw))
}
// session_preload_bullets render up to max_bullets nodes from a JSON array as
@@ -2583,7 +2620,7 @@ fn handle_chat_agentic(body: String) -> String {
let system: String = identity + bounded_persona_floor() + " You have access to tools: read files, write files, browse the web, search your memory, run commands. Use them when they add genuine value. Be direct.
" + ctx + ag_session_preload
" + ctx + ag_session_preload + receipt_rule()
let api_key: String = agentic_api_key()
let tools_json: String = agentic_tools_all()
@@ -3083,6 +3120,10 @@ fn agentic_loop(session_id: String, model: String, safe_sys: String, tools_json:
// genuine no-response (model returned an empty text block). The iteration cap
// means the task was too complex for the agentic loop depth surface it clearly
// so the caller/operator knows to increase the cap or break the task apart.
// FIX A follow-up: strip any receipt the MODEL wrote before this becomes the reply. Placed
// ABOVE the empty check on purpose a turn whose entire output was an imitated receipt has
// produced no answer, and must be reported as no answer rather than as a receipt.
let final_text = receipt_strip(final_text)
if str_eq(final_text, "") {
let hit_cap: Bool = iteration >= 12
let err_msg: String = if hit_cap {
+2
View File
@@ -32,6 +32,8 @@ extern fn provenance_scan_urls(arr: String, acc: String) -> String
extern fn provenance_add_sources(block: String, btype: String, has_cit: Bool, cit_raw: String, acc: String) -> String
extern fn provenance_names(tools_used: String) -> String
extern fn text_join_sep(accumulated: String, incoming: String, after_interruption: Bool) -> String
extern fn receipt_rule() -> String
extern fn receipt_strip(s: String) -> String
extern fn tool_receipt(tools_used: String, sources: String) -> String
extern fn hist_trim(hist: String) -> String
extern fn hist_trim_with_bell_guard(hist: String) -> String
Generated Vendored
+2
View File
@@ -11,6 +11,8 @@ el_val_t provenance_add_sources(el_val_t block, el_val_t btype, el_val_t has_cit
el_val_t provenance_names(el_val_t tools_used);
el_val_t provenance_scan_urls(el_val_t arr, el_val_t acc);
el_val_t text_join_sep(el_val_t accumulated, el_val_t incoming, el_val_t after_interruption);
el_val_t receipt_rule(void);
el_val_t receipt_strip(el_val_t s);
el_val_t tool_receipt(el_val_t tools_used, el_val_t sources);
el_val_t agent_number(el_val_t agent);
el_val_t agent_person(el_val_t agent);