fix(engine): excise the receipt, do not truncate at it — a leading receipt was erasing whole answers
CAUGHT BY A/B, AND ONLY BY A/B. The previous commit's receipt_strip assumed the receipt is
always TERMINAL and cut everything from the marker onward. It is not always terminal: once
receipt_rule told the model what [[RECEIPT ...]] means, the model sometimes LED with one and
wrote the answer underneath. Cutting at the marker then deleted the entire answer and the
turn returned {"error":"no response"}.
MEASURED, same prompt (two web searches, cited prose), fresh session each run:
round-7 brain 4 / 4 answered (471, 473, 473, 544 chars)
round-8 brain 2 / 7 answered (five {"error":"no response"})
This looked exactly like a flaky model. It was not — it was mine. Running the two brains
side by side on the same prompt is the only reason it was found, and it is the reason the
A/B is now part of how this class gets tested.
AFTER THE FIX, same protocol:
round-7 brain 4 / 4 (473, 473, 473, 544)
round-8 brain 4 / 4 (657, 657, 657, 673)
THE FIX: remove the [[...]] span and keep BOTH sides, instead of truncating at the marker.
An unterminated marker at position 0 is left completely alone — no rule about receipts is
worth erasing an answer over. Bounded four-pass loop rather than a conditional exit, because
rebinding the counter inside an if-expression is the block-expression shape that miscompiles
integer arithmetic under this elc (BUG-PLAINCHAT-1). Verified in the generated C:
str_slice(rest, (e + 2), str_len(rest)) <- integer addition, correct
el_str_concat(head, tail) <- string concat, correct
and zero el_str_concat(<ident>, str_len(...)) sites across all 49 modules.
SEAM PROOF (FIX C) rides on the same runs — a real two-search cited answer, inspected byte
by byte, in BOTH failure directions:
missing separator (the round-7 "to.Good", bytes 77 2e 47): 0 hits. Sentence boundaries
measure 2e 20 4d — "." SPACE "M".
over-separation (a cited sentence shattered across paragraphs): 0 hits. The answer is one
continuous paragraph with its sentences intact, which is the direction a blanket
separator would have broken.
BUILT: sha256 77115f2733e794c5bc4ad1f55b1acaf658f8f4a91cccd423a2d633d94a726cbc
Refs neuron#109
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -1057,17 +1057,43 @@ fn receipt_rule() -> String {
|
||||
//
|
||||
// Instruction (receipt_rule) reduces this; only a deterministic strip PREVENTS it. Both ship,
|
||||
// because a guard that depends on the model choosing to obey is the class of thing round 8 exists
|
||||
// to stop shipping. Truncation at the marker is safe: the receipt is always terminal, and a model
|
||||
// copying the format copies the leading blank line too.
|
||||
// to stop shipping.
|
||||
//
|
||||
// EXCISE THE RECEIPT, DO NOT TRUNCATE AT IT — bought with a measured regression, 2026-08-05.
|
||||
// The first version of this function assumed the receipt is always TERMINAL and cut everything
|
||||
// from the marker onward. It is not always terminal: told about the format in its system prompt,
|
||||
// the model sometimes LEADS with a receipt and then writes the answer underneath. Cutting at the
|
||||
// marker then deleted the entire answer, and the turn came back {"error":"no response"}.
|
||||
// Measured on a two-search cited prompt: round 7 answered 4/4; that version answered 2/7. The
|
||||
// A/B is the only reason this was caught — it looked like a flaky model, and it was not.
|
||||
// So: remove the [[...]] span and keep BOTH sides. An unterminated marker at position 0 is left
|
||||
// alone entirely, because no rule about receipts is worth erasing an answer over.
|
||||
fn receipt_strip(s: String) -> String {
|
||||
let p2: Int = str_index_of(s, "\n\n[[RECEIPT")
|
||||
if p2 >= 0 {
|
||||
return str_slice(s, 0, p2)
|
||||
let out: String = s
|
||||
// Bounded pass rather than a conditional exit: rebinding the counter inside an if-expression
|
||||
// is the block-expression shape that miscompiles integer arithmetic under this elc
|
||||
// (BUG-PLAINCHAT-1). Four straight passes is cheaper than being clever, and once no marker
|
||||
// remains every further pass is a no-op.
|
||||
let guard: Int = 0
|
||||
while guard < 4 {
|
||||
let p: Int = str_index_of(out, "[[RECEIPT")
|
||||
let found: Bool = p >= 0
|
||||
let rest: String = if found { str_slice(out, p, str_len(out)) } else { "" }
|
||||
let e: Int = if found { str_index_of(rest, "]]") } else { 0 - 1 }
|
||||
let head: String = if found { str_slice(out, 0, p) } else { "" }
|
||||
let tail: String = if e >= 0 { str_slice(rest, e + 2, str_len(rest)) } else { "" }
|
||||
let out = if !found {
|
||||
out
|
||||
} else {
|
||||
if e >= 0 {
|
||||
head + tail
|
||||
} else {
|
||||
if p == 0 { out } else { head }
|
||||
}
|
||||
}
|
||||
let guard = guard + 1
|
||||
}
|
||||
let p1: Int = str_index_of(s, "[[RECEIPT")
|
||||
if p1 < 0 { return s }
|
||||
if p1 == 0 { return "" }
|
||||
return str_slice(s, 0, p1)
|
||||
return str_trim(out)
|
||||
}
|
||||
|
||||
fn tool_receipt(tools_used: String, sources: String) -> String {
|
||||
|
||||
Reference in New Issue
Block a user