Tim Lingo
9717a4eeaf
Measured on the 75-query extended gold set (iteration 8's held-out extension) against the certified stack baseline results-stack-ext.json, on the embedded corpus. Three runs of the candidate, zero drift. A. CLAIM 24 WITHOUT THE THRESHOLD - net +4, NOT-SHOWN, kept in the tree. fixed : q14, q25 (in-sample paraphrase), q43, q52, q63, q67 (held-out) broken : q15 (paraphrase), q28 (associative) 8 discordant, McNemar exact p = 0.2891, floor is 6. heldout_paraphrase 16.7% -> 30.0%, paraphrase 61.5% -> 69.2%. Every regression guard held: exact_rare 6/6, phrase 7/7, nonsense 10/10, superseded 2/3. Latency FLAT: p50 641 -> 632 ms. The in-sample half (+q14 +q25 -q15 -q28 = 0) was already on record in iteration 7's cmp-nogate.json, so only the held-out +4 is new. B. ASYMMETRIC TASK PREFIXES ON THE EMBEDDER - net -5, REVERTED in this commit. Rationale was sound and the prediction was wrong, which is why it was worth measuring: nomic-embed-text is an asymmetric retrieval encoder and this file embedded query and document bare on both sides. Prefixing does exactly what the model card implies for the far-away cases - it rescued q42 (gold at GLOBAL COSINE RANK 25,564) and q39 - but it re-ranks the whole space and broke more than it fixed: fixed : q24, q39, q42 broken : q18, q19, q22, q31, q43, q44, q52, q63 heldout_paraphrase 30.0% -> 23.3%, paraphrase 69.2% -> 53.8%. The corpus and the reproducer are kept (embed-corpus-prefixed.py, snapshot-pre-repair-20260806-embedded-prefixed.json) so nobody re-runs it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>