Measured on the 75-query extended gold set (iteration 8's held-out extension)
against the certified stack baseline results-stack-ext.json, on the embedded
corpus. Three runs of the candidate, zero drift.
A. CLAIM 24 WITHOUT THE THRESHOLD - net +4, NOT-SHOWN, kept in the tree.
fixed : q14, q25 (in-sample paraphrase), q43, q52, q63, q67 (held-out)
broken : q15 (paraphrase), q28 (associative)
8 discordant, McNemar exact p = 0.2891, floor is 6.
heldout_paraphrase 16.7% -> 30.0%, paraphrase 61.5% -> 69.2%.
Every regression guard held: exact_rare 6/6, phrase 7/7, nonsense 10/10,
superseded 2/3. Latency FLAT: p50 641 -> 632 ms.
The in-sample half (+q14 +q25 -q15 -q28 = 0) was already on record in
iteration 7's cmp-nogate.json, so only the held-out +4 is new.
B. ASYMMETRIC TASK PREFIXES ON THE EMBEDDER - net -5, REVERTED in this commit.
Rationale was sound and the prediction was wrong, which is why it was worth
measuring: nomic-embed-text is an asymmetric retrieval encoder and this file
embedded query and document bare on both sides. Prefixing does exactly what
the model card implies for the far-away cases - it rescued q42 (gold at
GLOBAL COSINE RANK 25,564) and q39 - but it re-ranks the whole space and
broke more than it fixed:
fixed : q24, q39, q42
broken : q18, q19, q22, q31, q43, q44, q52, q63
heldout_paraphrase 30.0% -> 23.3%, paraphrase 69.2% -> 53.8%.
The corpus and the reproducer are kept (embed-corpus-prefixed.py,
snapshot-pre-repair-20260806-embedded-prefixed.json) so nobody re-runs it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two changes to the ONE leg that generalises. Iteration 8 measured that out of
sample the semantic leg contributes 100% of the stack's gain and the graph leg
contributes nothing, so this is where the remaining headroom is.
1. THE 0.60 FLOOR IS A PER-QUERY LOTTERY, AND CLAIM 24 HAS NO THRESHOLD IN IT.
06-claims.md l.148: "respond to embedding search queries by returning the
node records whose embedding vectors have the HIGHEST COSINE SIMILARITY to a
query vector, independently of the spreading activation traversal." A
ranking. ENGRAM_EMBED_SEED_MIN is defined at el_runtime.c l.6094 as the
HippoRAG seed-JOIN threshold and l.6102 admits the read-path leg merely
"reuses" it. Measured on the 30 held-out paraphrases: the query's own top-1
cosine ranges 0.564-0.680, so the constant keeps a rank-1 answer for one
query and discards a rank-1 answer for the next. Six golds sit at global
cosine rank 1-2 scoring 0.564-0.589 - discarded by nothing but the constant.
What holds the nonsense controls is the corpus-vocabulary gate (nhits == 0),
not this floor. Cosine clamped to [0,1] per 05-detailed-description l.69.
2. THE VECTORS THEMSELVES ANSWER THE WRONG QUESTION. EL_EMBED_MODEL defaults to
nomic-embed-text, an ASYMMETRIC retrieval encoder trained with task prefixes.
Embedding query and document bare - as this file did on both sides - measures
topical similarity rather than answer-hood. eg_embed_fetch now takes the task
prefix: EL_EMBED_QUERY_PREFIX on the three query call sites, EL_EMBED_DOC_PREFIX
on the two backfill sites. Restores no claim, and says so: Will specifies only
"computed by an embedding model over the node's content" (l.17), so the model
is his and its correct use is ours. It is the substrate under claim 24 -
the index is only as good as the vectors in it.
Reproducer for the derived corpus: tools/retrieval-eval/embed-corpus-prefixed.py.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tim Lingo