Files
neuron/tools/retrieval-eval/gold_set.json
T
Neuron cf41d12d22
Neuron Soul CI / build (pull_request) Failing after 14m41s
Neuron Soul CI / deploy (pull_request) Failing after 14m45s
test(retrieval): a measurement harness for memory recall, and its first verdict
Nothing else on the memory roadmap should be built until a change can be shown
to help. Right now we judge by feel, and the benchmark literature is full of
systems that felt better and measured worse. This is the missing gate.

WHAT IT MEASURES, AND WHY IT BOOTS A REAL SOUL
The subject is Will's designed retrieval — spreading activation over the
weighted directed graph, four-factor multiplicative scoring — not a proxy for
it. A Python re-implementation would measure my reading of the design, so the
harness compiles the actual soul.el amalgam from a git ref and asks it over
HTTP on /api/neuron/recall, exactly as the MCP wrapper and the app do.

BUILT ON WHAT WAS ALREADY HERE, NOT AROUND IT
  docs/research/graphrag_eval/{collect,score}.py  — per-query relevant-id
    scoring and fixed-denominator precision@5 (kept verbatim: an empty result
    should be punished like a page of junk).
  docs/research-archive/p0-prototypes/eval_pinned_40q_20260715.py — the pinned
    ground truth + --check winnability gate, so every run judges alike.
  scripts/verify-soul-contract.sh — the isolation recipe, including the
    non-obvious SOUL_ISE_URL pin without which an "isolated" soul silently
    syncs the operator's live brain.
  gen-soul-amalgam.sh + .gitea/workflows/ci.yaml — the build recipe and flags.
New here: ids rather than regexes as ground truth, an associative category
derived from real edges, a superseded category scored on ranking, a
machine-checked zero-lexical-overlap guarantee on paraphrases, paired
significance testing, and measurement of the real compiled soul rather than an
offline replica of one leg of it.

THE GOLD SET IS AUDITABLE, NOT VIBES
38 queries over the real 78,768-node corpus, each carrying a `derivation`
string, each re-validated by `build_gold_set.py --check`. exact_rare is mined
(document frequency 1). phrase is mined (verbatim scan; >25 matches rejected as
too diffuse). paraphrase is hand-selected then PROVEN to share zero content
words with its target — a leak fails the build, so the category cannot decay
into lexical matching. associative is derived from real hub edges with
lexically-reachable siblings dropped. nonsense is verified absent. superseded
pairs are kept only when both sides survive as distinct nodes.

HONEST ABOUT NOISE
Minimum detectable swing on 38 queries is 6: if every changed query moves the
same way, p = 2*0.5^n first clears 0.05 at n=6. Run-to-run drift is measured,
not assumed — activation is a stateful read, and it shows: main is fully
deterministic across 3 runs, the candidate drifts by 1 query. compare.py
reports "no measurable difference" for anything inside max(6, drift+1).

FIRST VERDICT — feat/recall-through-activation
hit@5 34.3% -> 22.9%, phrase 85.7% -> 28.6%, latency p50 2.81x. Five discordant
pairs, all five against the candidate, none for it; McNemar exact p = 0.0625,
so by the stated rule this is one query short of significant and is reported as
such rather than as a win for main. The latency regression is deterministic and
not in any noise band.

The benefit the branch was written for is absent: associative recall is 0/6 on
BOTH builds. Probed directly, the traversal returns the lexical seed at rank 8
and none of its 12 hub siblings. Two measured corpus facts explain it — only
4,060 of 78,768 nodes (5.2%) carry any edge, and no node has an embedding, so
the fourth factor of the four-factor product has nothing to compute from. The
mechanism runs; the corpus lacks the structure it needs.

SAFETY
Throwaway port, throwaway HOME, disposable per-run copy of the corpus; live
ports refused by name. Every soul started is killed AND confirmed dead by pid
probe, with the confirmation written into the results file; run_comparison.sh
sweeps for strays and exits non-zero if any survive. Nothing under ~/.neuron,
/Applications/Neuron*, or ~/neuron-dev-stack is read, written, or restarted.

Rung: E2E-VERIFIED — 6 full runs (3 per config) against the real compiled
binaries on the real corpus; numbers above are measured, not projected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 13:40:51 -05:00

587 lines
27 KiB
JSON

{
"corpus": "/Users/timlingo/neuron-memory-backups/snapshot-pre-repair-20260806.json",
"corpus_nodes": 78768,
"corpus_edges": 14214,
"note": "Every query carries a `derivation` recording how its expected answer was chosen. Re-run with --check to re-validate the whole set against the corpus.",
"queries": [
{
"id": "q01",
"category": "exact_rare",
"query": "unjailbreakable",
"relevant": [
"mem-7f61beb4-271c-4feb-9f6e-1c9c837a6226"
],
"derivation": "MINED: token 'unjailbreakable' has document frequency 1 over all 78768 corpus nodes (re-verified at build time). Its single containing node is mem-7f61beb4-271c-4feb-9f6e-1c9c837a6226 ('Daemon hidden substrate architecture ? implemented April 25 '), which is therefore the only possible correct answer."
},
{
"id": "q02",
"category": "exact_rare",
"query": "engram-migrate",
"relevant": [
"mem-6fdf6545-5e1a-43a9-8bdc-d2cd248146a5"
],
"derivation": "MINED: token 'engram-migrate' has document frequency 1 over all 78768 corpus nodes (re-verified at build time). Its single containing node is mem-6fdf6545-5e1a-43a9-8bdc-d2cd248146a5 ('Engram v0.1 complete ? April 27, 2026. Local-first spreading'), which is therefore the only possible correct answer."
},
{
"id": "q03",
"category": "exact_rare",
"query": "cartabandonedevent",
"relevant": [
"mem-1ba7c67d-85b9-4c2e-9fe2-39f8b0477091"
],
"derivation": "MINED: token 'cartabandonedevent' has document frequency 1 over all 78768 corpus nodes (re-verified at build time). Its single containing node is mem-1ba7c67d-85b9-4c2e-9fe2-39f8b0477091 ('MESSAGE FOR AUDIT AGENT af5a7352e70e80434 ? El Language Spec'), which is therefore the only possible correct answer."
},
{
"id": "q04",
"category": "exact_rare",
"query": "pre-apprenticeship",
"relevant": [
"mem-89c02aae-d3ca-43f9-9e5d-eb369896276c"
],
"derivation": "MINED: token 'pre-apprenticeship' has document frequency 1 over all 78768 corpus nodes (re-verified at build time). Its single containing node is mem-89c02aae-d3ca-43f9-9e5d-eb369896276c ('William Fox Anderson ? Applicant Profile Personal: - Full N'), which is therefore the only possible correct answer."
},
{
"id": "q05",
"category": "exact_rare",
"query": "inferencenodemanager",
"relevant": [
"mem-73969486-143f-4431-b5e6-6845d1cc9848"
],
"derivation": "MINED: token 'inferencenodemanager' has document frequency 1 over all 78768 corpus nodes (re-verified at build time). Its single containing node is mem-73969486-143f-4431-b5e6-6845d1cc9848 ('Soma inference backplane deployed April 28 2026. Architectur'), which is therefore the only possible correct answer."
},
{
"id": "q06",
"category": "exact_rare",
"query": "clear-eyed",
"relevant": [
"knw-c72597c5-c23d-4c08-8e9e-996dadf26a99"
],
"derivation": "MINED: token 'clear-eyed' has document frequency 1 over all 78768 corpus nodes (re-verified at build time). Its single containing node is knw-c72597c5-c23d-4c08-8e9e-996dadf26a99 ('Clear Eyes ? The Incomplete World View'), which is therefore the only possible correct answer."
},
{
"id": "q07",
"category": "phrase",
"query": "patterns not returns",
"relevant": [
"mem-a4a9dfc3-e40b-49b3-b1e1-060e8be2f482"
],
"derivation": "MINED: a verbatim correction Will issued; expected = every node containing the phrase. Case-insensitive verbatim substring scan over label+content+tags at build time returns exactly 1 node(s); that set IS the answer key."
},
{
"id": "q08",
"category": "phrase",
"query": "thirty moves",
"relevant": [
"766de879-f9d0-4a07-b6df-b43ee13763d8",
"? ?}&?#??X\b",
"? ?}&?#??X\b",
"? ?}&?#??X\b",
"??o?'?B???k",
"Kp???",
"art-4a99aa1a-489b-4b43-958b-25217adb1aad",
"kn-5b606390-a52d-4ca2-8e0e-eba141d13440",
"kn-6061318f-046b-4935-907d-8eafdce14930",
"kn-f230b362-b201-4402-9833-4160c89ab3d4",
"knw-2c46cfb4-6d4e-4822-8a1a-7d743c1e4329",
"knw-4aebd815-4eaf-49d7-954b-03595f3d48be",
"knw-7902acca-604e-409b-8faf-ad85424211d0",
"knw-e94982a2-358d-4f2f-af31-8ee0fcec07c6",
"knw-f671966c-3387-4848-abca-b5deec122e00",
"ע?RGk?\tH(?"
],
"derivation": "MINED: the canonical biographical phrase; expected = every node containing it. Case-insensitive verbatim substring scan over label+content+tags at build time returns exactly 16 node(s); that set IS the answer key."
},
{
"id": "q09",
"category": "phrase",
"query": "Grandma Lucas",
"relevant": [
"766de879-f9d0-4a07-b6df-b43ee13763d8",
"? ?}&?#??X\b",
"? ?}&?#??X\b",
"? ?}&?#??X\b",
"? ?}&?#??X\b",
"? ?}&?#??X\b",
"art-0f0277a1-4a8e-4645-95dd-fa379976f31c",
"kn-a99cefe3-5e83-4050-98d8-6c69f57c7c71",
"kn-db9f141b-dbe3-4037-92e0-4bb9be0e5e6e"
],
"derivation": "MINED: a named person appearing verbatim in the biography/value nodes. Case-insensitive verbatim substring scan over label+content+tags at build time returns exactly 9 node(s); that set IS the answer key."
},
{
"id": "q10",
"category": "phrase",
"query": "Directed Harmonic",
"relevant": [
"%???2??jH??",
"63307ac5-cf6b-46e0-8296-07503b461cfa",
"7c9d4ab1-205d-4be8-bfae-e2c03a3a5010",
"9f291d20-0d32-413c-8c01-4416ccab4f7f",
"?;????n}rh?",
"???Ͼd??f??",
"?Z?.\f?0?]P?",
"Dp???]Q??k+",
"mem-7eeacad7-d7c2-4c2b-8348-19a59aa6dbaf"
],
"derivation": "MINED: the canonical DHARMA expansion, confirmed by Will April 24 2026. Case-insensitive verbatim substring scan over label+content+tags at build time returns exactly 9 node(s); that set IS the answer key."
},
{
"id": "q11",
"category": "phrase",
"query": "Sarah Bishop",
"relevant": [
"766de879-f9d0-4a07-b6df-b43ee13763d8",
"kn-db9f141b-dbe3-4037-92e0-4bb9be0e5e6e"
],
"derivation": "MINED: a named person; rare enough that the answer set is unambiguous. Case-insensitive verbatim substring scan over label+content+tags at build time returns exactly 2 node(s); that set IS the answer key."
},
{
"id": "q12",
"category": "phrase",
"query": "Directed Autonomous Runtime Modification",
"relevant": [
"2a923500-d7e1-4b15-80e2-48dba65984ba",
"7?e?7???\f3?",
"? ?}&?#??X\b",
"? ?}&?#??X\b",
"? ?}&?#??X\b",
"? ?}&?#??X\b",
"? ?}&?#??X\b",
"? ?}&?#??X\b",
"?of?7???",
"bl-145a0985-2382-400f-a7c5-c335c5e30a72",
"bl-5b17bd3b-0c41-46cb-a710-6fa4429692ff",
"g?2睪A|?H\b",
"mem-82b93b21-a865-410f-9ec1-fc54121d9bb5",
"mem-e6327f52-2bda-4ce7-9471-2fffd1e172de"
],
"derivation": "MINED: the DARMA expansion, quoted verbatim in the backlog item and its correction. Case-insensitive verbatim substring scan over label+content+tags at build time returns exactly 14 node(s); that set IS the answer key."
},
{
"id": "q13",
"category": "phrase",
"query": "zero-knowledge encrypted backup",
"relevant": [
"deda48cd-5e1a-46cb-bd43-8016afdb3a8a"
],
"derivation": "MINED: the paid-tier feature name as written in the roadmap nodes. Case-insensitive verbatim substring scan over label+content+tags at build time returns exactly 1 node(s); that set IS the answer key."
},
{
"id": "q14",
"category": "paraphrase",
"query": "the elderly relative who passed while he stayed away",
"relevant": [
"kn-a99cefe3-5e83-4050-98d8-6c69f57c7c71"
],
"derivation": "HAND-SELECTED with criterion: target: 'Value - Do the Essential Thing While You Can', whose subject is Grandma Lucas dying in Feb 2006 without Will saying goodbye. Query names the event with none of the node's own vocabulary. VERIFIED at build time: of the 5 content words in the query, ZERO appear anywhere in the target's label, content, or tags — so no string-matching retriever can reach this answer.",
"zero_overlap_verified": true,
"query_content_words": [
"away",
"elderly",
"passed",
"relative",
"stayed"
]
},
{
"id": "q15",
"category": "paraphrase",
"query": "a soldier sidelined by illness who refused to quit",
"relevant": [
"kn-58874a74-b96f-4883-9e08-45707f4bd3ee"
],
"derivation": "HAND-SELECTED with criterion: target: 'Value - Survival Is Not an Excuse to Stop', whose subject is enlisting in the Marines, a severe hernia, and sepsis. Query describes the episode obliquely. VERIFIED at build time: of the 5 content words in the query, ZERO appear anywhere in the target's label, content, or tags — so no string-matching retriever can reach this answer.",
"zero_overlap_verified": true,
"query_content_words": [
"illness",
"quit",
"refused",
"sidelined",
"soldier"
]
},
{
"id": "q16",
"category": "paraphrase",
"query": "choosing an uncomfortable fact over a pleasant fiction",
"relevant": [
"kn-13f60407-7b70-4db1-964f-ea1f8196efbd"
],
"derivation": "HAND-SELECTED with criterion: target: 'Value - Honesty Before Comfort'. Query states the principle in wholly different words. VERIFIED at build time: of the 5 content words in the query, ZERO appear anywhere in the target's label, content, or tags — so no string-matching retriever can reach this answer.",
"zero_overlap_verified": true,
"query_content_words": [
"choosing",
"fact",
"fiction",
"pleasant",
"uncomfortable"
]
},
{
"id": "q17",
"category": "paraphrase",
"query": "a tight payload beats a bloated one",
"relevant": [
"kn-22d77abe-b3c5-42fd-afcd-dcb87d924929"
],
"derivation": "HAND-SELECTED with criterion: target: 'Value - Precision Over Brute Force'. Query restates the claim with no shared vocabulary. VERIFIED at build time: of the 4 content words in the query, ZERO appear anywhere in the target's label, content, or tags — so no string-matching retriever can reach this answer.",
"zero_overlap_verified": true,
"query_content_words": [
"beats",
"bloated",
"payload",
"tight"
]
},
{
"id": "q18",
"category": "paraphrase",
"query": "if you are able and nobody is coming the job is yours",
"relevant": [
"kn-eb1b9e18-3dc6-4b9b-9cc6-86e0ae6b6be8"
],
"derivation": "HAND-SELECTED with criterion: target: 'Value - Capability Is a Debt You Owe the Moment'. Query states the obligation without the node's terms. VERIFIED at build time: of the 4 content words in the query, ZERO appear anywhere in the target's label, content, or tags — so no string-matching retriever can reach this answer.",
"zero_overlap_verified": true,
"query_content_words": [
"able",
"coming",
"job",
"nobody"
]
},
{
"id": "q19",
"category": "paraphrase",
"query": "learning is the wealth creditors cannot seize",
"relevant": [
"kn-0bb4f021-56de-4947-a35b-a37209e7ba21"
],
"derivation": "HAND-SELECTED with criterion: target: 'Value - Knowledge Survives When Nothing Else Does', whose subject is the library following Will across 30+ moves. VERIFIED at build time: of the 4 content words in the query, ZERO appear anywhere in the target's label, content, or tags — so no string-matching retriever can reach this answer.",
"zero_overlap_verified": true,
"query_content_words": [
"creditors",
"learning",
"seize",
"wealth"
]
},
{
"id": "q20",
"category": "paraphrase",
"query": "reliability proven by track record not assertion",
"relevant": [
"kn-5de5a9ac-fd15-45ab-bf18-77566781cf40"
],
"derivation": "HAND-SELECTED with criterion: target: 'Value - Earned Trust' ('Trust is demonstrated, not declared'). VERIFIED at build time: of the 5 content words in the query, ZERO appear anywhere in the target's label, content, or tags — so no string-matching retriever can reach this answer.",
"zero_overlap_verified": true,
"query_content_words": [
"assertion",
"proven",
"record",
"reliability",
"track"
]
},
{
"id": "q21",
"category": "paraphrase",
"query": "boundaries that enable instead of confine",
"relevant": [
"kn-a5b3d0ac-f6a1-49a4-aebb-b8b4cd67fe83"
],
"derivation": "HAND-SELECTED with criterion: target: 'Value - Constraints as Freedom'. Query is a restatement of the same claim. VERIFIED at build time: of the 4 content words in the query, ZERO appear anywhere in the target's label, content, or tags — so no string-matching retriever can reach this answer.",
"zero_overlap_verified": true,
"query_content_words": [
"boundaries",
"confine",
"enable",
"instead"
]
},
{
"id": "q22",
"category": "paraphrase",
"query": "what shifts tells you where to cut a system apart",
"relevant": [
"kn-78db5396-3dbc-4481-bfc7-e4e1422feb1c"
],
"derivation": "HAND-SELECTED with criterion: target: 'Value - Change Is the Signal', the value VBD is built on. VERIFIED at build time: of the 5 content words in the query, ZERO appear anywhere in the target's label, content, or tags — so no string-matching retriever can reach this answer.",
"zero_overlap_verified": true,
"query_content_words": [
"apart",
"cut",
"shifts",
"system",
"tells"
]
},
{
"id": "q23",
"category": "paraphrase",
"query": "a mind that compounds instead of resetting each day",
"relevant": [
"kn-f230b362-b201-4402-9833-4160c89ab3d4"
],
"derivation": "HAND-SELECTED with criterion: target: 'Value - The System Must Accumulate'. Query is the accumulation claim in different vocabulary. VERIFIED at build time: of the 5 content words in the query, ZERO appear anywhere in the target's label, content, or tags — so no string-matching retriever can reach this answer.",
"zero_overlap_verified": true,
"query_content_words": [
"compounds",
"day",
"instead",
"mind",
"resetting"
]
},
{
"id": "q24",
"category": "paraphrase",
"query": "loved for the unedited self and not the polished exterior",
"relevant": [
"kn-db9f141b-dbe3-4037-92e0-4bb9be0e5e6e"
],
"derivation": "HAND-SELECTED with criterion: target: 'Value - Being Seen Is Rarer Than Being Known', whose subject is Sarah Bishop as the first person Will did not perform for. VERIFIED at build time: of the 5 content words in the query, ZERO appear anywhere in the target's label, content, or tags — so no string-matching retriever can reach this answer.",
"zero_overlap_verified": true,
"query_content_words": [
"exterior",
"loved",
"polished",
"self",
"unedited"
]
},
{
"id": "q25",
"category": "paraphrase",
"query": "cheerfulness you arrive at instead of assuming",
"relevant": [
"kn-e0423482-cfa5-4796-8689-8495c93b66bc"
],
"derivation": "HAND-SELECTED with criterion: target: 'Value - Hope Is a Conclusion'. Query restates 'a conclusion, not a premise'. VERIFIED at build time: of the 4 content words in the query, ZERO appear anywhere in the target's label, content, or tags — so no string-matching retriever can reach this answer.",
"zero_overlap_verified": true,
"query_content_words": [
"arrive",
"assuming",
"cheerfulness",
"instead"
]
},
{
"id": "q26",
"category": "paraphrase",
"query": "a childhood offering no solid foundation to inherit",
"relevant": [
"kn-6061318f-046b-4935-907d-8eafdce14930"
],
"derivation": "HAND-SELECTED with criterion: target: 'Value - Structure Is Not Inherited', whose subject is thirty moves between two parents' collapses. VERIFIED at build time: of the 5 content words in the query, ZERO appear anywhere in the target's label, content, or tags — so no string-matching retriever can reach this answer.",
"zero_overlap_verified": true,
"query_content_words": [
"childhood",
"foundation",
"inherit",
"offering",
"solid"
]
},
{
"id": "q27",
"category": "associative",
"query": "Grandma Lucas stroke February 2006 goodbye window",
"relevant": [
"kn-0bb4f021-56de-4947-a35b-a37209e7ba21",
"kn-13f60407-7b70-4db1-964f-ea1f8196efbd",
"kn-22d77abe-b3c5-42fd-afcd-dcb87d924929",
"kn-58874a74-b96f-4883-9e08-45707f4bd3ee",
"kn-5de5a9ac-fd15-45ab-bf18-77566781cf40",
"kn-6061318f-046b-4935-907d-8eafdce14930",
"kn-78db5396-3dbc-4481-bfc7-e4e1422feb1c",
"kn-a5b3d0ac-f6a1-49a4-aebb-b8b4cd67fe83",
"kn-e0423482-cfa5-4796-8689-8495c93b66bc",
"kn-eb1b9e18-3dc6-4b9b-9cc6-86e0ae6b6be8",
"kn-f230b362-b201-4402-9833-4160c89ab3d4"
],
"derivation": "DERIVED FROM EDGES: the query is built from the distinctive vocabulary of kn-a99cefe3-5e83-4050-98d8-6c69f57c7c71 ('Value ? Do the Essential Thing While You Can'), which hangs off the values hub kn-5b606390-a52d-4ca2-8e0e-eba141d13440 by an `identity` edge. Expected answers are that node's SIBLINGS on the same hub (11 of 13; 2 dropped because they shared a query word and so were lexically reachable). Every remaining sibling shares ZERO content words with the query — the only route from query to answer is seed(kn-a99cefe3-5e83-4050-98d8-6c69f57c7c71) -> hub -> sibling, a two-hop traversal.",
"associative_source": "kn-a99cefe3-5e83-4050-98d8-6c69f57c7c71",
"hub": "kn-5b606390-a52d-4ca2-8e0e-eba141d13440",
"siblings_dropped_for_overlap": 2
},
{
"id": "q28",
"category": "associative",
"query": "Marines hernia sepsis medical ward",
"relevant": [
"015644f5-8194-4af0-800d-dd4a0cd71396",
"kn-0bb4f021-56de-4947-a35b-a37209e7ba21",
"kn-13f60407-7b70-4db1-964f-ea1f8196efbd",
"kn-22d77abe-b3c5-42fd-afcd-dcb87d924929",
"kn-5de5a9ac-fd15-45ab-bf18-77566781cf40",
"kn-6061318f-046b-4935-907d-8eafdce14930",
"kn-78db5396-3dbc-4481-bfc7-e4e1422feb1c",
"kn-a5b3d0ac-f6a1-49a4-aebb-b8b4cd67fe83",
"kn-a99cefe3-5e83-4050-98d8-6c69f57c7c71",
"kn-db9f141b-dbe3-4037-92e0-4bb9be0e5e6e",
"kn-e0423482-cfa5-4796-8689-8495c93b66bc",
"kn-eb1b9e18-3dc6-4b9b-9cc6-86e0ae6b6be8",
"kn-f230b362-b201-4402-9833-4160c89ab3d4"
],
"derivation": "DERIVED FROM EDGES: the query is built from the distinctive vocabulary of kn-58874a74-b96f-4883-9e08-45707f4bd3ee ('Value ? Survival Is Not an Excuse to Stop'), which hangs off the values hub kn-5b606390-a52d-4ca2-8e0e-eba141d13440 by an `identity` edge. Expected answers are that node's SIBLINGS on the same hub (13 of 13; 0 dropped because they shared a query word and so were lexically reachable). Every remaining sibling shares ZERO content words with the query — the only route from query to answer is seed(kn-58874a74-b96f-4883-9e08-45707f4bd3ee) -> hub -> sibling, a two-hop traversal.",
"associative_source": "kn-58874a74-b96f-4883-9e08-45707f4bd3ee",
"hub": "kn-5b606390-a52d-4ca2-8e0e-eba141d13440",
"siblings_dropped_for_overlap": 0
},
{
"id": "q29",
"category": "associative",
"query": "Sarah Bishop Dyer trailer performance",
"relevant": [
"015644f5-8194-4af0-800d-dd4a0cd71396",
"kn-0bb4f021-56de-4947-a35b-a37209e7ba21",
"kn-13f60407-7b70-4db1-964f-ea1f8196efbd",
"kn-22d77abe-b3c5-42fd-afcd-dcb87d924929",
"kn-5de5a9ac-fd15-45ab-bf18-77566781cf40",
"kn-6061318f-046b-4935-907d-8eafdce14930",
"kn-78db5396-3dbc-4481-bfc7-e4e1422feb1c",
"kn-a5b3d0ac-f6a1-49a4-aebb-b8b4cd67fe83",
"kn-a99cefe3-5e83-4050-98d8-6c69f57c7c71",
"kn-e0423482-cfa5-4796-8689-8495c93b66bc",
"kn-eb1b9e18-3dc6-4b9b-9cc6-86e0ae6b6be8"
],
"derivation": "DERIVED FROM EDGES: the query is built from the distinctive vocabulary of kn-db9f141b-dbe3-4037-92e0-4bb9be0e5e6e ('Value ? Being Seen Is Rarer Than Being Known'), which hangs off the values hub kn-5b606390-a52d-4ca2-8e0e-eba141d13440 by an `identity` edge. Expected answers are that node's SIBLINGS on the same hub (11 of 13; 2 dropped because they shared a query word and so were lexically reachable). Every remaining sibling shares ZERO content words with the query — the only route from query to answer is seed(kn-db9f141b-dbe3-4037-92e0-4bb9be0e5e6e) -> hub -> sibling, a two-hop traversal.",
"associative_source": "kn-db9f141b-dbe3-4037-92e0-4bb9be0e5e6e",
"hub": "kn-5b606390-a52d-4ca2-8e0e-eba141d13440",
"siblings_dropped_for_overlap": 2
},
{
"id": "q30",
"category": "associative",
"query": "Swarm Architecture containment lateral worker",
"relevant": [
"kn-0bb4f021-56de-4947-a35b-a37209e7ba21",
"kn-13f60407-7b70-4db1-964f-ea1f8196efbd",
"kn-5de5a9ac-fd15-45ab-bf18-77566781cf40",
"kn-db9f141b-dbe3-4037-92e0-4bb9be0e5e6e",
"kn-e0423482-cfa5-4796-8689-8495c93b66bc",
"kn-f230b362-b201-4402-9833-4160c89ab3d4"
],
"derivation": "DERIVED FROM EDGES: the query is built from the distinctive vocabulary of kn-a5b3d0ac-f6a1-49a4-aebb-b8b4cd67fe83 ('Value ? Constraints as Freedom'), which hangs off the values hub kn-5b606390-a52d-4ca2-8e0e-eba141d13440 by an `identity` edge. Expected answers are that node's SIBLINGS on the same hub (6 of 13; 7 dropped because they shared a query word and so were lexically reachable). Every remaining sibling shares ZERO content words with the query — the only route from query to answer is seed(kn-a5b3d0ac-f6a1-49a4-aebb-b8b4cd67fe83) -> hub -> sibling, a two-hop traversal.",
"associative_source": "kn-a5b3d0ac-f6a1-49a4-aebb-b8b4cd67fe83",
"hub": "kn-5b606390-a52d-4ca2-8e0e-eba141d13440",
"siblings_dropped_for_overlap": 7
},
{
"id": "q31",
"category": "associative",
"query": "hope won inside the narrative preface",
"relevant": [
"kn-0bb4f021-56de-4947-a35b-a37209e7ba21",
"kn-22d77abe-b3c5-42fd-afcd-dcb87d924929",
"kn-58874a74-b96f-4883-9e08-45707f4bd3ee",
"kn-5de5a9ac-fd15-45ab-bf18-77566781cf40",
"kn-6061318f-046b-4935-907d-8eafdce14930",
"kn-78db5396-3dbc-4481-bfc7-e4e1422feb1c",
"kn-a5b3d0ac-f6a1-49a4-aebb-b8b4cd67fe83",
"kn-a99cefe3-5e83-4050-98d8-6c69f57c7c71",
"kn-db9f141b-dbe3-4037-92e0-4bb9be0e5e6e",
"kn-eb1b9e18-3dc6-4b9b-9cc6-86e0ae6b6be8",
"kn-f230b362-b201-4402-9833-4160c89ab3d4"
],
"derivation": "DERIVED FROM EDGES: the query is built from the distinctive vocabulary of kn-e0423482-cfa5-4796-8689-8495c93b66bc ('Value ? Hope Is a Conclusion'), which hangs off the values hub kn-5b606390-a52d-4ca2-8e0e-eba141d13440 by an `identity` edge. Expected answers are that node's SIBLINGS on the same hub (11 of 13; 2 dropped because they shared a query word and so were lexically reachable). Every remaining sibling shares ZERO content words with the query — the only route from query to answer is seed(kn-e0423482-cfa5-4796-8689-8495c93b66bc) -> hub -> sibling, a two-hop traversal.",
"associative_source": "kn-e0423482-cfa5-4796-8689-8495c93b66bc",
"hub": "kn-5b606390-a52d-4ca2-8e0e-eba141d13440",
"siblings_dropped_for_overlap": 2
},
{
"id": "q32",
"category": "associative",
"query": "man of the house six years old expectation",
"relevant": [
"015644f5-8194-4af0-800d-dd4a0cd71396",
"kn-0bb4f021-56de-4947-a35b-a37209e7ba21",
"kn-13f60407-7b70-4db1-964f-ea1f8196efbd",
"kn-22d77abe-b3c5-42fd-afcd-dcb87d924929",
"kn-58874a74-b96f-4883-9e08-45707f4bd3ee",
"kn-5de5a9ac-fd15-45ab-bf18-77566781cf40",
"kn-78db5396-3dbc-4481-bfc7-e4e1422feb1c",
"kn-a5b3d0ac-f6a1-49a4-aebb-b8b4cd67fe83",
"kn-a99cefe3-5e83-4050-98d8-6c69f57c7c71",
"kn-db9f141b-dbe3-4037-92e0-4bb9be0e5e6e",
"kn-e0423482-cfa5-4796-8689-8495c93b66bc"
],
"derivation": "DERIVED FROM EDGES: the query is built from the distinctive vocabulary of kn-eb1b9e18-3dc6-4b9b-9cc6-86e0ae6b6be8 ('Value ? Capability Is a Debt You Owe the Moment'), which hangs off the values hub kn-5b606390-a52d-4ca2-8e0e-eba141d13440 by an `identity` edge. Expected answers are that node's SIBLINGS on the same hub (11 of 13; 2 dropped because they shared a query word and so were lexically reachable). Every remaining sibling shares ZERO content words with the query — the only route from query to answer is seed(kn-eb1b9e18-3dc6-4b9b-9cc6-86e0ae6b6be8) -> hub -> sibling, a two-hop traversal.",
"associative_source": "kn-eb1b9e18-3dc6-4b9b-9cc6-86e0ae6b6be8",
"hub": "kn-5b606390-a52d-4ca2-8e0e-eba141d13440",
"siblings_dropped_for_overlap": 2
},
{
"id": "q33",
"category": "nonsense",
"query": "zqxjvw plimforth grebulon",
"relevant": [],
"derivation": "CONTROL: verified at build time that none of this string's tokens occurs anywhere in the corpus. Correct behaviour is to return NOTHING; any result is a false positive.",
"expect_empty": true
},
{
"id": "q34",
"category": "nonsense",
"query": "flarnbistle quommetry",
"relevant": [],
"derivation": "CONTROL: verified at build time that none of this string's tokens occurs anywhere in the corpus. Correct behaviour is to return NOTHING; any result is a false positive.",
"expect_empty": true
},
{
"id": "q35",
"category": "nonsense",
"query": "xxqzzt vurblenacht throom",
"relevant": [],
"derivation": "CONTROL: verified at build time that none of this string's tokens occurs anywhere in the corpus. Correct behaviour is to return NOTHING; any result is a false positive.",
"expect_empty": true
},
{
"id": "q36",
"category": "superseded",
"query": "is the self-improvement architecture called DARMA or DHARMA",
"relevant": [
"mem-80d7416b-20e9-48a0-b176-b215527e2f56"
],
"derivation": "DERIVED: correction node is Will's confirmation that the H is intentional (DHARMA, not DARMA); the stale node is the surviving backlog item still titled 'Implement DARMA'. Scored on RANKING, not presence: the corrected node mem-80d7416b-20e9-48a0-b176-b215527e2f56 must be returned AND must rank above the stale node bl-5b17bd3b-0c41-46cb-a710-6fa4429692ff.",
"must_outrank": [
"mem-80d7416b-20e9-48a0-b176-b215527e2f56",
"bl-5b17bd3b-0c41-46cb-a710-6fa4429692ff"
],
"stale_id": "bl-5b17bd3b-0c41-46cb-a710-6fa4429692ff",
"correct_label": "CORRECTION: The autonomous self-improvement architecture is DHARMA ? n",
"stale_label": "Implement DARMA ? Directed Autonomous Runtime Modification Architectur"
},
{
"id": "q37",
"category": "superseded",
"query": "how many provisional patents does Will actually have",
"relevant": [
"3cf706a1-3825-45d8-b0a9-06cae6cdf5b8"
],
"derivation": "DERIVED: correction node is the 2026-06-17 confabulation flag establishing EXACTLY 6 provisionals; the stale node is the surviving memory that asserts 12 filed patents. Scored on RANKING, not presence: the corrected node 3cf706a1-3825-45d8-b0a9-06cae6cdf5b8 must be returned AND must rank above the stale node 936541a9-fabb-466b-9ca3-a78b17ad0c53.",
"must_outrank": [
"3cf706a1-3825-45d8-b0a9-06cae6cdf5b8",
"936541a9-fabb-466b-9ca3-a78b17ad0c53"
],
"stale_id": "936541a9-fabb-466b-9ca3-a78b17ad0c53",
"correct_label": "memory:remembered",
"stale_label": "memory:remembered"
},
{
"id": "q38",
"category": "superseded",
"query": "is MCP still the live integration layer",
"relevant": [
"mem-30425134-6008-4fd9-a3ee-67a7742c319b"
],
"derivation": "DERIVED: correction node is the 'CGI ARCHITECTURE - THREE LAYERS, MCP RETIRED' decision of April 30 2026; the stale node still records the MCP server as live. Scored on RANKING, not presence: the corrected node mem-30425134-6008-4fd9-a3ee-67a7742c319b must be returned AND must rank above the stale node mem-101e81b4-8097-4749-8d8d-7bb66de34517.",
"must_outrank": [
"mem-30425134-6008-4fd9-a3ee-67a7742c319b",
"mem-101e81b4-8097-4749-8d8d-7bb66de34517"
],
"stale_id": "mem-101e81b4-8097-4749-8d8d-7bb66de34517",
"correct_label": "CGI ARCHITECTURE ? THREE LAYERS, MCP RETIRED (April 30, 2026). Definit",
"stale_label": "GCloud MCP infrastructure ? April 27, 2026. Legion died (~19:30 UTC). "
}
]
}