neuron-technologies/neuron
3
0
Fork 0
Code Issues 45 Pull Requests 8 Actions Packages Projects Releases Wiki Activity

feat(engram): restore claim 24 verbatim - unfloored semantic leg + corpus-vocabulary gate

Browse Source 
The read-path semantic leg was gated at ENGRAM_EMBED_SEED_MIN (0.60) and
rescaled onto [0.60,1]. That constant is defined as the HippoRAG SEED-JOIN
threshold; claim 24 authorises a ranking with no threshold at all. Measured:
the floor is not a quality gate (paraphrase targets 0.459-0.657 vs nonsense
nearest neighbours 0.553-0.622 - overlapping distributions). What holds the
nonsense controls is corpus vocabulary, so the floor is replaced by an
explicit nhits==0 gate: no record contains any query token -> return nothing.
This commit is contained in:
Tim Lingo
2026-08-07 16:23:32 -05:00
parent 55f9ee3cb0
commit 65c50073b8
1 changed files with 34 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
vendor/el-runtime/v1.0.0-20260501/el_runtime.c
Vendored
+34 -6
View File
@@ -9654,12 +9654,25 @@ el_val_t engram_search_json(el_val_t query, el_val_t limit) {
}
if (sem && n->emb && n->emb_dim == qdim) {
double c = eg_cosine(n->emb, qv, qdim);
/* Semantic leg: identical to eg_sem_term(), which is
* left in place and still used by engram_search().
* Inlined here only so one cosine serves both uses. */
if (c > ENGRAM_EMBED_SEED_MIN) {
double sv = (c - ENGRAM_EMBED_SEED_MIN) / (1.0 - ENGRAM_EMBED_SEED_MIN);
if (sv > 1.0) sv = 1.0;
/* Semantic leg, claim 24 verbatim: "returning the node
* records whose embedding vectors have the HIGHEST
* COSINE SIMILARITY to a query vector" — a ranking, with
* no threshold anywhere in the claim. The leg used to be
* gated at ENGRAM_EMBED_SEED_MIN and rescaled onto
* [SEED_MIN,1]; that constant is defined (l.6083) as the
* SEED-JOIN threshold for the HippoRAG pass, and reusing
* it as a result filter is not authorised by claim 24.
* Measured on this corpus, it is also not a quality
* gate: true paraphrase targets score 0.459-0.657 while
* the nonsense controls' own nearest neighbours score
* 0.553-0.622 — the distributions overlap, so no value
* of the constant separates them. What actually holds
* the nonsense control is corpus vocabulary (see the
* nhits==0 gate below), not cosine magnitude.
* Claim 32: clamp the cosine to [0,1] rather than let a
* negative value invert the signal. */
if (c > 0.0) {
double sv = c > 1.0 ? 1.0 : c;
sem[nsem].idx = i; sem[nsem].sem = sv; nsem++;
}
/* Graph seeds: top-K by RAW cosine, insertion-ordered. */
@@ -9677,6 +9690,21 @@ el_val_t engram_search_json(el_val_t query, el_val_t limit) {
}
}
}
/* CORPUS-VOCABULARY GATE — the thing that actually keeps an
* unfloored semantic leg from answering gibberish.
* nhits == 0 means NO stored record contains ANY query token
* anywhere in its content, label or tags: the query is outside
* the graph's vocabulary entirely. A vector index always has a
* nearest neighbour, so without this gate the semantic leg
* answers "zqxjvw plimforth grebulon" with its 0.55-cosine
* garbage. It is also the honest reading of Will's retrieval
* contract: 05-detailed-description l.64 has the caller supply
* "one or more seed node UUIDs representing the current active
* context", and every leg here is downstream of finding those
* seeds. No seeds, no retrieval — the graph declines rather
* than confabulates. Suppressing the graph seeds too keeps the
* associative leg from running off the semantic top-K alone. */
if (nhits == 0) { nsem = 0; nsemseed = 0; }
/* BM25-shaped lexical score. Binary term frequency (the match
* primitive is a substring test, not a count), Lucene-form IDF,
* and length normalisation over the corpus mean. A token that