neuron-technologies/neuron
3
0
Fork 0
Code Issues 44 Pull Requests 7 Actions Packages Projects Releases Wiki Activity

feat(engram): Will's Layer-2 executive filter (claims 44/45) on the recall read path

Browse Source 
Claim 44 requires a second pass that computes a working memory weight
(background activation x goal-state attentional bias x confidence) and
promotes only what clears a per-type threshold; claim 45 keeps the
un-promoted field retained rather than discarded. That pass exists in
engram_activate and nowhere on the route /api/neuron/recall reaches.

engram_search_json now splits each of its three legs into promoted and
suppressed sublists and rotates the promoted material into the head of
the result, background-only behind it (05-detailed-description l.221:
'promoted nodes first ... followed by background-only nodes').

MEASURED: net +0 queries vs its parent feat/bm25-lexical-leg on the
38-query gold set. NOT-SHOWN. hit@5 74.3% both sides; recall@10
61.8 -> 63.2%, MRR@10 0.502 -> 0.524, latency p50 1184 -> 1198ms.

The first cut (results-execfilter.json, kept as evidence) fed pass 2 the
semantic leg's shift-and-floor value instead of raw cosine and lost 6
queries (paraphrase 61.5 -> 23.1%, p=0.0312).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Tim Lingo
2026-08-07 16:10:17 -05:00
parent 55f9ee3cb0
commit cc4c9345f2
6 changed files with 2439 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
vendor/el-runtime/v1.0.0-20260501/el_runtime.c
Vendored
+118 -4
View File
@@ -7696,11 +7696,15 @@ static int64_t engram_assoc_leg(EngramStore* g,
* survivable here only because the structural-relation filter leaves the
* associative list EMPTY for most queries — a Memory node whose only edges are
* `tagged` and `related` expands to nothing, so its ranking is untouched. */
/* `no` is the count ALREADY in `out` — the promoted pass fills the head, the
* suppressed pass appends behind it and must dedup against the whole prefix
* (the same node can be promoted in one leg and suppressed in another, since
* its background activation differs per leg). (2026-08-07, claim 44/45) */
static int64_t engram_interleave3(const EngramRankEntry* L, int64_t nL,
const EngramSemEntry* S, int64_t nS,
const EngramSemEntry* A, int64_t nA,
int64_t lim, int64_t* out) {
int64_t no = 0, li = 0, si = 0, ai = 0;
int64_t lim, int64_t* out, int64_t no) {
int64_t li = 0, si = 0, ai = 0;
while (no < lim && (li < nL || si < nS || ai < nA)) {
if (li < nL) {
int dup = 0;
@@ -9591,6 +9595,47 @@ el_val_t engram_get_node_by_label(el_val_t label) {
return el_wrap_str(el_strdup("{}"));
}
/* ── Layer 2: the executive filter, on the recall read path (claims 44/45) ──
*
* 06-claims.md claim 44 (verbatim): "execute a first activation pass that
* propagates spreading activation from query-matched seed node records ...
* WITHOUT ANY THRESHOLD FILTERING, recording a background activation score for
* every reachable node record; and execute a second executive filter pass that
* computes a working memory weight for each background-activated node record by
* multiplying the background activation score by a goal-state attentional bias
* derived from the current query and by the node record's confidence value ...
* wherein context compilation uses only node records whose working memory
* weight exceeds a per-type threshold, and node records that do not exceed the
* threshold retain their background activation scores and are not discarded."
* Claim 45 keeps the un-promoted field available to callers.
* 05-detailed-description l.221: "Results are sorted with promoted nodes first
* ... followed by background-only nodes."
*
* This pass exists in engram_activate and NOWHERE on the route the app calls.
* /api/neuron/recall reaches engram_search_json, whose three legs each get a
* fixed share of the output slots by rotation — so on a query where a leg is
* structurally incapable of being right, that leg still consumes its slots.
* The promotion gate is Will's own answer to that: a candidate that does not
* clear its per-type threshold is not discarded, it is moved behind the ones
* that do, and the freed slots go to whichever leg still has promoted material.
*
* ENGRAM_WM_LEG_SCAN bounds the per-leg work: only the head of each already
* sorted leg can reach a result slot at any sane limit.
*/
#define ENGRAM_WM_LEG_SCAN 64
static int eg_wm_promote(const EngramNode* n, const char* q, double bg,
double* wm_out) {
/* Same product engram_activate's pass 2 computes (l.8519), minus the
* inhibitory / inhibition-of-return terms, which need activation state
* this read path does not carry. */
double bias = engram_goal_bias(n, q);
double impf = (n->importance > 0.0) ? (0.5 + n->importance) : 1.0;
double wm = bg * bias * n->confidence * impf;
if (wm_out) *wm_out = wm;
return wm > engram_type_threshold(n->node_type, n->tier);
}
el_val_t engram_search_json(el_val_t query, el_val_t limit) {
EngramStore* g = engram_get();
const char* q = EL_CSTR(query);
@@ -9714,8 +9759,77 @@ el_val_t engram_search_json(el_val_t query, el_val_t limit) {
: 0;
int64_t* order = malloc((size_t)lim * sizeof(int64_t));
if (order) {
int64_t no = engram_interleave3(hits, nhits, sem, nsem,
assoc, nassoc, lim, order);
/* ── Layer 1 → background activation, per leg ──
* Each leg's raw score is put on a common [0,1] footing
* WITHOUT blending the legs against each other (that
* failed in the score-fusion cut of the semantic leg —
* two rankings with different spreads cannot be summed).
* Lexical: BM25 over the score a node would earn covering
* every query token at mean field length, so the scale is
* "how much of this query's rare vocabulary did you
* actually account for". Semantic: the shift-and-floor
* value, already in [0,1]. Associative: cosine to the
* query, which is what orders that leg. */
double idf_sum = 0.0;
for (int t = 0; t < ntok; t++) idf_sum += idf[t];
double w_ideal = idf_sum * (ENGRAM_BM25_K1 + 1.0)
/ (1.0 + ENGRAM_BM25_K1);
if (w_ideal <= 0.0) w_ideal = 1.0;
int64_t nLs = nhits < ENGRAM_WM_LEG_SCAN ? nhits : ENGRAM_WM_LEG_SCAN;
int64_t nSs = nsem < ENGRAM_WM_LEG_SCAN ? nsem : ENGRAM_WM_LEG_SCAN;
int64_t nAs = nassoc < ENGRAM_WM_LEG_SCAN ? nassoc : ENGRAM_WM_LEG_SCAN;
EngramRankEntry Lp[ENGRAM_WM_LEG_SCAN], Lq[ENGRAM_WM_LEG_SCAN];
EngramSemEntry Sp[ENGRAM_WM_LEG_SCAN], Sq[ENGRAM_WM_LEG_SCAN];
EngramSemEntry Ap[ENGRAM_WM_LEG_SCAN], Aq[ENGRAM_WM_LEG_SCAN];
int64_t nLp = 0, nLq = 0, nSp = 0, nSq = 0, nAp = 0, nAq = 0;
/* ── Layer 2 → promote or suppress. Nothing is dropped. */
for (int64_t i = 0; i < nLs; i++) {
double bg = hits[i].w / w_ideal;
if (bg > 1.0) bg = 1.0;
if (eg_wm_promote(&g->nodes[hits[i].idx], q, bg, NULL))
Lp[nLp++] = hits[i];
else
Lq[nLq++] = hits[i];
}
for (int64_t i = 0; i < nSs; i++) {
/* RAW cosine, not the shift-and-floor value. The
* first cut of this filter fed pass 2 the shifted
* value, which for a genuine match (c .60-.70) is
* 0.02-0.25 — under every per-type threshold — so the
* whole semantic leg was suppressed while lexical
* junk cleared its gate. Pass 2's thresholds are
* calibrated against activation strengths in [0,1],
* which is the scale raw cosine is on. Measured cost
* of getting this wrong: paraphrase 61.5% -> 23.1%. */
double bg = sem[i].sem * (1.0 - ENGRAM_EMBED_SEED_MIN)
+ ENGRAM_EMBED_SEED_MIN;
if (eg_wm_promote(&g->nodes[sem[i].idx], q, bg, NULL))
Sp[nSp++] = sem[i];
else
Sq[nSq++] = sem[i];
}
for (int64_t i = 0; i < nAs; i++) {
if (eg_wm_promote(&g->nodes[assoc[i].idx], q, assoc[i].sem, NULL))
Ap[nAp++] = assoc[i];
else
Aq[nAq++] = assoc[i];
}
/* Promoted material fills the head, in leg order; the
* background-only field follows behind it (claim 45). */
int64_t no = engram_interleave3(Lp, nLp, Sp, nSp,
Ap, nAp, lim, order, 0);
if (no < lim)
no = engram_interleave3(Lq, nLq, Sq, nSq,
Aq, nAq, lim, order, no);
/* Anything past the scanned head of each leg, only if the
* filter left the result short of the caller's limit. */
if (no < lim)
no = engram_interleave3(hits, nhits, sem, nsem,
assoc, nassoc, lim, order, no);
for (int64_t k = 0; k < no; k++) {
if (!first) jb_putc(&b, ',');
engram_emit_node_json(&b, &g->nodes[order[k]], 0);