feat(engine): recall now spreads activation through the graph instead of matching substrings #135

Closed

tim.lingo wants to merge 1 commits from feat/recall-through-activation into main

Author SHA1 Message Date

Author	SHA1	Message	Date
Tim Lingo	bca7d8ac99	feat(recall): retrieve through spreading activation, not substring matching Neuron Soul CI / build (pull_request) Failing after 14m29s Details Neuron Soul CI / deploy (pull_request) Has been skipped Details recall and searchKnowledge both ended at engram_search_json — a case- insensitive substring matcher scored by how many distinct query tokens appear in a node's content/label/tags, tie-broken by raw salience. It never read a single edge. Meanwhile engram_activate / engram_activate_json — real BFS spreading activation over the weighted directed graph, four-factor multiplicative scoring, two-layer background/working-memory filter — has been implemented and compiled into the shipped runtime the whole time, called from four places, none of them retrieval. This wires retrieval to the traversal, restoring the designed mechanism: Engram provisional 64/064,260 claim 1, "no data is retrieved from the weighted directed graph except through the spreading activation traversal." Seeding follows the runtime's own convention (all four existing call sites pass query TEXT, not seed ids): engram_activate seeds lexically — every node matching >=1 query token, initial activation = salience x temporal_decay x dampening x token_coverage — then supplements with the top-K nodes by cosine against the query embedding. So the lexical surface recall used to RETURN is now the SEED SET of the traversal, and what comes back is what those seeds activate. Exact lookup is not regressed. engram_activate's collector drops any reached node whose background_activation x confidence < 0.1 unless it was promoted to working memory, so a rare token on a dormant node can seed and still go unreported. Retrieval therefore appends the lexical seed list after the activated ranking, deduped by id, until `limit` is filled — the same seed set the traversal already computed, restored to the tail, not a parallel search. searchKnowledge gets the identical path. Its existing "activate fallback" was unreachable dead code: it fired only when engram_search_json's return did not start with '[' or '{', and that function always emits a '['-prefixed array. Response shape is unchanged — a bare array of full engram node objects, so the MCP wrapper, tools/telegram-gateway.sh (.value.content) and cli/neuron_mcp.py keep working. Activation strength is a ranking input here, not a payload change. Measured, cold-start, two builds of this tree against the same 79,250-node / 14,214-edge graph (main @ `18714e6` vs this branch): "volatility-based decomposition" before: 1 of 10 results relevant after: 6 of 10, incl. architecture/styles/ vbd/glossary.md and project-design foundations "Structure is not inherited" before: persona boilerplate, "1", a Disneyland fragment, a corrupted node after: self/voice registers, neuron/ user-imprint/boundary-definition, diagrams/vbd.md "inherited" Value - Structure Is Not Inherited: rank 23 -> rank 3; Self - Values hub: rank 32 -> rank 4 searchKnowledge, same query 2 of 5 relevant -> 5 of 5 "HNSW" / "Fayetteville" (rare) 1 result both builds - no regression nonsense control 0 results both builds Known limit, unchanged by this commit: the 12 sibling Value nodes still do not surface. A ~58-day-dormant seed's activation (0.7 salience x 0.05 decay floor x 0.34 dampening ~= 0.012) lands below the runtime's 0.02 firing threshold, so it cannot propagate to its neighbours at all. That is runtime tuning inside the vendored el_runtime.c, not the wiring. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-07 12:27:51 -05:00

Tim Lingo

bca7d8ac99

feat(recall): retrieve through spreading activation, not substring matching

Neuron Soul CI / build (pull_request) Failing after 14m29s

Details

Neuron Soul CI / deploy (pull_request) Has been skipped

Details

recall and searchKnowledge both ended at engram_search_json — a case-
insensitive substring matcher scored by how many distinct query tokens appear
in a node's content/label/tags, tie-broken by raw salience. It never read a
single edge. Meanwhile engram_activate / engram_activate_json — real BFS
spreading activation over the weighted directed graph, four-factor
multiplicative scoring, two-layer background/working-memory filter — has been
implemented and compiled into the shipped runtime the whole time, called from
four places, none of them retrieval.

This wires retrieval to the traversal, restoring the designed mechanism:
Engram provisional 64/064,260 claim 1, "no data is retrieved from the weighted
directed graph except through the spreading activation traversal."

Seeding follows the runtime's own convention (all four existing call sites pass
query TEXT, not seed ids): engram_activate seeds lexically — every node
matching >=1 query token, initial activation = salience x temporal_decay x
dampening x token_coverage — then supplements with the top-K nodes by cosine
against the query embedding. So the lexical surface recall used to RETURN is
now the SEED SET of the traversal, and what comes back is what those seeds
activate.

Exact lookup is not regressed. engram_activate's collector drops any reached
node whose background_activation x confidence < 0.1 unless it was promoted to
working memory, so a rare token on a dormant node can seed and still go
unreported. Retrieval therefore appends the lexical seed list after the
activated ranking, deduped by id, until `limit` is filled — the same seed set
the traversal already computed, restored to the tail, not a parallel search.

searchKnowledge gets the identical path. Its existing "activate fallback" was
unreachable dead code: it fired only when engram_search_json's return did not
start with '[' or '{', and that function always emits a '['-prefixed array.

Response shape is unchanged — a bare array of full engram node objects, so the
MCP wrapper, tools/telegram-gateway.sh (.value.content) and cli/neuron_mcp.py
keep working. Activation strength is a ranking input here, not a payload change.

Measured, cold-start, two builds of this tree against the same 79,250-node /
14,214-edge graph (main @ 18714e6 vs this branch):

  "volatility-based decomposition"  before: 1 of 10 results relevant
                                    after:  6 of 10, incl. architecture/styles/
                                    vbd/glossary.md and project-design
                                    foundations
  "Structure is not inherited"      before: persona boilerplate, "1", a
                                    Disneyland fragment, a corrupted node
                                    after:  self/voice registers, neuron/
                                    user-imprint/boundary-definition,
                                    diagrams/vbd.md
  "inherited"                       Value - Structure Is Not Inherited:
                                    rank 23 -> rank 3; Self - Values hub:
                                    rank 32 -> rank 4
  searchKnowledge, same query       2 of 5 relevant -> 5 of 5
  "HNSW" / "Fayetteville" (rare)    1 result both builds - no regression
  nonsense control                  0 results both builds

Known limit, unchanged by this commit: the 12 sibling Value nodes still do not
surface. A ~58-day-dormant seed's activation (0.7 salience x 0.05 decay floor x
0.34 dampening ~= 0.012) lands below the runtime's 0.02 firing threshold, so it
cannot propagate to its neighbours at all. That is runtime tuning inside the
vendored el_runtime.c, not the wiring.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-07 12:27:51 -05:00

feat(engine): recall now spreads activation through the graph instead of matching substrings #135

1 Commits