Two improvements:
1. BM25 search corpus coverage (server.el) — raised scan cap from 500 to
5000 nodes. On the 161K-node graph, 500 was 0.3% coverage; 5000 is 3%.
engram_scan_nodes_json returns nodes sorted by salience DESC, so ISEs
(salience 0.3) fall below Knowledge/Memory (0.5–0.8) naturally — the
effective corpus stays content-dense. Also updated stale comment on the
ISE route (no longer need high offset; recent-first ordering from May 23).
2. Traversal inference guard (el_runtime.c) — two changes:
- INFER_CAP reduced 256→32: proactive curiosity runs engram_activate
every ~30s. At 256 edges/call the soul daemon accumulated 107K edges
in 23h (5× BFS slowdown). At 32 the rate drops 8×.
- Edge count guard: skip inference entirely when snap_ec ≥ 40,000.
At that density most A→C paths are already explicit; marginal inference
value is low and the O(edge_count²) inner-loop cost is high. Self-limits
unbounded accumulation across restarts.
Two improvements:
1. ISE scan ordering — engram_scan_nodes_by_type_json now sorts InternalStateEvent
nodes by created_at DESC (most-recent-first) instead of salience DESC. Old
high-salience ISEs (session-start, wm-promotion) no longer dominate offset 0,
burying recent heartbeat and curiosity_scan events at offsets 20000+. New
behavior: ?limit=10 returns the 10 most recent ISEs regardless of salience.
All other node types retain existing salience-sorted behavior.
2. http_serve_async registered as elc builtin — added to builtin_arity() in both
codegen.el (EL compiler source) and dist/platform/elc.c (compiled C). Also
rebuilt elc binary from updated elc.c. This closes the fragile-patch gap from
2026-05-21: elc previously treated http_serve_async as an unknown identifier,
and the gap description noted elc would 'silently revert to blocking http_serve'
on next soul rebuild. Now http_serve_async has a proper 2-arg arity entry and
will survive all future soul recompiles without a manual neuron.c patch.
Semantic seed floor raised 0.65 → 0.70: field literature (SuperLocalMemory
arXiv:2604.04514) validates 0.70-0.75 as the noise floor for sentence-transformer
embeddings on non-trivial corpora. The 0.65 threshold was admitting false
positives that diluted BFS traversal quality on the 13K-node post-ingestor graph.
Top-30 cap retained — sparse graph (1.26 edges/node) needs more semantic entry
points than a dense graph would.
ACT-R Base-Level Learning insight applied to engram_temporal_decay: current
purely time-based formula treated a node activated 100 times identically to a
node never activated (same decay rate). ACT-R BLL (Anderson 1993) shows
frequently-accessed memories resist temporal decay. Fix: freq_resist multiplier
= 1.0 + 0.1 * log(1 + activation_count). count=0→1.0×, count=9→1.23×,
count=99→1.46×. Existing activation_dampening continues to reduce per-query
novelty for well-known nodes — the two mechanisms are complementary: resist
decay (durable in graph), dampen per-query (don't dominate any single turn).
Three improvements from daily review:
1. Add semantic seed supplement to Pass 1 activation (el_runtime.c).
Previously, engram_cosine_sim was only called in goal_bias (Pass 2) for
nodes that already matched lexically. Nodes semantically close but
lexically disjoint were completely invisible to activation. With 8K+
world-ingestor nodes added overnight, this was a critical gap. Now: after
lexical seeding, scan un-seeded nodes for cosine sim ≥ 0.65 and inject
top-30 as additional seeds. Sem seeds get 85% of full act to stay weaker
than exact lexical matches.
2. Lower WM promotion ISE threshold from >0.5 to >0.1 (el_runtime.c).
Only one wm-promotion ISE was ever logged — the 0.5 floor was too high.
Most practical Knowledge/Memory promotions are in the 0.1-0.5 range.
Lowering to 0.1 makes working memory activity visible in state events.
Three research-grounded improvements:
1. Tier-based temporal decay in el_runtime.c (engram_node_full, engram_node_layered):
Working=48h, Episodic=72h, Semantic=336h, Procedural=720h half-lives.
Grounded in ACT-R literature — differentiated decay by chunk type. The
temporal_decay_rate field existed but was always 0 (global 168h for everything).
New nodes now carry the correct half-life for their tier from creation.
2. Implement route_neuron_knowledge_promote in server.el (was a silent stub):
Reads existing node, creates promoted-tier copy with supersedes edge,
checkpoints. promote_knowledge MCP tool now has real effect.
3. ISE label extraction + offset support in route_neuron_state_events:
POST now extracts 'event' field from content JSON as label (heartbeat,
wm_promotion, etc.) instead of always writing 'state-event'. GET now
accepts ?offset= for pagination to reach recent ISEs.
BM25+ (k1=1.2, b=0.75, delta=1.0) now powers all search routes in EL.
No external dependencies in the activation/search path.
- bm25_tokenize/bm25_count_term/bm25_score_doc/bm25_search_json in server.el
- route_search, route_neuron_recall: engram_search_json -> bm25_search_json
- route_activate: BM25 pre-bias (strengthen top-10) before spreading activation
- Remove standalone /api/bm25/search endpoint (BM25 is the engine, not a feature)
- Fix zero-score filter: float comparison not string match
- Add + to tokenizer for URL-encoded query params
- Scan floor 200 nodes regardless of limit size
- Revert Ollama engram_embed_query from 9af2482 (no Ollama at query time)
- Add list_set and math_exp builtins to el_runtime.c
- Add list_set, math_exp, and float_add/sub/mul/div/gt/lt/eq/gte/lte builtins to
el_runtime.c + el_runtime.h (float arithmetic builtins needed because EL operators
+*/ operate on raw el_val_t bits, not IEEE 754 doubles)
- Remove engram_embed_query() and its forward declaration from el_runtime.c
- Remove Ollama cosine-similarity blend from activation scoring (reverts 9af2482):
drops query_emb/query_edim variables, bias *= (1 + 0.3 * sim) block, and all
free(query_emb) calls from the activation loop
- Implement BM25+ scoring in server.el (k1=1.2, b=0.75, delta=1.0):
bm25_tokenize, bm25_count_term, bm25_score_doc, bm25_search_json
V1 uses n_t=1 approximation (constant IDF per corpus size); acceptable as a
first pass without an inverted index
- Wire /api/bm25/search POST/GET route in server.el dispatcher
- Zero Ollama calls in the activation/search path; embeddings on nodes are
untouched (still written at node-creation time)
el_strdup tracks pointers in the arena. The BFS arrays in
engram_neighbors_json are manually freed — using el_strdup caused a
double-free when the arena was later popped. Changed to plain strdup
for those allocations.
engram/dist/engram.c rebuilt from engram/src/server.el with current
elc (minor codegen diff: parenthesisation and _argc/_argv rename).
Dharma's EngramDB client calls /nodes/list to retrieve all nodes.
Add this as an alias for the existing /nodes (and /api/nodes) route
so downstream clients don't need to be updated when the API drifts.
Also update dist/engram.c to match server.el.
Engram is now a thin HTTP face over the El runtime's in-process graph
store. The C runtime owns the data; engram_*_json builtins serialize
results directly. There is no SQL, no SQLite, no db layer, no state
machine — the runtime IS the database.
src/server.el (348 lines, replacing 5797 lines across 15 legacy files):
GET /health
GET /api/stats
POST /api/nodes (auth required)
GET /api/nodes
GET /api/nodes/:id
DELETE /api/nodes/:id (auth required)
POST /api/edges (auth required)
GET /api/neighbors/:id
POST /api/activate
GET /api/activate
POST /api/search
GET /api/search
POST /api/strengthen (auth required)
POST /api/save (auth required)
POST /api/load (auth required)
Auth: ENGRAM_API_KEY in env. GET routes pass through (read-only).
Mutating routes require {"_auth": "<key>"} in the JSON body until
http_serve surfaces request headers and we can switch to Bearer.
Persistence: engram_save / engram_load via JSON snapshot at
$ENGRAM_DATA_DIR/snapshot.json. Loaded best-effort on startup.
Build: dist/platform/elc src/server.el > dist/engram.c
cc -std=c11 -O2 -I <runtime> -lcurl -lpthread -o dist/engram
dist/engram.c <runtime>/el_runtime.c
Live: native binary at dist/engram (113 KB), running under
~/Library/LaunchAgents/ai.neuron.engram.plist on :8742. Verified:
GET /api/stats returns counts; POST /api/nodes with auth creates
node with UUID; GET /api/search returns full node JSON; spreading
activation returns hop-decayed strengths (0.8 × edge × decay per
hop) with epistemic confidence filtering.
Legacy (5797 lines of SQLite-era src) sealed at
~/Archives/engram-src-legacy-20260430.tar.gz and removed from disk.
will.anderson
Will Anderson