Content-hash census of the live graph: 1,858 redundant copies, 44.9% of the
non-ISE store, all from a June id-scheme migration that re-added nodes under
fresh UUIDs instead of matching on content. Generation stopped in June; the
copies did not. Being byte-identical they carry identical embeddings, so they
score identically against any query.
Measured over 50 real query probes against the live 3,998-vector set:
40.2% of semantic seed slots were consumed by redundant copies of content
already in the seed set, 92% of retrievals affected, effective distinct seeds
4.78 of 8. Two fifths of every retrieval was spent re-reading the same page.
Deleting nodes is a separate operation with its own backup discipline. This
change makes the runtime immune to the condition instead: redundancy can never
buy a scarce slot, whatever state the graph is in. Enforced at both scarcity
points — semantic seed selection (a rejected copy does not consume one of the K
slots; the loop retries for the next distinct node) and WM admission via a new
Pass 3+1/2 ahead of the capacity cap, so 24 slots are contested by 24 distinct
meanings rather than by however many copies of one document exist.
Identity is exact content hash first, then cosine >= 0.995 for copies that
differ only in insignificant characters. At 768 dimensions that admits only
near-verbatim text: this suppresses redundancy, never similarity.
Live after restart: ~8.8 redundant seed candidates rejected per activation.
New dup_seeds/dup_wm gauges in act-stats.
WM continuity (the significant one). A node reached by the current query but
scoring under its type threshold was zeroed outright, while a node the query
did NOT reach got the full ACT-R carry-over treatment. Being found was punished
relative to not being found. Measured consequence: WM turned over 100% every
call — three activations of a byte-identical query gave |A∩B| = |B∩C| = 0 — and
wm_evicted stayed 0 the whole time because that path never counted. WM was not
a working set; it was six suppression-breakthrough nodes re-drawn per call.
Both exits from a WM slot now share one extracted retention rule.
Result: WM 6 -> 24 nodes (the designed Cowan capacity), top weight 0.097 ->
0.748 (natural promotion, not the breakthrough floor), and contents that are
actually query-relevant.
Hebbian learning. Edge weights were written once at engram_connect and never
changed; last_fired's only writer in 12.5k lines was an unrelated dharma path.
Every learning mechanism operated on nodes — the wiring between them was
frozen. Adds co-activation potentiation (HeLa-Mem arXiv:2604.16839) in a
separate `hebb` field so authored structure is never mutated, with homeostatic
per-node scaling the source lacks (PNAS 2422602122) to prevent hub saturation.
Measuring it produced the finding that mattered: zero edges existed between
co-active WM members, so reweighting existing edges was a no-op. This graph's
41k edges were all authored by explicit tool calls — nothing had ever formed an
association from experience. So Hebb literally: if the wire is absent, grow it.
Consolidation is gated hard (sustained EWMA past 0.15, <=2/call, 5% ceiling,
in-memory candidates discarded on restart) because it permanently mutates the
graph.
Two bugs caught only by instrumenting rather than assuming: the snap-to-zero
floor sat above the per-step increment, so nothing could ever accumulate; and
the reached-but-sub-threshold eviction above. Verified live end to end — 53
links formed under load, then discarded with the test snapshot.
Also exposes engram_act_stats_json over GET /api/act-stats. It had existed
since 2026-07-27 but was reachable only through the soul daemon, so diagnosing
the activation layer required a working soul. This review needed it and could
not get at it.
The soul's curiosity auto-term extractor takes the first word of a top-WM
node label. It has no term-quality scoring, so three prior self-reviews each
bolted on another hand-curated blocklist (genre words 07-23, quoted titles
07-25, stopwords 07-30). Every one was written reactively, after a flood was
already observed. A list can only contain floods that already happened.
Two were in flight and unfixed when this review ran:
"<!--" label df 220 -> 252 nodes activated
"SELF" label df 175 -> 541 nodes activated (list has "Self" Title-case;
str_eq is case-sensitive, so the uppercase token sailed through)
engram_label_df(term) counts nodes whose label contains term. Low-specificity
tokens are corpus-frequent by definition, so this catches the flood class
prospectively and tracks the corpus as the world-ingestor changes it. This is
Sparck Jones (1972), which introduced IDF under the name 'term specificity';
automatic stopword compilation from it is the textbook application.
NOT a replacement for the stopword list -- verified against all 86 listed
terms, not assumed. Catches 13 (Will:306, Self:175, Over:116, Knowledge:112),
misses 73 (Whose:0, Would:0, Could:0, This:9). Labels are terse titles, so
English function words are genuinely rare in them. The gates cover disjoint
failure modes; both are required.
Policy lives in awareness.el, not here: the runtime measures, the soul decides.
Working memory was thrashing behind a healthy-looking gauge. wm_active sat
at 22-24 while breakthroughs ran 661-903 and evictions 485-717 PER 60s tick
- roughly 825-1125 nodes cycling in 5-call lockstep.
Root cause: the breakthrough path was an anti-starvation mechanism that reset
its own counter on firing, with no budget and no refractory. A node failing
its type threshold 5 times was force-promoted at exactly 0.10 and had its
suppression_count reset to 0, so it immediately restarted the identical
climb. Since BREAKTHROUGH_WEIGHT (0.10) > WM_FLOOR (0.05), every one of them
cleared the admission floor and entered the rank contest tied at 0.10, where
the tie-break degenerated to node-array index order. Cap-evicted nodes are
skipped by retrieval reinforcement, so they never got an access_ts record and
the STI inhibition-of-return damper never applied to them. That closed the
loop: re-suppressed, completely unmarked, forever.
An anti-starvation rule that resets its own counter without a bound is not a
fairness valve, it is an oscillator.
Fixes in engram_activate Pass 2:
- ENGRAM_BREAKTHROUGH_BUDGET (WM_CAP/4 = 6) caps intrusive thoughts per call.
- ENGRAM_BREAKTHROUGH_COOLDOWN (55) via NEGATIVE suppression_count. The field
already serializes as %d and parses through eg_get_int_field, so negatives
round-trip through snapshots with no struct or format change.
- Blocked breakthroughs no longer reset the counter; it saturates so a starved
node surfaces on a later call instead of restarting from zero.
- Graded breakthrough weight by nearness to own threshold, so the rank
tie-break is cognitive rather than insertion order. Invariant preserved:
WM_FLOOR < weight < min(type_threshold).
Also: moved the additive cosine term AFTER the STI multiplier. It was applied
before, so an incumbent re-reached 30s later took t_n/(t_n+120) = 0.2x, which
cut the semantic term's ceiling from 0.20 to 0.04 - below every per-type
threshold. Meaning-match was being punished for having been recently useful.
Inhibition-of-return should rotate the structural score, not the semantic one.
Also: _eg_act_wm_evicted counted 3 of 5 eviction paths. The two carry-over
paths were silent, so the reported rate was an undercount of unknown
magnitude - while being used to diagnose an eviction pathology. All five now
increment.
Also: route_sync returned {"nodes":[],"edges":[]} when the snapshot export
failed. The soul's sync_ok check only tests for "" and "{}", so that
placeholder passed as a healthy sync: last_sync_ok_ts stamped, sync_age_ms
green, sync_empty never fired, added:0 forever. A broken sync was
indistinguishable from a quiet healthy one - the exact class this route was
added to fix. Returns a real error now.
Verified live (boot 20 vs boot 19): breakthroughs 661-903 -> 36/tick,
evictions 485-717 -> 12-46/tick against a counter that now covers more paths,
wm_active unchanged at 22-24, wm_avg_weight 0.138-0.273 -> 0.186-0.446.
Working memory is holding strong nodes instead of breakthrough-floor filler.
Every node object on consumer read routes (/api/nodes, /api/search,
activation results, neighbors, compiled context) carried the full ~5.7KB
emb vector — responses 10-50x oversized, blowing MCP token limits.
engram_emit_node_json now takes include_emb; only engram_save passes 1,
so persistence and the /api/sync//api/edges replication paths (which
serve engram_save output) keep embeddings intact.
_eg_act_wm_evicted/_eg_act_breakthroughs were reset at the top of every
engram_activate, so act_stats reported only the last call and the 60s
heartbeat missed nearly all events (curiosity runs 2 activates per 30s).
Both are now monotonic process-lifetime totals; consumers diff readings.
Working memory was pinned saturated (24/24, wm_saturated:1 on every
heartbeat) because every cap path only trimmed the population down TO
the cap — rank-based eviction guarantees a full WM whenever >=24 nodes
hold any weight, so sub-cap fill was unreachable and the saturation
flag carried no information.
- ENGRAM_WM_FLOOR 0.05: absolute admission bar (Soar WM forgetting,
Derbinsky & Laird ICCM 2012 — removal by absolute threshold, not
rank) applied in Pass 4, carry-over, Pass 5, and load-cap. Fill can
now drain below 24 during quiet periods.
- Zero wm_anchor at every eviction site: stale anchors on evicted
nodes were a latent resurrection bug.
- Context centroid folds only NEW WM entrants: incumbents re-promoted
every scan no longer re-entrench the centroid each call, breaking
the WM->centroid->e_eff->re-selection positive feedback (fixation
driver behind the wm_top0_streak=1407 incident).
Verified live: wm_active 3->22->23, wm_saturated:0 post-restart.
n->importance was stored, serialized, and clamped at creation but never
read by any activation path — a curated importance=1.0 node competed
identically with a default note. Multiply raw_wm by (0.5 + importance):
default 0.5 nodes are unchanged (x1.0), critical x1.5, low x0.6;
importance<=0 from legacy snapshots stays neutral. Verified activation
and WM promotion unchanged for default-importance candidates.
- Gate each spreading-activation increment by target-node query similarity
(arXiv:2606.30133): soft gate FLOOR+(1-FLOOR)*clip(cos), FLOOR=0.25, for
embedded targets; ungated for unembedded; disabled when embedder is down.
Prior spreading was query-blind — hubs relayed activation into branches
unrelated to the query.
- Stats: add embed_eligible_count so embedding coverage is measured against
the true denominator (ISE/Tag/short nodes can never embed). Today's review
misread 3753/12693 as a 30% coverage gap; eligible coverage is 100%.
- Observability: per-call wm_evicted + breakthroughs counters and embed
circuit-breaker state exposed via engram_act_stats_json() — the three
highest-value previously-invisible executive-filter transitions.
- Carry-over branch: occupancy inhibition m = t_c/(t_c+t_hold), t_c=3600s
(ENGRAM_CARRY_TC). An unreached incumbent held its wm_anchor verbatim
(keep~1.0 for BLL inflated in the pre-07-25 era) — observed 23h at WM
top while every reached node rotated at the 0.10 breakthrough floor.
STI only runs in the reached branch; inhibition must key on occupancy,
not retrieval recency (Morita 2021 / Lebiere & Best 2009).
- engram_strengthen: drop the 07-22 BLL access record — the 07-25 STI
multiplier reads the same ring, so novelty reinforcement self-inhibited
its target for ~2 minutes.
- engram_load reset: free n->emb (~3KB/embedded node leaked per reload).
- engram_wm_top_json: emit id — its absence made the heartbeat's
wm_top0_streak compare ""=="" and measure uptime, not fixation.
Working memory was winner-take-all: suppression_count never entered the
promotion score and was reset on promotion, so two high-salience nodes
pinned a saturated 24-slot WM for hours. Add Lebiere-Best (CogSci 2009)
short-term inhibition — raw_wm *= t_n/(t_n + 120s) from the most recent
recorded access — producing emergent round-robin over WM candidates.
embedded_count stalled at 93/12175 after restart: the lazy backfill only
runs inside engram_activate, which nothing calls on the authoritative
store in production, and in-RAM vectors were never snapshotted. Add
engram_embed_backfill(n) + GET/POST /api/embed-backfill route that
persists the canonical snapshot whenever it embeds anything; the soul
heartbeat pumps it at 32/min.
Semantic activation was spec-only since 2026-06-30 — the seed loop used
istr_contains and nothing else. Per the 07-21 integration brief:
- EngramNode gains a lazily-backfilled nomic-embed-text vector (8/call
inside engram_activate, newest-first; no create-path latency, no bulk
Ollama hammering during sync seeds)
- query embedding (cached) drives a top-K cosine seed supplement
(HippoRAG use-similarity-twice) plus an additive WM term with
shift-and-floor at 0.45 — raw cosine is a constant bias in anisotropic
spaces (unrelated pairs read 0.4-0.7), floor-and-ramp makes it a signal
- 4s embed timeout (http_do_t) + 3-strike circuit breaker: activation
never wedges on a dead embedder; everything degrades to lexical
- embeddings persist as %.4g comma lists in snapshots, parsed by both
loaders; embedded_count in /api/stats tracks coverage
- engram_cosine_sim + http_delete_json exposed (DELETE now carries a
body — the server's _auth scheme requires it)
- route_create_node honored only content/node_type/salience; label,
importance, tier, tags were silently dropped (label defaulted to
content). Now honored via engram_node_full.
Verified live: embedded_count 0->96 across activations, semantic-only
promotion observed (zero token overlap), snapshot round-trip intact.
Durability: the 2026-07-21 fix stopped read routes writing the canonical
snapshot but left no save on ANY write path — every mutation lived in RAM
until a manual POST /api/save. Observed live: two restarts reverted the
store to a 17h-old snapshot, destroying same-day writes. persist_canonical()
now runs after node/edge create, knowledge capture, forget, strengthen, and
load-merge. ISE telemetry excluded deliberately (48h-pruned, loss-tolerant,
~2/min; snapshotting 28MB per heartbeat is waste).
Listing order: scan routes sort by salience with store-order ties, so
equal-salience telemetry (all ISEs are 0.3) returned OLDEST first — a
limited /api/nodes query silently returned a stale window, and a 41h-old
heartbeat series read as a live outage during this review. Ties now break
newest-first by created_at.
The old carry-over (weight *= 0.7 per engram_activate call) was call-rate-
dependent — carried context died in seconds under rapid curiosity scans and
lingered for hours under quiet loops — and a decayed scalar cannot represent
access frequency at all.
Now: k=10 access-timestamp ring + Petrov (2006) closed-form tail, d=0.5.
WM promotion and engram_strengthen record presentations; carry-over evicts
at base-level tau=-3.0 (Soar forgetting, ~403s single-touch) and shapes the
weight held at promotion (wm_anchor) with the ACT-R retrieval logistic
(s=0.4) — a pure function of wall-clock time, idempotent per call.
Persisted as access_ts/wm_anchor in snapshots; legacy nodes fall back to
the optimized form ln(n/(1-d)) - d*ln(L). base_level exposed in both node
serializers for observability.
Backing spec: 2026-07-21 integration brief (bl-b17facdd). Verified live:
carried weight ~anchor seconds after two disjoint activations (old code:
0.49x); frequency-hot nodes hold B=1.9 vs -0.14 single-touch.
Three fixes that existed elsewhere but never reached the runtime the engram
binary actually builds against:
- tokenized + ranked query matching (search/search_json/activate seeds/
goal_bias) ported from the el-compiler copy (e3dabe3, 2026-07-14) — the
production engram kept whole-query Ctrl-F for 5 days after the fix
'shipped'. Multi-word curiosity seeds went 0 -> 36 activated. Kept the
ISE seed exclusion the el-compiler copy dropped.
- Knowledge -> 0.20 WM threshold after tier checks (dev-line 4bf7716):
Semantic/Episodic Knowledge nodes fell to the 0.40 note default and only
entered WM via breakthrough.
- goal_bias: Knowledge in is_knowledge + curiosity-seed technical terms
(dev-line d53516b).
Also: seed_epoch was a running pairwise average, not the mean it claimed —
exponentially over-weighted later seeds in the temporal-proximity bonus.
Fixed to a true int64-sum mean. Stale INHIBITION_FACTOR comment corrected.
Root cause captured as knowledge: two runtime copies + branch-per-fix
without merge discipline stranded the entire dev semantic layer (cosine
activation, embeddings) out of production. Reconciliation planned as P1.
1. engram_neighbors_json (release runtime): BFS frontier/visited strings were
el_strdup'd (arena-tracked) but manually freed, so el_request_end()
double-freed every one — SIGABRT in http_worker under load (2 prod crashes
today via /api/neuron/session/begin and /api/neuron/graph; reproduced and
verified fixed with ASAN). Introduced when porting from the dev runtime,
which correctly uses plain strdup. Third instance of the
arena-vs-manual-free class (after EngramNode 07-15 and idmap keys 07-16).
2. server.el: let-in-if scoping sweep — defaults assigned inside if-blocks
never mutated the outer binding, so /api/search and /api/activate always
ran with q="", created nodes got node_type=""/salience=0.0, edges got
relation=""/weight=0.0, and save/load with no path hit engram_save("").
Rewritten to the let-if-else expression form. /api/activate now also
rejects empty queries instead of wiping carried WM weights.
3. engram_activate: retrieval reinforcement (ACT-R base-level learning) —
nodes promoted to WM that survive both capacity caps now get
last_activated/activation_count updated, so frequently retrieved memories
decay slower than abandoned ones. Scoped to promoted-only to avoid
flattening dampening across BFS fan-out.
engram search/activate/goal-bias matched the ENTIRE raw query string as a
single case-insensitive substring (istr_contains(field, q)). Multi-word
queries like "windows msi signing" only matched a node containing that exact
contiguous run, so real multi-word queries returned ZERO on a graph saturated
with the answer. This is Ctrl-F, not search — and search is the core of the
engram being useful.
Fix: split the query on whitespace into distinct tokens; a node matches if it
contains ANY token in content/label/tags. Rank by distinct tokens matched
(desc) then salience (desc). istr_contains is kept unchanged as the per-token
primitive. Single-token queries are a strict special case (score 0 or 1) so
the many single-word callers do not regress.
Sites changed (all in el_runtime.c):
- new helpers engram_tokenize_query / engram_node_match_score / engram_rank_cmp
- engram_search (internal el_val_t path)
- engram_search_json (HTTP /api/search path)
- engram_activate seed loop (HTTP /api/activate path; seed activation scaled
by token coverage so full-query matches seed more strongly)
- engram_goal_bias overlap bonus upgraded to graded token coverage
Proof (6591-node snapshot copy, rebuilt binary on :8799, POST JSON path):
windows msi signing 0 -> 20 Will Anderson 0 -> 20
windows msi 0 -> 20 tokenized search fix 0 -> 20
Single-word parity preserved (VBD/volatility/elc capped at limit; unkey = all
matching nodes). Top hits are relevant (e.g. "Will Anderson" surfaces the
Project Design and VBD whitepapers).
Note: GET ?q=a%20b still returns 0 because query_param (server.el) does not
URL-decode — a separate EL-layer bug; the soul's POST-JSON path is fixed here.
Out-of-range tok_kind/tok_value reads returned runtime null (el_list_get OOB
-> 0) rather than the Eof sentinel, so the inner parse loops (parse_block,
call-arg, array-literal, match-arm) that terminate only on their close
delimiter or k=="Eof" never saw Eof once the cursor ran past the single
trailing Eof token. On unclosed-delimiter input the parser then appended AST
nodes forever -> unbounded allocation -> ~700GB -> OOM (observed compiling
neuron/sessions.el).
Fix at the choke point: tok_kind returns "Eof" and tok_value returns "" for
out-of-range positions, restoring the parser-wide contract that reads at/after
the end yield Eof. expect() no longer steps past the Eof sentinel on mismatch.
This terminates every overrun loop simultaneously; a malformed program now
surfaces as a normal (best-effort) parse end instead of exhausting memory.
Requires a self-hosted bootstrap rebuild of elc to take effect.
Two independent investigations, one runtime, complementary halves:
1. Leak (Jul 2, this machine): JsonBuf buffers returned via el_wrap_str
were raw malloc, never arena-tracked — every engram_*_json call leaked
its output unconditionally. Added jb_finish() arena-tracking across all
~30 return sites. Plus el_arena_push/pop per-tick bracketing support
for the soul's awareness loop (the loop ran outside any request arena,
so even correctly-tracked allocations were permanent — 7.5GB RSS in
under a minute at 1s tick).
2. Corruption (Tim's container soak, docs findings/container-migration):
stored engram node/edge fields (content, node_type, label, tier, tags,
metadata, from/to ids) were arena el_strdup — freed at request end,
leaving dangling pointers that read back as recycled request-buffer
bytes one request later. This is the June corruption root cause and
the mechanism that grew snapshot.json to 18GB of empty-type junk
(21.6M nodes, 3,335 real). 39 sites switched to el_strdup_persist,
plus a latent double-free fix in engram_load metadata fixup.
Interaction note: fix 1's per-tick arena reclamation makes fix 2
mandatory — more aggressive arena recycling widens the use-after-free
window if stored fields still live in the arena. Apply as a pair, never
separately.
Verified live: soul + engram rebuilt from this runtime, booted against
the recovered real snapshot (3,335 nodes/40,146 edges), 5h stable at
<100MB RSS, write-then-next-request field-integrity test passes (the
June corruption fingerprint does not reproduce). engram/dist/engram
binary updated from this build.
Investigation credit: leak diagnosis this machine Jul 2-6; corruption
diagnosis + persist-fix patch by Tim's instance (docs PR #4).
engram_load_merge was added to el-compiler/runtime in 35c1897 but never
ported to the released runtime used by Engram and the soul daemon.
awareness.el calls engram_load_merge in its sync refresh cycle; without
this function in lang/releases/v1.0.0-20260501/el_runtime.c the soul
daemon fails to compile.
Also adds header declarations for engram_wm_count, engram_wm_avg_weight,
engram_wm_top_json, and engram_load_merge — all four were added as
implementations (da116b2 / 35c1897) but their prototypes were missing from
el_runtime.h, causing implicit-function-declaration warnings and potential
ABI breakage on stricter compilers.
Identified during self-review 2026-06-30.
Port critical WM fixes from self-review 2026-06-26 branch (f7bd99a) that were
never merged to HEAD. Running binary had these fixes; source did not — rebuild
would have silently regressed all three improvements.
1. ENGRAM_BREAKTHROUGH_WEIGHT 0.25→0.10
With 0.25, naturally-promoted nodes (threshold ≥0.15) decayed below the
breakthrough floor within one activation call and lost their WM slot to
fresh breakthrough candidates. All 524/525 WM nodes were at floor = useless.
Invariant: BREAKTHROUGH_WEIGHT < min(type_thresholds = 0.15 Canonical).
2. ENGRAM_WM_CAP=24 with Pass 4 (per-call) + Pass 5 (global) enforcement
Without cap, broad curiosity seeds promote 500+ nodes simultaneously.
wm_avg_weight collapses, goal-bias differentiation is lost. Verified:
"knowledge" query now promotes exactly 24 nodes (was 525). Cowan (2001)
cognitive basis: WM capacity ~4 chunks; 24 allows rich multi-topic context.
3. ISE exclusion from WM (Pass 2 guard)
InternalStateEvent JSON content ("knowledge", "memory", etc.) triggered
lexical seeding → suppression accumulation → breakthrough at floor. ISEs
are observability-only and must never surface in context compilation.
suppression_count cleared so ISEs never build toward breakthrough.
4. route_create_ise importance fix (0.5→0.3)
Corrects mismatch between HTTP route and awareness.el in-process fallback.
Also adds body comment clarifying auth-exempt rationale.
SYNAPSE (arXiv 2601.02744) validates WM cap design and ISE exclusion principle.
Next priority: cosine similarity seeding to complement lexical BFS.
ElBridge.java: Android Java companion to el_android.c — all public methods are
static, dispatches View mutations to the UI thread via runOnUiThread/CountDownLatch,
and exposes native callbacks (nativeOnClick, nativeOnChange, nativeOnSubmit).
PLATFORM_BRIDGE_SPEC.md: authoritative spec for implementing new platform bridges
(slot table contract, required __* functions, callback dispatch pattern).
detect-platforms: shell script that probes for available bridge toolchains and
prints what can be built on the current machine.
new-platform: scaffold generator that creates a new el_<name>.c with all 33
required stubs wired up.
Add seven platform bridge implementations and the shared native target header:
el_native_target.h, el_appkit.m, el_uikit.m, el_android.c, el_gtk4.c,
el_sdl2.c, el_lvgl.c, el_win32.c, el_runtime_win32.c. Each bridge implements
the 33 __widget_* C builtins declared in el_native_target.h for its platform
toolkit. el_runtime_win32.c provides a POSIX-free runtime stub for cross-compiled
Win32 targets.
Three improvements from today's self-review:
1. ENGRAM_BREAKTHROUGH_WEIGHT 0.25→0.10
Live data showed 524/525 WM nodes at breakthrough floor (0.25). Knowledge
nodes promoted at 0.21 decayed to 0.147 in one call, fell below the old
0.25 floor, and were immediately evicted for fresh breakthrough candidates.
Natural promotion was invisible. Invariant maintained: 0.10 < all
per-type thresholds (min=0.15 Canonical).
2. ENGRAM_WM_CAP=24 with Pass 4 (per-call) + Pass 5 (global) enforcement
Without a cap, broad queries like 'knowledge' promote 525+ nodes
simultaneously. WM is now bounded to 24 nodes. Algorithm: qsort on
promoted weights, keep top-24 by cutoff, evict the rest. Global pass
enforces cap across nodes that were promoted in prior calls and persist
via working_memory_weight. Validated: WM promoted goes 525→24.
Cognitive basis: Cowan (2001) WM ~4 chunks; 24 gives richer multi-topic
context while preventing flooding.
3. ISE exclusion from WM + /api/neuron/state-events route
InternalStateEvent nodes were reaching WM via breakthrough (5 suppression
cycles) because their content (curiosity seed JSON with 'knowledge',
'memory', etc.) triggered lexical seeding. ISEs are observability-only
and must never surface in context. Fix: guard in Pass 2 clears
suppression_count and skips to wm_weights[i]=0.0.
Also added POST /api/neuron/state-events route to server.el (auth-exempt,
internal endpoint). The main soul daemon posts ISEs here but the route
was missing — all ise_post() calls were silently returning 'not found'.
Research: SYNAPSE (arXiv 2601.02744) validates spreading factor 0.8 (our
0.7), top-M WM cap design, and cosine similarity seeding. Next priority:
implement cosine similarity initial seeding from the other branch.
Implements the accumulation layer from the Layered Consciousness architecture
(provisional 64/064,262) and answers the deferred design question. Per the spec
and Will's design: new user-facing nodes (memories, knowledge, conversations) are
created in an accumulation layer at the TOP of the consciousness stack — the engram
the user sees — while the layers below (safety, core-identity, domain, imprint,
suit) shape behavior but are hidden from the user.
- Adds ENGRAM_LAYER_ACCUMULATION (5) + the layer record in engram_init_layers
(activation_priority 50, suppressible, not injectable, transparent=0).
- engram_node and engram_node_full now assign new nodes to ENGRAM_LAYER_ACCUMULATION.
- ENGRAM_LAYER_DEFAULT stays CORE_IDENTITY ON PURPOSE: it is the fallback for LEGACY
nodes loaded from snapshots without a layer_id, so existing data (the originator
corpus) is NEVER migrated. New-nodes-only — the immutable-originator rule.
This is the foundation for fixing the identity-bleed / customer-isolation issue
(user data was landing in Neuron's core-identity layer). The retrieval-side
provenance filter (introspection should compile from accumulation, not the
originator corpus — Persona 64/036,574) is a follow-on, pending the batch-2
Layered Consciousness + Engram spec docs for exact semantics. Compiles clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ports the fixes that until now lived only in the un-versioned el-sdk source the live
macOS soul was hand-built from (captured in the [DO NOT MERGE] live-darwin-runtime
snapshot) FORWARD onto main, faithfully and minimally — without dragging in the
snapshot's deletions of main's newer engram_wm_/engram_load_merge/http_serve_async.
1. UAF (hallucinated/lost-saves root cause): engram_new_id + engram_node_full now use
el_strdup_persist, NOT el_strdup. el_strdup tracks into the per-request arena that
el_request_end() frees when the creating HTTP request completes — leaving stored
nodes with dangling pointers (corrupted ids, 'saved but never listed'). Transplanted
verbatim from the live runtime; el_strdup_persist sites 19->27, matching live.
2. Atomic engram_save: write <path>.tmp, fflush+fsync, rename() over target (atomic on
POSIX) so a booting soul's engram_load never reads a truncated/0-byte snapshot — the
genesis -> nodes=1 -> 63-node-clobber loop. Plus a sparse-write floor: refuse to
overwrite a >200KB snapshot with one < 1/16 its size. (Validated in isolation:
harness 11/11; rebuilt+booted the darwin soul, round-tripped 5113 nodes, no clobber.)
The response-truncation fix is already on main (_tl_fs_read_len binary-safe length).
Compiles clean. For Will to build through CI/elb and deploy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
http_handler_fn / http_handler4_fn were defined only inside el_runtime.c, so soul
modules (routes/chat/...) that reference them via cross-module forward declarations
couldn't see the types — which broke the Windows link of every module. Moving the
public function-pointer types to the shared header is the correct home and unblocks
the build on all platforms (identical typedef, C11-safe redefinition in el_runtime.c).
With this, the soul links into a native Windows neuron.exe (mingw, static) that boots
and serves HTTP on :7770 — verified /health → 200 {"status":"alive",...} in a Win11 VM.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
handle_api_consolidate writes a "SessionSummary" node, but engram_valid_node_type
omitted it — so once this validation ships, every consolidate() would be silently
REJECTED at the engram boundary. Add SessionSummary to the allowlist.
Found in Will's PR review of neuron #1 / el #52.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
llm_call_system / llm_call accepted a model argument and discarded it:
they called llm_chain_call(system, user) with no model, and the legacy
ANTHROPIC_API_KEY fallback passed NULL to llm_provider_request, so every
non-agentic chat was pinned to LLM_DEFAULT_MODEL (claude-sonnet-4-5)
regardless of the caller's selection.
Thread model_pref through llm_chain_call: provider-chain entries still
honor their own NEURON_LLM_N_MODEL override and fall back to the
requested model otherwise; the legacy Anthropic path now uses the
requested model. NULL/empty preserves prior default behavior.
Effect: the soul's model selection (state soul_model / SOUL_LLM_MODEL,
e.g. claude-opus-4-8) now reaches api.anthropic.com. Previously the
chat response echoed the selected model in its label while the request
billed Sonnet 4.5.
Not built locally (no elc/cc toolchain on this checkout); needs stage CI.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Validate UTF-8 continuation bytes in jb_emit_escaped; pass valid
sequences through and escape orphaned/invalid start bytes as \u00xx.
Pre-existing change found uncommitted in the working tree; committed
here so it is reviewable rather than lost.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The wrapper signature was stale and didn't match the C primitive
__engram_node_full(content, node_type, label, salience, importance, confidence, tier, tags).
Because el_val_t is an untyped machine word, the compiler coerced caller args to the
wrong declared param types and forwarded them BY POSITION — so tier received an int,
importance/confidence received strings, label received a float, etc. (~100 corrupt nodes).
- Correct the wrapper to match the C contract 1:1 (no coercion, no reorder).
- Add engram_valid_node_type / engram_valid_tier allowlists; engram_node and
engram_node_full now reject invalid values with __println + return "" (fail loud,
no silent malformed write).
See neuron repo: HANDOFF-engram-write-corruption.md for the full write-up + deploy runbook.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
elb.el:
- Auto-detect Homebrew OpenSSL (-L$(brew --prefix openssl)/lib) so -lssl
resolves on macOS without manual flags; no-op on Linux
- Add -include elp-c-decls.h when present in out_dir: resolves undeclared
cross-module calls in packages like ELP that lack explicit imports
ELP source:
- Add import "morphology.el" to all 29 language morphology modules
- Add language module imports to morphology.el (all langs it dispatches to)
These were missing since ELP was originally built as a monolithic unit
parse_html_children consumed the closing `}` of the outer El function as
HTML text content when a tag was left open across a function boundary
(e.g. `page_open()` opens `<body>` without a closing `</body>`). Fix:
stop the children loop when the current token is RBrace — that token
belongs to the El function, not the HTML tree.
Add html_raw() and html_escape() builtins to el_runtime so templates
can interpolate trusted raw HTML and safely escape user-supplied content.
Rename elc-new.c → elc.c as the canonical compiler source; rebuild
elc binary from it.
Add `c_source "path"` in manifest.el build block — lets packages link
extra C files (platform stubs, native glue) without touching elb source.
On macOS, homebrew OpenSSL isn't on the default linker path. Detect it
via `brew --prefix` and inject -L/-I flags; no-op on Linux.
Rebuild elb binary; remove elc-new binary (elc is now canonical).
Rebuilt from fix/elc-oom-checkout: scan_fn_sigs_el() --emit-header path
+ el_mem_check() guard. Verified on checkout.el: all 3 sigs in .elh,
clean exit under normal load, exit(1) on memory limit exceeded.
Add el_mem_check() to el_runtime.c: reads ELC_MAX_MEM_MB (default 512),
checks RSS via getrusage (macOS bytes / Linux KB normalised to MB), prints
a clear diagnostic to stderr and exits(1) if exceeded.
Wire it into two places:
- compiler.el: upfront check at --emit-header entry point
- codegen.el: per-function check in the streaming loop after each
el_arena_pop, so runaway growth is caught at the earliest function
boundary rather than after the machine is already dying.
The --emit-header path previously called parse() which builds the entire
program AST in memory before writing the .elh file. For checkout.el (~491
lines with HTML template trees and deep BinOp string-concat chains), this
exhausted memory before the header could be written.
Fix: replace parse() + emit_header() with scan_fn_sigs_el() +
emit_header_from_sigs(). The new path tokenises the source once, then
walks the flat token list skipping over function bodies entirely — peak
memory is O(tokens) instead of O(whole-program AST).
New functions in parser.el:
- scan_type_el: reads a type annotation and returns its El source string
- scan_params_el: reads (name: Type, ...) and returns El params string
- scan_fn_sigs_el: token-level scan that collects El-style fn signatures
without building any expression AST nodes
New function in compiler.el:
- emit_header_from_sigs: writes .elh from scan_fn_sigs_el output
Self-hosting check: elc compiled with new elc, diff of outputs is
identical (zero difference).
Smoke test: elc --emit-header checkout.el produces correct three-entry
.elh (previously truncated at two entries due to mid-parse OOM).
will.anderson
tim.lingo
Tim Lingo