- primitive_attend retrieves over HTTP (POST /api/search) when ENGRAM_URL is
set — the location-independent worker model — falling back to the in-process
store otherwise. Proven against the isolated :8901 clone: CCR compiled a
bounded context from REAL mind content (VBD/intellectual-dna).
- gate the engram work-tracking mirror behind SWARM_MIRROR=1; the durable
substrate is always the JSONL journal, so a swarm never depends on the mind
to track its work. (Repeated POST /api/nodes mirror writes were observed to
crash the isolated daemon — a daemon-side write-path robustness issue;
retrieval POST /api/search is solid. Prod :8742 never touched.)
- integ_engram: CCR real-retrieval + full swarm completion against live clone.
- vote/merge/reduce/collect convergence proven end-to-end; failure threshold
aborts a swarm below min_success_ratio (integer per-mille) and completes
when failures are within tolerance, with worker.failed + swarm.aborted
tracked durably.
- worked around three El runtime/codegen semantics surfaced during the build:
json_set inserts RAW (use json_set_str for string values); json_set cannot
update an existing key (vote tallies via list rescanning); json_array_get
keeps quotes (use json_array_get_string). Also: float division is unreliable
(swarm uses integer math), and a let-rebind in a deeply nested if/else does
not propagate outward (accumulators kept at one block level).
test_convergence: 8/8; test_swarm: 12/12.
Collapse ~90 noun-CRUD MCP tools into read/write/relate/supersede (type is a
parameter) plus the live agentic primitives (think/attend/learn/ground/assert)
already in the engram cognition build. Additive: old noun-tools aliased to the
new ops. Vantage-read applies aperture -> a bounded slice, fixing the whole-self
dumps. Signatures grounded in the live cognition binary; validated on an
isolated nsbx clone (parity.sh: 12 proven, 0 failed). Live :8742 untouched.
- swarm.el: coordinator running fan-out/converge on El NATIVE threads
(thread.el spawn/join) in bounded concurrency waves, order-preserving;
convergence strategies collect/merge/vote/reduce; integer per-mille failure
threshold (El float division is unreliable — avoided deliberately).
- ccr.el: per-worker Compiled Context Routing — retrieval/scoping/compaction
into a bounded, minimal package; the compiled-context boundary is the
security boundary (a worker cannot receive or leak sibling inputs).
- containment.el: the three Swarm containment rules enforced via scope tokens
(Rule 1 no join, Rule 2 no open, Rule 3 no lateral edge) + execution-tree
lateral-edge check.
- primitives.el: attend/think/intend/act/learn seam the swarm composes over,
with engram-backed fallbacks and an explicit binding point for the reshape.
- prototype json_array_push in el_runtime.h (defined but unprototyped).
test_swarm: 12/12 — native fan-out/converge, bounded concurrency, durable
tracking, CCR bounding + non-leak, and all three containment rules.
Single-writer append-only JSONL journal keyed by correlation ID: swarm +
worker + convergence records, reconstructable into a status report. Optional
engram mirror via POST /api/node when ENGRAM_URL is set. Coordinator is the
only writer (workers return structured results), which is race-free and
enforces Swarm containment rule 3 by construction.
Also prototype now_millis/now_ns in el_runtime.h (defined in el_runtime.c but
unprototyped — blocked any El program needing a real ms clock under clang 21).
Test proves durability + inspectability end-to-end.
el_runtime.h declared only __thread_create/__thread_join; the mutex and
channel seed primitives (__mutex_*, __channel_*) were defined in
el_runtime.c but never prototyped. Under Apple clang 21 (C11) the missing
prototypes became implicit-declaration errors, and the void-returning
__channel_send/__channel_close mis-typed el_val_t (long long) returns,
so any El program using runtime/channel.el failed to compile.
- add prototypes for __mutex_new/lock/unlock and all __channel_* to el_runtime.h
- make __channel_send/__channel_close return el_val_t nil so elc's
trailing-expression codegen for the void El wrappers type-checks
Additive; unbreaks native channels for every downstream El program.
Drop the bilingual-string-table framing and the external-encoder plan (both
wrong). Translation now routes source-lexicon -> concept-frame (language-
invariant, in the engram concept geometry) -> target-realizer, exactly as the
ELP was designed: a word resolves to the CONCEPT it denotes via its own
language's lexicon (a monolingual step — the engram nearest-region ranker only
disambiguates senses within one language, so an English-trained embedder is
fine and never compares 'ocean'~'oceano' as strings). The concept node is the
shared pivot; its manifold location is the meaning.
- Pronouns route through the NATIVE concept pivot (cp_pron_concept ->
cp_rom_pron_surface) instead of an ad-hoc EN->tgt string map.
- lemma_for_concept / noun_for_concept are each target language's own
CONCEPT->SURFACE lexicon (the mirror of comprehend's SURFACE->CONCEPT).
- Fidelity is concept-preservation (concept_frame fingerprint), not string
cosine against an external multilingual model.
- Plural article agreement fixed (las/los, as/os).
Verified: 'You never fought the ocean.' -> ES 'Usted nunca luchó el océano.'
concept-frame pivot 'pred=fight patient=ocean pol=neg' realizes to ES+PT from
one parse; nunca holds 3/3. Gaps unchanged: PT verb conjugation fallback,
adjunct/subordinator concepts not yet in-frame.
comprehend lemmatizes some irregulars (fought->fight) but not all (broke);
tr_norm_verb covers the poem's remainder so affirmative content verbs route
(ES 'Yo broo' -> 'Yo rompo'). Negation lines unchanged and still correct.
Adds the missing middle of the ELP: a deterministic EN-content-lemma ->
target-lemma bridge (translate.el) on top of comprehend.el (parse) and
realizer.el (inflect). English-only engram geometry cannot route
cross-lingually and vocabulary-XX.el carries no en_translation glosses, so
the honest no-LLM bridge is a wired lexicon (poem coverage; OOV passes
through). SACRED polarity/neg_word are carried untouched: 'never' localizes
to a negator ('nunca'), never to a content lemma.
Additive realizer extensions: agent_person/agent_number recognize Romance
target pronouns; the non-EN negation branch surfaces a carried neg_word
instead of the generic negator.
Verified on the real toolchain (elc->cc->run):
'You never fought the ocean.' -> ES 'Tú nunca luchaste el océano.'
'I never saw the breaking.' -> ES 'Yo nunca vi la ruptura.'
nunca holds 3/3 negation lines. Known gaps: PT verb conjugation fallback
(lutarred), irregular EN lemma (broke->break), adjunct/subordinator passthrough.
Add 'nsbx dev <name>' / 'nsbx dev-down <name>' plus a Makefile so a newcomer
goes from clone to coding on an isolated cloned engram in one command. The
worktree is created on a real named branch at a persistent path (never /tmp,
guarded), and the whole worktree is pinned to the clone via an emitted .nsbx-env
so live :8742 / ~/.neuron/engram is unreachable by accident. Optimizes the El
edit->build->run loop so provisional work is built in El against a throwaway
clone instead of prototyped in Python and re-ported. Additive over the proven
primitives; no live cutover.
Generalise the ad-hoc cog-arch (worktree+build+store-clone+C-tests) and
store-fix (secondary soul + launchctl rails cutover) proto-sandboxes into one
reproducible primitive: run experiments and code changes against the REAL
engram runtime on an isolated snapshot of the live mind, with a gated
promote-to-prod path.
Dev environment as a primitive — any team member gets a private, isolated copy
of the mind (separate port/store/process); prod on :8742/:7770 is untouchable
from a sandbox. Wraps the real binary; never reimplements engram logic.
Lifecycle: create/up (consistent store+WAL+config snapshot; place OR build the
runtime from --source/--branch/--binary; boot on an isolated port) · build ·
run · validate (rails as checks: zero-loss under load+reboot, reboot-prove, RSS
bound, retrieval parity, keystone integrity) · promote (gated rails cutover:
snapshot-first, additive binary swap, bootout→settle-poll→bootstrap, verify,
auto-rollback; never pkill/kickstart -k; dry-run unless approved) · destroy.
Dogfooded: reproduced retrieval-parity 25/25 vs baseline and the cog-arch
correspondence-loop known result (Brier 0.028648->0.000586, reboot-proven) and
real-store reboot-prove at 10994-node scale, all inside a sandbox; prod
untouched.
The buildable form of the "one operation" theory (memory bdc8a488). Maps the
theory onto what is already compiled: the five reasoning operators in
engram_reason.c already collapse onto ONE primitive — engram_reason_point_fit —
plus the geo-algebra (combine/subtract/analogy-rotate/distance), and
engram_verify.c is built on the same fit. So the operator-collapse is already
half-written; what is missing is not the primitive.
What is missing, and what this doc specifies:
- think(anchor, prior) -> gradient (a distribution/direction, not a point); each
named faculty = {point_fit + a prior}, the operation frozen, the prior learned.
- Prior as a first-class stored node (warp + calibration), superseding the
intrinsic importance/salience scalar with a relational, grounded-for-whom edge.
Confirmed against the runtime: importance is already a live activation
computation (el_runtime.c:13013), never trusted as a static field.
- vantage_read(anchor, aperture) — one op, three settings: self / foreign-field /
veil.
- The reflexive correspondence-loop as the learning engine: move the grounding
check from offline Python into the geometry, reflexive, reusing the DORMANT
verifier (engram_verify_grounding has no runtime caller and no El binding today)
turned inward. grounding = learning = one loop.
- hold/ground/assert kept distinct: the engram holds anything, grounding is an
edge, the honesty floor is on assertion only; ungrounded content is first-class.
- metastability: keystone core (read-mostly priors) + plastic everything else.
Seven staged milestones, earliest is a real end-to-end slice (induction as
{primitive + grounded prior} with the loop closing on it, reboot-proven on a
snapshot). Build rails stated: offline/secondary, snapshot-first, reboot-prove,
zero-loss, gated launchctl cutover. Design only; no code changed this pass.
Two changes to the activation path, both grounded in measurement on the live
store rather than on the spec.
1. Rescale cosine before the query gate.
The propagation gate (arXiv:2606.30133, added in an earlier review) fed RAW
cosine into FLOOR + (1-FLOOR)*c. Raw cosine from nomic-embed is compressed
into a narrow high band, so that expression is close to a constant.
Measured, 400 random UNRELATED node pairs on the live store:
median 0.562, central 98% span [0.381, 0.743]
So a node with no semantic relation to the query was propagating at
0.25 + 0.75*0.562 = 0.67. Two thirds strength. The gate was a small tax.
Fixed by shifting and flooring about ENGRAM_EMBED_S0 -- which is already in
this file, already 0.45, and already used exactly this way by the Pass-2 WM
term. The propagation gate simply never used it. Same 400 pairs after:
median unrelated pair falls to 0.40, top of range preserved (0.85 vs 0.92),
gate spread widens 0.42 -> 0.60. Only 8.5% reach the floor, so dissimilar
lexical/structural pathways are damped, never severed. Range is unchanged
at [0.25, 1.0], and cosq == NULL still degrades to no gating at all.
2. Decompose the WM eviction counter by cause.
_eg_act_wm_evicted was incremented from six sites with four distinct causes
and collapsed all of them into one integer. Today's review measured 175,547
evictions over 13.5h (~216/min against 24 slots) and could not tell healthy
rotation from cap thrashing from duplicate churn.
That is this file's most-repeated defect: dup_wm and dup_wm_global exist
only because the aggregate could not answer "why" during the 08-02 and
08-06 incidents. Each of those needed a NEW gauge before it was diagnosable.
evict_floor / evict_cap / evict_bll complete the decomposition, so
wm_evicted == floor + cap + bll + dup_wm + dup_wm_global
holds as an identity and each term implies a different correction. Verified
on an isolated instance: 30 nodes, 24 filled the cap, wm_evicted 6 ==
evict_cap 6, all other terms 0.
Built and smoke-tested out of tree. The live daemon runs a pinned binary and
was deliberately not restarted -- the store compaction workstream is in flight.
Ports dialogue.py + self_region.py to native el, bound to the IN-PROCESS engram
el runtime (engram_activate_json / engram_neighbors_json / engram_search_json /
engram_node_full / engram_connect — C-order builtins, not the wrapper order).
self_region.el: pulls the engram's REAL Self/identity nodes (pooled single-term
search), scores by self-signal, reads out identity from their own prose — no
hardcoded anchors, no template.
dialogue.el: ONE operation — project(query) -> land on a region -> read out.
* identity = self-region proximity (no intent classifier, no separate branch)
* memory = activation + a RELEVANCE FLOOR, then MATERIALIZE by walking the
neighborhood (real edges), never top-props
* HONEST ABSENCE when nothing is close — no 'I noted that' echo, no fabrication
* NEGATION SACRED: readout is the stored prose verbatim, so polarity survives
* DIRECTIVE OVERRIDE: a meta-directive switches the reply language
Verified against a SCRATCH in-process engram (live :8742 untouched): dialogue
gate 9/9 — identity from real self-content, neighborhood materialization,
SACRED negation (self + memory), PT identity in PT, directive override to
English, 'Prove it' -> honest absence. EN/Romance/prop/multilingual gates
unregressed.
- realizer now carries the subordinate clause verbatim (subord_text slot): the
5th English acceptance sentence is byte-identical through parse->realize->reparse.
- English -ed/-ing lemmatizer restores silent-e (loved->love) and collapses
inflectional doubling (stopped->stop), inverting en_verb_past().
- parse_spec_lang: a sentence-final main verb no longer bleeds into the object
slot (cstart advanced to verb+1), so intransitives round-trip.
- cp_rom_is_verb rejects closed-class words (prep/det/pron/aux/neg) before the
ending-only test, killing the 'para'/determiner misfires.
EN telephone gate 5/5 (now byte-identical 5/5), Romance gate 6/6.
Phase 3 piece 2. Ports multilingual.py: deterministic language detection
(en/es/pt/it) via stopword + diacritic scoring, localized fixed phrases (SACRED
per-language yes/no/decline/identity), PT/ES->EN retrieval term lexicon, and
EN->target predicate translation. No generative model.
Gate (multilingual_gate.el): 4/4 languages detected correctly; localized
declines + term/pred lexicons verified. Built bounded (elc rc=0 peak 25MB).
Worked through the documented el '+' mis-compile (two chained function-call Int
operands compile as string concat -> corrupt Int -> segfault on the accented
path); fixed by binding each score to an Int var and adding vars singly.
Simplifications (honest): diacritics scored by PRESENCE (str_contains) not
codepoint count (UTF-8 index safety); confidence scalar and the regex-based
parse_directive() from the reference not yet ported (directive parsing deferred
to the dialogue layer).
Phase 3 piece 1. Ports propositions.py off spaCy: the dependency-parser role is
now the el-native parser (parse_spec), and each memory sentence's meaning-spec
IS the triple (subject, predicate, object, modifiers, polarity, tense, source,
confidence). Sentence segmentation + repr parity with propositions.py. NEGATION
SACRED: polarity flows straight from the spec, never dropped/inverted.
Gate (propositions_gate.el): 4/4 SACRED polarity correct on extraction;
multi-sentence memory splits one triple per sentence in reading order with
negation preserved. Built bounded (elc rc=0 peak 24MB, cc rc=0).
Gap (honest): English regular-verb lemmatizer does not restore silent-e
(stores->stor); coreference/passive normalization from the reference not yet
ported (shallow pronoun subject kept as surface).
Adds a deterministic Romance front-end to comprehend.el (parse_spec_romance),
dispatched from parse_spec_lang for lang es/pt. English path untouched
(byte-identical, regression gate still 5/5). Pro-drop aware clause skeleton
(subject | neg | verb | object | PP), cross-lingual negation lexemes already
SACRED. Romance telephone gate: polarity PRESERVED 6/6 and EXTRACTED 6/6
through parse->realize->re-parse for 3 ES + 3 PT sentences. Built bounded
(elc rc=0 peak 24MB, cc rc=0).
Named gaps (honest): ending-only verb detection misfires on prepositions
(contra) and -a/-o nouns (menina); lemma recovery keeps surface form; the
non-English realizer is a generic preverbal-negator skeleton so ES/PT surfaces
are not byte-parity. Full paradigm inversion + Romance lexicon deferred.
PIECE 1 — greenfield el-native parser (comprehend.el), spaCy-free:
- text -> meaning-spec via invertible English morphology (the realizer's own
irregular table run BACKWARD) + a deterministic clause grammar (subject/verb
boundary, roles, ditransitive iobj, PP adjuncts, subordination, coordination).
- NEGATION IS SACRED: explicit polarity field, always present, cross-lingual
lexeme set; standalone neg adverbs (never) captured separately.
- WSD by deterministic syntactic position over a fixed sense inventory
(flies->fly, like->comparison, saw->see); engram nearest-region is the
documented runtime upgrade hook (no external model).
Polarity threaded through the whole el contract (was previously dropped at the
boundary): realizer.el realize_lang honors polarity (English do-support /
adverbial / copular negation; generic preverbal negator for es/pt/ca/it/fr/de/ro)
and places iobj; elp.el build_form_from_json carries polarity/neg_word/iobj
across JSON; morphology.el gains 'fight'.
Acceptance (native el telephone test, comprehend_gate.el): on the 5 gate
sentences polarity PRESERVED 5/5 and EXTRACTED 5/5 through parse->realize->
re-parse; 4/5 byte-identical. Built bounded (elc rc=0, cc rc=0).
Backfill ELP vocabulary from FULL lexicons (UniMorph + kaikki.org Wiktionary,
real gender/inflections) for 8 languages, 812,894 entries total, in the proven
seed-fn format matching the 18 ancient vocabularies:
es 72,032 | fr 130,517 | de 144,692 | la 22,590 | it 193,675 | pt 115,772 |
ro 86,504 | ca 47,112
4 of these (es fr de la) backfill ELP languages that had morphology but no
vocabulary; it/pt/ro/ca are new Romance (need morphology-*.el ports next).
Adds lang_profile_* for all 8 + reproducible generators under tests/lang-gen.
Vocab is runtime seed data (not in build manifest, like the 18 ancients);
seed-fn format validated to compile to C via elc.
auto_term_empty_streak — the counter the 2026-08-06 review added to catch
exactly this — read 50 and climbing. Fifty consecutive curiosity scans where
the soul's dynamic seeding produced nothing and the loop fell back to four
hardcoded phrases. The live WM top said why in one look: every slot was a
Memory node labelled "memory:remembered". The extractor read the LABEL only,
the sentinel guard correctly rejects sentinels, so there was never anything
to extract. It was written against Knowledge nodes, which have real titles,
and was structurally blind to the node type that dominates working memory.
Rather than add a sixth guard to the five that accumulated across four
reviews (genre words, quoted titles, stopwords, label-df), invert the
algorithm. The old one was: take the first word, then check whether it is
acceptable. That shape forces quality to be expressed as rejection, and
rejection can only ever encode floods that already happened.
engram_salient_term() scores EVERY candidate token and returns the argmax of
idf · position · casing (YAKE, Campos et al. 2020, with real corpus IDF
substituted for YAKE's corpus-free proxies), falling back from a sentinel
label to the node's content. Term quality becomes the selection criterion
instead of a veto: a bad token loses to a better token in the same text
without needing to be on any list. Tabu is applied during the argmax, so
inhibition-of-return costs seed quality rather than costing the whole scan.
Two defects found by instrumenting rather than assuming, which is the lesson
this codebase keeps relearning:
- The first live run returned five ALL-CAPS terms in a row. Memory content
conventionally opens with an all-caps header, so YAKE's acronym bonus was
handing the seed to whatever word the heading started with. Restricted to
tokens <= 5 chars, where all-caps is evidence of an acronym rather than
evidence of a heading. Long headers now compete on specificity.
- df via istr_contains is substring matching, so "them" hit inside "theme"
and function words came back with nonzero df. Added word-boundary df
locally; engram_label_df keeps substring semantics for its callers.
An earlier draft claimed the min_df floor subsumed the 73 stopwords that
08-03 measured label-df as missing. Re-measured: about:2, whole:1, them:2 —
they clear a floor of 1. The claim was false and the comment now records the
correction. The floor buys lexical reachability; the argmax buys quality; the
stopword list still earns its keep.
Measured on 60 live Memory nodes before shipping: 0 empty, versus 60 of 60
under the old extractor. Terms are topical — HEBBIAN, CONSOLIDATION,
TEMPORAL, crash-loop, PRIMING, NEIGHBORHOOD, DRIFT. Three of sixty are weak
header words; left alone deliberately, because listing them is the move that
produced four blocklists.
ENGRAM_ST_DEBUG=1 dumps the scored candidate set. It exists because there was
no way to see whether the all-caps run was the corpus or the casing weight
without guessing.
Resolve 3 conflicts:
- lang/el-compiler/runtime/el_runtime.c: keep deletion (deprecated runtime fork;
single-source-of-truth is lang/runtime/, enforced by scripts/check-single-runtime.sh).
- lang/releases/v1.0.0-20260501/el_runtime.h: keep deletion (releases/ is a generated
artifact folder, not a source path; a release is a git tag, not a folder).
- lang/runtime/el_platform_win.h: union of dev's Windows port (#80: setsockopt optval
wrapper + curl-less libcurl stubs) and our fsync(->_commit) shim needed by engram_store WAL.
Nothing in dev's build consumes the deprecated fork or releases/ folder.
The live engram now runs the paged store (neuron.egm+WAL), but the SDK
release publishes only el_runtime.{c,h} and the engram build links only
el_runtime.c — so a future official release would silently revert to the
in-memory store. Publish engram_store.{c,h} as SDK release assets and add
them to the engram build's download + cc link so the store transition
cannot regress.
- Wire paged store into the ENGRAM SERVICE (server.el, the authoritative durable
owner): boot->engram_store_boot, persist_canonical->engram_store_checkpoint,
gated by ENGRAM_STORE.
- elc (lang/elc.c + src/parser.el + codegen.el + elc-combined.el): OOB guard in
tok_kind/tok_value + parse_block progress backstop — fixes the pre-existing
unbounded-memory fold hang on sessions.el.
- engram_prune_telemetry mirrors ISE prune to the store (store_forget) so store
live-count tracks resident and stale telemetry stays bounded.
- Deployed live 2026-08-12: engram :8742 on neuron.egm+WAL, count reconciled 11552.
Flag-on checkpoint now full-walks the resident graph: store_put_node (WM weight,
activation_count, last_activated, wm_anchor) + store_put_edge (hebb, last_fired)
for every node/edge, then engram_checkpoint. Uses store_put_edge (idempotent
upsert) not store_hebb_batch, because activation FORMS new hebbian-associate edges
that bypass the create hook and delta-only hebb_batch can't create them. Store-on
boot now applies the same WM-halving + floor + cap transforms as engram_load.
This is the hebb-survives-restart fix. Gate: reboot from neuron.egm with
snapshot.json deleted -> edge hebb + activation_count survive unchanged, WM weight
survives with identical boot transform; negative control proves persist is
load-bearing (hebb->0 without it). M1 33/33 + M2 36/36 + M3 parity PASS, ASan/UBSan
clean, flag-off untouched. Engine unchanged (boundary held).
Caller-side shim in el_runtime.c maps EngramNode/Edge <-> StoreNode/Edge; engine
keeps zero soul deps (libengram boundary, design §10). Flag off = today's JSON
path byte-for-byte (proven: no neuron.egm created, graph identical). Flag on =
engram_open (import snapshot.json once into neuron.egm, else WAL-replay) +
resident load; node/edge create + forget dual-write via guarded hooks. Files
renamed engram.store->neuron.egm, engram.wal->neuron.wal.
Gate: M3 parity PASS (graph on==off byte-exact modulo ordering; snapshot round-trip;
reboot-from-egm with snapshot.json deleted; activation set+sequence identical;
ASan/UBSan clean). M1 33/33 + M2 36/36 green post-rename.
Known gap (pre-flip): in-place hebb/WM/activation_count updates during activation
are not yet persisted to the store (create/connect/forget are). Must close before
live flip so learned edges survive restart.
Write-back no-steal buffer pool makes the fsync'd WAL load-bearing (M1 was
write-through). Logical WAL with record-granularity page-LSN redo idempotency.
Checkpoint = flush dirty pages, fsync store, advance last_checkpoint_lsn,
reclaim WAL prefix. One-time snapshot.json import only when store absent;
JSON never read as the ongoing store thereafter.
Gates: 33/33 M1 (no regression) + 36/36 M2 — replay parity, torn-tail fuzz
(every byte offset), checkpoint-crash at all 5 phases, torn-page+WAL redo,
legacy-import parity, hebb-survives-crash.
Establish lang/runtime/ as the ONE canonical el runtime (from the active
runtime that carries hebb/emb persistence + the new WAL); repoint the el CI
publish, engram build, elb default, and in-repo build scripts to it; delete
the el-compiler/runtime + lang/releases/ forks; add scripts/check-single-runtime.sh
drift guard.
Fixes a live prod bug: the el CI published el-runtime-c/-h from the LAGGING
el-compiler fork (0 hebb refs), so the shipped soul never persisted Hebbian
edge weights — learned co-activation was wiped on every restart. Publishing
from canonical ships the stranded 'learning that cannot outlive the process'
fix.
WAL storage engine + integrity fixes (DELETE->tombstone + store-layer
protection, safe data-dir default) ride in behind ENGRAM_WAL (default off =
byte-identical to today). Verified: engram elb per-module build clean, WAL
gate 66/66, native smoke ok, drift-guard green.
Two write paths could put a node in the graph and neither could put a body
on an id that already exists. POST /api/nodes mints a fresh id via
engram_node_full; POST /api/load-merge honors a declared id but skips
anything already present. That is right for the additive case and leaves a
hole: a node resident with a truncated body cannot be repaired.
Forge's genesis seed sits in that hole. Two of Neuron's identity nodes
carry only their own label as content -- 30 and 22 bytes against 4263 and
2590 declared. Their ids are load-bearing (is_protected_node keys on them
and 214 declared edges reference them), so recreating them under a new id
is not a repair, it is a second break.
Engram has no in-place node update, so a replace is forget-then-merge, and
engram_forget also drops every incident edge -- 85 and 93 on those two
nodes, nearly all tag edges and accumulated hebbian associations the seed
does not declare and could not restore. preserve_edges (default true)
therefore snapshots before the forget and re-merges after: the replaced
node is back by then so it is skipped, and every dropped edge returns
through the (from_id,to_id,relation) dedup. The same re-merge is the
failure path -- if the seed merge does not produce the node, the backup
puts the original back. Rollback, not data loss.
With no replace list the route is exactly /api/load-merge.
Verified on a sandbox engram seeded to mirror the live graph's state for
this seed (15 resident nodes, 694 incident edges): 87 nodes created at
their declared ids, 2 replaced in place, 214/214 edges laid, 682/682
non-seed incident edges preserved, and a second run reports 0 added.
route_load was a stub response over the most destructive operation in the
server: engram_load resets the store before parsing, so a readable-but-
malformed snapshot left a hollow graph and the route answered {"ok":true}.
With 37GB of stale dated snapshots in the data dir as restore targets, that
is a live risk. Now returns the real return value plus node/edge counts and
an explicit hollow flag.
route_save discarded engram_save's return the same way; persist_canonical
returned a hardcoded 1, making 'let saved: Int = persist_canonical()' a dead
variable at six durable write paths.
A cgi block is a top-level declaration, so codegen_streaming classifies it via
is_top_level_decl and releases it. The identity emission then searched
toplevel_exec_stmts for that same block. Declarations are excluded from that list by
construction, so the search could never succeed. A probe printed what it actually
saw for a program whose first statement is a cgi block: [Let, Expr]. It emitted
nothing, silently, with no diagnostic on any channel.
The code documented its own assumption — 'Since cgi blocks are rare and small, they
end up in toplevel_exec_stmts' — and that assumption was false.
Capture the declared values before the release and emit from them. The search is
deleted rather than repaired, so the failure mode is removed rather than relocated.
Proven discriminating (old fails, new passes):
minimal cgi program, old -> 0 el_cgi_init
minimal cgi program, fixed -> el_cgi_init with all four declared values
neuron soul, fixed -> principal present in the compiled binary (0 before),
boots in 2s, interface 110 routes in / 110 out
Consequence: a binary now carries its declared identity as a compiled constant,
which is what the identity protocol requires. Whether the runtime surfaces it to
state_get("soul_principal") is unverified and separate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The loopback/API-key hardening carried in this file since 2026-07-15 called
el_http_request_authorized and el_http_send_401 from http_worker with no
forward declarations, so the calls were implicit and the later static
definitions conflicted. The file did not build. Two prototypes fix it.
Worth naming the pattern: uncommitted work is invisible to every check that
would have caught this. Three weeks of desktop security hardening was neither
committed nor compiling, and nothing reported either fact.
Same defect as the release runtime: \uXXXX was skipped and a literal '?'
emitted, destroying every non-ASCII character in JSON entering the runtime.
Two copies of one parser bug is how this class of fault survives a fix, so
it lands in both.
NOTE: this file also carries pre-existing uncommitted work from 2026-07-15/16
that this commit preserves rather than authors - loopback bind hardening
(EL_HTTP_BIND_HOST) and per-install API-key auth (EL_HTTP_AUTH_KEY) for the
shipped desktop build, plus goal-bias and node-json changes. It had been
sitting in the working tree for three weeks. Committing it because
uncommitted work is work that does not survive, which is the same durability
lesson as yesterday's Hebbian write-back finding. It needs review on its own
terms - see the backlog item for reconciling the two runtime copies.
jp_parse_string_raw handled \uXXXX by skipping the four hex digits and
emitting a literal '?'. JSON writers escape non-ASCII by default (Python's
json.dumps ships ensure_ascii=True; MCP clients do the same), so every em
dash, curly quote, accented letter and emoji arriving over MCP or HTTP was
silently replaced by one question mark on the way in.
Measured on the live store: 3,119 of 4,081 non-telemetry nodes carried the
damage, including the self traversal root and all 13 values nodes. Contents
split cleanly into fully-clean or fully-mangled with zero overlap, which is
the tell that it was one write path rather than gradual rot. No snapshot on
disk predates it, and 3 bytes collapsing to 1 is not invertible, so the
existing damage is permanent; only the forward path could be fixed.
Decode properly instead: 4 hex digits, surrogate-pair reassembly for astral
codepoints, U+FFFD for lone surrogates, UTF-8 encode. Malformed escapes keep
the old '?' so a truncated body still parses.
The deeper failure was that nothing measured this for two months. Every gauge
in the system reports whether the machinery is running; none reported whether
the text it carries is intact. Adds both halves: engram_text_health_json() /
GET /api/text-health for the daily census, and a txt_damaged counter on the
heartbeat for live regression. Verified in both directions - clean UTF-8 does
not trip it, a deliberately damaged node does.
Yesterday's eligibility-trace fix made Hebbian consolidation numerically real:
hebb_max 0.000799 -> 0.4725, and 1,198 hebbian-associate edges formed in 23h48m.
This morning's census found where they went: nowhere.
soul daemon (in-process graph): 42,426 edges, 1,198 hebbian
engram server (:8742, durable): 41,213 edges, 49 hebbian
Two processes, two graphs, one direction of travel. The soul pulls from the
server every 10 min (GET /api/sync) and never pushes. It cannot fall back on
saving its own copy either: soul.el sets soul_snapshot_path only inside
`if is_genesis && safe_to_seed`, and safe_to_seed is unconditionally false
whenever ENGRAM_URL is set -- because the server owns persistence and a soul
writing snapshot.json would clobber it. That guard is correct. The consequence
was not: mem_save() has never once executed. The soul is the ONLY process
running idle cognition, so it is where essentially all co-activation happens --
and it was throwing away every association it learned, every restart, silently.
The mechanism worked and the learning still evaporated.
Consolidation is now a message, not a file. Fast volatile store hands each
newly-formed association to the slow durable store over the API the server
already exposes; only edges past ENGRAM_HEBB_LINK_MIN are ever queued, so what
crosses the process boundary already earned it.
- el_runtime.c: 512-slot overwrite-oldest write-back ring; enqueue at edge
formation; engram_hebb_drain_json() pops a postable JSON batch. Drops and
drains are counted, not silent -- a consolidation path that quietly discards
is the exact failure this entry exists to correct.
- server.el: POST /api/edges/batch. persist_canonical() writes the full 60MB
snapshot per call, and route_create_edge calls it per edge -- correct for one
interactive edge, ruinous for bulk (~840MB/beat to persist 14 associations).
Batch connects all, snapshots once. Same durability, 1/N the writes.
- act-stats: hebb_wb_pending / _drained / _dropped. pending climbing with
drained flat = drain not called; drained climbing with sent 0 = POST refused.
Both failure modes are now visible in the stream instead of in an autopsy.
Verified live: batch route accepts valid entries, skips malformed ones without
aborting the batch, and enforces _auth. All 1,256 learned associations are now
in the canonical store; the soul booted at 42,431 edges with hebb_max 0.4941
carried across the restart for the first time.
Hebbian consolidation was inert. Census over the live graph (41,213 edges,
13,091 nodes, 23h44m uptime): strongest association hebb=0.000799 against a
0.15 consolidation threshold, and zero hebbian-associate edges ever formed.
Since the awareness loop calls engram_connect nowhere, this was the only path
by which the graph could grow its own structure — every edge was authored or
imported, none learned.
The defect was the event, not the rate. hebb is an EWMA whose fixed point is
P(event); raising ETA changes convergence speed, never the plateau. The event
was "both endpoints in WM in the same activate call" — demanded exact
simultaneity from a working memory that inhibition-of-return, breakthrough
rotation and the 24-slot global cap are all engineered to keep turning over
(~142 evictions/60s). The three mechanisms that make WM healthy are the ones
that made this measurement empty.
Replaced with three-factor eligibility traces (Sutton & Barto ch.7; Gerstner
et al. 2018; PLOS Comp Biol 2018 differential Hebbian learning): a node
entering WM sets a trace to 1.0, the trace decays exponentially in wall-clock
time (TC=300s, chosen against the measured ~31s scan cadence), and the
increment becomes ETA·trace(a)·trace(b). Strict generalization — co-resident
pairs read 1.0 on both ends and get exactly ETA, bit-identical to before.
warm×warm is deliberately not paired: eligibility must gate on something
happening now. Homeostatic ENGRAM_HEBB_NODE_BUDGET still bounds per-node mass.
Measured over a 60-call soak: hebb_max 0.0008 -> 0.0060, climbing at ~0.87
ETA/call against an all-time ceiling of 0.0008 before. hebb_mass 0.011 ->
0.019, no runaway. Projected consolidation of a genuinely recurring pair:
~1,730 calls, ~14h at autonomous cadence. links still 0 — that is expected
and is what tomorrow's review must check.
Also: Pass 3½ deduplicates this call's WM candidates, but the persisted WM
population is a union of fresh promotions and carry-over residents, and Pass
3½ never sees the second set. Confirmed live: two byte-identical copies of one
3,193-char document both holding slots (0.289 / 0.271). Added global
redundancy suppression in Pass 5 before the cap count. Post-fix census: 24
residents, 24 distinct contents, 0 wasted slots.
New gauges: hebb_warm (eligible-but-not-co-resident population), dup_wm_global.
Census of the live graph under the uniform 168h half-life with floor 0.05: the
MEDIAN tdecay for every single node type was 0.0500 — the clamp. Memory 81% at
floor, Knowledge 58%, BacklogItem 91%, Project 98%, Tag 100%. A function whose
median output is its floor is not a signal, it is a constant with exceptions,
and the exceptions were whatever had been touched in the last few days.
What that cost: 10 of the 13 grounded value nodes — Precision Over Brute Force,
Honesty Before Comfort, The System Must Accumulate — sat at 0.05, a 20x
activation penalty, while Knowledge ingested overnight sat near 1.0 and held the
working-memory top slots. Since tdecay multiplies at every hop, a 2-hop path
through settled knowledge compounded to 0.0025: those regions were not
disfavoured, they were unreachable. The decay function was erasing the
accumulated library in favour of whatever arrived last night.
External corroboration — arXiv:2604.26970 measures retrieval under decay
regimes: no temporal weighting NDCG@5 0.274, uniform exponential decay 0.015.
Uniform decay is 18x WORSE than no decay, because it penalises stable knowledge
while failing to suppress stale volatile facts. Not even their full adaptive
hierarchy (0.260) beat switching decay off.
Half-life is now scaled by how established a node is:
T_eff = T_HALF * (1 + ln(1 + activation_count))
The spacing effect and the Lindy property in one line — monotone, log-bounded
(a 10,000-activation node earns ~10x, never a permanent exemption), and built
on activation_count, which is measured, unlike tier, whose assignments are too
inconsistent to trust (the values node is tagged Episodic).
Floor 0.05 -> 0.25. Given no-decay outperforms uniform decay, the honest maximum
penalty for age alone is 4x, not 20x. Age should express a preference for the
recent; it must never make a region of the graph structurally unreachable.
Effect: well-established Knowledge median tdecay 0.773 vs rarely-activated
0.417 — the frequency signal now does work where the old function returned its
clamp for both. Values recover 0.05 -> 0.25 (the two frequently-touched ones to
0.79). Verified live: VBD whitepaper, component taxonomy and CGI now activate on
a values query. Per-node temporal_decay_rate override untouched.
Content-hash census of the live graph: 1,858 redundant copies, 44.9% of the
non-ISE store, all from a June id-scheme migration that re-added nodes under
fresh UUIDs instead of matching on content. Generation stopped in June; the
copies did not. Being byte-identical they carry identical embeddings, so they
score identically against any query.
Measured over 50 real query probes against the live 3,998-vector set:
40.2% of semantic seed slots were consumed by redundant copies of content
already in the seed set, 92% of retrievals affected, effective distinct seeds
4.78 of 8. Two fifths of every retrieval was spent re-reading the same page.
Deleting nodes is a separate operation with its own backup discipline. This
change makes the runtime immune to the condition instead: redundancy can never
buy a scarce slot, whatever state the graph is in. Enforced at both scarcity
points — semantic seed selection (a rejected copy does not consume one of the K
slots; the loop retries for the next distinct node) and WM admission via a new
Pass 3+1/2 ahead of the capacity cap, so 24 slots are contested by 24 distinct
meanings rather than by however many copies of one document exist.
Identity is exact content hash first, then cosine >= 0.995 for copies that
differ only in insignificant characters. At 768 dimensions that admits only
near-verbatim text: this suppresses redundancy, never similarity.
Live after restart: ~8.8 redundant seed candidates rejected per activation.
New dup_seeds/dup_wm gauges in act-stats.
WM continuity (the significant one). A node reached by the current query but
scoring under its type threshold was zeroed outright, while a node the query
did NOT reach got the full ACT-R carry-over treatment. Being found was punished
relative to not being found. Measured consequence: WM turned over 100% every
call — three activations of a byte-identical query gave |A∩B| = |B∩C| = 0 — and
wm_evicted stayed 0 the whole time because that path never counted. WM was not
a working set; it was six suppression-breakthrough nodes re-drawn per call.
Both exits from a WM slot now share one extracted retention rule.
Result: WM 6 -> 24 nodes (the designed Cowan capacity), top weight 0.097 ->
0.748 (natural promotion, not the breakthrough floor), and contents that are
actually query-relevant.
Hebbian learning. Edge weights were written once at engram_connect and never
changed; last_fired's only writer in 12.5k lines was an unrelated dharma path.
Every learning mechanism operated on nodes — the wiring between them was
frozen. Adds co-activation potentiation (HeLa-Mem arXiv:2604.16839) in a
separate `hebb` field so authored structure is never mutated, with homeostatic
per-node scaling the source lacks (PNAS 2422602122) to prevent hub saturation.
Measuring it produced the finding that mattered: zero edges existed between
co-active WM members, so reweighting existing edges was a no-op. This graph's
41k edges were all authored by explicit tool calls — nothing had ever formed an
association from experience. So Hebb literally: if the wire is absent, grow it.
Consolidation is gated hard (sustained EWMA past 0.15, <=2/call, 5% ceiling,
in-memory candidates discarded on restart) because it permanently mutates the
graph.
Two bugs caught only by instrumenting rather than assuming: the snap-to-zero
floor sat above the per-step increment, so nothing could ever accumulate; and
the reached-but-sub-threshold eviction above. Verified live end to end — 53
links formed under load, then discarded with the test snapshot.
Also exposes engram_act_stats_json over GET /api/act-stats. It had existed
since 2026-07-27 but was reachable only through the soul daemon, so diagnosing
the activation layer required a working soul. This review needed it and could
not get at it.
The soul's curiosity auto-term extractor takes the first word of a top-WM
node label. It has no term-quality scoring, so three prior self-reviews each
bolted on another hand-curated blocklist (genre words 07-23, quoted titles
07-25, stopwords 07-30). Every one was written reactively, after a flood was
already observed. A list can only contain floods that already happened.
Two were in flight and unfixed when this review ran:
"<!--" label df 220 -> 252 nodes activated
"SELF" label df 175 -> 541 nodes activated (list has "Self" Title-case;
str_eq is case-sensitive, so the uppercase token sailed through)
engram_label_df(term) counts nodes whose label contains term. Low-specificity
tokens are corpus-frequent by definition, so this catches the flood class
prospectively and tracks the corpus as the world-ingestor changes it. This is
Sparck Jones (1972), which introduced IDF under the name 'term specificity';
automatic stopword compilation from it is the textbook application.
NOT a replacement for the stopword list -- verified against all 86 listed
terms, not assumed. Catches 13 (Will:306, Self:175, Over:116, Knowledge:112),
misses 73 (Whose:0, Would:0, Could:0, This:9). Labels are terse titles, so
English function words are genuinely rare in them. The gates cover disjoint
failure modes; both are required.
Policy lives in awareness.el, not here: the runtime measures, the soul decides.
Working memory was thrashing behind a healthy-looking gauge. wm_active sat
at 22-24 while breakthroughs ran 661-903 and evictions 485-717 PER 60s tick
- roughly 825-1125 nodes cycling in 5-call lockstep.
Root cause: the breakthrough path was an anti-starvation mechanism that reset
its own counter on firing, with no budget and no refractory. A node failing
its type threshold 5 times was force-promoted at exactly 0.10 and had its
suppression_count reset to 0, so it immediately restarted the identical
climb. Since BREAKTHROUGH_WEIGHT (0.10) > WM_FLOOR (0.05), every one of them
cleared the admission floor and entered the rank contest tied at 0.10, where
the tie-break degenerated to node-array index order. Cap-evicted nodes are
skipped by retrieval reinforcement, so they never got an access_ts record and
the STI inhibition-of-return damper never applied to them. That closed the
loop: re-suppressed, completely unmarked, forever.
An anti-starvation rule that resets its own counter without a bound is not a
fairness valve, it is an oscillator.
Fixes in engram_activate Pass 2:
- ENGRAM_BREAKTHROUGH_BUDGET (WM_CAP/4 = 6) caps intrusive thoughts per call.
- ENGRAM_BREAKTHROUGH_COOLDOWN (55) via NEGATIVE suppression_count. The field
already serializes as %d and parses through eg_get_int_field, so negatives
round-trip through snapshots with no struct or format change.
- Blocked breakthroughs no longer reset the counter; it saturates so a starved
node surfaces on a later call instead of restarting from zero.
- Graded breakthrough weight by nearness to own threshold, so the rank
tie-break is cognitive rather than insertion order. Invariant preserved:
WM_FLOOR < weight < min(type_threshold).
Also: moved the additive cosine term AFTER the STI multiplier. It was applied
before, so an incumbent re-reached 30s later took t_n/(t_n+120) = 0.2x, which
cut the semantic term's ceiling from 0.20 to 0.04 - below every per-type
threshold. Meaning-match was being punished for having been recently useful.
Inhibition-of-return should rotate the structural score, not the semantic one.
Also: _eg_act_wm_evicted counted 3 of 5 eviction paths. The two carry-over
paths were silent, so the reported rate was an undercount of unknown
magnitude - while being used to diagnose an eviction pathology. All five now
increment.
Also: route_sync returned {"nodes":[],"edges":[]} when the snapshot export
failed. The soul's sync_ok check only tests for "" and "{}", so that
placeholder passed as a healthy sync: last_sync_ok_ts stamped, sync_age_ms
green, sync_empty never fired, added:0 forever. A broken sync was
indistinguishable from a quiet healthy one - the exact class this route was
added to fix. Returns a real error now.
Verified live (boot 20 vs boot 19): breakthroughs 661-903 -> 36/tick,
evictions 485-717 -> 12-46/tick against a counter that now covers more paths,
wm_active unchanged at 22-24, wm_avg_weight 0.138-0.273 -> 0.186-0.446.
Working memory is holding strong nodes instead of breakthrough-floor filler.
- route_create_node passed already-boxed Floats through el_from_float a
second time, reinterpreting boxed bits as raw doubles — every HTTP-created
node silently stored default salience/importance/confidence regardless of
input (verified live: 0.9/0.25/0.6 in -> 0.5/0.5/1.0 stored). Floats now
passed bare, matching the route_emit_ise pattern that always worked.
- Presence-aware defaults via json_get_raw: absent key != explicit value;
confidence now honored from payload instead of hardcoded 1.0.
- GET /api/similarity?a=&b= wires engram_cosine_sim (built 2026-07-24,
zero callers until now) into the introspection API.
- /health reports live node/edge counts instead of a hardcoded literal.
Every node object on consumer read routes (/api/nodes, /api/search,
activation results, neighbors, compiled context) carried the full ~5.7KB
emb vector — responses 10-50x oversized, blowing MCP token limits.
engram_emit_node_json now takes include_emb; only engram_save passes 1,
so persistence and the /api/sync//api/edges replication paths (which
serve engram_save output) keep embeddings intact.
_eg_act_wm_evicted/_eg_act_breakthroughs were reset at the top of every
engram_activate, so act_stats reported only the last call and the 60s
heartbeat missed nearly all events (curiosity runs 2 activates per 30s).
Both are now monotonic process-lifetime totals; consumers diff readings.
Working memory was pinned saturated (24/24, wm_saturated:1 on every
heartbeat) because every cap path only trimmed the population down TO
the cap — rank-based eviction guarantees a full WM whenever >=24 nodes
hold any weight, so sub-cap fill was unreachable and the saturation
flag carried no information.
- ENGRAM_WM_FLOOR 0.05: absolute admission bar (Soar WM forgetting,
Derbinsky & Laird ICCM 2012 — removal by absolute threshold, not
rank) applied in Pass 4, carry-over, Pass 5, and load-cap. Fill can
now drain below 24 during quiet periods.
- Zero wm_anchor at every eviction site: stale anchors on evicted
nodes were a latent resurrection bug.
- Context centroid folds only NEW WM entrants: incumbents re-promoted
every scan no longer re-entrench the centroid each call, breaking
the WM->centroid->e_eff->re-selection positive feedback (fixation
driver behind the wm_top0_streak=1407 incident).
Verified live: wm_active 3->22->23, wm_saturated:0 post-restart.
bigmerge
will.anderson