promote stage -> main: transduce unification + HNSW + ggml adapter + reconciliation (2026-08-15) #120
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: neuron-technologies/el#120
Reference in New Issue
Block a user
Delete Branch "stage"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Final pipeline hop. Local build is the bar tonight, not CI — CI itself is known-unreliable pending sprawl cleanup.
A cgi block is a top-level declaration, so codegen_streaming classifies it via is_top_level_decl and releases it. The identity emission then searched toplevel_exec_stmts for that same block. Declarations are excluded from that list by construction, so the search could never succeed. A probe printed what it actually saw for a program whose first statement is a cgi block: [Let, Expr]. It emitted nothing, silently, with no diagnostic on any channel. The code documented its own assumption — 'Since cgi blocks are rare and small, they end up in toplevel_exec_stmts' — and that assumption was false. Capture the declared values before the release and emit from them. The search is deleted rather than repaired, so the failure mode is removed rather than relocated. Proven discriminating (old fails, new passes): minimal cgi program, old -> 0 el_cgi_init minimal cgi program, fixed -> el_cgi_init with all four declared values neuron soul, fixed -> principal present in the compiled binary (0 before), boots in 2s, interface 110 routes in / 110 out Consequence: a binary now carries its declared identity as a compiled constant, which is what the identity protocol requires. Whether the runtime surfaces it to state_get("soul_principal") is unverified and separate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>auto_term_empty_streak — the counter the 2026-08-06 review added to catch exactly this — read 50 and climbing. Fifty consecutive curiosity scans where the soul's dynamic seeding produced nothing and the loop fell back to four hardcoded phrases. The live WM top said why in one look: every slot was a Memory node labelled "memory:remembered". The extractor read the LABEL only, the sentinel guard correctly rejects sentinels, so there was never anything to extract. It was written against Knowledge nodes, which have real titles, and was structurally blind to the node type that dominates working memory. Rather than add a sixth guard to the five that accumulated across four reviews (genre words, quoted titles, stopwords, label-df), invert the algorithm. The old one was: take the first word, then check whether it is acceptable. That shape forces quality to be expressed as rejection, and rejection can only ever encode floods that already happened. engram_salient_term() scores EVERY candidate token and returns the argmax of idf · position · casing (YAKE, Campos et al. 2020, with real corpus IDF substituted for YAKE's corpus-free proxies), falling back from a sentinel label to the node's content. Term quality becomes the selection criterion instead of a veto: a bad token loses to a better token in the same text without needing to be on any list. Tabu is applied during the argmax, so inhibition-of-return costs seed quality rather than costing the whole scan. Two defects found by instrumenting rather than assuming, which is the lesson this codebase keeps relearning: - The first live run returned five ALL-CAPS terms in a row. Memory content conventionally opens with an all-caps header, so YAKE's acronym bonus was handing the seed to whatever word the heading started with. Restricted to tokens <= 5 chars, where all-caps is evidence of an acronym rather than evidence of a heading. Long headers now compete on specificity. - df via istr_contains is substring matching, so "them" hit inside "theme" and function words came back with nonzero df. Added word-boundary df locally; engram_label_df keeps substring semantics for its callers. An earlier draft claimed the min_df floor subsumed the 73 stopwords that 08-03 measured label-df as missing. Re-measured: about:2, whole:1, them:2 — they clear a floor of 1. The claim was false and the comment now records the correction. The floor buys lexical reachability; the argmax buys quality; the stopword list still earns its keep. Measured on 60 live Memory nodes before shipping: 0 empty, versus 60 of 60 under the old extractor. Terms are topical — HEBBIAN, CONSOLIDATION, TEMPORAL, crash-loop, PRIMING, NEIGHBORHOOD, DRIFT. Three of sixty are weak header words; left alone deliberately, because listing them is the move that produced four blocklists. ENGRAM_ST_DEBUG=1 dumps the scored candidate set. It exists because there was no way to see whether the all-caps run was the corpus or the casing weight without guessing.Ports dialogue.py + self_region.py to native el, bound to the IN-PROCESS engram el runtime (engram_activate_json / engram_neighbors_json / engram_search_json / engram_node_full / engram_connect — C-order builtins, not the wrapper order). self_region.el: pulls the engram's REAL Self/identity nodes (pooled single-term search), scores by self-signal, reads out identity from their own prose — no hardcoded anchors, no template. dialogue.el: ONE operation — project(query) -> land on a region -> read out. * identity = self-region proximity (no intent classifier, no separate branch) * memory = activation + a RELEVANCE FLOOR, then MATERIALIZE by walking the neighborhood (real edges), never top-props * HONEST ABSENCE when nothing is close — no 'I noted that' echo, no fabrication * NEGATION SACRED: readout is the stored prose verbatim, so polarity survives * DIRECTIVE OVERRIDE: a meta-directive switches the reply language Verified against a SCRATCH in-process engram (live :8742 untouched): dialogue gate 9/9 — identity from real self-content, neighborhood materialization, SACRED negation (self + memory), PT identity in PT, directive override to English, 'Prove it' -> honest absence. EN/Romance/prop/multilingual gates unregressed.Two changes to the activation path, both grounded in measurement on the live store rather than on the spec. 1. Rescale cosine before the query gate. The propagation gate (arXiv:2606.30133, added in an earlier review) fed RAW cosine into FLOOR + (1-FLOOR)*c. Raw cosine from nomic-embed is compressed into a narrow high band, so that expression is close to a constant. Measured, 400 random UNRELATED node pairs on the live store: median 0.562, central 98% span [0.381, 0.743] So a node with no semantic relation to the query was propagating at 0.25 + 0.75*0.562 = 0.67. Two thirds strength. The gate was a small tax. Fixed by shifting and flooring about ENGRAM_EMBED_S0 -- which is already in this file, already 0.45, and already used exactly this way by the Pass-2 WM term. The propagation gate simply never used it. Same 400 pairs after: median unrelated pair falls to 0.40, top of range preserved (0.85 vs 0.92), gate spread widens 0.42 -> 0.60. Only 8.5% reach the floor, so dissimilar lexical/structural pathways are damped, never severed. Range is unchanged at [0.25, 1.0], and cosq == NULL still degrades to no gating at all. 2. Decompose the WM eviction counter by cause. _eg_act_wm_evicted was incremented from six sites with four distinct causes and collapsed all of them into one integer. Today's review measured 175,547 evictions over 13.5h (~216/min against 24 slots) and could not tell healthy rotation from cap thrashing from duplicate churn. That is this file's most-repeated defect: dup_wm and dup_wm_global exist only because the aggregate could not answer "why" during the 08-02 and 08-06 incidents. Each of those needed a NEW gauge before it was diagnosable. evict_floor / evict_cap / evict_bll complete the decomposition, so wm_evicted == floor + cap + bll + dup_wm + dup_wm_global holds as an identity and each term implies a different correction. Verified on an isolated instance: 30 nodes, 24 filled the cap, wm_evicted 6 == evict_cap 6, all other terms 0. Built and smoke-tested out of tree. The live daemon runs a pinned binary and was deliberately not restarted -- the store compaction workstream is in flight.The buildable form of the "one operation" theory (memory bdc8a488). Maps the theory onto what is already compiled: the five reasoning operators in engram_reason.c already collapse onto ONE primitive — engram_reason_point_fit — plus the geo-algebra (combine/subtract/analogy-rotate/distance), and engram_verify.c is built on the same fit. So the operator-collapse is already half-written; what is missing is not the primitive. What is missing, and what this doc specifies: - think(anchor, prior) -> gradient (a distribution/direction, not a point); each named faculty = {point_fit + a prior}, the operation frozen, the prior learned. - Prior as a first-class stored node (warp + calibration), superseding the intrinsic importance/salience scalar with a relational, grounded-for-whom edge. Confirmed against the runtime: importance is already a live activation computation (el_runtime.c:13013), never trusted as a static field. - vantage_read(anchor, aperture) — one op, three settings: self / foreign-field / veil. - The reflexive correspondence-loop as the learning engine: move the grounding check from offline Python into the geometry, reflexive, reusing the DORMANT verifier (engram_verify_grounding has no runtime caller and no El binding today) turned inward. grounding = learning = one loop. - hold/ground/assert kept distinct: the engram holds anything, grounding is an edge, the honesty floor is on assertion only; ungrounded content is first-class. - metastability: keystone core (read-mostly priors) + plastic everything else. Seven staged milestones, earliest is a real end-to-end slice (induction as {primitive + grounded prior} with the loop closing on it, reboot-proven on a snapshot). Build rails stated: offline/secondary, snapshot-first, reboot-prove, zero-loss, gated launchctl cutover. Design only; no code changed this pass.Adds the missing middle of the ELP: a deterministic EN-content-lemma -> target-lemma bridge (translate.el) on top of comprehend.el (parse) and realizer.el (inflect). English-only engram geometry cannot route cross-lingually and vocabulary-XX.el carries no en_translation glosses, so the honest no-LLM bridge is a wired lexicon (poem coverage; OOV passes through). SACRED polarity/neg_word are carried untouched: 'never' localizes to a negator ('nunca'), never to a content lemma. Additive realizer extensions: agent_person/agent_number recognize Romance target pronouns; the non-EN negation branch surfaces a carried neg_word instead of the generic negator. Verified on the real toolchain (elc->cc->run): 'You never fought the ocean.' -> ES 'Tú nunca luchaste el océano.' 'I never saw the breaking.' -> ES 'Yo nunca vi la ruptura.' nunca holds 3/3 negation lines. Known gaps: PT verb conjugation fallback (lutarred), irregular EN lemma (broke->break), adjunct/subordinator passthrough.audio-surface.el / image-surface.el: own-core additive-synthesis WAV and raster-PNG renderers (integer-only DSP, since EL has no floats), rendered from learned engram signatures via a pluggable surface-profile abstraction (surface-profile.el). audio-demo.el / image-demo.el are drivers. NOTE: demo files hardcode absolute paths to this worktree's own directory — will need a path fixup before landing. elp/projector/ is a Python package the author's own README marks as "STAGING/PROOF-OF-SHAPE — not the deliverable", superseded by the native .el surface-profile work above; kept as a validated architecture proof. Generated output (elp/faculty/{out,sig}, elp/projector/out, __pycache__) intentionally excluded.speech.el: formant/glottal integer DSP synthesis + voice-analyze-by- imitation. voice-profile.el / voice-ingest.el: voice-profile plumbing. accent.el: British-RP as an ingested transform-geometry (explicitly marked provisional/citation-pending by its own comments). organ-read.el: engram read-through for the speech organ. Includes demo/test drivers and non-personal reference data (British-RP phonetics/lexicon derived data, a public-domain LibriVox RP reference recording). Deliberately excludes elp/data/live/ (raw recorded voice + face-photo samples of the repo owner) and the will-*.{json,psv} derived voiceprint files — personal biometric data that shouldn't be committed to a shared repo without an explicit decision from the owner. Also excludes this worktree's elp/src/surface-profile.el, which diverges from the copy in other worktrees (agent-aaf04b0a9714c4070, main) — needs manual reconciliation before landing, left out here to avoid silently picking a version.ingest and transduce are complements, not synonyms: ingest is the conscious, deliberate act of pointing at a source (ingest_file/dir/url/llm/stream stay named exactly that); transduce is the automatic, invisible mechanism inside it that converts extracted surface content into geometry (renamed build_prose/build_structured -> transduce_prose/transduce_structured, the functions that actually turn raw text into a node+edge manifold). Real bug found and fixed along the way: the final /api/load-merge response was never checked for an error. A total failure (bad auth, network down, anything) silently reported nodes_added:0/edges_added:0 — indistinguishable from a benign 'everything was already known' outcome. Verified live: with a wrong key, the tool now honestly returns {"error":"load-merge failed: unauthorized",...} instead of a misleading zero. Also dropped a CRUD-verb smell: the per-decision println said CREATE (a database-log verb for something that hasn't actually been written to the server yet — it's a local, tentative decision pending the batch merge). Renamed to FORM. The dead-code eg_create_node (defined, never called) renamed to eg_crystallize_node and annotated honestly as unused, since if it's ever wired up it represents the real server-confirmed write, unlike the local FORM guess. Not yet re-verified end-to-end against a real successful write: the ingest-test sandbox (nsbx up ingest-test) is itself currently broken — it prints a green "ready" banner after its own readiness check fails, and nothing is actually listening. Filed separately; not in scope here.