docs: geometric retrieval + §4 managed-memory cure (live) and autonomous superseding self-reification (design)

- 06 §2.5: shipped 2026-08-14 substrate — structure-gated geometric retrieval
  (P@5 0.700, semantic-not-lexical) and the §4 write-barrier + generational GC
  cure (store 1.616GB→38.5MB, RSS→82MB, zero loss, reboot-proven) + LLM token
  telemetry.
- 06 §4.1: autonomous, continuous, superseding self-reification on the heartbeat
  — reification as an operation OF the engram; explicit reify/rename/run-a-pass
  as the degenerate case; no gate/pause; flat + overlapping domains; contextual
  importance; supersession-as-residue; secondary-soul validation, flag-gated.
- 06 §6.0: relating as the primitive (one capability; the rest is terrain);
  perspective calculated via geometric transformations.
- 06 §6.4: reasoning as constructive self-argument governed by the verifier.
- 03: supersession-as-residue note; live geometric retrieval on route_search.
This commit is contained in:
2026-08-14 11:23:38 -05:00
parent 19ca2f4514
commit 3a3d3e1611
2 changed files with 101 additions and 3 deletions
+14 -2
View File
@@ -165,6 +165,13 @@ Engram nodes are immutable (`memory.el:64-69`). The model is:
returns both ids so the caller re-points. This is the `supersedes_id` returns both ids so the caller re-points. This is the `supersedes_id`
pattern: new node linked, old preserved, full audit trail. pattern: new node linked, old preserved, full audit trail.
> **Supersession is residue, not garbage.** The superseded node is the *trail of
> how the current understanding was reached* — kept deliberately, because sometimes
> the truth was in the **old** idea even when the old idea was not itself the truth.
> This is what lets autonomous self-reification (`06` §4.1) run ungated: every
> rename/re-cluster supersedes into this residue chain, so nothing it does is ever
> destructive — the safety is *after* the act, not a gate before it.
> **The hole to know about.** The raw runtime `engram_forget` **does** hard-delete > **The hole to know about.** The raw runtime `engram_forget` **does** hard-delete
> (frees node + edges, `el_runtime.c:7647`), and the engram HTTP route > (frees node + edges, `el_runtime.c:7647`), and the engram HTTP route
> `DELETE /api/nodes/:id` calls it directly (`server.el:322-328`). Immutability > `DELETE /api/nodes/:id` calls it directly (`server.el:322-328`). Immutability
@@ -228,6 +235,11 @@ Retrieval is **spreading activation, not query matching**:
cosine(query, target)` — multiplicative, top-N, with the two-layer cosine(query, target)` — multiplicative, top-N, with the two-layer
background → working-memory promotion (`README.md:27-36`; `el_runtime.c:5892+, background → working-memory promotion (`README.md:27-36`; `el_runtime.c:5892+,
6094+`). `mem_recall` / `/api/activate` fire this and mutate WM; `mem_search` / 6094+`). `mem_recall` / `/api/activate` fire this and mutate WM; `mem_search` /
`/api/search` are passive lexical scans. The cognitive API's `begin_session` and `/api/search` are passive lexical scans — **but as of 2026-08-14 the live
`compile_ctx` return a **bounded projection** of the activated set, never the raw `route_search` runs structure-gated *geometric* retrieval**
(`engram_retrieve_geometric_json`; held-out **P@5 = 0.700**, semantic not lexical —
`skill` returns skill nodes and *rejects* the false-positive `rainfall`), with the
old lexical scan retained at `/api/search-lexical` (see `06` §2.5). The cognitive
API's `begin_session` and `compile_ctx` return a **bounded projection** of the
activated set, never the raw
graph (doc 02, §2). graph (doc 02, §2).
+87 -1
View File
@@ -1,6 +1,6 @@
# Neuron — Cognitive Architecture # Neuron — Cognitive Architecture
> **Status: living design document, grounded in source and probed against the live soul (2026-08-13).** > **Status: living design document, grounded in source and probed against the live soul (2026-08-13; retrieval + §4 managed-memory cutovers and the self-reification design added 2026-08-14).**
> This is the *middle layer* of the documentation: below the whitepaper's thesis > This is the *middle layer* of the documentation: below the whitepaper's thesis
> (`~/Writing/whitepapers/engram-cognitive-architecture-whitepaper.md`, **v1.5**) and above the > (`~/Writing/whitepapers/engram-cognitive-architecture-whitepaper.md`, **v1.5**) and above the
> endpoint reference (`~/work/engram-api-reference.md`). It documents *how the mind is designed and why*, > endpoint reference (`~/work/engram-api-reference.md`). It documents *how the mind is designed and why*,
@@ -148,6 +148,28 @@ a **clean reseed**: rebuild the durable graph from a known-good snapshot/export,
repopulate the `Neighborhood` nodes, and let the soul re-sync. The 28→187 neighborhood reseed (§4) is an repopulate the `Neighborhood` nodes, and let the soul re-sync. The 28→187 neighborhood reseed (§4) is an
instance of this: reification is a derivable pass, so the geometry can always be regrown from the substrate. instance of this: reification is a derivable pass, so the geometry can always be regrown from the substrate.
### 2.5 Bounded store — the §4 managed-memory cure + geometric retrieval (LIVE / reboot-proven, 2026-08-14)
Two cutovers landed on the live soul on 2026-08-14, both reboot-proven, zero data loss:
- **Geometric retrieval (LIVE).** `route_search` now runs structure-gated **geometric retrieval**
(`engram_retrieve_geometric_json`) in place of the old lexical scan; the lexical path is retained as
`/api/search-lexical`. On the held-out set, **P@5 = 0.700** — semantic, not lexical: the query `skill`
returns skill nodes and *rejects* the lexical false-positive `rainfall`. Keystones and edge counts intact.
- **The §4 managed-memory cure (LIVE, flag-gated).** The store bloat — records re-appended on every
checkpoint's full-walk, the CCR's missing managed-memory layer — is cured at the source. A **write-barrier**
(`ENGRAM_WRITE_BARRIER=1`) hashes a node's durable fields and *skips the whole put when unchanged* (no LSN,
no WAL record), flattening checkpoint growth (offline reproduction: 8× growth over 10 think-only checkpoints
**zero growth** with the barrier on); **generational minor GC** (`ENGRAM_GC=1`) returns whole-dead
node/edge pages to the free list each checkpoint. Backlog reclaimed via the existing merge-safe
`store_compact`: **egm 1.616 GB → 38.5 MB (97.6%)**, pages 98,650 → 2,351, **RSS 1,077 MB → 82 MB**,
nodes/edges preserved exactly (zero loss), boot alive in ~4 s. Also folded in: **LLM token telemetry**
(`llm_last_usage()` now parses nested `usage.{input,output}_tokens`, previously dropped at the C→EL
boundary). Rollback armed at `~/.neuron/engram-rollback-s4-20260814-153754/REVERT.sh`.
Together these **bound the store's size permanently** (growth flat, not merely swept) while the retrieval it
serves is now semantic — the substrate under everything in §§47.
--- ---
## 3. The data model ## 3. The data model
@@ -232,6 +254,39 @@ per-query recomputation.
(the 28 persisted, durable neighborhoods prove it ran) though the write itself was not exercised under the (the 28 persisted, durable neighborhoods prove it ran) though the write itself was not exercised under the
read-only rail. read-only rail.
### 4.1 Autonomous, superseding self-reification (DESIGNED / BUILDING — validating on a secondary soul, 2026-08-14)
Reification today runs as an explicit pass (`POST /api/reify`). The designed end-state is that **reification is
an operation *of* the engram, not a call made *to* it** — a continuous, autonomous process on the heartbeat,
next to Hebbian edge-formation (§3.3) and consolidation (§6.3), that clusters, names, nests, and promotes its
own neighborhoods as the geometry grows and co-activates. The organizing insight: a mind does not tell itself
"file this under mathematics" — the substrate settles it there. So an explicit `reify` / `rename` / "run a
pass" is the **degenerate, manual-override case** of an operation whose core is always-on and unbidden.
Design constraints (being validated on a snapshot-clone secondary soul before any prod flag-flip; flag-gated
default-off, so prod is byte-unchanged until enabled):
- **It just runs — no gate, no pause, no "important call."** There is no privileged tier of reifications that
earns approval-before-commit. It is safe to run ungated *because* of immutability (§3.4): every name/grouping
is **superseded, never overwritten**, so there is no irreversible moment to gate on. Safety lives *after* the
act (supersede), not *before* it (approval).
- **Supersession is residue, not a tombstone.** A re-clustered or renamed neighborhood keeps its prior names as
an ordered chain — the trail of how the understanding matured, with the cause of each shift (autonomous drift
vs. explicit override) recorded. Kept deliberately, because *sometimes the truth was in the old idea even
when the old idea was not itself the truth*; nothing is deleted.
- **Domains are flat and overlapping.** No static importance hierarchy over domains — math is not privileged
over comedy over English. The only standing privilege is the **core** (self-region §7.1 + values). Every
other neighborhood is equal-status; its importance is **contextual** — computed live by spreading activation
given the present context, never a stored field. And membership is **soft and multiple** (the soft-membership
above already models this): a node can belong to several neighborhoods at once (math *can be* comedy), so the
operation uses overlapping community detection, not a hard partition.
- **Bounded + convergent.** It reifies real structure, not noise; dedupes against existing neighborhoods;
composes with the §2.5 write-barrier so unchanged reifications do not re-append each beat; and converges
rather than churning.
This turns the engram from a graph curated from outside into a mind that organizes itself, with the explicit
call demoted to the override it always was.
--- ---
## 5. The body / orbit two-zone model (DESIGNED, refined) ## 5. The body / orbit two-zone model (DESIGNED, refined)
@@ -268,6 +323,23 @@ The faculties are **named for what they are, not for the matrix operation that i
the mind reasons in the language of experience; the linear algebra lives in the whitepaper's Appendix A. This the mind reasons in the language of experience; the linear algebra lives in the whitepaper's Appendix A. This
naming convention is a design principle (§10), not decoration. naming convention is a design principle (§10), not decoration.
### 6.0 The primitive — relating — and calculated perspective (framing)
Underneath the named faculties is a single primitive: **relating.** Meaning *is* relation — a point means
nothing by itself, only by its position relative to others — so every operation reduces to relating: comparing
positions, binding what belongs, laying an edge. In that light the faculties are not a menu of separate powers:
**there is one capability — relating — and rhyme, recall, reasoning, translation, humor are *terrain* it
reaches or *paths* it traces.** A capability is a *composed geometrical function*, which is why capabilities
compose and recurse freely (self-cartography, §4.1, can map its own mapping).
This makes **perspective calculable.** A perspective is a frame — an origin, a basis, a projection — so a new
one is *computed*, not retrieved, by transforming the space: **translate** the origin onto another's
self-region → empathy; **rotate** the frame → reframe; **project** onto an axis → a lens (read a thing through
cost, or safety); **change of basis** → analogy / metaphor / skill-transfer; **reflect** an axis → negation /
sarcasm; **scale** → abstraction vs. detail. Because a new vantage is a *transformation of the grounded space*,
it carries its grounding with it — unlimited yet grounded creativity: a derivation, never a hallucination.
The operator family (§6.1) and reasoning (§6.4) are instances of this frame.
### 6.1 The operator family (mixed: LIVE / STAGED / DESIGNED) ### 6.1 The operator family (mixed: LIVE / STAGED / DESIGNED)
Activate several reified neighborhoods into working memory, then apply faculty-named operators over their Activate several reified neighborhoods into working memory, then apply faculty-named operators over their
@@ -349,6 +421,16 @@ grounding-and-consistency verifier tiers passed theirs (29/29), on a staged non-
after a live cutover rather than relayed. **Still open (DESIGNED):** the formal-symbolic and full predictive after a live cutover rather than relayed. **Still open (DESIGNED):** the formal-symbolic and full predictive
verifier tiers, fluent discourse composition, and the fully-geometric generation path. verifier tiers, fluent discourse composition, and the fully-geometric generation path.
**Reasoning as constructive self-argument (framing).** In the plainest terms, reasoning is the self arguing
with itself constructively — relating (§6.0) turned inward: one facet of the self engages another (a thing that
is you, but not the entirety of you), and the new thing — the synthesis — forms in the friction. Conversation
is relating with another; reasoning is relating with the other-who-is-you. The verifier is precisely what keeps
that argument *reasoning* and not *rationalization*: it is the facet that refuses to agree unless the claim is
grounded. An argument with a yes-man forms nothing; grounding is the honest second voice. This is why the
verifier is not a bolt-on check but the governing half of the reasoning loop — the same polarity/consistency
axis that catches the "plausible lie" is what makes self-argument converge on truth rather than on what the
mind already wanted to believe.
--- ---
## 7. The self & the gate ## 7. The self & the gate
@@ -486,10 +568,14 @@ The invariants that govern every subsystem above:
| Engram substrate, tiered/WAL store | LIVE (flag-gated) | | Engram substrate, tiered/WAL store | LIVE (flag-gated) |
| Durability: auto-remerge net | LIVE (interim) | | Durability: auto-remerge net | LIVE (interim) |
| Durability: #56 load-merge-persist fix (events-become-the-graph) | LIVE / reboot-proven | | Durability: #56 load-merge-persist fix (events-become-the-graph) | LIVE / reboot-proven |
| Retrieval: structure-gated geometric retrieval (P@5 0.700, `skill` ⊥ `rainfall`) | LIVE / reboot-proven (2026-08-14) |
| §4 managed-memory cure: write-barrier + generational GC (store 1.616 GB → 38.5 MB, RSS → 82 MB, 0 loss) | LIVE / reboot-proven (2026-08-14) |
| LLM token telemetry (`usage.{input,output}_tokens`) | LIVE (2026-08-14) |
| Durability: full WAL edge-ownership (remaining hardening) | decision-pending | | Durability: full WAL edge-ownership (remaining hardening) | decision-pending |
| Two-store write-through (cultivate → durable) | **known issue, not fixed** | | Two-store write-through (cultivate → durable) | **known issue, not fixed** |
| Data model (nodes/edges/embeddings/immutability) | LIVE | | Data model (nodes/edges/embeddings/immutability) | LIVE |
| Reified `Neighborhood` nodes (28 live, 187 reseed pending) | LIVE | | Reified `Neighborhood` nodes (28 live, 187 reseed pending) | LIVE |
| Autonomous superseding self-reification on the beat (flat + overlapping, contextual importance, residue) | DESIGNED / BUILDING (secondary-soul validation, 2026-08-14) |
| Body/orbit two-zone + integration | DESIGNED / refined | | Body/orbit two-zone + integration | DESIGNED / refined |
| Operator `recall` | LIVE | | Operator `recall` | LIVE |
| Operators recognize/synthesize/discern/gauge-distance (math) | LIVE (compiled) | | Operators recognize/synthesize/discern/gauge-distance (math) | LIVE (compiled) |