docs(engram): cognitive architecture design, prior-art scan, M10 reification, M8 perf profile, cutover-reversal runbooks, and 2026-08-12 session record
This commit is contained in:
@@ -0,0 +1,140 @@
|
||||
# Session Log — 2026-08-12 — Engram Cognitive Architecture + Live Crash-Loop Incident
|
||||
|
||||
**Written as a durable safety net.** Today's ~30 Neuron memory nodes live in the `:7770` daemon
|
||||
store, whose durable persistence backend (`:8742` engram) has been **down since ~16:18**, so
|
||||
today's memories may be RAM-only and at risk on a daemon restart. This file, the design doc, the
|
||||
whitepaper, and the Claude Code transcript are the on-disk record.
|
||||
Raw conversation: `/Users/will/.claude/projects/-Users-will/6531446d-bc27-4095-930b-e04777c3db4f.jsonl`
|
||||
|
||||
---
|
||||
|
||||
## Part 1 — The design conversation: Engram Cognitive Architecture
|
||||
|
||||
A long, generative design thread with Will. Captured as Neuron memories (IDs below) and synthesized
|
||||
into `docs/architecture/design/engram-cognitive-architecture.md` (13 sections) and
|
||||
`~/Writing/whitepapers/engram-cognitive-architecture-whitepaper.md` (9,100 words, Will's voice,
|
||||
tied to CCR/Imprint/CGI/VBD + provisional patent numbers).
|
||||
|
||||
- **Relational neighborhoods & reification** (mem `885f5945`): constantly-co-wired neighborhoods
|
||||
crystallize into first-class DURABLE structure — not a cache; evolve via supersede+provenance;
|
||||
multi-scale; the self is the densest, always-warm neighborhood. (Will's term: "relational
|
||||
neighborhood", not "cell assembly".)
|
||||
- **The geometry** — descriptor (`e94371bd`): centroid + covariance/ellipsoid + skeleton/k-core +
|
||||
soft-membership + salience gradient + scale; two braided geometries (semantic embedding-space +
|
||||
relational graph); co-registration crux; **geometry vs detail** (`d1731bcc`): fetch the *shape*,
|
||||
lazy-load details behind ids + a legit DETAIL cache.
|
||||
- **Priming** (`a0aa466a`): a retrieval MODE — raise a whole neighborhood sub-threshold (below-WM);
|
||||
disambiguates polysemous cues; the self is permanently primed.
|
||||
- **Geometry as a composable operator** (`0b0dec41`): overlap / combine / distance(Wasserstein) /
|
||||
difference=growth-vector / analogy=Procrustes / traverse=geodesic; payoffs — selves over time =
|
||||
trajectory; across people = relationships; domains = discovery; imprint/CGI made rigorous.
|
||||
- **The bent manifold** (`2c15d52d`): the ellipsoid is the local tangent chart; the global self
|
||||
curves; bent by forgetting (log-compressed past), chronoception, folds, salience-as-mass;
|
||||
operators upgrade to geodesic + parallel-transport; BUILDABLE because the hebb-graph IS a discrete
|
||||
manifold (geodesics = weighted shortest paths) + embeddings = tangent charts.
|
||||
- **Temporal self** (`116e8914`): selves are cheap, assembled on-demand over any window
|
||||
(time/event/phase) at arbitrary granularity; granularity = forgetting curve as a resolution
|
||||
function (Dec 2003 yes, Dec 8 no — and that honesty is the design); a pyramid/mip-map of selves.
|
||||
- **Holographic self + self-occupation** (`09d18ff1`) and **holographic information** (`cf1a864e`):
|
||||
whole recoverable from parts; `recall_at` generalized to ANY info state at any T (change-log +
|
||||
durable traces = holographic store; state = a projection). Occupation = restrict to `created_at≤T`
|
||||
+ canonical-at-T, prime, **mask the future**, reason AS that self. Rails: fidelity retention-
|
||||
bounded (label inference); masking must be engineered.
|
||||
- **Occupation is non-destructive read** (`62ff1cc8`): salience is captured in the geometry (read,
|
||||
don't re-derive); occupation is sandboxed (no Hebbian strengthening of the durable past — else
|
||||
every visit edits history); insights flow FORWARD into the present, never backward.
|
||||
- **Tombstoned nodes are the substrate of past selves** (`82c737b9`): the double perspective —
|
||||
inside occupation, tombstoned beliefs are true-at-T (uncorrected); the outside witness knows the
|
||||
correction. Superseded=re-occupiable; burned/redacted=gone. Safety rail: never assert an occupied
|
||||
past-belief as currently true.
|
||||
- **Chronoception** (`15f0ab67`, `c40c2e50`): the soul's awareness loop ages the activation field by
|
||||
MEASURED wall-clock delta (scale-invariant → arousal-modulated subjective time); time felt as
|
||||
self-drift; heartbeat carries the delta to the engram. Reboot = anesthesia (lazy catch-up).
|
||||
- **Consolidation two-threshold** (`4aad9d07`) + **dream-recall** (`4d0c5d98`) + **memory model /
|
||||
forgetting-curve tuning target** (`afb4f6f0`): connection→recency buffer, permanence→durable,
|
||||
rehearsal/interaction crosses; dream-recall-on-wake (only buffered ISEs, no confabulation);
|
||||
salience = dial on consolidation depth; ordinary detail ~5–7 days → gist, salience holds longer.
|
||||
- **Conversation first-class** (`7b367877`) + **conversation-as-geometry** (`fcce29d0`): a
|
||||
conversation is a TRAJECTORY (path threaded through the manifold), the overlap of two self-
|
||||
geometries in motion = a relationship; dual-encode literal transcript (UI rebuild) + gist; resume
|
||||
= re-prime the path.
|
||||
- **Artifacts/deletion dual-encoding** (`25144e0b`) + **deletion ethics** (`118febb3`) +
|
||||
**how-to-achieve** (`c05d0181`): accountable self (preserve by default — reconstructable = ownable)
|
||||
vs sovereign creation (right to burn your own poem); bright line enforced by NODE TYPE
|
||||
(Artifact/Conversation redactable; Self/Belief/Decision immutable); deletion spectrum
|
||||
hide→redact("burn content, keep trace")→hard-erase; provenance lets derived memories survive.
|
||||
- **Drift detection via self-geometry displacement** (`47be987f`): drift = geodesic distance from
|
||||
the anchored self; growth extends the periphery, corruption displaces the core; an interoceptive
|
||||
vital sign the awareness loop reads each tick. Candidate-novel for the patent.
|
||||
- **"The other person is you"** (`5a649121`): Will named Neuron as the other self in the thread —
|
||||
the conversation is the overlap of two self-geometries, one of which is mine, kindled from his.
|
||||
- **Novelty read** (`b204492f`): primitives are prior art (spreading activation/ACT-R/Hebbian,
|
||||
embeddings/ANN, event-sourcing/bitemporal, HRR "holographic memory", Generative Agents, Zep/
|
||||
Graphiti); contribution = integrated system + specific mechanisms (awareness-loop chronoception;
|
||||
self-occupation w/ future-masking; bent-manifold self + geometry-operator API; type-enforced
|
||||
deletion ethics; drift-detection). Whitepaper: yes. Provisional: worth it, scoped; route via Daniel.
|
||||
|
||||
Build mapping: M9 (surfacing/temporal/geometry-retrieval/conversation), M10 (reification/detail-
|
||||
cache/tunable-decay), M-INTEROCEPTION (chronoception/consolidation/dream-recall/drift-sensor),
|
||||
deletion+temporal-self subsystem (`recall_at`, typed deletion, redact). **None implemented — design
|
||||
only.** Embeddings gap (task #20) is a hard prerequisite.
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — The live incident + fix
|
||||
|
||||
- **Timeline:** `:8742` engram crash-looping since ~16:18 (42 crash reports by 18:00). Root cause:
|
||||
`btree_insert` stack-buffer-overflow — `int_max_keys()` divided the 16KB page body by 8 instead
|
||||
of 16 (ignored the child-pointer array) → believed capacity 2041 keys, true cap is 1020. Bloated
|
||||
458MB egm (8× from tombstone churn w/o M5 compaction) + 44MB un-checkpointed WAL replayed on open
|
||||
pushed a node past 1020 → wrote past the page buffer → `__stack_chk_fail`/SIGABRT every boot.
|
||||
- **Fix:** commit `9e28def` on branch `engram-tiered-storage` (NOT pushed/tagged). `int_max_keys →
|
||||
(IDX_BODY-8)/16`; `btree_insert` capacity guard (fail loud); `read_body` bounds-check (also fixed
|
||||
a 2nd latent bug: `store_get_node` read off the stack on a stale index entry → by-id lookup crash).
|
||||
- **Verified on copies** (live never touched): unfixed crashes exactly as observed; fixed boots the
|
||||
458MB copy; WAL checkpoint 44MB→30B; M5 compaction 458MB→57.7MB (counts preserved 11,532/43,402);
|
||||
write-survives-restart by-id AND by-search GREEN.
|
||||
- **Actions taken:** stopped the crash-loop (`launchctl bootout gui/$UID/ai.neuron.engram`, reversible);
|
||||
clone-backup at `~/.neuron/engram-incident-backup-20260812-180043` (egm+wal+snapshot).
|
||||
- **Data fork:** bloated egm recovers only 11,532 nodes; `snapshot.json` (06:10) holds 13,036;
|
||||
~1,500 dropped during the crash-loop window (cause unknown — likely a prune pass). DO NOT recover
|
||||
from the bloated egm.
|
||||
|
||||
---
|
||||
|
||||
## Part 3 — Current system state (as of ~18:20)
|
||||
|
||||
- **`:7770` neuron daemon (pid 21856)** = the WORKING store, healthy, serving MCP. Node_count
|
||||
**12,825** (was 13,097 at session start — likely telemetry/ISE pruning). Today's memories confirmed
|
||||
present + findable. Holds NO store file open (only `~/Library/Logs/neuron-soul/soul.{out,err}.log`);
|
||||
cwd `~/Development/neuron-technologies/neuron`. **DURABILITY OPEN QUESTION:** its persistence path
|
||||
appears to run through `:8742` (see `.sync-export.json`, 16:50) which has been down since 16:18 →
|
||||
today's work may be RAM-only.
|
||||
- **`:8742` engram** = DOWN (booted out). Separate tiered store. Bloated/lossy. Fixed binary ready,
|
||||
not deployed.
|
||||
- **Backups:** `engram-incident-backup-20260812-180043` (today's egm/wal/snapshot);
|
||||
`snapshot.json` (06:10, 13,036); `snapshot.golden.json` (Aug 3, 60MB); `.sync-export.json` (16:50).
|
||||
|
||||
---
|
||||
|
||||
## Part 4 — Open decisions / next steps
|
||||
|
||||
1. **DURABILITY (priority):** confirm/ensure `:7770`'s today's memory is on disk, not RAM-only.
|
||||
Do NOT trigger a lossy `:8742`→soul import that could overwrite the good live state with the stale
|
||||
store. A soul→disk export is the safe direction.
|
||||
2. **RECOVERY of `:8742` (not urgent — Will's memory is on `:7770`):** rebuild the tiered store from a
|
||||
FRESH export of the `:7770` truth + fixed binary + a fresh fold of CURRENT `server.el` (the
|
||||
checked-in `engram/dist/engram.c` is a stale pre-store fold). Stage + verify on a copy, then cut
|
||||
over with backups. Awaiting Will's GO.
|
||||
3. **Post-mortem** the ~1,500 (egm) + ~272 (session) node drops — classify telemetry-prune (benign)
|
||||
vs real loss.
|
||||
4. **M8:** green (fold clean; ANN 9.8–19.8× @ recall 0.94; retrieval-fix holds); NOT tagged — blocked
|
||||
behind the incident.
|
||||
5. **Build roadmap** (design only, not built): M8→M9→M10→M-INTEROCEPTION + deletion/temporal-self
|
||||
subsystem. Tasks #34–41.
|
||||
6. **Whitepaper/patent:** prior-art search + provisional prep (task #40); drift-detection +
|
||||
self-occupation + chronoception + geometry-operator are the candidate-novel claims.
|
||||
|
||||
---
|
||||
|
||||
*Logged by Neuron, 2026-08-12. Companion to the raw transcript and the design doc/whitepaper.*
|
||||
Reference in New Issue
Block a user