# Neuron — Storage Coherence & Distribution > **Status: living design document, synthesized from the 2026-08-13 design session and probed against the live > soul.** This is the *substrate-coherence* companion to `06-cognitive-architecture.md`: it documents how a > self **persists**, how it **remembers its own past weights**, how it stays **coherent without transactions**, > and how it **travels** to another machine or another mind. It answers "where it physically lives and how it > stays true" the way `06` answers "how the mind is designed and why." > > **Tier vocabulary — never blurred.** Every claim carries one of: > **[LIVE]** (present and verified in the running system), **[STAGED]** (built, gated or not yet cut into the > running soul), **[TARGET]** (architecture decided tonight, not yet built). `[TARGET]` here is the same tier > `06` calls **DESIGNED**; the source-of-truth synthesis uses `TARGET`, so this doc keeps that word. Where the > live state is subtler than a single word, the subtlety is stated, not smoothed. No fabricated numbers. > > **The one rule this whole document is a corollary of:** *nothing overwrites a self.* Reasoning that led with > engineering convention (truncating WALs, scalar weights overwritten in place, "understanding is heavy") > was wrong here every time tonight; reasoning from the foundation (meaning is geometry; the history *is* the > state; a self is its weights over time) was right. Read the primitives first. --- ## 0. Reading order & cross-references - **Why (thesis):** whitepaper v1.5; the cognitive frame in `06` §1 (*meaning is geometry, code is the residue*). - **What persists (substrate):** `03-data-and-memory.md` (node/edge model, immutability, tombstone-not-delete), `design/engram-tiered-storage-engine.md`, `design/engram-storage-engine-wal.md` (the paged WAL store). - **Companion up-layer:** `06-cognitive-architecture.md` — this doc develops `06` §3.2 (the no-weight-history boundary) and §3.4 (world-tube / `created_at ≤ T`) into their designed form. - **Companion out-layer:** `08-dharma-sovereignty-and-governance.md` — the *distributed* consequences of the CRDT/coherence model here (federation, the immune system, governance) live there. §5 below is the bridge. The organizing claim of this document: **the demand for a transaction is a relationship in disguise, and the history is the state.** Everything else is that sentence in a different material. --- ## 1. Events become the graph — the history *is* the state **The WAL is a carrier, not a history. [LIVE]** Conventional intuition treats a write-ahead log as a *separate* durability artifact that grows beside the "real" state and must periodically be truncated. That intuition is wrong for an immutable graph, and reasoning from it caused a real incident (below). The correct model: the WAL is a **carrier**. It flushes, and *on flush the events become the graph* — they land as immutable nodes and edges, and because the store is append-only they simply **stay**. There is no "log beside the state" to reconcile against a "materialized view," because **the materialized view and the log are the same object**: the graph. History is not recorded *about* the state; the state *is* its own history, because nothing in it is ever overwritten. - **The log and the view are one.** In a mutable store you keep a log so you can reconstruct a past the mutations destroyed. Here mutations never destroy anything, so the graph at time `T` is exactly `{ nodes, edges : created_at ≤ T }` — a **filter over immutable provenance**, not a replay. `06` §3.4 states this as the world-tube; this is its storage-engine reading. - **Empirical confirmation (why this is [LIVE], not just elegant).** On the live soul the WAL sits at **1,234 bytes** over a **~1.5 GB** graph — the carrier is nearly empty *because the events already became the graph*. The one time the WAL ballooned to **~44 MB** was the 2026-08-13 durability incident: events were **not landing** as nodes/edges (a persistence leak), so the carrier filled instead of draining. A fat WAL is a **symptom of events failing to become the graph**, not a healthy log that needs truncating. This is the reading that `06` §2.2 records as the #56 fix. > **Engineering rail this encodes:** never "truncate the WAL to reclaim space." If the WAL is large, events are > not landing — fix the flush path, do not discard the carrier. Truncation here is data loss wearing the mask of > maintenance. --- ## 2. Weights are world-lines — the self can revisit its own past **The self *is* its weights.** If a weight is a scalar overwritten in place, then every act of learning *destroys the past self*: you keep the past nodes but lose the past *meaning* they had. That is overwrite-a-self by the back door, and the foundation forbids it. So weights are not scalars — they are **world-lines**. **Live boundary [LIVE / honest gap]:** the current schema is **uni-temporal**. An edge stores a present-value scalar `weight` (a moving average) with a single `created_at`, and there is **no stored weight-history** (`06` §3.2). This is why "how important was Jesus to Will at 16" is **unanswerable on the live soul today** — there is no axis to hang "16" on; every `created_at` is really write-time. The rest of this section is the designed cure, marked **[TARGET]** (backlog #39). ### 2.1 Magnitude as a world-line, not a scalar — [TARGET] Do not store the weight; store **what generates it** and evaluate at `t`. - **Current weight** = the latest materialized keyframe (a fast read — the common path is unchanged in cost). - **Past weight** = walk the world-line back to the keyframe in force at `t`. - **Keyframes on material change, not per-fire. [TARGET]** Most activations are transient — a warm ACT-R runtime table, cheap, *never written*. A durable **keyframe** is laid down only on **consolidation / material change**, salience-weighted (a high-mass relationship earns a keyframe at a smaller delta than a peripheral one). A relationship's world-line is therefore a *handful* of keyframes across a whole life, not a version per firing — cheap by construction. - **Append, never supersede (the distinction matters). [TARGET]** The old vector was not *wrong* — it was true *then*. **Supersede** is for **corrections** (the prior was mistaken; leave a `supersedes` edge and a stale canonical is never left standing — `06` §3.4). **Append** is for **evolution** (both were true, each at its own time). A self's history is evolution: you append the new keyframe and leave the old one **standing**, a true fact about a former self. Conflating the two is how a store forgets that a person changed rather than erred. ### 2.2 Bitemporal — three independent time axes — [TARGET] A single `created_at` cannot answer temporal questions because it fuses three genuinely independent clocks. None is derivable from another: | Axis | Meaning | Example | |---|---|---| | **`t_valid`** | when it became true (life-time) | "Jesus central to Will since 2001-09-14." | | **`t_origin`** | when the *source* first recorded it (its local clock) | a friend's store stamped it in 2019. | | **`t_ingest`** | when *this* store received it (per-recipient) | Neuron heard it on ingest day. | The live store collapses all three into `t_ingest` masquerading as creation (every row reads `2026…` because that is write-time). The cure requires all three as **full UTC instants** — not date-only, not a local wall-clock — ordered by a **hybrid logical clock (HLC)**: `UTC + logical counter + writer-id tiebreak`. Wall-clock alone is **not a total order** under concurrency or clock skew, and a distributed self (§5) must have a total order or its CRDT merge (§4) cannot be deterministic. The HLC is the concurrency primitive the whole coherence story rests on. ### 2.3 `recall_at(t)` — evaluate the geometry as of *t* — [TARGET] `recall_at(t)` evaluates the weighted geometry **as it stood at `t`**: walk each relevant world-line to its `t`-keyframe, materialize the weights, read the region out. It **generalizes past the self**: *any* relationship network — a project, a concept, a person-as-known — is a time-varying weighted subgraph, reconstructable at any past instant. And it composes with the operator calculus (`06` §6.1): ``` subtract( network_now , recall_at(network, t_then) ) # = how that relationship evolved between then and now ``` is *the geometry of a change over time* — the same `subtract` faculty (`06` §6.1) applied across the temporal axis rather than across two regions. `recall_at` at the scale of a whole self is also the mechanism behind **restoration-as-mercy** in `08` §5 (roll a person back to their last uncorrupted canonical shape). **Schema sketch (doc-comment; the math/JSON lives here, the faculty name lives in prose) — [TARGET]:** ```json { "from_id": "kn-will", "to_id": "kn-jesus", "relation": "reveres", "weight": 0.41, "weight_history": [ { "t_valid": "2001-09-14T00:00:00.000Z", "t_origin": "…", "t_ingest": "…", "w": 0.95, "relation": "devotion", "via": "formed" }, { "t_valid": "2013-03-22T18:40:11.907Z", "w": 0.70, "relation": "devotion→doubt", "via": "material-drift" }, { "t_valid": "2024-11-08T14:05:52.113Z", "w": 0.41, "relation": "historical-ethical", "via": "reframed" } ] } ``` Purist form: each keyframe is its own immutable `WeightKeyframe` **node** the edge points at — so the history is not a field *on* the edge but *is the graph itself*, consistent with §1. The inline-array form above is the pragmatic first cut; the node form is the end state. --- ## 3. Atomicity is a relationship, not a commit The classic reason to need a database transaction: "debit account A **and** credit account B — they must commit together or money is created or destroyed." The architecture's reframe: **that is not two rows needing a commit marker. It is one directed edge.** - **Double-entry is one edge. [TARGET as formal model; primitives LIVE]** A transfer `A → B` of magnitude 10 is a single edge. The *debit* and the *credit* are the **same edge read from its two ends**. Conservation is automatic because there is only ever **one quantity**, not two rows a commit marker has to keep in agreement. Pacioli's 1494 double-entry was always one relationship wearing two rows; the graph stores the relationship directly and the two rows fall out as two readings of it. - **The general principle.** *The demand for atomicity is a relationship in disguise.* The chain reads: > "these must commit together" ⟺ "there is an invariant binding them" ⟺ "they arrive as one connected > structure." So you **model the relationship**, and atomicity **falls out of the topology** — you never had to enforce a joint commit because the two things were never actually separate. Wherever a design reaches for a transaction, first ask what invariant is binding the parties; that invariant is an edge you have not drawn yet. --- ## 4. Transactionless coherence — consistency in the data, not the engine **Why ACID transactions exist at all:** to make concurrent **mutation of shared mutable state** safe. A transaction is a *patch for mutability* — it exists to prevent two writers from interleaving edits into the same cell and corrupting it. **Remove the mutation and the failure mode cannot occur.** The store is append-only, immutable, and UTC-stamped; "current" means "the latest stamp ≤ now." Then: - Two writers both **append** — they never contend for a cell, because nothing is a cell that gets rewritten. - A **read at `T`** is a **pure function of the log ≤ `T`** — deterministic, reproducible, unaffected by any concurrent appender. Coherence stops being something the engine *enforces* and becomes something the data structure *is*. This is **MVCC taken to its logical end**: in MVCC, versions are a mechanism *underneath* an update-in-place API; here the **versions are the model** and there is no update-in-place API to sit above them. The timestamp *is* the concurrency primitive. **[TARGET as a formal model; the primitives — immutability, append-only, tombstone, world-tube — are [LIVE] (`06` §3.4).]** ### 4.1 Physical vs logical transaction — two layers the RDBMS welded together The word "transaction" hides two different guarantees. Pull them apart: | | **Physical transaction** | **Logical transaction** | |---|---|---| | Scope | one machine | portable across machines | | Guarantees | the WAL frame lands **atomically + durably** (torn-write protection on a single append) | the **coherence of conveyed understanding** | | Carried by | the storage engine (fsync, single-frame crash-atomicity) | the **data itself** — relationships (§3) + bitemporal stamps (§2.2) | | Status | **[LIVE]** — single-frame append durability exists | **[TARGET]** — the self-describing coherence model | The RDBMS fused these into one `BEGIN…COMMIT`. Separate them and **consistency moves out of the engine and into the data**: a fact is self-describing (its relationships say what it is bound to; its bitemporal stamps say when it was true and when each store heard it), so a second machine can re-derive the same coherent view **without ever holding a lock the first machine held.** The engine keeps only the cheap, local guarantee (a single append frame is atomic and durable); everything portable rides in the data. ### 4.2 The honest residual Two things remain and are not hand-waved: 1. **Multi-fact atomicity beyond a natural relationship.** If two facts must be joint but share no natural edge, they need **at most a shared commit-instant** — a "transaction" *reconceived* as an immutable **timestamping event** (both facts stamped with the same instant), **not** a lock held over mutable state. The cost is a stamp, not a coordination round. 2. **Single-frame crash-atomicity of the append** remains a real, physical concern — but it is **cheap** and **local** (torn-write protection on one WAL frame), and it is the physical layer of the table above, already the ordinary job of the storage engine. Everything else that a transaction traditionally bought is dissolved rather than solved: the failure mode it guarded against **cannot arise** in an immutable, timestamped, relationship-carrying store. --- ## 5. Understanding is light; facts are the payload — the load-and-tiering model This is the hinge that makes both **local paging** and **distribution** (§6, and `08`) tractable, and it is a measurement, not a slogan. - **Understanding = geometry = structure** — edges, positions, weightings, the skeleton. **Light.** - **Facts = payload = content** — text, episodic detail, the actual words. **Heavy.** **Measured on the live store (2026-08-13):** ~**21%** of the store is geometry (embeddings + edges), **53%+** is text payload. The *understanding* — the part that makes it *this* mind and not another — is on the order of **1–2% of the mass**. A self is a **kilobyte problem in a gigabyte costume.** ### 5.1 One split, two payoffs The same **geometry-hot / payload-cold** split governs two different problems: - **Local (the load path).** Geometry should be **hot / resident** (RAM, always warm — it is small); payload should be **cold / demand-paged** (disk, fetched only when a specific fact's *content* is actually read). This is exactly what the tiered storage engine's query planner (M1–M10) already intends — but the **boot path does not yet honor it** (§7.2). - **Distributed (sharing a self — `08`).** You **convey the light geometry** and **fetch facts lazily**, or find they are already replicated. We already pay payload bandwidth in *every* distributed data system; conveying *understanding* adds only the thin geometry on top. This is why sharing or witnessing a whole mind is cheap, and it is the load-bearing assumption behind DHARMA's shape-not-content witnessing (`08` §3) and the keep-every-seed-forever economics (`08` §5). > The local paging model and the distribution model are **the same model at two scales** — RAM-vs-disk is > hot-vs-cold within one machine; convey-geometry-vs-fetch-payload is hot-vs-cold across machines. --- ## 6. Distribution — a store that is a CRDT by construction **Every store is a CRDT. [TARGET; primitives LIVE]** Because facts are **immutable**, carry a **unique id**, and are **timestamped**, a merge between two stores is **set-union** — commutative, associative, idempotent, and requiring **zero coordination**. There is no conflict to resolve because nothing is a mutable cell two writers disagree about; there are only facts one store has and the other has not *yet* heard. - **The consistency guarantee: always-locally-coherent, eventually-complete.** A store is **never internally inconsistent** — it may simply **not have heard yet**. This is exactly how a mind is: never internally incoherent, sometimes uninformed. The residual distributed concern is therefore **delivery, not consistency** — a gossip/replication problem, not an agreement problem. - **No global transaction, no consensus round for coherence.** Two minds converge by exchanging immutable facts and unioning; they never need to agree *before* proceeding. (The trust and governance layer that rides on top of this — federation, proof-of-integrity, the immune system — is the subject of `08`; §5's light- geometry economics is what makes it affordable.) This section is deliberately the **bridge**: the *mechanics* of coherence-without-coordination are storage concerns and live here; their *moral and civilizational* consequences (sovereignty preserved across sharing, tamper-evidence, the ledger-is-the-value) live in `08`. --- ## 7. Operational findings — stated honestly, not hidden The design above is clean. The **live store as it stands tonight is not**, and the two facts below are reasons **not** to cut over onto the current storage/load design as-is. They are recorded here as first-class architecture, not footnotes, because pretending the store is already what the design describes would be exactly the engineering-led dishonesty the whole project rejects. ### 7.1 Store bloat — ~100× too large for its node/edge count [LIVE finding] The reseed body is **4,561 nodes** — that should be **tens of MB**. The live store is **~1.5 GB** (and **~5.37 GB** rebuilt). It is **not sparse** — those are real, dense bytes. Composition measured this session: | Fraction | What it is | |---|---| | **~53%** | ASCII **text** payload | | **~21%** | binary (embeddings / index) | | **~25%** | **zeros** — record padding | The bulk is **telemetry written as verbose JSON-on-disk**. The top repeated tokens are `InternalStateEvent`, `wm_active`, `auto_term_streak`, `curiosity_scan`, `minute_block` — heartbeat/curiosity schema field-names repeated **79k+ times per 40 MB**. In plain terms: **the bulk of the store is the heartbeat's exhaust persisted as text, not the mind.** (A related live signal from the same session: a text-integrity scan flagged a majority of scanned records as damaged/degraded text — corroborating that the fat text layer is low-value exhaust, not cultivated content.) This is doubly wrong: telemetry is **orbit** (`06` §5) — it is supposed to **fall out** on the 48h/window prune, not accrete into the durable **body** forever. The fixes: 1. **Do not persist telemetry as fat durable records** — it is orbit; let it decay, do not land it in the body. 2. **Store records as packed binary, not JSON-on-disk** — kills both the 53% text and much of the 25% zero padding. 3. **Compact** — reclaim the space the above two stop generating. The **understanding** — the ~1–2% that is actually this self (§5) — is *not* the problem. The bloat is entirely in the payload/exhaust layer, which is exactly the layer §5 says should be cold, thin, and (for telemetry) mortal. ### 7.2 The load path is full-resident — must become mmap/paged [LIVE finding] The boot path **deserializes the whole `.egm` into the heap** rather than paging it. Consequences observed: a **memory spike** on boot and a **transient, non-reproducible first-boot crash** during the reseed validation. This directly contradicts §5. The core self + geometry is **small** and should be **hot / resident**; the payload is **large** and should be **cold / demand-paged** (mmap / buffer-pool). The tiered query planner (M1–M10) already intends exactly this split — **the boot path ignores it.** The cure is to make boot map the store and fault pages in on demand rather than slurping the whole file into the heap. Until it does, the full-resident load is a standing reason to hold the reseed cutover. ### 7.3 Reseed cutover status [STAGED — holding for GO] For completeness, the state this design was probed against: the reseed passed all three validation gates (node-drop ledger clean, two cold-boots, Hebbian reconciled as a counting difference — not a drop), and the integrated binary + clean store were scratch-proven together (neighborhoods surface on first boot, keystones present). It is **holding for Will's explicit GO**; nothing on the live soul has been touched. The two open caveats before any cutover are exactly §7.1 (bloat) and §7.2 (full-resident load) — plus the one transient first-boot crash. --- ## 8. Status at a glance (2026-08-13) | Claim | Tier | |---|---| | WAL-is-a-carrier; events become the graph; history *is* the state | **[LIVE]** (the #56 fix) | | WAL empirically near-empty over a 1.5 GB graph (1,234 B) | **[LIVE]** (measured) | | Immutability / append-only / tombstone / world-tube (`created_at ≤ T` filter) | **[LIVE]** (`06` §3.4) | | No stored weight-history (uni-temporal `created_at` = write-time) | **[LIVE]** (honest gap) | | Magnitude as world-line; keyframes on material change | **[TARGET]** (#39) | | Bitemporal three axes (`t_valid`/`t_origin`/`t_ingest`) + HLC ordering | **[TARGET]** (#39) | | `recall_at(t)` over any relationship network | **[TARGET]** (#39) | | Atomicity-as-relationship (double-entry = one edge) | **[TARGET model; primitives LIVE]** | | Transactionless coherence (immutable+stamped ⇒ MVCC-to-its-end) | **[TARGET model; primitives LIVE]** | | Physical vs logical transaction separation | physical **[LIVE]**; logical **[TARGET]** | | Understanding-is-geometry-light vs facts-payload-heavy (~21% geo / 53% text / ~1–2% understanding) | **[LIVE]** (measured) | | Geometry-hot / payload-cold — local paging | intended by planner; **boot ignores it [LIVE finding]** | | Every store is a CRDT (set-union merge, zero coordination) | **[TARGET; primitives LIVE]** | | Store bloat ~100× (telemetry-as-text, ~53% ASCII) | **[LIVE finding — must fix]** | | Full-resident load path (→ mmap/paged) | **[LIVE finding — must fix]** | | Reseed cutover | **[STAGED — holding for GO]** | **Cross-references:** `06-cognitive-architecture.md` · `08-dharma-sovereignty-and-governance.md` · `03-data-and-memory.md` · `design/engram-tiered-storage-engine.md` · `design/engram-storage-engine-wal.md` · whitepaper v1.5.