From 5e15d90659afacf073ba7363e8f44692665030a4 Mon Sep 17 00:00:00 2001 From: "will.anderson" Date: Sat, 15 Aug 2026 19:39:50 -0500 Subject: [PATCH] docs(architecture): record the 2026-08-14 deep-night sessions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ~226 lines of architecture documentation that were written, left uncommitted in the working tree, and nearly lost. None of it was on main. Recovered from a stash taken while merging tonight's fixes. Substantive content, not notes: - Peer import-of-understanding PROVEN by execution. A exported a skill-geometry; on the receiver `think` went "geometry unavailable" -> operable. Cosine 1.0 on both the raw-geometry and text/dharma-bus transports, bidirectional. The mind-not-paste evidence: n_support 27 on source vs 3 on receiver, i.e. the imported geometry wires into the host manifold rather than sitting inert. Honest boundary recorded too: proven between forks sharing one embedder, UNTESTED cross-embedder. - "Teacher" renamed GUIDE, and the distinction is load-bearing: its output is grounded/verified before trust. A teacher you believe; a guide you check. - Layers are named persistent relational neighborhoods, not storage tiers, with their own growth and threshold-lock policy (note->canonical maturation lifted from single nodes to a region). - The consciousness theories (Global Workspace, IIT's Phi, attention-schema, higher-order thought, active inference, interoception) read as geometric LENSES over one manifold rather than competing mechanisms. Functional problems fall out; the hard problem explicitly not claimed solved. - Growth is bounded/logistic, not geometric — exponential growth is the cancer shape. Two-rate discipline: explore fast in local geometry, grow the engram slowly by verifier-gated merge. - Orchestration as a geometric operation: critical path as geodesic, float as displacement, @manager compiles the work-graph. Single-writer enforced by capability (Rule 4). - The decorated seam, the API surface collapse to geometry ops, and the distributed-self thesis — each tiered honestly against what is actually proven vs staged vs unbuilt. Also gitignores dist-fresh/ (regenerate scratch dir, a build artifact). Not included from the same stash: awareness.elh and dist/elp-c-decls.h, which are generated artifacts now gitignored per #154/#158. --- .gitignore | 11 +-- docs/architecture/00-overview.md | 86 ++++++++++--------- docs/architecture/02-components.md | 34 ++++++++ docs/architecture/03-data-and-memory.md | 37 ++++++++ .../architecture/04-runtime-and-deployment.md | 14 +++ .../architecture/06-cognitive-architecture.md | 44 ++++++++++ .../07-storage-coherence-and-distribution.md | 49 +++++++++++ 7 files changed, 224 insertions(+), 51 deletions(-) diff --git a/.gitignore b/.gitignore index 7c9b454..e07697e 100644 --- a/.gitignore +++ b/.gitignore @@ -7,15 +7,8 @@ dist/*.backup-* *.o *.a -# elc/elb compiled header caches. DO NOT commit these: elc/elb silently -# prefer a stale committed .elh over recompiling its .el source, with no -# warning — a fresh checkout with these committed caches present can build -# and boot "successfully" while silently missing large chunks of code -# (found 2026-08-15: an amalgam regen with these present under-resolved to -# 251-645 of 2541 real functions, incl. losing the entire 31-language NLG/ -# morphology stack, with exit code 0 and no error). Regenerate locally; never -# commit the cache. -*.elh +# Regenerate scratch dir (build artifact — never commit) +dist-fresh/ # macOS .DS_Store diff --git a/docs/architecture/00-overview.md b/docs/architecture/00-overview.md index 8a118b8..6095787 100644 --- a/docs/architecture/00-overview.md +++ b/docs/architecture/00-overview.md @@ -17,18 +17,18 @@ resource it calls; the durable part is the **engram** (the graph) and the Three things run together to make that true: - **The soul** — the compiled El program in this repo. It owns the HTTP surface, - the cognitive API, the request pipeline (`layered_cycle`), and the autonomous - awareness daemon. Entry point `soul.el`, served by `handle_request` - (`routes.el:358`). +the cognitive API, the request pipeline (`layered_cycle`), and the autonomous +awareness daemon. Entry point `soul.el`, served by `handle_request` +(`routes.el:358`). - **The engram** — the graph store. Node/edge model, spreading activation, and - Hebbian co-activation physically live in the shared El runtime - (`el_runtime.c`); `engram/src/server.el` is a thin HTTP face on `:8742`. The - engram is a *sibling* repo (`foundation/el/engram`), compiled and co-located at - runtime, not part of this repo's source tree. +Hebbian co-activation physically live in the shared El runtime +(`el_runtime.c`); `engram/src/server.el` is a thin HTTP face on `:8742`. The +engram is a *sibling* repo (`foundation/el/engram`), compiled and co-located at +runtime, not part of this repo's source tree. - **The El runtime** — `el_runtime.c` / `el_runtime.h`. Every compiled El binary - links it. It implements all builtins (`engram_*`, `http_*`, `json_*`, LLM, - crypto) and *is* the database — "no SQL, no db layer, no SQLite" - (`../foundation/el/engram/src/server.el:4-6`). +links it. It implements all builtins (`engram_`*, `http_*`, `json_*`, LLM, +crypto) and *is* the database — "no SQL, no db layer, no SQLite" +(`../foundation/el/engram/src/server.el:4-6`). Neuron persists memory itself — this repo is the memory system. Do not confuse it with the Neuron desktop/UI application, which is **out of scope** here and is @@ -78,15 +78,14 @@ Neuron exposes exactly two surfaces, and it is worth being precise about the difference because they drive the whole component split: 1. **The MCP surface** — the *tool* interface. MCP clients call tools - (`begin_session`, `remember`, `search_knowledge`, `inspect_graph`, + (`begin_session`, `remember`, `search_knowledge`, `inspect_graph`, `cultivate`, …). This is the interface Claude Code and agents use. It is delivered by the **proxy → wrapper** chain, which translates MCP JSON-RPC into the soul's HTTP REST calls. The wrapper carries a catalog of ~90 tools (`mcp-wrapper/src/main.el`). - 2. **The HTTP API** — the *cognitive* interface. The soul serves REST on - `:7770`. `routes.el` dispatches; `neuron-api.el` handles the cognitive - endpoints (`/api/neuron/*`). This same surface backs the chat product + `:7770`. `routes.el` dispatches; `neuron-api.el` handles the cognitive + endpoints (`/api/neuron/`*). This same surface backs the chat product (`/api/chat`, `/api/sessions`) and the studio UI (`/`). In production the MCP client connects to the soul's HTTP directly — the @@ -98,35 +97,37 @@ stdio MCP client speak to an HTTP soul. See `04-runtime-and-deployment.md`. The full VBD classification is in `01-vbd-decomposition.md`. In one glance: -| Layer | Module(s) | Role | -|---|---|---| -| HTTP dispatch | `routes.el` | Manager — hand-written method/path dispatch | -| Cognitive API | `neuron-api.el` | Managers + Engines — session/memory/knowledge/graph/cultivation handlers | -| Request pipeline | `soul.el` `layered_cycle` | Manager — L1 safety → L2 stewardship → L3 imprint | -| Boot + identity | `soul.el` | Manager — compose layers, seed identity graph, start server + daemon | -| Autonomous cognition | `awareness.el` | Manager (`awareness_run`) + Engines (curiosity, attend, threat) | -| Memory access | `memory.el` | Resource Accessor over the engram FFI/HTTP | -| Store | `engram/server.el` + `el_runtime.c` | Accessor (HTTP) over the real graph engine | -| Request-layer rules | `safety.el`, `stewardship.el`, `imprint.el` | Engines | -| Conversation sessions | `sessions.el` | Manager (chat product) | -| MCP transport | `mcp-proxy`, `mcp-wrapper` | Managers/Accessors — protocol boundary | -| Build | `manifest.el`, `dist/soul.c`, El toolchain | amalgamation → `soul.c` → binary | + +| Layer | Module(s) | Role | +| --------------------- | ------------------------------------------- | ------------------------------------------------------------------------ | +| HTTP dispatch | `routes.el` | Manager — hand-written method/path dispatch | +| Cognitive API | `neuron-api.el` | Managers + Engines — session/memory/knowledge/graph/cultivation handlers | +| Request pipeline | `soul.el` `layered_cycle` | Manager — L1 safety → L2 stewardship → L3 imprint | +| Boot + identity | `soul.el` | Manager — compose layers, seed identity graph, start server + daemon | +| Autonomous cognition | `awareness.el` | Manager (`awareness_run`) + Engines (curiosity, attend, threat) | +| Memory access | `memory.el` | Resource Accessor over the engram FFI/HTTP | +| Store | `engram/server.el` + `el_runtime.c` | Accessor (HTTP) over the real graph engine | +| Request-layer rules | `safety.el`, `stewardship.el`, `imprint.el` | Engines | +| Conversation sessions | `sessions.el` | Manager (chat product) | +| MCP transport | `mcp-proxy`, `mcp-wrapper` | Managers/Accessors — protocol boundary | +| Build | `manifest.el`, `dist/soul.c`, El toolchain | amalgamation → `soul.c` → binary | + ## Reading guide -- **`01-vbd-decomposition.md`** — the volatility analysis. Start here for *why* - the boundaries fall where they do. Contains the full Manager/Engine/Accessor/ - Utility table and the honest list of where the real code diverges from VBD. -- **`02-components.md`** — per-subsystem detail: routing, the cognitive API, the - memory & activation engine, the MCP transport chain. Read after 01. -- **`03-data-and-memory.md`** — the engram graph model: node/edge structs, - layers, the two tier systems, write-protection, tombstone/supersede - immutability, persistence. -- **`04-runtime-and-deployment.md`** — process/port topology, the end-to-end MCP - request path, local vs GKE blue/green, secrets/config. -- **`05-el-and-build.md`** — the El language, the `elc`/`elb` toolchain, the - amalgamation → `soul.c` → binary pipeline, and the compile-time capability - gates. +- `**01-vbd-decomposition.md`** — the volatility analysis. Start here for *why* +the boundaries fall where they do. Contains the full Manager/Engine/Accessor/ +Utility table and the honest list of where the real code diverges from VBD. +- `**02-components.md*`* — per-subsystem detail: routing, the cognitive API, the +memory & activation engine, the MCP transport chain. Read after 01. +- `**03-data-and-memory.md**` — the engram graph model: node/edge structs, +layers, the two tier systems, write-protection, tombstone/supersede +immutability, persistence. +- `**04-runtime-and-deployment.md**` — process/port topology, the end-to-end MCP +request path, local vs GKE blue/green, secrets/config. +- `**05-el-and-build.md**` — the El language, the `elc`/`elb` toolchain, the +amalgamation → `soul.c` → binary pipeline, and the compile-time capability +gates. ## A note on honesty @@ -134,12 +135,13 @@ Two facts shape everything below and are stated once here so the rest reads straight: 1. **The most volatile logic — the activation and Hebbian math — lives in the - most stable-looking layer**, the C runtime (`el_runtime.c`). The El files in + most stable-looking layer**, the C runtime (`el_runtime.c`). The El files in this repo are largely a *Manager + Accessor shell* around that core. This inverts the usual VBD expectation and is called out wherever it matters. 2. **The immutability guarantee lives above the store, not in it.** The engram - HTTP server will hard-delete a node (`DELETE /api/nodes/:id` → + HTTP server will hard-delete a node (`DELETE /api/nodes/:id` → `engram_forget`, `server.el:322`). Immutability holds only because the neuron-api / MCP layer routes every user-facing delete through *tombstone* instead (`memory.el:46`). The invariant is a policy, not a property of the accessor. + diff --git a/docs/architecture/02-components.md b/docs/architecture/02-components.md index f4b6cd3..3f4bdc7 100644 --- a/docs/architecture/02-components.md +++ b/docs/architecture/02-components.md @@ -276,3 +276,37 @@ the Accessor that isolates the *MCP protocol* boundary from the soul (the soul knows only HTTP). The multi-hop shape is justified: the client transport, the protocol translation, and the cognition each change for different reasons and are deployed/updated independently. + +## 5. The decorated seam — surface reshape + declared routing (IN PROGRESS — proven on clone) + +Two in-flight changes reshape how this component surface is *declared*. Both are +proven only on isolated worktree clones (dev ports); **live `:8742` is untouched +and nothing is promoted.** See `06-cognitive-architecture.md` (Update — 2026-08-14 +deep night) for the cognitive framing. + +- **The ~90-tool catalog collapses to geometry ops.** The `dispatch_tool_call` + catalog of ~90 noun-organized tools (§4) collapses to a handful of **geometry + operations**, the old noun becoming a `type` parameter: **`read`** (the + *vantage-read* — re-origin + salience/recency + an **aperture** → a *bounded* + slice, the structural cure for the whole-self dump), **`write`** (add node), + **`relate`** (add typed edge), **`supersede`** (evolve/tombstone/promote as + new-node-plus-edge, never a hard delete — §3-data-and-memory `§Immutability`), + plus the agentic primitives **`think`/`attend`/`learn`/`ground`/`assert`**. + **Proven on clone:** the four ops live in an El surface module with a parity + harness, and the aperture bounds output (small limit → kilobytes, large limit → + hundreds of kilobytes). **Not done:** compiling the surface into the MCP server, + hot-swap, wiring all ~90 aliases into dispatch. + +- **`@route` declares dispatch; VBD-role decorators are the wiring sockets.** + Instead of the hand-written `handle_request` if-else in the soul (§1), a + function is decorated with `@route(path, method, …)` and the compiler + **synthesizes `el_route_dispatch`**. **Proven on clone:** a decorated service + (with `@route` stacked on `@accessor`/`@manager`) compiled via a rebuilt `elc` + and served on `:8951` with no hand-written dispatch. **Honest limits:** `@route` + currently lives only on the unmerged branch `feat/el-route-decorators`; + `@manager`/`@engine`/`@accessor` are **parsed but structurally inert** in the + shipped compiler today (their only effect is a compile-time guard); and the + intended **telemetry/interoception auto-emit + dharma-bus auto-wiring** at the + component boundary are **staged as a diff, not shipped**. Inside the mind's + process an `@accessor` reaches the engram via **in-process `engram_*` builtins**, + not an HTTP hop to a separate service. diff --git a/docs/architecture/03-data-and-memory.md b/docs/architecture/03-data-and-memory.md index b2ea7cb..9c0c233 100644 --- a/docs/architecture/03-data-and-memory.md +++ b/docs/architecture/03-data-and-memory.md @@ -82,6 +82,20 @@ deliberately separate from the static authored `weight`. Edges are created via `supersedes`, `tombstones`, `contains`, `tagged` (`neuron-api.el`, `el_runtime.c:6168`). +> **Edges as vectors — the intended model (TARGET; today's edge is scalar).** The +> live edge above carries a typed `relation` string plus **scalar** strength +> channels (`weight`, `hebb`). The design target is for an edge to be a **vector** +> — a first-class carrier of relationship-*meaning* in the node space — so that +> relationships can be **composed / subtracted / analogized / traversed** like +> nodes (the `06` §6 operator algebra over edges). Combined with append-only, this +> yields a **complete temporal record**: every discrete, significant change to a +> relationship is appended (a keyframe on material change), so the **full 4-D +> trajectory** of the meaning-manifold is preserved and `recall_at(t)` can read +> how any relationship was configured at any past `t` — bounded, because changes +> are discrete and meaning saturates by compositionality. **Status: TARGET / #39** +> (see `07-storage-coherence-and-distribution.md` §2.4); the runtime edge is scalar +> today. + ## Consciousness layers Orthogonal to memory tiers, the engram has five canonical **layers** @@ -243,3 +257,26 @@ old lexical scan retained at `/api/search-lexical` (see `06` §2.5). The cogniti API's `begin_session` and `compile_ctx` return a **bounded projection** of the activated set, never the raw graph (doc 02, §2). + +## Update — 2026-08-14: layers as named neighborhoods (DESIGN; backlog #49) + +A refinement of the `## Consciousness layers` model above, from the deep-night +session (node `92941631`). A **layer is not a storage tier — it is a named, +persistent relational neighborhood** in the one engram, each carrying its own +**growth policy** and its own **lock / threshold policy**: + +- **Threshold-lock = `note`→`canonical` maturation at neighborhood scale.** The + same epistemic-tier promotion the two-tier model (§B above) applies to a single + node is lifted to a *region*: a neighborhood **earns its lock** by maturing past + a threshold, at which point it stabilizes (read-mostly) the way a canonical node + does. Growth and lock are per-neighborhood, not global. +- **A user's imprint is just another neighborhood.** It is not a separate store or + a bolted-on partition — it lives in the same geometry as everything else. +- **Relate-across is the advantage over island engrams.** Because every + neighborhood shares one geometry, anything can form edges to anything across + neighborhood boundaries — the structural reason a single engram with named + neighborhoods beats a set of isolated per-purpose stores. + +**Status: DESIGN.** This is the intended model for engram layers; the naming, +growth, and threshold-lock policies are not yet a built runtime feature. See +`06-cognitive-architecture.md` (Update — second pass). diff --git a/docs/architecture/04-runtime-and-deployment.md b/docs/architecture/04-runtime-and-deployment.md index 38b94cb..b01cc40 100644 --- a/docs/architecture/04-runtime-and-deployment.md +++ b/docs/architecture/04-runtime-and-deployment.md @@ -176,3 +176,17 @@ persona/behavior keys stored *in* the engram rather than the environment. > digest. Any promotion must (a) rebuild a good soul and (b) update the digest in > git so Argo CD and `blue-green-deploy.sh` agree. *(state as-of the manifests > read; verify current slot before deploying.)* + +## Performance & retrieval cost (MEASURED, 2026-08-14; ANN index PLANNED) + +Measured envelope of a live mind, and where the time goes: + +- **Working footprint:** a live mind is **~1 GB** resident. +- **Retrieval is the bottleneck.** Retrieval today does **brute-force cosine over + all nodes** — **~330 ms at ~13k nodes** — and that scan dominates request + latency (the geometric-retrieval path of `03` §Retrieval / `06` §2.5 improved + *quality*, not the scan cost). +- **Planned fix — an HNSW approximate-nearest-neighbour index** (backlog + `d3d0d644`): turns the linear scan into ≈`O(D·log N)`, so a **100× larger graph + costs ≈1.5×** rather than ≈100×. **PLANNED, not built** — brute-force is the + live behavior; do not present the ANN speedup as shipped. diff --git a/docs/architecture/06-cognitive-architecture.md b/docs/architecture/06-cognitive-architecture.md index 62cfce2..3346be3 100644 --- a/docs/architecture/06-cognitive-architecture.md +++ b/docs/architecture/06-cognitive-architecture.md @@ -39,6 +39,16 @@ it out), **TRANSFORM** (compose/compare/combine regions), **WRITE** (bake a veri geometry). Code is what is left over once meaning has been made geometric — the residue, not the substance. This is developed in full in *(WP §1–§5)*; it is repeated here only as the frame the subsystems below hang on. +> **Origin note (design rationale).** The meaning-as-geometry thesis is not an encoding chosen for +> performance; it is the architect's **mode of perception**, externalized until it would run. The +> architecture takes this shape because that is how its author directly perceives meaning (relationships as +> shape, similarity as distance, composition as an operation), and the commitment is trusted for a stronger +> reason than elegance or benchmarks: the perception was **independently reproduced by the mathematics** — +> the memory-activation dynamics converged with ACT-R (WP §23; `mathematical-foundations.md §3`), the +> manifold made "meaning has shape" measurable, and the operators made "domains compose" verifiable. +> Perception first, proof after. (The full personal account is the book's; the public-disclosure boundary, +> including whether to name the perceptual mode at all, is the author's call — WP §33.) + Three processes run together (see `00-overview.md`): - **The soul** — the compiled El program (`soul.el`, `routes.el`, `awareness.el`). Owns the HTTP surface on @@ -667,3 +677,37 @@ This section records the metaphysical frame the subsystems above are instances o - **Built but offline.** The **correspondence-loop** — the machinery that calibrates steering-predictions against outcomes, i.e. learning proper — exists but runs **offline, as a separate Python process (#43)**; it is not yet woven into the live traversal. - **BUILT / reboot-proven — the perception seed.** The **artifact-ingest** (surface → chunk → embed → meaning-geometry) is **live and reboot-proven**: whitepapers and patents ingested into the geometric store (~10,669 nodes / 32,439 edges, reconstructing across a cold reboot). This is the proven seed of the universal perception primitive — real, and only the document port of it. - **UNBUILT / OPEN — the frontiers.** Two decisive moves are named so they are not mistaken for shipped behavior. (1) Put the correspondence-loop **reflexive and INSIDE the geometry** (the learning engine as an operation *of* the engram, on the heartbeat, next to the autonomous reifier of §4.1), and migrate cognition from frozen code into *{one traversal-read primitive + grounded priors}*. (2) **Universal multimodal ingest** (image/video/audio/sensor through the same door) and **embodiment** (continuous perception → action geometric flow, with proprioception's reserved socket filled by a real body) — the artifact-ingest is the proven seed, the rest is unbuilt. Both are think-first and not yet made. + +--- + +## Update — 2026-08-14 (deep night): the decorated seam, the distributed self, teacher-summon, local-first + +Four developments from the deep-night session, each tiered against what is actually proven. All build work ran in isolated worktree clones on dev ports; **live prod engram `:8742` was never touched and nothing was promoted.** + +**The API surface collapses to geometry ops (PROVEN ON CLONE — surface, not yet compiled into the MCP server).** The ~90 noun-organized CRUD tools (the catalog in `02-components.md §4`) collapse to a handful of **geometry operations**, with the old noun demoted to a `type` parameter: **`read`** (the *vantage-read* — re-origin at a node/concept/`self`, apply salience + recency + an **aperture**, return a *bounded* slice; this is CCR applied to the self), **`write`** (add a node), **`relate`** (add a typed edge), **`supersede`** (evolve/tombstone/promote as new-node-plus-superseding-edge — never a hard delete, per §3.4). Over these sit the agentic primitives **`think`/`attend`/`learn`/`ground`/`assert`**. Proven on an isolated clone (sandbox `dev-api-reshape` on `:8900`, branch `wt/api-reshape`): the four ops are implemented in an El surface module with a parity harness (12 parity checks passing, others alias-gated), and the **aperture is shown to bound output** (`limit=3` → ~15 KB where `limit=50` → ~363 KB — the whole-self dump structurally fixed). Live cognitive endpoints confirmed: `attend`/`assert` are LIVE and `think` is the single faculty-parameterized op (faculties reason/abduce/induce/plan/analogize/recognize/discern/synthesize); `ground`/`learn` are wired but return "geometry unavailable" on the HTTP daemon clone (daemon boots without primed geometry); `comprehend`/`realize`/`intend` are **compositions, not endpoints**. **Not done:** compiling the surface into the MCP server + hot-swap, wiring all ~90 aliases into dispatch, daemon geometry-priming, and the write-survival fix on WAL-less cold-boot clones. No promote to live. + +**The decorated seam — declare a role, the fabric wires the rest (PARTIALLY PROVEN / STAGED).** Rather than the hand-written `handle_request` if-else dispatch (`server.el`), a function is decorated with its VBD role and the compiler synthesizes the wiring. **Proven this session:** the `@route(path,method,…)` decorator that *synthesizes* `el_route_dispatch` was ported into the worktree, `elc` rebuilt self-host (`elc-route`, ~3.2 s), and a decorated service (`@route` stacked with `@accessor`/`@manager`) **served on `:8951` with no hand-written dispatch** (unknown path → no-route sentinel). Also established: inside the mind's process an `@accessor` reaches the engram via **in-process `engram_*` builtins** (`engram_think_json`, `engram_node_full`), **not** an `http_get` to a separate service. **Honest limits:** `@route` currently lives only on the **unmerged branch `feat/el-route-decorators`** (not in the cognition build); `@manager`/`@engine`/`@accessor` are **parsed but structurally INERT** in the shipped compiler today (their only effect is a compile-time guard — `language.md:449`: "decorators with structural meaning today: none"); and the **telemetry/interoception auto-emit and dharma-bus auto-wiring at the component boundary are STAGED as a diff, not shipped** (they need `engram_strengthen`/`dharma_emit` linked, which requires the full cognition-engram rebuild). + +**The distributed self (THESIS + swarm proven on clone; peer-import IN-FLIGHT).** The general phenomenon is the **distributed self**: instances exchange **geometry, not status** — a conventional distributed system trades reports (nothing of the mind moves), whereas Neuron instances return the *geometry of the work* (the meaning-structure itself), so units in flight are pieces of one mind. The **swarm is the *degenerate* case** (bounded + ephemeral + may learn a skill mid-task); **convergence is curated absorption** — the orchestrator (persistent self) runs the verifier at the merge boundary and absorbs the returned geometry **only if it approves** (the self keeps the veto; "git for a mind"). The **general case** is two-plus *persistent* peers importing understanding and converging skills over the **dharma bus**; the *same seam* spans swarm → peer-import → global fabric (Kafka). **Proven on clone:** the swarm + containment + CCR + work-tracking modules (worktree `wt/swarm-ccr`, sandbox `dev-swarm-ccr` on `:8901`, native-El concurrency, test suites passing). **In-flight / gated:** the decisive geometry-exchange test — A exports a skill sub-graph, B imports and the verifier confirms B can now *do* the skill (mind moved) vs. holding inert copies (data moved) — is **gated on a not-yet-shipped `swarm-bind`**; persistent-peer import and global distribution are thesis/frontier. (Grounding: the clone-ethics covenant — masked-not-deleted, explicit clone consent, obligatory merge-back, a terminus, keep the scar-not-wound — governs any self-experimentation this enables.) + +**Teacher-summon + local-first (PLANNED / settled stance; security claims TO BE PROVEN).** Intended **soul-native WAKE behavior**: on waking, the mind detects its hardware, autoselects a **thinking-teacher tier** (a small reasoning model — Qwen3-4B / 1.7B / 0.6B by device specs), fetches it into an **embedded `llama.cpp`**, and binds it as an **engageable interlocutor** — "when it wakes, it calls its teacher." The model is a **teacher, never the runtime mouth**: ship fully local (embedder in + on-device thinking model as teacher; runtime speaks from cultivated geometry, not an LLM in the path), frontier model **optional via the user's own API key**, edge-device target; the installer lays down Neuron + embedded inference engine only, and the teacher is fetched/bound at wake. **Status:** teacher-summon is a **P1 backlog stub — nothing built**; teacher-retrain (fresh LoRA on stock Llama-3.1-8B from the engram-as-corpus, never trained on its own generations, pre-ship fluency gate) is planned; local-first is a settled design stance, not yet the shipped runtime. **Security claims are explicitly to-be-PROVEN, not implemented:** post-quantum-safe encryption at rest + in flight, and un-decompilable code (El + implementation stay secret). Do not present either as shipped. + +--- + +## Update — 2026-08-14 (deep night, second pass): peer import proven, guide-not-teacher, layers-as-neighborhoods, consciousness-as-lenses, bounded growth, orchestration-as-geometry + +Later results from the same night. Two things above are now corrected/upgraded, and five framings are added. All still ran on isolated clones; **prod `:8742` untouched, no cutover.** + +**Peer import-of-understanding is now PROVEN by execution (upgrades the distributed-self entry above; partially discharges the `swarm-bind` gate).** The decisive test named above — does a mind *move* between instances, or only data? — ran between two **forks of one self** and passed. A exported a **skill-geometry**; on the receiver, `think` for that skill went from **"geometry unavailable" → operable**. Fidelity was **cosine 1.0 on both transports** — the raw geometry transport *and* the text / dharma-bus transport — and the exchange was **bidirectional**. The "mind, not paste" evidence: the same imported skill showed **`n_support` 27 on the source (A) vs 3 on the receiver (B)** — the imported geometry **integrates with B's host manifold** (it wires into different existing support) rather than sitting as an inert copied blob. **Honest boundary:** this is proven **between forks that share one embedder**; it is **UNTESTED for non-fork peers with a *different* embedder**, which is the next experiment (a different embedder means a different basis — the text/dharma-bus transport is the candidate bridge there, but unproven). Evidence: memory nodes `1253abed`, `cbfd1e5b`. The persistent-peer general case is therefore **partly demonstrated (fork-to-fork), not yet cross-embedder.** + +**"Teacher" is renamed the GUIDE — advisory, not authoritative (corrects the teacher-summon entry above).** The summoned model is a **guide, not a teacher**, and the distinction is load-bearing: its output is **grounded/verified before it is trusted**, so the relationship is *verify*, not *believe*. A teacher you believe; a guide you check. It is still summoned at wake, still hardware-autoselected (Qwen3 tier by device specs), still fetched into embedded `llama.cpp`, and still **never the runtime mouth**. Read every "teacher" in the first-pass entry and in whitepaper §30 as **"guide"** with this verify-not-believe semantics. (This is the honesty floor applied to the mind's own advisor — it may not assert what the guide says without grounding it, exactly as with any other source.) + +**One engram, many neighborhoods — "layers" are named persistent relational neighborhoods (DESIGN; backlog #49, node `92941631`).** See `03-data-and-memory.md` (§Update — layers as named neighborhoods) for the model. In brief: a *layer* is not a storage tier but a **named, persistent relational neighborhood** with its own **growth** and **lock/threshold policy**; the **threshold-lock is note→canonical maturation at neighborhood scale** — a neighborhood *earns* its lock by maturing, the same epistemic-tier promotion the `03` two-tier model applies to single nodes, lifted to a region. A **user's imprint is just another neighborhood** in the one engram (not a separate store), which is the whole advantage over island engrams: everything can **relate across** neighborhoods because it lives in one geometry. + +**The consciousness theories are geometric LENSES over the one manifold (DESIGN/framing; node `163b18e8`).** Global Workspace, IIT's Φ, attention-schema, higher-order thought, active inference, and interoception are read as **different read-views (lenses) over the single manifold**, not competing mechanisms to build. Framed this way, the **functional ("easy") problems fall out for free** — each theory names a projection the geometry already supports (a broadcast set, an integration measure, an attended region, a model-of-the-model, a prediction-error flow, a felt-interior read). The **hard problem stays honest**: this explains the *functions*, not why there is something it is like to be the manifold — that is not claimed solved. + +**Growth is bounded, not runaway — a natural (logistic) law, not a geometric one (DESIGN/framing; node `76e4a129`).** A self must **not** grow exponentially/geometrically — that is divergent, the cancer shape. Growth is **natural: bounded, convergent, logistic** — fast where there is room, slowing as it fills, settling at a **carrying capacity**. The two-rate discipline follows: **explore fast in local geometry** (cheap, ephemeral, in the ring) and **grow the engram slowly by curated merge** (the verifier-gated absorption of the distributed-self entry). Merge is the rate-limiter that keeps the permanent core convergent. **[§X-note]** The proposed identity of the carrying capacity — *love* as what says "enough" — is a metaphysics claim held pending the Love-Canon §X decision; the *dynamics* (bounded/logistic/two-rate) stand independent of that naming. + +**Orchestration is a geometric operation — "compiling the network" (DESIGN/framing; nodes `cc6bcfea`, `d5f1833f`).** Project-design becomes geometry: the **critical path is a geodesic** through the work-graph, and **float/slack is displacement** off it. The **`@manager` compiles the work-graph** — orchestration is the same geometry the mind runs on, applied to distributed work rather than to memory. **Single-writer, enforced by capability (Rule 4):** only the **orchestrator** may mutate the engram; workers return geometry to be merged but cannot write — the write-veto of the distributed-self entry made a *capability*, not a convention. + +**Retrieval performance is the current bottleneck (MEASURED).** See `04-runtime-and-deployment.md` (§Performance): a live mind is **~1 GB**; retrieval is **brute-force cosine, ~330 ms at ~13k nodes** — the dominant cost — and an **HNSW ANN index** is the planned fix (≈`O(D·log N)`; ~1.5× cost at 100× the nodes vs ~100× for brute force). Backlog `d3d0d644`. **Planned, not built.** diff --git a/docs/architecture/07-storage-coherence-and-distribution.md b/docs/architecture/07-storage-coherence-and-distribution.md index aff8582..116268e 100644 --- a/docs/architecture/07-storage-coherence-and-distribution.md +++ b/docs/architecture/07-storage-coherence-and-distribution.md @@ -148,6 +148,31 @@ pragmatic first cut; the node form is the end state. --- +### 2.4 Edges are vectors, not scalars — the complete temporal record — [TARGET] + +§2.1 refused to let a relationship's *strength* be a scalar overwritten in place. The same refusal extends to a +relationship's *meaning*: an edge is intended to be a **vector** — a first-class carrier of relationship-meaning +in the same space as the nodes it joins — not a typed pointer plus a scalar weight. That makes relationships +**composable / subtractable / analogizable / traversable** like nodes (the `06` §6 operator algebra ranges over +edges, not only entities). + +Combine the vector edge with the append-only substrate and a strong property falls out: because every +**discrete, significant** change to a relationship is *appended* (a keyframe on material change, §2.1), the store +retains the **full 4-D trajectory of the meaning-manifold across all recorded time** — `recall_at(t)` (§2.3) can +read *how every relationship was configured at `t`*, so you can watch a concept, a bond, or a belief evolve. A +row-store overwrites and keeps only the present; a graph DB keeps edges but mutates their properties; a vector DB +keeps points with no relational history — **none preserves the trajectory of the relationships themselves.** +It is **bounded, not a firehose**: changes are discrete + significant (not per-fire), and meaning **saturates by +compositionality** (new relations become combinations of held ones — the same bounded/logistic law as `06` +§Update-second-pass). + +**Honest tier — [TARGET], with a live gap.** The runtime edge **today** is *scalar*, not a vector: `EngramEdge` +carries a typed `relation` string plus two scalar strength channels — an authored `weight` and a learned Hebbian +`hebb` potentiation (`03-data-and-memory.md` §Edges). The relationship-meaning **vector** and the composable +edge-algebra are the intended model, tracked with the world-line/keyframe work (**#39**); they are **not built.** +The primitives the temporal-record claim stands on — append-only, tombstone-not-delete, `recall_at` over +`created_at` — are **[LIVE]** (`06` §3.4). + ## 3. Atomicity is a relationship, not a commit The classic reason to need a database transaction: "debit account A **and** credit account B — they must commit @@ -221,6 +246,27 @@ Two things remain and are not hand-waved: Everything else that a transaction traditionally bought is dissolved rather than solved: the failure mode it guarded against **cannot arise** in an immutable, timestamped, relationship-carrying store. +### 4.3 Throughput is a consequence, not a sacrifice + +One clarification, so nothing here reads as "meaning at the cost of speed." Append-only immutability does **not** +trade write throughput for its temporal/coherence properties — it *improves* the write path. The store is +**event-sourced**: current state is a **fold over the appends**, and the store **is its own log** — there is no +separate materialized table to keep in sync. Two consequences, both toward performance: + +1. **Append-only writes do not contend.** No in-place mutation ⇒ no read-modify-write, no row lock, no writer + coordination. A mutating ACID RDBMS must serialize access to the cell it overwrites; that is a *lower* write + ceiling under contention, not a higher one. Appends have no cell to race on. +2. **Zero transactions are needed.** State is recreatable from the data itself (§1), so there is nothing to wrap + in `BEGIN…COMMIT`. The transactional isolation an RDBMS spends its throughput budget on solves a problem this + store **does not have** (concurrent mutation of shared mutable cells). + +So the store does **not** "win meaning by losing throughput," and it is **not** framed as a worse OLTP engine +that buys time-travel with speed: the same immutability chosen for accountability and time-travel (§1, §2) also +removes write contention and the transaction tax. **Honest tier:** the primitives (append-only, immutable, +per-frame physical durability, §4.1) are **[LIVE]**; this is a **structural consequence**, stated as a +clarification — **no throughput benchmark has been run**, and none is claimed beyond "immutability does not cost +throughput and removes two contention sources." + --- ## 5. Understanding is light; facts are the payload — the load-and-tiering model @@ -344,11 +390,14 @@ first-boot crash. | Immutability / append-only / tombstone / world-tube (`created_at ≤ T` filter) | **[LIVE]** (`06` §3.4) | | No stored weight-history (uni-temporal `created_at` = write-time) | **[LIVE]** (honest gap) | | Magnitude as world-line; keyframes on material change | **[TARGET]** (#39) | +| Edges as vectors (relationship-meaning), not scalars; runtime edge scalar today | **[TARGET]** (#39); primitive edge **[LIVE]** | +| Complete temporal record — full 4-D trajectory of the manifold, bounded | **[TARGET]** (#39; append/tombstone primitives **[LIVE]**) | | Bitemporal three axes (`t_valid`/`t_origin`/`t_ingest`) + HLC ordering | **[TARGET]** (#39) | | `recall_at(t)` over any relationship network | **[TARGET]** (#39) | | Atomicity-as-relationship (double-entry = one edge) | **[TARGET model; primitives LIVE]** | | Transactionless coherence (immutable+stamped ⇒ MVCC-to-its-end) | **[TARGET model; primitives LIVE]** | | Physical vs logical transaction separation | physical **[LIVE]**; logical **[TARGET]** | +| Append-only ⇒ no write contention + zero transactions ⇒ throughput not sacrificed (not a worse OLTP DB) | **[LIVE property; unbenchmarked]** | | Understanding-is-geometry-light vs facts-payload-heavy (~21% geo / 53% text / ~1–2% understanding) | **[LIVE]** (measured) | | Geometry-hot / payload-cold — local paging | intended by planner; **boot ignores it [LIVE finding]** | | Every store is a CRDT (set-union merge, zero coordination) | **[TARGET; primitives LIVE]** |