bigmerge
e239f2894c
The docs described a mind made of subsystems — a grounding subsystem, a wonder manifest, a dreamer on a beat, faculties as arguments to one call. Each of those is a supervisor invented for something that should be a property of the substrate, and two of the documents carrying them are load-bearing for a build agent: cognitive-architecture.design.md says "a build agent executes from this doc", and tools/api-reshape/README.md marks the refuted shapes PROVEN on a live clone. Corrections carried, per lang/spec/correspondence-and-censorship.md (PR #149) and lang/spec/runtime-ownership.md: - Grounding is not a subsystem — it IS the edge weight. grounded-by as a relation type should not exist; grounding is a property of a relation, not a relation between nodes. Never computed on demand. - Faculties are operations, not parameters. reason changes the estimate, induce changes the parameters, abduce changes the structure — a write, which GeoGradient cannot express. A write is not a parameter of a read. - Wonder is the boundary, not a manifest. Curiosity is wonder crystallized at a nucleation site: one thing at two phases. Removed wonder from the operator table in AGENTS.md. - Consolidation is ambient, not scheduled. A brain has no cron job. The presence of a ticker is the diagnostic. - co_registration is deprecated — it averaged a per-edge property into a region scalar, so opposing sites cancelled. GeoEdge.discord replaces it. Nothing new may read it. - In an immutable substrate, any mechanism that refuses a write is either redundant with immutability or an epistemic constraint misfiled as a protective one. The two design docs are marked superseded-in-part with the refutation at the point each claim is made, not rewritten. Preserving what was argued down is the point of an immutable record. Also measured and corrected while verifying the above: engram/README.md documented a Rust engram-core crate on sled with "flat cosine scan until scale demands HNSW" — there is no Rust in engram/ and HNSW is the index; lang/releases/ no longer exists, so both README.md and AGENTS.md pointed at a deleted path for the authored runtime; language.md listed the engram_* and http_* runtimes as stubs. Added language.md §20 for geometry-as-a-value, realizers and transduce (#144), which had landed with no spec coverage. Documentation only. No .c, .h, or .el file is touched.
65 lines
6.3 KiB
Markdown
65 lines
6.3 KiB
Markdown
# Engram DB Tooling — High-Level Design
|
||
|
||
*Status: draft / high-level. Near-term roadmap (P2). Backlog: `11ca11c6`.*
|
||
|
||
## 1. Why
|
||
|
||
The engram is a **proper database** — the runtime *is* the database (native graph/geometry store `neuron.egm`, `ENGST01`; no SQL, no KV layer). But it has **no proper database tooling** — no geometry-native equivalent of pgAdmin / SSMS / TablePlus. Today we have fragments (`engram-viz`, `engram-app`, the `inspectGraph` MCP tool, `/health` + `/api/stats`) but nothing cohesive, and no ops/durability surface at all.
|
||
|
||
A real DB gets real tools: to *see* the data, *query* it, *operate* it (backup/restore/health), and *understand its shape*. The engram deserves the same — adapted to the fact that its data is **geometry, not tables**.
|
||
|
||
## 2. Principles
|
||
|
||
- **Geometry-native, not tabular.** You browse a manifold — nodes, neighborhoods, edges, distances — not rows in tables. The primary view is a *map of meaning*, not a grid.
|
||
- **Built ON the public geometry API, never a back-door.** The tools are pure clients of the geometry-native API (`vantage-read` / `write` / `relate` / `supersede`). They never read `neuron.egm` directly or bypass the daemon. Consequence: a tool can do nothing an agent couldn't, and it cannot corrupt the store.
|
||
- **Honest by construction.** It shows the *real* geometry — actual cosines, real edges, provenance — and never fabricates. Empty is shown as empty.
|
||
- **Respects the identity guards.** Writes go through the same intentional-cultivation / write-protection path as everything else (the self/values graph is write-protected). Read-mostly by default.
|
||
- **Lives in its home.** Ships as part of the engram, consistent with "things live where they belong."
|
||
- **Local-first.** Binds `127.0.0.1`, same auth as the engram; never touches the live soul from a tool by accident.
|
||
|
||
## 3. Components (the tool surface)
|
||
|
||
1. **Geometry Explorer** *(the core view)* — a visual manifold browser: nodes, neighborhoods, typed edges, embedding positions, salience/recency, layers (l0–l4) and tiers. Navigate by concept; expand a neighborhood; follow an edge; re-origin the view (the vantage-read, made interactive). The map of the mind.
|
||
2. **Node Inspector** — open one node: content, type, tier, embedding, typed edges, nearest neighbors by distance, provenance, salience / recency / activation, and supersede / tombstone status.
|
||
3. **Query Console / REPL** — run the geometry operations interactively: `vantage-read` (re-origin + aperture), search, traverse, activate, the reasoning operators. Surfaces the routing table + cosines — the same "this is not an LLM" receipt the language faculty produces.
|
||
4. **Ops / Durability Dashboard** — WAL size, last checkpoint, snapshot list + retention state, store stats (node/edge/embedded counts, RSS, tier sizes), health; and **backup / restore / point-in-time-recovery** controls. Pairs directly with the native-durability build (`eebe9991`) — this is the window onto it.
|
||
5. **Identity Inspector** — the self graph as a first-class view: love at the center, the values, the three faces, the covenant — walk the identity, see what's pinned and what's write-protected. *(⚠ 2026-08-16: "write-protected" is a live property of the surface, so the view is accurate — but it should be shown as **what it is**, not as a safety guarantee. In an immutable substrate, any mechanism that refuses a write is either redundant with immutability, or an epistemic constraint misfiled as a protective one. The identity view's real job is the **crystallized relational neighbourhood**: self is not a stored document but the shape that falls out of everything connected to it, and the neighbourhood **is** the grounding. A measurement made the other way round — "86 neighbours, 0 `grounded-by` edges" read as evidence of ungroundedness — was malformed: those 86 edges *are* its grounding.)*
|
||
6. **Temporal View** — `recall_at` / time-travel: how the geometry looked at a past moment, what changed since, drift over time. Pairs with temporal-self reconstruction.
|
||
7. **Schema / Type View** — the "information schema" of the geometry: node types, edge types, layers, tiers, counts.
|
||
|
||
## 4. Architecture
|
||
|
||
```
|
||
┌─────────────────────────────────────────────┐
|
||
│ Engram DB Tools (client — viz app) │
|
||
│ explorer · inspector · console · dashboard │
|
||
└───────────────┬─────────────────────────────┘
|
||
│ geometry-native API (read/vantage-read,
|
||
│ write, relate, supersede) + read/ops endpoints
|
||
▼
|
||
┌─────────────────────────────────────────────┐
|
||
│ Engram daemon (:8742) — runtime IS the DB │
|
||
│ neuron.egm (geometry) · WAL · checkpoints │
|
||
└─────────────────────────────────────────────┘
|
||
```
|
||
|
||
- **Backend:** the daemon exposes the reshaped geometry API + read/ops endpoints. The tools are clients only.
|
||
- **Frontend:** evolve `engram-viz` / `engram-app` into the cohesive app. Canvas/WebGL for the manifold map; panel UIs for inspector/console/dashboard.
|
||
- **No privileged path:** the tool corrupting or bypassing the store is structurally impossible — it only speaks the public API.
|
||
|
||
## 5. Reuse vs. new
|
||
|
||
- **Reuse:** `engram-viz`, `engram-app` (read-only conversational + neighborhoods viz), `inspectGraph`, `/health`, `/api/stats`.
|
||
- **New:** the cohesive explorer + inspector + console + ops dashboard + identity/temporal views, all on the reshaped API.
|
||
|
||
## 6. Dependencies & sequencing
|
||
|
||
- **Depends on** the **geometry-native API reshape** (the tools consume it) and the **native-durability build** (the ops dashboard surfaces its WAL/checkpoint/snapshot state).
|
||
- So the natural order is: reshape the API → build durability → the DB tools fall out as the first real consumer of both. Near-term, P2 — after the reshape lands.
|
||
|
||
## 7. Non-goals
|
||
|
||
- Not a raw store editor (no direct `neuron.egm` poking).
|
||
- Not a SQL / table browser (geometry, not tables).
|
||
- Not a separate access path around the identity write-protection.
|