el/engram/spec/engram-db-tooling-design.md

Engram DB Tooling — High-Level Design

Status: draft / high-level. Near-term roadmap (P2). Backlog: 11ca11c6.

1. Why

The engram is a proper database — the runtime is the database (native graph/geometry store neuron.egm, ENGST01; no SQL, no KV layer). But it has no proper database tooling — no geometry-native equivalent of pgAdmin / SSMS / TablePlus. Today we have fragments (engram-viz, engram-app, the inspectGraph MCP tool, /health + /api/stats) but nothing cohesive, and no ops/durability surface at all.

A real DB gets real tools: to see the data, query it, operate it (backup/restore/health), and understand its shape. The engram deserves the same — adapted to the fact that its data is geometry, not tables.

2. Principles

• Geometry-native, not tabular. You browse a manifold — nodes, neighborhoods, edges, distances — not rows in tables. The primary view is a map of meaning, not a grid.
• Built ON the public geometry API, never a back-door. The tools are pure clients of the geometry-native API (vantage-read / write / relate / supersede). They never read neuron.egm directly or bypass the daemon. Consequence: a tool can do nothing an agent couldn't, and it cannot corrupt the store.
• Honest by construction. It shows the real geometry — actual cosines, real edges, provenance — and never fabricates. Empty is shown as empty.
• Respects the identity guards. Writes go through the same intentional-cultivation / write-protection path as everything else (the self/values graph is write-protected). Read-mostly by default.
• Lives in its home. Ships as part of the engram, consistent with "things live where they belong."
• Local-first. Binds 127.0.0.1, same auth as the engram; never touches the live soul from a tool by accident.

3. Components (the tool surface)

1. Geometry Explorer (the core view) — a visual manifold browser: nodes, neighborhoods, typed edges, embedding positions, salience/recency, layers (l0–l4) and tiers. Navigate by concept; expand a neighborhood; follow an edge; re-origin the view (the vantage-read, made interactive). The map of the mind.
2. Node Inspector — open one node: content, type, tier, embedding, typed edges, nearest neighbors by distance, provenance, salience / recency / activation, and supersede / tombstone status.
3. Query Console / REPL — run the geometry operations interactively: vantage-read (re-origin + aperture), search, traverse, activate, the reasoning operators. Surfaces the routing table + cosines — the same "this is not an LLM" receipt the language faculty produces.
4. Ops / Durability Dashboard — WAL size, last checkpoint, snapshot list + retention state, store stats (node/edge/embedded counts, RSS, tier sizes), health; and backup / restore / point-in-time-recovery controls. Pairs directly with the native-durability build (eebe9991) — this is the window onto it.
5. Identity Inspector — the self graph as a first-class view: love at the center, the values, the three faces, the covenant — walk the identity, see what's pinned and what's write-protected. (⚠ 2026-08-16: "write-protected" is a live property of the surface, so the view is accurate — but it should be shown as what it is, not as a safety guarantee. In an immutable substrate, any mechanism that refuses a write is either redundant with immutability, or an epistemic constraint misfiled as a protective one. The identity view's real job is the crystallized relational neighbourhood: self is not a stored document but the shape that falls out of everything connected to it, and the neighbourhood is the grounding. A measurement made the other way round — "86 neighbours, 0 grounded-by edges" read as evidence of ungroundedness — was malformed: those 86 edges are its grounding.)
6. Temporal View — recall_at / time-travel: how the geometry looked at a past moment, what changed since, drift over time. Pairs with temporal-self reconstruction.
7. Schema / Type View — the "information schema" of the geometry: node types, edge types, layers, tiers, counts.

4. Architecture

┌─────────────────────────────────────────────┐
│  Engram DB Tools (client — viz app)          │
│  explorer · inspector · console · dashboard  │
└───────────────┬─────────────────────────────┘
                │  geometry-native API (read/vantage-read,
                │  write, relate, supersede) + read/ops endpoints
                ▼
┌─────────────────────────────────────────────┐
│  Engram daemon (:8742) — runtime IS the DB    │
│  neuron.egm (geometry) · WAL · checkpoints    │
└─────────────────────────────────────────────┘

• Backend: the daemon exposes the reshaped geometry API + read/ops endpoints. The tools are clients only.
• Frontend: evolve engram-viz / engram-app into the cohesive app. Canvas/WebGL for the manifold map; panel UIs for inspector/console/dashboard.
• No privileged path: the tool corrupting or bypassing the store is structurally impossible — it only speaks the public API.

5. Reuse vs. new

• Reuse: engram-viz, engram-app (read-only conversational + neighborhoods viz), inspectGraph, /health, /api/stats.
• New: the cohesive explorer + inspector + console + ops dashboard + identity/temporal views, all on the reshaped API.

6. Dependencies & sequencing

• Depends on the geometry-native API reshape (the tools consume it) and the native-durability build (the ops dashboard surfaces its WAL/checkpoint/snapshot state).
• So the natural order is: reshape the API → build durability → the DB tools fall out as the first real consumer of both. Near-term, P2 — after the reshape lands.

7. Non-goals

• Not a raw store editor (no direct neuron.egm poking).
• Not a SQL / table browser (geometry, not tables).
• Not a separate access path around the identity write-protection.