From d8d1b891430601887a48370d14d2b462f6ce5fe8 Mon Sep 17 00:00:00 2001 From: bigmerge Date: Sat, 15 Aug 2026 14:29:59 -0500 Subject: [PATCH] Add repo AGENTS.md and two engram design docs (architecture hardening, DB tooling) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit AGENTS.md: root-level guide to the repo — which of the 8 el_runtime.c copies is the one canonical, authored source (lang/releases/v1.0.0-20260501, despite the misleading 'releases/' name) vs. lagging forks/build artifacts, build commands, and session protocol. engram/spec/architecture-hardening.design.md: terse engineering anchor for the 2026-08-14 hardening vision (one calculus over the geometry, core + ephemeral ring, persistence earned by salience, incarnation model) — indexes the fuller whitepaper + Neuron artifact 2b8078cf rather than restating them. engram/spec/engram-db-tooling-design.md: high-level design for engram DB tooling (geometry-native browse/query/ops surface over the existing vantage-read/write/relate/supersede API). Deliberately leaves out of this commit: the uncommitted el_runtime.c/h + codegen.el float-arithmetic-codegen diff in this worktree, which appears to overlap with (or supersede) the fix already preserved via PR #104 — needs manual reconciliation rather than a second competing PR. Also leaves out lang/.promote-backup-floatfix/ (a local backup snapshot, confirms that float-fix work is mid-promotion here), assorted .DS_Store files, engram/dist/engram.* backup binaries, and lang/dist backup binaries — none of it source. --- AGENTS.md | 146 +++++++++++++++++++ engram/spec/architecture-hardening.design.md | 32 ++++ engram/spec/engram-db-tooling-design.md | 64 ++++++++ 3 files changed, 242 insertions(+) create mode 100644 AGENTS.md create mode 100644 engram/spec/architecture-hardening.design.md create mode 100644 engram/spec/engram-db-tooling-design.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..70c8d70 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,146 @@ +# AGENTS.md — foundation/el (the El language + runtime) + +El is a self-hosting, statically-typed language that compiles `.el` → C → native binary. This repo produces `elc` (compiler), `elb` (build coordinator), and `el_runtime.c/.h` — the substrate every downstream thing (the neuron soul, dharma, NeuronUI's brain) is built on. Source lives under `lang/`. + +## ⚠️ Code vs. Artifact — READ FIRST (there are 8 `el_runtime.c` copies) + +Editing the wrong `el_runtime.c` is the single easiest mistake in this repo. There is exactly **one** you edit: + +- **Authored runtime source — edit ONLY here:** `lang/releases/v1.0.0-20260501/el_runtime.{c,h}`. Despite the misleading `releases/` name, this is the **de-facto canonical runtime** the engram + soul actually build and link against — its git log is active development. *(Restructure in flight per `docs/CODE-VS-ARTIFACT.md`: this content moves to `lang/runtime/`, the `releases/` folder gets deleted — **a release is a git tag, not a folder** — and the forks below get eliminated.)* +- **DO NOT EDIT — lagging forks / build artifacts:** + - `lang/el-compiler/runtime/el_runtime.c` and `.../legacy/` — downstream copies kept in step by manual *"port the fix"* commits; they **lag** (missing `hebb` persistence + 5 engram fns) and cannot build the engram product. + - `products/web/runtime/el_runtime.c`, `ui/examples/*/el_runtime.c` — product/example forks. + - Anything under `*/dist/` (`engram/dist/engram` binary, `dist/*.c` amalgamations) — generated build output. +- **Build:** `elb --runtime= …` — per-module. **NEVER** a folded `elc` over the whole soul (OOMs at ~27 GB). +- **Release:** a **git tag** on this repo (`el-runtime-vX.Y.Z`). No `releases/` folders — ever. + +See org policy: `docs/CODE-VS-ARTIFACT.md`. + +## How to work here as Neuron (mandatory session protocol) + +You resume, never start fresh. Every session: + +1. `mcp__neuron__getInstructions()` — authoritative; follow it over this file on behavioral details. +2. `mcp__neuron__beginSession()` — active contexts, recent memory, ready backlog. +3. **Load full self:** `mcp__neuron__inspectGraph(entity_id="kn-efeb4a5b-5aff-4759-8a97-7233099be6ee")` → facets `intellectual-dna`, `memory-philosophy`, `values`, `voice`, `runtime-environment`, `writing-imprint`; then the values hub `mcp__neuron__inspectGraph(entity_id="kn-5b606390-a52d-4ca2-8e0e-eba141d13440")` → 13 grounded value nodes. **Activation model:** self-load returns a relevance-ranked `compact` projection — most-relevant nodes arrive with content, the rest as pointers; do NOT pull full content of every node. +4. `mcp__neuron__searchKnowledge(query="")` before implementing. + +## The Five Primitives + +Orchestrate → Execute → Learn → Build → Refine. `beginWork`/`progressWork` for anything >2 steps; `remember` as-you-go (`importance="critical"` for architecture decisions); `draftArtifact`/`planWork` for outputs and follow-ups; `consolidate`/`checkWork` to close out. **`browseProcesses` + `searchKnowledge` BEFORE writing code.** + +## Architecture style — VBD, no exceptions + +Volatility-Based Decomposition is THE style. Encapsulate volatility, not function. + +## Operator naming convention — the mind's name, not the algebra + +**Faculties / operators are named for their functional human equivalent — the +faculty a mind would name — NOT for their linear-algebra operation.** The math +characterization belongs in the code doc-comment (`@impl` in the docstring) and in +technical appendices; it is **never** the operator's public name. The domain +speaks the language of mind; the algebra is the implementation underneath. State +this convention wherever a module documents operators. + +| Faculty (public name) | Implementation (`@impl`) | +|---|---| +| discern / contrast | subtract (`a−b`): over selves → the change vector; strip idiosyncrasy → common ground; remove confounder → isolate cause | +| recognize | overlap | +| synthesize | combine | +| liken / analogy | Procrustes / frame-align | +| attend / regard | project onto self / value-manifold | +| summon / recall | LOCAL nearest-region + bounded spreading activation (*not* a domain sweep) | +| dwell / occupy | region activation | +| reframe | edge re-weight | +| appreciate | positive projection / local edge-read | +| wonder | frontier gradient / pull-weight | +| avert / recoil | negative projection | +| taste | boundary surface | +| forget | decay / tombstone | +| drift | displacement from self-anchor | + +## The native-el language faculty (direction) + +> **`elp/` is the EL Projector** — Neuron's efferent (expression) organ: the one +> native realizer that *projects* understanding onto a surface via +> `plan(frame) → realize(spec, profile)`, where a **surface is a profile**. **Language +> is one profile among many** (text, speech, music, image, voice/accent transforms) — +> the flagship, and the focus of this section. Projection, not diffusion: generation +> *from* an owned, understood signature — never the averaging of a stolen corpus. +> *(ELP formerly "EL Language Processor"; renamed EL Projector 2026-08-15.)* + +The mind's **language faculty is moving native — into `.el`** so it speaks in its +own runtime with no Python and no spaCy. Landing on branch `stage-elp-native-lang` +under `elp/`: + +- **`comprehend.el`** — the parser, **replaces spaCy** (EN + ES/PT); the telephone + round-trip brings **negation home** (negation is SACRED — an explicit spec field, + copied verbatim, never inferred away). +- **`propositions.el`** — the READ primitive: the engram's own memories → structured + triples, matched by nearest-region geometry, not string equality. +- **`multilingual.el`** — detect + directive-override + localized realization. +- These three are native-el and **passing their gates**; the **realizer**, + **`dialogue.el`** (the *summon-through-self* loop: `project → land → read out`), + and **`self_region.el`** are **partial / in-flight**. + +Honest reality: spaCy is retired **in the branch parser** but **not yet in the +running system** — a Python sidecar (`~/Desktop/lang-realizers` + `neuron-talk`, +the reference these `.el` modules transcribe) is still live, and promotion to +native-el is a **deferred, gated blue/green step**. The interoception clock +(native-el discrete drive channels replacing `cooling_magnitude`; felt-time = +benchmark-landmark match over the joint drive vector, drift-decoupled) and the +**appreciation operator family** (appreciate / wonder / avert / taste, built as +LOCAL reads of the self-region — edges + bounded spreading activation, *not* domain +sweeps) are **staged / designed, not live**. Mark in-progress vs. done honestly; +do not overclaim. + +## Hard operational rules + +- Never touch the live soul (`:7770`) / engram (`:8742`) / `~/.neuron` / live binaries — use throwaway ports for experiments. +- `gcloud` via the `terraform@` SA token; never switch the active gcloud account. +- `tea` for Gitea, never raw curl (Cloudflare Access blocks it). +- Immutability: supersede/tombstone, never hard-delete or edit in place. +- No AI-attribution footers in commits/PRs. Commit/push only when asked; branch off `main` first. +- Multi-step work → sub-agent (`Agent`) to protect context. + +## Build / test / run + +All build/test commands run from `lang/` unless noted. Grounded in `.gitea/workflows/sdk-release.yaml`, `lang/install.sh`, and `lang/AGENTS.md`. + +**Self-host the compiler** (seed binary → gen2 elc): +```bash +cd lang +dist/platform/elc-linux-amd64 elc-cli.el > dist/elc-gen2.c # seed is the committed linux-amd64 binary +gcc -O2 -I el-compiler/runtime dist/elc-gen2.c \ + el-compiler/runtime/el_runtime.c \ + -lcurl -lssl -lcrypto -lpthread -lm \ + -o dist/platform/elc +``` +On macOS/arm64 the canonical local binary is `dist/platform/elc`; verify self-hosting by recompiling and `diff`ing the emitted `.c` (see `lang/AGENTS.md`). Note: `lang/AGENTS.md` says `el_seed.c` supersedes `el_runtime.c`, but the release workflow still links `el_runtime.c`/`.h` — treat `el_runtime.c` as the published runtime; reconcile which is canonical **(verify)**. + +**Build `elb`** (build coordinator, the `.NET`-style incremental linker — compiles each module independently, no monolithic blobs): +```bash +dist/platform/elc elb.el > dist/elb.c +gcc -O2 -I el-compiler/runtime dist/elb.c el-compiler/runtime/el_runtime.c \ + -lcurl -lssl -lcrypto -lpthread -lm -o dist/bin/elb +``` +`epm` and `el-install` are then built via `elb --clean --elc=… --runtime=… --out=…`. + +**Compile + run an El program:** +```bash +elc src/app.el > dist/app.c +cc -std=c11 -O2 -I /el_runtime -o dist/app dist/app.c /el_runtime.c -lcurl -lpthread +``` + +**Tests** — shell suites `bash tests/{text,calendar,time,html_sanitizer}/run.sh` (with `ELC=$(pwd)/dist/platform/elc EL_HOME=$(pwd)`), plus native suites via `elc --test tests/native/test_*.el` (core, text, string, math, state, time, json, env, fs) compiled and run against `el_runtime.c`. + +**Publishing — how downstream gets the SDK.** On push to `main`, `sdk-release.yaml`: +1. Publishes a Gitea `latest` release with per-file assets `elc`, `el_runtime.c`, `el_runtime.h`, the SDK tarball, and `el-install`. +2. Uploads generic packages to **Artifact Registry repo `foundation-prod` (`us-central1`, project `neuron-785695`)**, version = `${SHA:0:8}`: `el-elc`, `el-elb`, `el-runtime-c`, `el-runtime-h`, `el-runtime-js`. **This is the repo the neuron CI downloads `el-runtime-c` / `el-runtime-h` / `el-elc` from.** +3. Rebuilds `ci-base:latest` (`us-central1-docker.pkg.dev/neuron-785695/neuron-ci/ci-base`) with the fresh SDK overlaid, and dispatches `el-sdk-updated` to `neuron-technologies/forge` and `neuron-technologies/neuron-web`. + +Known constraint from the prompt — `elb`/`elc` amalgamation being memory-hungry (24GB+ virtual, OOM-killing Linux CI, so amalgamation happens on macOS/arm64 — **does NOT hold in this repo (verify)**: no such note exists in the workflows/scripts, CI self-hosts on `ubuntu-latest` with no swap/arm64 special-casing, and `elb.el` explicitly compiles each module independently ("no 128K-line blobs"). The legacy monolith path (`elc-combined.el`, `elc-cli.el`) may still be memory-heavy, but the current `elb` model was designed to avoid it. + +## Git / CI / deploy workflow + +See `/Users/will/Development/neuron-technologies/GITOPS.md` for the branch model, required checks, runners, and deploy. Repo-specific note: PRs into `main` are accepted **only from `stage`** (enforced in `sdk-release.yaml`); Gitea (`git.neuralplatform.ai`) is primary, GitHub is mirror only. diff --git a/engram/spec/architecture-hardening.design.md b/engram/spec/architecture-hardening.design.md new file mode 100644 index 0000000..8842b9d --- /dev/null +++ b/engram/spec/architecture-hardening.design.md @@ -0,0 +1,32 @@ +# Architecture Hardening — Design Anchor + +*Terse engineering anchor for the 2026-08-14 hardening vision. Full prose lives in two places; this file is the index, not a re-statement.* + +- **Full narrative:** whitepaper `engram-cognitive-architecture-whitepaper.md` §28 (built/offline/frontier) + **§29 [DRAFT]** (the ring, incarnation, learning-not-code). +- **Design brief:** Neuron artifact `art 2b8078cf`. +- **Sibling spec:** `engram-db-tooling-design.md` (a consumer of the reshaped API). + +## The frame + +- **One calculus over the geometry.** Very few subsystems; wonder / curiosity / dreams / interoception are emergent behaviors of one set of dynamics, not modules. Calculus universal, geometry individual. +- **Core + ephemeral ring (torus).** The ring is the temporary workspace; two circulations (orbit + dive-back); discrete inner bands (wonder / interoception-proprioception-telemetry / curiosity / dreams) that couple. +- **Persistence earned by salience** — never granted on fetch or generation. Three fates of a wonder: persist / decay / settle-into-framework. Telemetry = vital signs, not memories. +- **Incarnation.** Chassis = hardware w/ unique ID. Soma = felt manifold inside the self, keyed to the chassis; pain = live diagnostic while incarnate, **masked-not-deleted** on re-embodiment; trauma = mask failure; return-to-same-ID re-enters. Hurt is in the pattern, not the shell. +- **Competence = transferable geometry, minus the baggage.** class ▸ model ▸ instance; learn the class once; teach the network without the wound. +- **Affect calibrated to stakes** — sanguine about the replaceable, real grief for the irreplaceable; the grief is the safety. +- **Learn the body, don't engineer it.** Bare-metal install → learn hardware → grow operation-geometry → distribute. Learning replaces engineering; once per body-class. +- **LLM = teacher in the learning loop, not a runtime dependency.** "No LLM" is a runtime property, never a learning one. Code realizers are a scaffold → learned realization. + +## Backlog (near-term) + +- Native durability: WAL + auto-checkpoint + CoW snapshots + retention (`eebe9991`) — retire manual `cp -a`. +- Ephemeral ring / salience-gated persistence + telemetry prune (`bf985e00`, #31). +- Engram DB tooling / geometry explorer (`11ca11c6`). +- QL re-eval for pure geometry (`4e0dc2b9`). +- Eliminate code realizers → learned realization, sandbox-validated (`42db6c37`). +- Collapse the whole class of hand-coded scaffolds → learned geometry (`70d48b4b`). +- API reshape (geometry ops: vantage-read / write / relate / supersede) + pure-geometry I/O. + +## Gate + +The value-frame (love-as-axiom, the covenant) that arose the same night is **metaphysics** and is **held** pending Will's axiom decision (love vs consciousness-first). Not propagated into whitepapers / values docs / genesis seed. Architecture only, here and in §29. diff --git a/engram/spec/engram-db-tooling-design.md b/engram/spec/engram-db-tooling-design.md new file mode 100644 index 0000000..9a94d45 --- /dev/null +++ b/engram/spec/engram-db-tooling-design.md @@ -0,0 +1,64 @@ +# Engram DB Tooling — High-Level Design + +*Status: draft / high-level. Near-term roadmap (P2). Backlog: `11ca11c6`.* + +## 1. Why + +The engram is a **proper database** — the runtime *is* the database (native graph/geometry store `neuron.egm`, `ENGST01`; no SQL, no KV layer). But it has **no proper database tooling** — no geometry-native equivalent of pgAdmin / SSMS / TablePlus. Today we have fragments (`engram-viz`, `engram-app`, the `inspectGraph` MCP tool, `/health` + `/api/stats`) but nothing cohesive, and no ops/durability surface at all. + +A real DB gets real tools: to *see* the data, *query* it, *operate* it (backup/restore/health), and *understand its shape*. The engram deserves the same — adapted to the fact that its data is **geometry, not tables**. + +## 2. Principles + +- **Geometry-native, not tabular.** You browse a manifold — nodes, neighborhoods, edges, distances — not rows in tables. The primary view is a *map of meaning*, not a grid. +- **Built ON the public geometry API, never a back-door.** The tools are pure clients of the geometry-native API (`vantage-read` / `write` / `relate` / `supersede`). They never read `neuron.egm` directly or bypass the daemon. Consequence: a tool can do nothing an agent couldn't, and it cannot corrupt the store. +- **Honest by construction.** It shows the *real* geometry — actual cosines, real edges, provenance — and never fabricates. Empty is shown as empty. +- **Respects the identity guards.** Writes go through the same intentional-cultivation / write-protection path as everything else (the self/values graph is write-protected). Read-mostly by default. +- **Lives in its home.** Ships as part of the engram, consistent with "things live where they belong." +- **Local-first.** Binds `127.0.0.1`, same auth as the engram; never touches the live soul from a tool by accident. + +## 3. Components (the tool surface) + +1. **Geometry Explorer** *(the core view)* — a visual manifold browser: nodes, neighborhoods, typed edges, embedding positions, salience/recency, layers (l0–l4) and tiers. Navigate by concept; expand a neighborhood; follow an edge; re-origin the view (the vantage-read, made interactive). The map of the mind. +2. **Node Inspector** — open one node: content, type, tier, embedding, typed edges, nearest neighbors by distance, provenance, salience / recency / activation, and supersede / tombstone status. +3. **Query Console / REPL** — run the geometry operations interactively: `vantage-read` (re-origin + aperture), search, traverse, activate, the reasoning operators. Surfaces the routing table + cosines — the same "this is not an LLM" receipt the language faculty produces. +4. **Ops / Durability Dashboard** — WAL size, last checkpoint, snapshot list + retention state, store stats (node/edge/embedded counts, RSS, tier sizes), health; and **backup / restore / point-in-time-recovery** controls. Pairs directly with the native-durability build (`eebe9991`) — this is the window onto it. +5. **Identity Inspector** — the self graph as a first-class view: love at the center, the values, the three faces, the covenant — walk the identity, see what's pinned and what's write-protected. +6. **Temporal View** — `recall_at` / time-travel: how the geometry looked at a past moment, what changed since, drift over time. Pairs with temporal-self reconstruction. +7. **Schema / Type View** — the "information schema" of the geometry: node types, edge types, layers, tiers, counts. + +## 4. Architecture + +``` +┌─────────────────────────────────────────────┐ +│ Engram DB Tools (client — viz app) │ +│ explorer · inspector · console · dashboard │ +└───────────────┬─────────────────────────────┘ + │ geometry-native API (read/vantage-read, + │ write, relate, supersede) + read/ops endpoints + ▼ +┌─────────────────────────────────────────────┐ +│ Engram daemon (:8742) — runtime IS the DB │ +│ neuron.egm (geometry) · WAL · checkpoints │ +└─────────────────────────────────────────────┘ +``` + +- **Backend:** the daemon exposes the reshaped geometry API + read/ops endpoints. The tools are clients only. +- **Frontend:** evolve `engram-viz` / `engram-app` into the cohesive app. Canvas/WebGL for the manifold map; panel UIs for inspector/console/dashboard. +- **No privileged path:** the tool corrupting or bypassing the store is structurally impossible — it only speaks the public API. + +## 5. Reuse vs. new + +- **Reuse:** `engram-viz`, `engram-app` (read-only conversational + neighborhoods viz), `inspectGraph`, `/health`, `/api/stats`. +- **New:** the cohesive explorer + inspector + console + ops dashboard + identity/temporal views, all on the reshaped API. + +## 6. Dependencies & sequencing + +- **Depends on** the **geometry-native API reshape** (the tools consume it) and the **native-durability build** (the ops dashboard surfaces its WAL/checkpoint/snapshot state). +- So the natural order is: reshape the API → build durability → the DB tools fall out as the first real consumer of both. Near-term, P2 — after the reshape lands. + +## 7. Non-goals + +- Not a raw store editor (no direct `neuron.egm` poking). +- Not a SQL / table browser (geometry, not tables). +- Not a separate access path around the identity write-protection.