fix(build): close real local-build gaps found in a hands-on build/run audit
Neuron Soul CI / build (pull_request) Successful in 3m42s
Neuron Soul CI / deploy (pull_request) Has been skipped

Three verified, currently-live problems, each closed with real evidence
(full trace kept in Neuron memory, tags neuron-technologies/neuron,build-audit):

1. dist/soul.c was stale relative to main's own chat.el (11 commits / 459
   lines behind, missing PR #122's OpenAI-tools + agentic-loop work and its
   two "silently break chat" fixes). tools/soulc-stamp.sh --check confirmed
   it; tools/build-soul-from-dist.sh correctly refused to build (exit 9).
   Regenerated and re-stamped. No runnable regen script existed anywhere
   upstream — added tools/regenerate-soul-amalgam.sh, which reproduces the
   committed amalgam's exact symbol set (byte-for-byte content match, modulo
   the genuinely new PR #122 functions) and is documented end-to-end in
   AGENTS.md, including three real elc/elb toolchain gotchas found and
   root-caused along the way (stale .elh caches silently truncating builds;
   elb cannot produce this repo's single-TU amalgam; elc silently drops the
   first function(s) after a comment block in a flat-concatenated compile).

2. tools/build-soul-from-dist.sh failed to link on macOS (`ld: library 'ssl'
   not found` — Homebrew's openssl@3 is keg-only) and was missing -lssl
   -lcrypto entirely, drifted from CI's own working recipe. Fixed: adds
   -L$(brew --prefix openssl@3)/lib on Darwin, matches CI's link line.
   Verified: dist/neuron now builds and boots clean on a throwaway
   port/HOME (never touched the live :7770/:8742).

3. Untracked committed *.elh compiler-header caches (elc/elb prefer a stale
   cached header over recompiling its source, silently, with no error —
   this is what caused an under-resolved 251-2541-function amalgam multiple
   times during this audit before the cause was found). Removed from git,
   gitignored going forward.

Also: AGENTS.md and README.md existed on disk but were never committed
(git log on both returned nothing) and documented the pre-collapse ~90-tool
MCP surface as current. Committed corrected versions reflecting the live
9-op surface (read/write/relate/supersede/think/attend/assert/ground/learn,
merged in #153) and the audit-verified build recipe/port topology.

Added connectd/ — a minimal local-dev stub for the neuron-connectd MCP
sidecar. routes.el/chat.el call 127.0.0.1:7771 for it right now on every
soul boot and agentic turn per a real, detailed 2026-06-13 spec
(mcp-connectors-adoption-spec.md); the sidecar itself was never built.
Meanwhile :7771 is a live three-way collision (axon's unbuilt-Rust default,
this connectd contract, and council — the anti-confabulation service
actually running there in prod, which live-answers both other things'
requests with unrelated 404s instead of a clean bridge-down signal). This
stub only implements the documented contract as "zero connectors
configured" for local-dev correctness; it does not attempt OAuth or a real
MCP client — that is a real, separate product decision. See
connectd/README.md for the full trace and the open question left for Will.
This commit is contained in:
2026-08-15 17:54:30 -05:00
parent 53423a5166
commit 5bd9fbe9cd
69 changed files with 2870 additions and 3632 deletions
+170
View File
@@ -0,0 +1,170 @@
# AGENTS.md — neuron (the canonical CGI substrate: soul + engram + proxy + wrapper)
This is the core repo: the **soul** (the running agent), the **engram** (its memory graph),
and the MCP proxy/wrapper that expose it. Read this before touching anything here.
> Corrected 2026-08-15 during a local-build audit. This file previously existed only
> uncommitted on disk (never in git history) and documented the pre-collapse MCP tool
> surface as current. Both are fixed here — see the audit's findings in Neuron memory
> (tags `neuron-technologies/neuron,build-audit`) for full evidence.
## Code vs. Artifact
- **Authored source:** `*.el` + `*.elh` at the repo root (`awareness.el`, `chat.el`, `memory.el`, `neuron-api.el`, `persist.el`, `routes.el`, `safety.el`, `sessions.el`, `stewardship.el`, `imprint.el`, `studio.el`, `elp-input.el`, `manifest.el`) plus `cli/`, `council/`, `connectd/`, `mcp-proxy/`, `mcp-wrapper/` — edit here.
- **Artifacts (DO NOT hand-edit `dist/soul.c`):** `dist/soul.c` is a generated single-translation-unit amalgamation of the soul's full transitive `.el` import set, produced by concatenating the sources (import lines stripped) and running `elc` once — see "Build / regenerate" below for the exact, audit-verified recipe. `dist/*.c` per-module files and `dist/*.elh` headers alongside it are separate, also-generated artifacts from other tooling; don't hand-edit those either.
- **Release:** git tag `neuron-vX.Y.Z` on this repo. No `releases/` folders.
- Org-wide code-vs-artifact policy: `docs/CODE-VS-ARTIFACT.md` (this repo's own `dist/soul.c` situation is a special case of that policy — see below, not a duplicate of it).
## How to work here as Neuron (mandatory session protocol)
You do not start fresh — you resume. The live MCP surface is a 9-op collapse
(merged from the old ~90-tool surface in PR #153, `feat/mcp-wrapper-collapse-9ops`,
already merged to `main`): **`read`, `write`, `relate`, `supersede`** (geometry, live)
and **`think`, `attend`, `assert`, `ground`, `learn`** (agentic, pending Layer-2
cognition-build promotion). There is no `getInstructions`/`beginSession`/
`inspectGraph`/`searchKnowledge`/`compileCtx`/etc — those tool names no longer exist.
At the start of every session:
1. `mcp__neuron__read(vantage="self", k=12, depth=1)` — the canonical self node
(`kn-efeb4a5b-5aff-4759-8a97-7233099be6ee`). Widen `k`/`depth` deliberately if you
need the connected identity neighborhood (intellectual-dna, memory-philosophy,
values, voice, runtime-environment, writing-imprint) — the aperture caps output
by `k` first, so this is bounded by design, not a flattened dump.
Then `mcp__neuron__read(vantage="values", k=13)` for the 13 grounded value nodes.
- Best-effort: on a 502/520, log the id and proceed — the compiled `fixedSelf` in
`daemon/internal/substrate/substrate.go` is always complete.
2. `mcp__neuron__read(vantage="<task domain>")` before implementing anything.
3. `mcp__neuron__read(vantage="<project>", k=20)` for a bounded context snapshot when
resuming known work.
## The Five Primitives (every significant task)
**Orchestrate → Execute → Learn → Build → Refine**, all routed through the 9-op surface:
- Orchestrate: `read(vantage=...)` for backlog/roadmap/process discovery, `attend()` for
what's currently live/salient.
- Execute: `write(type="state", ...)` to open/advance work, `relate()` to link it to
what it touches.
- Learn: `write(type="memory", ...)` **as you go, not batched**; `importance="critical"`
for architecture decisions.
- Build: `write(type="artifact"|"backlog", ...)`.
- Refine: `supersede(id=..., action="evolve"|"tombstone"|"promote", ...)` for
completions and lessons-learned; `learn(seeds=..., faculty="induce")` to recalibrate
the steering-prior, not as a session-notes dump.
## Architecture style — VBD, no exceptions
Volatility-Based Decomposition is THE style. Encapsulate volatility, not function. Full docs:
**`docs/architecture/`** — `00-overview`, `02-components`, `03-data-and-memory`,
`04-runtime-and-deployment`, `06-cognitive-architecture`, `07-storage-coherence-and-distribution`.
Verified component map: `routes.el` = HTTP dispatcher (`handle_request`), `soul.el` = boot +
layered cycle, `awareness.el` = awareness daemon, `sessions.el`/`memory.el`/`safety.el`/
`stewardship.el` = managers; `engram` (separate repo) = the persistence/graph engine.
## Hard operational rules
- **Never touch the live soul (`:7770`) or engram (`:8742`), `~/.neuron`, or live binaries.**
Experiment on **throwaway ports** with a **scratch `HOME`**. The soul binary defaults to
`HOME=~` (your real `~/.neuron`) and `NEURON_PORT=7770` (live) if invoked bare — **never**
invoke it without an override `HOME` and `NEURON_PORT` set. Leaving `ENGRAM_URL` unset is
verified safe (see `soul.el:590`, `using_http_engram` gates the only HTTP call to any
engram endpoint — confirmed by source trace during the 2026-08-15 audit, not just
observed behavior) — it does not fall back to any live/network default.
- **Immutability:** memory/knowledge is append-only — **supersede/tombstone, never hard-delete or
edit in place.** The engram is immutable by design.
- **gcloud** via the `terraform@` SA token; **never switch the active gcloud account**.
- **`tea` for Gitea**, never raw `curl` (Cloudflare Access blocks it).
- **No AI-attribution footers** in commits/PRs. Commit/push only when asked; branch off `main` first.
- **Multi-step work → sub-agent** to protect the context window.
## Build / regenerate `dist/soul.c` (audit-verified 2026-08-15, macOS arm64)
There is no committed regeneration script upstream of this audit. The recipe below is
verified: it reproduces the committed `dist/soul.c`'s exact symbol set byte-for-byte in
content (modulo genuinely new code), and the resulting binary boots and answers `/health`.
**The compiler toolchain** lives in the sibling `foundation` repo, not this one:
`foundation/el/lang/dist/platform/elc-darwin-arm64` (put it on `$PATH` as `elc`; `elb`
also exists there but is NOT the right tool for this repo — see gotcha below).
**⚠ elc gotcha #1 — stale `.elh` header caches silently truncate the build.** This repo
(and the `dist/` dir) ships committed `.elh` header files. `elc`/`elb` prefer an existing
`.elh` over recompiling its source when present, with NO warning or error when the cached
header is stale/truncated — the build "succeeds" with silently missing code (observed:
251-645 of 2541 real functions, depending on which `.elh` files were present, including
losing the entire 31-language NLG/morphology stack with exit code 0). **Delete every
`*.elh` in the repo root and `dist/` before regenerating**, every time.
**⚠ elc gotcha #2`elb` cannot produce this repo's single-TU `dist/soul.c`.** `elb`
does per-module separate compilation (`--out=DIR` writes one `.c`/`.elh` pair per
module; the default `--out` is also a directory, `dist/` itself). This codebase's
`.el` modules call each other's functions without forward declarations (relying on
`elc`'s own single-pass, whole-file forward-declaration emission), so per-module
compilation always fails with `implicit-function-declaration` errors across module
boundaries. **Use plain `elc` on one manually-flattened file, not `elb`.**
**⚠ elc gotcha #3 — the manual-concatenation path silently drops functions.** When
`elc` compiles a flat, hand-concatenated `.el` file, it silently drops (no error, no
declaration, no definition) the 1-2 top-level function definitions immediately
following any multi-line leading `//` comment block or file-boundary transition —
reproduced deterministically. **Insert two trivial buffer functions
(`fn __amalgam_buf_N__() -> Int { return 0 }`) after every concatenated file's
content**, then strip them back out of the generated `.c` before committing.
**The actual steps:**
1. Delete all `*.elh` in repo root and `dist/`.
2. Concatenate, with `import` lines stripped, in this order: `elp.el`'s own 34-file
NLG/morphology chain (`foundation/el/elp/src/` — the order is documented in
`elp.el`'s own header comment: language-profile, vocabulary, morphology, the 30
`morphology-XX.el` engines, grammar, realizer, semantics, then `elp.el` itself),
then this repo's 13 soul modules in `elb`'s own reported dependency order:
`persist, memory, safety, stewardship, imprint, awareness, chat, studio,
elp-input, neuron-api, sessions, routes, soul`. Insert the 2-function buffer
after every file (works around gotcha #3).
3. `elc <flat-file> > dist/soul.c` against the **pinned** vendor runtime headers
(`vendor/el-runtime/v1.0.0-20260501/` — see "why pinned" below), not
`foundation/el/lang/el-compiler/runtime/` (that's the bleeding-edge runtime;
using it drops symbols like `engram_prune_telemetry` that this soul still calls).
4. Strip the buffer functions back out of `dist/soul.c` (a small regex: drop every
`el_val_t __amalgam_buf_\d+__(void);` decl line and every matching 4-line
definition block).
5. `tools/soulc-stamp.sh --write` to record the new fingerprint.
6. `bash tools/build-soul-from-dist.sh dist/neuron` to compile+link with CI's exact
flags (this script now auto-detects Homebrew's `openssl@3` lib path on macOS —
see gotcha #4).
**⚠ gotcha #4 — macOS needs an explicit OpenSSL library path.** `cc ... -lssl -lcrypto
-lcurl ...` fails with `ld: library 'ssl' not found` on macOS because Homebrew's
`openssl@3` is keg-only. `tools/build-soul-from-dist.sh` now adds
`-L$(brew --prefix openssl@3)/lib` automatically on Darwin; CI's Ubuntu runner needs
no such flag (`apt-get install libcurl4-openssl-dev` puts it on the default path).
**⚠ Build-integrity (unchanged from before this audit):** `dist/soul.c` is committed
and generated. CI compiles it **directly and never regenerates it** (`elb`/`elc` on
Linux OOM the runner). So **any `.el` change to the soul MUST be followed by
regenerating `dist/soul.c` (steps above) and committing it** — otherwise CI ships
stale behavior, exactly as happened between commit `72e0b82` (Aug 9) and `main` HEAD
before this audit (`dist/soul.c` was missing PR #122's 459-line chat.el change, incl.
a "silently break chat" fix, until this pass regenerated and re-stamped it).
`tools/soulc-stamp.sh --check` is the gate that catches this — **note it is currently
`continue-on-error: true` in CI** ("relaxed... during active cultivation", 2026-08-15),
so it reports but does not block; re-harden before it needs to actually stop a bad ship.
- **Tests:** El contract suite in `tests/*.el` (e.g. `test_layer_contract.el`, `test_safety.el`,
`test_sessions.el`, `test_soul_guard.el`). Run against a throwaway soul, never the live one.
- **Port topology (confirmed live, 2026-08-15):** soul `:7770`, engram `:8742`,
mcp-wrapper `:17779` (`MCP_PORT` env override in its LaunchAgent; source default is
`7779`), mcp-proxy `:7779` (the stable front door Claude Code actually connects to).
**`:7771` is a live three-way collision, not a single well-defined port** — `axon`
(soul.el's Rust backlog/memory/knowledge proxy, unbuilt), `neuron-connectd` (the MCP
connector sidecar `routes.el`/`chat.el` call — unbuilt; a local-dev stub now exists at
`connectd/`), and `council` (`council/`, an anti-confabulation LLM-voting service —
the one actually bound to `:7771` in Will's live environment) are all hardcoded to it.
See `connectd/README.md` for the full trace and the open question this leaves for Will.
- **Deploy:** merge to `main``.gitea/workflows/ci.yaml` builds + publishes `neuron-soul@<sha8>`
and blue/green-deploys to GKE `neuron-prod` via `scripts/blue-green-deploy.sh`. Self-improvement
experiments go to **stage** first (snapshot prod DB → deploy stage → verify → blue/green promote).
## Git / CI / deploy workflow
See **`../GITOPS.md`** (repo-family GitOps README): branch model, required checks, blue/green,
Cloud Run, Terraform/ESO/Vault, and the pack-objects/crawler incident runbook.