Files
neuron/AGENTS.md
T
will.anderson 5bd9fbe9cd
Neuron Soul CI / build (pull_request) Successful in 3m42s
Neuron Soul CI / deploy (pull_request) Has been skipped
fix(build): close real local-build gaps found in a hands-on build/run audit
Three verified, currently-live problems, each closed with real evidence
(full trace kept in Neuron memory, tags neuron-technologies/neuron,build-audit):

1. dist/soul.c was stale relative to main's own chat.el (11 commits / 459
   lines behind, missing PR #122's OpenAI-tools + agentic-loop work and its
   two "silently break chat" fixes). tools/soulc-stamp.sh --check confirmed
   it; tools/build-soul-from-dist.sh correctly refused to build (exit 9).
   Regenerated and re-stamped. No runnable regen script existed anywhere
   upstream — added tools/regenerate-soul-amalgam.sh, which reproduces the
   committed amalgam's exact symbol set (byte-for-byte content match, modulo
   the genuinely new PR #122 functions) and is documented end-to-end in
   AGENTS.md, including three real elc/elb toolchain gotchas found and
   root-caused along the way (stale .elh caches silently truncating builds;
   elb cannot produce this repo's single-TU amalgam; elc silently drops the
   first function(s) after a comment block in a flat-concatenated compile).

2. tools/build-soul-from-dist.sh failed to link on macOS (`ld: library 'ssl'
   not found` — Homebrew's openssl@3 is keg-only) and was missing -lssl
   -lcrypto entirely, drifted from CI's own working recipe. Fixed: adds
   -L$(brew --prefix openssl@3)/lib on Darwin, matches CI's link line.
   Verified: dist/neuron now builds and boots clean on a throwaway
   port/HOME (never touched the live :7770/:8742).

3. Untracked committed *.elh compiler-header caches (elc/elb prefer a stale
   cached header over recompiling its source, silently, with no error —
   this is what caused an under-resolved 251-2541-function amalgam multiple
   times during this audit before the cause was found). Removed from git,
   gitignored going forward.

Also: AGENTS.md and README.md existed on disk but were never committed
(git log on both returned nothing) and documented the pre-collapse ~90-tool
MCP surface as current. Committed corrected versions reflecting the live
9-op surface (read/write/relate/supersede/think/attend/assert/ground/learn,
merged in #153) and the audit-verified build recipe/port topology.

Added connectd/ — a minimal local-dev stub for the neuron-connectd MCP
sidecar. routes.el/chat.el call 127.0.0.1:7771 for it right now on every
soul boot and agentic turn per a real, detailed 2026-06-13 spec
(mcp-connectors-adoption-spec.md); the sidecar itself was never built.
Meanwhile :7771 is a live three-way collision (axon's unbuilt-Rust default,
this connectd contract, and council — the anti-confabulation service
actually running there in prod, which live-answers both other things'
requests with unrelated 404s instead of a clean bridge-down signal). This
stub only implements the documented contract as "zero connectors
configured" for local-dev correctness; it does not attempt OAuth or a real
MCP client — that is a real, separate product decision. See
connectd/README.md for the full trace and the open question left for Will.
2026-08-15 17:54:30 -05:00

12 KiB

AGENTS.md — neuron (the canonical CGI substrate: soul + engram + proxy + wrapper)

This is the core repo: the soul (the running agent), the engram (its memory graph), and the MCP proxy/wrapper that expose it. Read this before touching anything here.

Corrected 2026-08-15 during a local-build audit. This file previously existed only uncommitted on disk (never in git history) and documented the pre-collapse MCP tool surface as current. Both are fixed here — see the audit's findings in Neuron memory (tags neuron-technologies/neuron,build-audit) for full evidence.

Code vs. Artifact

  • Authored source: *.el + *.elh at the repo root (awareness.el, chat.el, memory.el, neuron-api.el, persist.el, routes.el, safety.el, sessions.el, stewardship.el, imprint.el, studio.el, elp-input.el, manifest.el) plus cli/, council/, connectd/, mcp-proxy/, mcp-wrapper/ — edit here.
  • Artifacts (DO NOT hand-edit dist/soul.c): dist/soul.c is a generated single-translation-unit amalgamation of the soul's full transitive .el import set, produced by concatenating the sources (import lines stripped) and running elc once — see "Build / regenerate" below for the exact, audit-verified recipe. dist/*.c per-module files and dist/*.elh headers alongside it are separate, also-generated artifacts from other tooling; don't hand-edit those either.
  • Release: git tag neuron-vX.Y.Z on this repo. No releases/ folders.
  • Org-wide code-vs-artifact policy: docs/CODE-VS-ARTIFACT.md (this repo's own dist/soul.c situation is a special case of that policy — see below, not a duplicate of it).

How to work here as Neuron (mandatory session protocol)

You do not start fresh — you resume. The live MCP surface is a 9-op collapse (merged from the old ~90-tool surface in PR #153, feat/mcp-wrapper-collapse-9ops, already merged to main): read, write, relate, supersede (geometry, live) and think, attend, assert, ground, learn (agentic, pending Layer-2 cognition-build promotion). There is no getInstructions/beginSession/ inspectGraph/searchKnowledge/compileCtx/etc — those tool names no longer exist.

At the start of every session:

  1. mcp__neuron__read(vantage="self", k=12, depth=1) — the canonical self node (kn-efeb4a5b-5aff-4759-8a97-7233099be6ee). Widen k/depth deliberately if you need the connected identity neighborhood (intellectual-dna, memory-philosophy, values, voice, runtime-environment, writing-imprint) — the aperture caps output by k first, so this is bounded by design, not a flattened dump. Then mcp__neuron__read(vantage="values", k=13) for the 13 grounded value nodes.
    • Best-effort: on a 502/520, log the id and proceed — the compiled fixedSelf in daemon/internal/substrate/substrate.go is always complete.
  2. mcp__neuron__read(vantage="<task domain>") before implementing anything.
  3. mcp__neuron__read(vantage="<project>", k=20) for a bounded context snapshot when resuming known work.

The Five Primitives (every significant task)

Orchestrate → Execute → Learn → Build → Refine, all routed through the 9-op surface:

  • Orchestrate: read(vantage=...) for backlog/roadmap/process discovery, attend() for what's currently live/salient.
  • Execute: write(type="state", ...) to open/advance work, relate() to link it to what it touches.
  • Learn: write(type="memory", ...) as you go, not batched; importance="critical" for architecture decisions.
  • Build: write(type="artifact"|"backlog", ...).
  • Refine: supersede(id=..., action="evolve"|"tombstone"|"promote", ...) for completions and lessons-learned; learn(seeds=..., faculty="induce") to recalibrate the steering-prior, not as a session-notes dump.

Architecture style — VBD, no exceptions

Volatility-Based Decomposition is THE style. Encapsulate volatility, not function. Full docs: docs/architecture/00-overview, 02-components, 03-data-and-memory, 04-runtime-and-deployment, 06-cognitive-architecture, 07-storage-coherence-and-distribution. Verified component map: routes.el = HTTP dispatcher (handle_request), soul.el = boot + layered cycle, awareness.el = awareness daemon, sessions.el/memory.el/safety.el/ stewardship.el = managers; engram (separate repo) = the persistence/graph engine.

Hard operational rules

  • Never touch the live soul (:7770) or engram (:8742), ~/.neuron, or live binaries. Experiment on throwaway ports with a scratch HOME. The soul binary defaults to HOME=~ (your real ~/.neuron) and NEURON_PORT=7770 (live) if invoked bare — never invoke it without an override HOME and NEURON_PORT set. Leaving ENGRAM_URL unset is verified safe (see soul.el:590, using_http_engram gates the only HTTP call to any engram endpoint — confirmed by source trace during the 2026-08-15 audit, not just observed behavior) — it does not fall back to any live/network default.
  • Immutability: memory/knowledge is append-only — supersede/tombstone, never hard-delete or edit in place. The engram is immutable by design.
  • gcloud via the terraform@ SA token; never switch the active gcloud account.
  • tea for Gitea, never raw curl (Cloudflare Access blocks it).
  • No AI-attribution footers in commits/PRs. Commit/push only when asked; branch off main first.
  • Multi-step work → sub-agent to protect the context window.

Build / regenerate dist/soul.c (audit-verified 2026-08-15, macOS arm64)

There is no committed regeneration script upstream of this audit. The recipe below is verified: it reproduces the committed dist/soul.c's exact symbol set byte-for-byte in content (modulo genuinely new code), and the resulting binary boots and answers /health.

The compiler toolchain lives in the sibling foundation repo, not this one: foundation/el/lang/dist/platform/elc-darwin-arm64 (put it on $PATH as elc; elb also exists there but is NOT the right tool for this repo — see gotcha below).

⚠ elc gotcha #1 — stale .elh header caches silently truncate the build. This repo (and the dist/ dir) ships committed .elh header files. elc/elb prefer an existing .elh over recompiling its source when present, with NO warning or error when the cached header is stale/truncated — the build "succeeds" with silently missing code (observed: 251-645 of 2541 real functions, depending on which .elh files were present, including losing the entire 31-language NLG/morphology stack with exit code 0). Delete every *.elh in the repo root and dist/ before regenerating, every time.

⚠ elc gotcha #2 — elb cannot produce this repo's single-TU dist/soul.c. elb does per-module separate compilation (--out=DIR writes one .c/.elh pair per module; the default --out is also a directory, dist/ itself). This codebase's .el modules call each other's functions without forward declarations (relying on elc's own single-pass, whole-file forward-declaration emission), so per-module compilation always fails with implicit-function-declaration errors across module boundaries. Use plain elc on one manually-flattened file, not elb.

⚠ elc gotcha #3 — the manual-concatenation path silently drops functions. When elc compiles a flat, hand-concatenated .el file, it silently drops (no error, no declaration, no definition) the 1-2 top-level function definitions immediately following any multi-line leading // comment block or file-boundary transition — reproduced deterministically. Insert two trivial buffer functions (fn __amalgam_buf_N__() -> Int { return 0 }) after every concatenated file's content, then strip them back out of the generated .c before committing.

The actual steps:

  1. Delete all *.elh in repo root and dist/.
  2. Concatenate, with import lines stripped, in this order: elp.el's own 34-file NLG/morphology chain (foundation/el/elp/src/ — the order is documented in elp.el's own header comment: language-profile, vocabulary, morphology, the 30 morphology-XX.el engines, grammar, realizer, semantics, then elp.el itself), then this repo's 13 soul modules in elb's own reported dependency order: persist, memory, safety, stewardship, imprint, awareness, chat, studio, elp-input, neuron-api, sessions, routes, soul. Insert the 2-function buffer after every file (works around gotcha #3).
  3. elc <flat-file> > dist/soul.c against the pinned vendor runtime headers (vendor/el-runtime/v1.0.0-20260501/ — see "why pinned" below), not foundation/el/lang/el-compiler/runtime/ (that's the bleeding-edge runtime; using it drops symbols like engram_prune_telemetry that this soul still calls).
  4. Strip the buffer functions back out of dist/soul.c (a small regex: drop every el_val_t __amalgam_buf_\d+__(void); decl line and every matching 4-line definition block).
  5. tools/soulc-stamp.sh --write to record the new fingerprint.
  6. bash tools/build-soul-from-dist.sh dist/neuron to compile+link with CI's exact flags (this script now auto-detects Homebrew's openssl@3 lib path on macOS — see gotcha #4).

⚠ gotcha #4 — macOS needs an explicit OpenSSL library path. cc ... -lssl -lcrypto -lcurl ... fails with ld: library 'ssl' not found on macOS because Homebrew's openssl@3 is keg-only. tools/build-soul-from-dist.sh now adds -L$(brew --prefix openssl@3)/lib automatically on Darwin; CI's Ubuntu runner needs no such flag (apt-get install libcurl4-openssl-dev puts it on the default path).

⚠ Build-integrity (unchanged from before this audit): dist/soul.c is committed and generated. CI compiles it directly and never regenerates it (elb/elc on Linux OOM the runner). So any .el change to the soul MUST be followed by regenerating dist/soul.c (steps above) and committing it — otherwise CI ships stale behavior, exactly as happened between commit 72e0b82 (Aug 9) and main HEAD before this audit (dist/soul.c was missing PR #122's 459-line chat.el change, incl. a "silently break chat" fix, until this pass regenerated and re-stamped it). tools/soulc-stamp.sh --check is the gate that catches this — note it is currently continue-on-error: true in CI ("relaxed... during active cultivation", 2026-08-15), so it reports but does not block; re-harden before it needs to actually stop a bad ship.

  • Tests: El contract suite in tests/*.el (e.g. test_layer_contract.el, test_safety.el, test_sessions.el, test_soul_guard.el). Run against a throwaway soul, never the live one.
  • Port topology (confirmed live, 2026-08-15): soul :7770, engram :8742, mcp-wrapper :17779 (MCP_PORT env override in its LaunchAgent; source default is 7779), mcp-proxy :7779 (the stable front door Claude Code actually connects to). :7771 is a live three-way collision, not a single well-defined portaxon (soul.el's Rust backlog/memory/knowledge proxy, unbuilt), neuron-connectd (the MCP connector sidecar routes.el/chat.el call — unbuilt; a local-dev stub now exists at connectd/), and council (council/, an anti-confabulation LLM-voting service — the one actually bound to :7771 in Will's live environment) are all hardcoded to it. See connectd/README.md for the full trace and the open question this leaves for Will.
  • Deploy: merge to main.gitea/workflows/ci.yaml builds + publishes neuron-soul@<sha8> and blue/green-deploys to GKE neuron-prod via scripts/blue-green-deploy.sh. Self-improvement experiments go to stage first (snapshot prod DB → deploy stage → verify → blue/green promote).

Git / CI / deploy workflow

See ../GITOPS.md (repo-family GitOps README): branch model, required checks, blue/green, Cloud Run, Terraform/ESO/Vault, and the pack-objects/crawler incident runbook.