diff --git a/neuron-dev-setup/README.md b/neuron-dev-setup/README.md new file mode 100644 index 0000000..c84f507 --- /dev/null +++ b/neuron-dev-setup/README.md @@ -0,0 +1,171 @@ +# neuron-dev-setup — one-command Neuron CORE dev stack + +Stand up an identical **Neuron brain + agent** on a fresh Mac so any developer +gets the same local runtime to build against. This is the **CORE** dev stack +only — the four native `launchd` services that make Neuron think, remember, and +speak MCP to Claude Code. Will's personal automations (catalyst, telegram, +vessels, studio, self-review, world-integrator, council, compressor, snapshots, +act-runner, …) are **deliberately excluded**. + +``` + ┌─────────────┐ ┌──────────────┐ + │ soul :7770 │ ─────► │ engram :8742 │ the mind ──► its memory substrate + └─────────────┘ └──────────────┘ + ▲ + │ + ┌───────────────────┐ + │ mcp-wrapper :17779│ ─── MCP surface over the soul HTTP API (internal) + └───────────────────┘ + ▲ + │ + ┌────────────────┐ + │ mcp-proxy :7779│ ◄─── Claude Code connects here (stable front door) + └────────────────┘ +``` + +Claude Code's `neuron` MCP server points at `http://127.0.0.1:7779/` — the proxy. +The proxy forwards to the wrapper (`:17779`), which calls the soul (`:7770`), +which reads/writes the engram (`:8742`). The engram is the persistent brain. + +## Quick start + +```bash +git clone && cd neuron-dev-setup +cp config.env.example config.env # optional — edit ports/paths if you like +./install.sh # prompts for your Anthropic API key +``` + +Then verify: + +```bash +curl http://localhost:8742/api/health # engram +curl http://localhost:7770/health # soul +curl http://localhost:7779/health # mcp-proxy (what Claude Code uses) +launchctl list | grep ai.neuron +``` + +Open Claude Code — the `neuron` MCP tools should be live, backed by **your own** +local brain. `./install.sh --dry-run` shows every action without touching anything. + +## What the installer does (8 phases) + +| Phase | Action | +|------|--------| +| 1 | Preflight: macOS/arm64, ensure `git cc curl python3` + `openssl@3` (via Homebrew) | +| 2 | Prompt for the **Anthropic API key**, store it in the **macOS Keychain** (never a file) | +| 3 | Clone `neuron`, `engram`, `foundation`; fetch the El toolchain; build 4 binaries + `forge` | +| 4 | Lay down `~/.neuron/{bin,logs,engram}` and the templated `soul-wrapper.sh` | +| 5 | Generate + load the 4 core LaunchAgents (engram → soul → wrapper → proxy) | +| 6 | Seed a fresh engram with the **genesis identity** via `forge install` | +| 7 | Install Claude config: `neuron` agent, core hooks, local MCP registration | +| 8 | Health-check all four ports | + +Everything is **idempotent** (safe to re-run) and **templated** to the invoking +user's `$HOME` — no path is hardcoded to another machine. + +## Prerequisites + +- macOS on Apple Silicon (uses `launchd`; soul build flags assume arm64). +- **Xcode Command Line Tools** (`xcode-select --install`) — provides `cc`, `git`. +- **Homebrew** — for `openssl@3`, `curl`. +- An **Anthropic API key** — the soul's inference provider. Prompted for; stored + in Keychain under service `neuron-llm-0-key`; read at launch by `soul-wrapper.sh`. +- **Git access** to Gitea (`git.neuralplatform.ai`) for the source repos. +- **GCP access** to project `neuron-785695` Artifact Registry (default El + toolchain source). Ask Will to grant it, or set `EL_TOOLCHAIN_SOURCE=local`. + +## Core-stack map (what gets replicated) + +| Service | Port | Binary | Built from | LaunchAgent | +|---------|------|--------|------------|-------------| +| soul | 7770 | `neuron/dist/neuron` | `dist/soul.c` + El runtime, `cc` (CI recipe) | `ai.neuron.soul` | +| engram | 8742 | `engram/dist/engram` | `engram` repo `src/server.el` via `elc`→`cc` | `ai.neuron.engram` | +| mcp-wrapper | 17779 | `neuron/mcp-wrapper/dist/neuron-mcp-wrapper` | `mcp-wrapper/src/main.el` | `ai.neuron.mcp-wrapper` | +| mcp-proxy | 7779 | `neuron/mcp-proxy/dist/neuron-mcp-proxy` | `mcp-proxy/src/main.el` | `ai.neuron.mcp-proxy` | + +**`~/.neuron` layout the installer creates** + +``` +~/.neuron/ + bin/soul-wrapper.sh # reads Anthropic key from Keychain, execs the soul binary + logs/ # soul.*.log, engram.log, mcp-*.log + engram/ # ENGRAM_DATA_DIR — the persistent brain (snapshot.json + db) +``` + +**Identity seed.** `foundation/forge/seeds/neuron-genesis-seed.json` carries +`identity_nodes[]` + `edges[]` with **fixed** knowledge-node IDs (e.g. +`kn-efeb4a5b-5aff-4759-8a97-7233099be6ee`, the "self" traversal root). Those exact +IDs are referenced by the SessionStart self-load hook and the neuron agent, so +seeding must **preserve IDs** — `forge install ` is the mechanism. + +**Claude config installed** (`~/.claude/`) + +- `agents/neuron.md` — the Neuron agent (identity, session protocol, five primitives). +- `mcp.json` — registers `neuron` → `http://127.0.0.1:7779/`. +- `settings.json` hooks (CORE subset only): + - `SessionStart` → `neuron-self-load.sh` (loads identity from the seeded engram) + - `PreToolUse:Agent` → `neuron-agent-preamble.sh` (subagents load substrate first) + - `PreCompact` → `pre-compact.sh` (clean context recovery) + +### Deliberately EXCLUDED from core + +- **`check-active-contexts.sh`** and **`require-execution-context.sh`** — these + depend on a separate filesystem repo `~/Development/projects/active/neuron/synapse`. + `require-execution-context.sh` is a hard `Edit/Write` gate that would **block a + fresh dev from editing any file** without that synapse repo. Not core; excluded. +- `engram-mirror.py` (PostToolUse) — optional; mirrors MCP writes to engram. +- All Will-personal LaunchAgents: `catalyst-*`, `telegram-gateway`, `vessel.*`, + `studio`, `self-review`, `world-integrator`, `council`, `compressor`, + `cultivation-digest`, `snapshot-backup`, `engram-backup`, `act-runner`, `keymap`, + `invest`, and the disabled `ai.neuron.api` (`:7771` is a personal Python + perception helper — confirmed not core). + +## Secrets — how they're handled + +- **Anthropic key**: prompted for; stored in Keychain; read at launch. Never in a + plist, this repo, or a log. +- **Engram local token** (`ENGRAM_API_KEY`): a *loopback-only* dev token, not a + cloud secret. Defaults to a generated `ntn-dev-*` value; override in `config.env`. +- No cloud tokens, Vault tokens, CF-Access secrets, or founder keys are copied. + (Will's live `start-daemon.sh`/`neuron-api-launch.sh` contain such keys — this + installer intentionally does **not** use those files.) + +## Uninstall + +```bash +./uninstall.sh # stop + remove the 4 LaunchAgents and added Claude hooks +./uninstall.sh --purge-data # ALSO delete ~/.neuron/engram (destroys the brain) +``` + +## OPEN QUESTIONS (need Will to confirm) + +1. **El toolchain acquisition.** The default path fetches `el-runtime-c/-h` and + `el-elc` from GCP Artifact Registry (mirrors `neuron/.gitea/workflows/ci.yaml`). + A new dev needs GCP access to `neuron-785695`. Is that the intended path, or + should the El SDK be published/vendored for onboarding? +2. **`elc` invocation for engram/wrapper/proxy.** The soul build (`cc dist/soul.c + + el_runtime.c`) is verified from CI. The `.el → .c` transpile step for engram, + mcp-wrapper, and mcp-proxy is inferred (`elc -o `). Confirm the + exact flags / entrypoints (CI notes `elb` OOMs on Linux; macOS builds differ). +3. **`forge install` ID preservation.** Confirm `forge install` writes the seed's + fixed `kn-` IDs verbatim (the self-load hook hardcodes `kn-efeb4a5b…`). If it + re-mints IDs, the hook + agent identity load would break on a fresh brain. +4. **engram repo layout.** The live engram binary is built from `src/server.el` + (Gitea repo `neuron-technologies/engram`, cloned in CI). Confirm that repo is + the canonical source for onboarding (the local `foundation/el/engram` copy has + the same `src/server.el`). +5. **Home for this bundle** — see below. + +## Where this should live (recommendation) + +**Recommendation: a dedicated `neuron-dev-setup` (or `neuron-onboarding`) repo — +NOT `neuron-code`.** `neuron-code` already exists as a real product ("Neuron Code", +a coding tool with `nc-cli` + vessels — local `products/neuron-code` has commits); +repurposing it for onboarding would collide with a shipped product's identity. + +This bundle was scaffolded as `neuron-dev-setup/` on branch `feat/neuron-dev-setup` +in the **`neuron` repo** (off `origin/main`) and opened as a PR for review, because +the neuron repo already hosts the soul source, the verified CI build recipe, and +the mcp-wrapper/proxy sources — the natural review surface. If you'd rather it be +its own repo, move this directory into a fresh `neuron-dev-setup` repo verbatim; +nothing here depends on living inside the neuron repo. diff --git a/neuron-dev-setup/config.env.example b/neuron-dev-setup/config.env.example new file mode 100644 index 0000000..e0d02c8 --- /dev/null +++ b/neuron-dev-setup/config.env.example @@ -0,0 +1,42 @@ +# neuron-dev-setup — configuration +# Copy to config.env and edit if you want non-default paths/ports. +# install.sh sources this file if it exists; otherwise it uses these defaults. +# NOTHING here is a secret. The Anthropic API key is read from your Keychain, +# never from this file. See README.md. + +# ── Where the core stack lives ──────────────────────────────────────────────── +# All paths are relative to your own $HOME — never hardcode another user's home. +NEURON_HOME="${HOME}/.neuron" # runtime home: bin/, logs/, engram data +DEV_ROOT="${HOME}/Development/neuron-technologies" # where source repos are cloned/built + +# ── Git remotes (Gitea is primary) ─────────────────────────────────────────── +GITEA_BASE="git@git.neuralplatform.ai:neuron-technologies" +NEURON_REPO_URL="${GITEA_BASE}/neuron.git" # soul + mcp-wrapper + mcp-proxy source +ENGRAM_REPO_URL="${GITEA_BASE}/engram.git" # engram memory substrate +FOUNDATION_REPO_URL="${GITEA_BASE}/foundation.git" # El SDK + forge (seed installer) +NEURON_REPO_BRANCH="main" + +# ── Ports (must match across services; change only if a port clashes) ───────── +SOUL_PORT="7770" # soul daemon HTTP API +ENGRAM_PORT="8742" # engram memory substrate +WRAPPER_PORT="17779" # mcp-wrapper (internal, talks to soul) +PROXY_PORT="7779" # mcp-proxy (stable front door Claude Code connects to) + +# ── Engram ──────────────────────────────────────────────────────────────────── +ENGRAM_DATA_DIR="${NEURON_HOME}/engram" +# Local shared auth token for the engram/soul HTTP APIs on loopback. This is a +# LOCAL dev token (not a cloud secret); override it if you like. install.sh will +# generate a random one if you leave it empty. +ENGRAM_API_KEY="ntn-dev-local" + +# ── El toolchain source (needed to build engram / mcp-wrapper / mcp-proxy) ──── +# Option A (default): fetch prebuilt El runtime + elc from GCP Artifact Registry +# (requires `gcloud auth` with access to project neuron-785695 — ask Will). +# Option B: build El from the foundation/el checkout locally. +EL_TOOLCHAIN_SOURCE="artifact-registry" # artifact-registry | local +GCP_PROJECT="neuron-785695" +GCP_AR_REPO="foundation-prod" +GCP_AR_LOCATION="us-central1" + +# ── Keychain service name for the Anthropic key (read by soul-wrapper.sh) ───── +KEYCHAIN_SERVICE="neuron-llm-0-key" diff --git a/neuron-dev-setup/install.sh b/neuron-dev-setup/install.sh new file mode 100755 index 0000000..d7608bd --- /dev/null +++ b/neuron-dev-setup/install.sh @@ -0,0 +1,395 @@ +#!/usr/bin/env bash +# +# neuron-dev-setup / install.sh +# ───────────────────────────────────────────────────────────────────────────── +# One-command onboarding for the Neuron CORE dev stack on a fresh Mac. +# +# Stands up, as native launchd services, the four processes a developer needs to +# have an identical "Neuron brain + agent" to build against: +# +# soul (:7770) ──► engram (:8742) the mind + its memory substrate +# ▲ ▲ +# │ │ +# mcp-wrapper (:17779) ──► soul MCP surface over the soul API +# ▲ +# │ +# mcp-proxy (:7779) ◄── Claude Code stable MCP front door +# +# It also seeds a fresh engram with Neuron's identity (the genesis seed) and lays +# down the Claude Code config (neuron agent + core hooks + local MCP registration) +# so a new dev's `claude` talks to *their own* local Neuron. +# +# DESIGN RULES +# * Idempotent: safe to re-run. Existing state is detected and reused. +# * Templated: every path/port/user is derived from $HOME and config.env. +# Nothing is hardcoded to another developer's machine. +# * Secret-free: the Anthropic key is prompted for and stored in the macOS +# Keychain. No key is ever written to a plist, this repo, or a logfile. +# +# USAGE +# ./install.sh # full install +# ./install.sh --dry-run # print what would happen, touch nothing +# ./install.sh --skip-build # assume binaries already built (see --use-local) +# ./install.sh --skip-services # lay down files but don't load LaunchAgents +# ./install.sh --help +# ───────────────────────────────────────────────────────────────────────────── +set -euo pipefail + +# ── Locate ourselves ───────────────────────────────────────────────────────── +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +TEMPLATES="${SCRIPT_DIR}/templates" + +# ── Flags ──────────────────────────────────────────────────────────────────── +DRY_RUN=0; SKIP_BUILD=0; SKIP_SERVICES=0; USE_LOCAL_BINARIES=0 +for arg in "$@"; do + case "$arg" in + --dry-run) DRY_RUN=1 ;; + --skip-build) SKIP_BUILD=1 ;; + --skip-services) SKIP_SERVICES=1 ;; + --use-local) USE_LOCAL_BINARIES=1 ;; + --help|-h) + sed -n '2,40p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//' + exit 0 ;; + *) echo "unknown flag: $arg" >&2; exit 2 ;; + esac +done + +# ── Pretty logging ─────────────────────────────────────────────────────────── +c_blue=$'\033[1;34m'; c_grn=$'\033[1;32m'; c_yel=$'\033[1;33m'; c_red=$'\033[1;31m'; c_off=$'\033[0m' +step() { echo "${c_blue}▶${c_off} $*"; } +ok() { echo "${c_grn}✓${c_off} $*"; } +warn() { echo "${c_yel}!${c_off} $*"; } +die() { echo "${c_red}✗ $*${c_off}" >&2; exit 1; } +run() { if [ "$DRY_RUN" = 1 ]; then echo " [dry-run] $*"; else eval "$*"; fi; } + +# ── Load config ────────────────────────────────────────────────────────────── +if [ -f "${SCRIPT_DIR}/config.env" ]; then + # shellcheck disable=SC1091 + source "${SCRIPT_DIR}/config.env" +else + # shellcheck disable=SC1091 + source "${SCRIPT_DIR}/config.env.example" + warn "No config.env found — using defaults from config.env.example." +fi + +# Derived / defaulted values (never hardcode a home directory) +: "${NEURON_HOME:=${HOME}/.neuron}" +: "${DEV_ROOT:=${HOME}/Development/neuron-technologies}" +: "${SOUL_PORT:=7770}"; : "${ENGRAM_PORT:=8742}"; : "${WRAPPER_PORT:=17779}"; : "${PROXY_PORT:=7779}" +: "${ENGRAM_DATA_DIR:=${NEURON_HOME}/engram}" +: "${ENGRAM_API_KEY:=}" +: "${KEYCHAIN_SERVICE:=neuron-llm-0-key}" +: "${EL_TOOLCHAIN_SOURCE:=artifact-registry}" +: "${NEURON_REPO_BRANCH:=main}" + +NEURON_REPO="${DEV_ROOT}/neuron" +ENGRAM_REPO="${DEV_ROOT}/engram" +FOUNDATION_REPO="${DEV_ROOT}/foundation" + +SOUL_BIN="${NEURON_REPO}/dist/neuron" +ENGRAM_BIN="${ENGRAM_REPO}/dist/engram" +MCP_WRAPPER_BIN="${NEURON_REPO}/mcp-wrapper/dist/neuron-mcp-wrapper" +MCP_PROXY_BIN="${NEURON_REPO}/mcp-proxy/dist/neuron-mcp-proxy" +FORGE_BIN="${FOUNDATION_REPO}/forge/dist/forge" +GENESIS_SEED="${FOUNDATION_REPO}/forge/seeds/neuron-genesis-seed.json" + +LAUNCHAGENTS="${HOME}/Library/LaunchAgents" +CLAUDE_DIR="${HOME}/.claude" + +# Generate a local engram token if none was supplied. +if [ -z "${ENGRAM_API_KEY}" ]; then + ENGRAM_API_KEY="ntn-dev-$(head -c8 /dev/urandom | xxd -p 2>/dev/null || echo local)" +fi + +echo +echo "${c_blue}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${c_off}" +echo "${c_blue} Neuron CORE dev stack installer${c_off}" +echo "${c_blue}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${c_off}" +echo " user : ${USER}" +echo " NEURON_HOME : ${NEURON_HOME}" +echo " source repos : ${DEV_ROOT}" +echo " ports : soul=${SOUL_PORT} engram=${ENGRAM_PORT} wrapper=${WRAPPER_PORT} proxy=${PROXY_PORT}" +echo " dry-run : ${DRY_RUN}" +echo + +# render