el/tools/neuron-sandbox/README.md

nsbx — the Neuron Sandbox

Dev environment as a primitive. A reproducible way to run experiments and code changes against the real engram runtime on an isolated snapshot of the live mind — with a gated promote-to-prod path built on the proven rails.

Everyone (Tim, any team member, any agent) gets their own private, safe copy of the mind to build against. Prod — the live Neuron on :8742 (engram) / :7770 (soul) — is untouchable from a sandbox. A sandbox runs a separate engram process, on a separate port, against a separate clone of the store. The only op that can ever reach prod is promote, which is explicit, gated, and per-use approved.

It wraps the real engram binary — it never reimplements any engram logic. It generalises two proven proto-sandboxes into one primitive:

• the cog-arch build — isolated git worktree + build + clone of the live .egm + real C tests
• the store-fix cutover — secondary soul + launchctl bootout → settle → bootstrap rails

Quickstart

export PATH="$PWD:$PATH"          # or symlink nsbx onto your PATH

nsbx up                          # your private copy of the mind (auto-named <user>-dev)
nsbx run <name> api /api/stats   # poke it
nsbx validate <name>             # prove it: zero-loss, reboot, RSS, retrieval, keystones
nsbx destroy <name>              # cheap teardown; live untouched

That is the whole loop. Sane defaults: stock prod binary, auto-allocated port (8900+, never 8742/7770), snapshot of the live store.

The code-change dev loop (first-class)

Run your changed runtime, not just the stock binary, against a snapshot:

# build a runtime from a working tree, a git branch, or a prebuilt binary:
nsbx create feat --source /path/to/worktree           # elc + cc build from source
nsbx create feat --branch feat/my-change --repo <r>   # worktree the branch, then build
nsbx create feat --binary /path/to/engram             # use a prebuilt binary

nsbx build feat --source /path/to/worktree            # rebuild + hot-restart in place
nsbx validate feat                                    # prove the change is safe
nsbx promote feat --i-approve-prod-cutover            # gated rails cutover (see below)

The build replicates the engram release recipe exactly: elc engram/src/server.el > engram.c then cc -std=c11 -O2 -I lang/runtime engram.c el_runtime.c engram_*.c -lcurl -lpthread.

Lifecycle

op	what it does
create <name> [--port N] [--source|--branch|--binary]	consistent snapshot of the live store+WAL+config into an isolated dir; place or build the runtime; boot the real engram daemon on an isolated port. Named, versioned (binary sha + egm sha in manifest.json), reproducible.
up [name]	one command: create-if-missing then start; prints the URL.
build <name> --source|--branch	rebuild the runtime from a code change and hot-restart on the same clone+port.
run <name> <cmd…> / run <name> api <path> [json]	run an experiment against the real runtime; capture output + before/after stats + wall time. Env: $SBX_URL $SBX_PORT $SBX_KEY $SBX_DATA $SBX_BIN.
validate <name>	the rails as first-class checks (below).
promote <name> [--data] [--i-approve-prod-cutover]	the only prod-touching op. Gated rails cutover. DRY-RUN plan unless approved.
destroy <name>	stop the isolated daemon, free the port, remove the clone. Live untouched.
list / status <name>	inspect.

validate — the rails as checks

• zero-loss-under-load — node/edge counts hold at/above baseline through ~15s of sustained tick+read load
• reboot-prove — counts survive a real stop→start of the daemon
• rss-bound — daemon RSS under NSBX_RSS_BOUND_MB (default 550 MB, from the store-fix reboot-proof)
• retrieval-parity — top-k node ids for a fixed probe set match the create-time baseline
• keystone-integrity — kn-efeb4a5b… and kn-5b606390… present and intact

A PASS writes validate.json stamped with the binary sha; promote refuses unless the current binary has a fresh PASS on record.

promote — gated cutover (rails only)

Default is a dry-run plan. With --i-approve-prod-cutover it, in order:

1. snapshot-first — back up live egm+wal+plist to ~/.neuron/backups/promote-<name>-<ts>/ with a rollback.txt
2. additive binary install — copy the validated binary to a new file, update the plist ENGRAM_REAL_BIN (old binary retained — additive/supersede, never destructive)
3. rails cutover — launchctl bootout → settle-poll (prints until the job is gone) → launchctl bootstrap. Never pkill, never kickstart -k.
4. verify — /api/stats returns, edges ≥ baseline, keystones intact
5. auto-rollback armed — any verify failure restores the plist (and data, if --data) and boots the prior binary back via the same rails

Isolation guarantees

• separate port (8900+; refuses 8742/7770), separate store clone, separate process
• a hard guard refuses to boot a sandbox daemon whose data dir resolves to the live store
• sandboxes are plain supervised background processes (not launchd), so teardown is a signal + settle-poll — it can never touch the prod launchd job
• prod is read exactly twice: once for the snapshot, and (only if you approve) during promote

Layout

• tool: tools/neuron-sandbox/nsbx (this repo, branch feat/neuron-sandbox)
• runtime state: ~/.neuron/sandboxes/<name>/ — data/ (clone), bin/engram, build/, logs/, manifest.json, validate.json, baseline/

Validated (dogfood)

Standing up a sandbox from a live-store clone and reproducing a known result:

• retrieval-parity 25/25 top-k id overlap vs baseline; sandbox boot-stats exactly matched the live baseline captured at snapshot time (10 672 nodes / 32 439 edges) — the wrapped real binary faithfully reloads the live mind
• reboot-prove + zero-loss PASS; RSS 379 MB < 550 MB; keystones intact
• the cog-arch correspondence-loop re-run inside the sandbox reproduced the known calibration numbers exactly: held-Brier 0.028648 → 0.000586 (98.0% reduction), monotone, reboot bit-identical, metastability holds; and the real-store Stance persistence reboot-proved at 10 994-node scale (think() on real 768-dim embeddings) against a scratch copy of the sandbox's own clone — never live
• promote dry-run refused to touch prod; teardown freed the port; live :8742/:7770 never perturbed (soul uptime unbroken)

Migrating existing experiments

Each ad-hoc harness becomes nsbx run <name> … (or --source build) against a sandbox:

• cog-arch — nsbx create x --source <worktree> then nsbx run x -- bash cogarch_dogfood.sh (compiles + runs the real C cognition tests against $SBX_DATA)
• codec / ingest / faculty — nsbx run x api /api/<endpoint> '<json>' against the isolated daemon, or a script using $SBX_URL/$SBX_KEY; measure with the built-in before/after stats

Env knobs

NSBX_ROOT, NSBX_PORT_BASE, NSBX_RSS_BOUND_MB, NSBX_REMERGE_THRESHOLD, EL_REPO (for elc + runtime sources), ENGRAM_LIVE_DATA_DIR, ENGRAM_LIVE_PLIST.