bigmerge
112bb2540f
Generalise the ad-hoc cog-arch (worktree+build+store-clone+C-tests) and store-fix (secondary soul + launchctl rails cutover) proto-sandboxes into one reproducible primitive: run experiments and code changes against the REAL engram runtime on an isolated snapshot of the live mind, with a gated promote-to-prod path. Dev environment as a primitive — any team member gets a private, isolated copy of the mind (separate port/store/process); prod on :8742/:7770 is untouchable from a sandbox. Wraps the real binary; never reimplements engram logic. Lifecycle: create/up (consistent store+WAL+config snapshot; place OR build the runtime from --source/--branch/--binary; boot on an isolated port) · build · run · validate (rails as checks: zero-loss under load+reboot, reboot-prove, RSS bound, retrieval parity, keystone integrity) · promote (gated rails cutover: snapshot-first, additive binary swap, bootout→settle-poll→bootstrap, verify, auto-rollback; never pkill/kickstart -k; dry-run unless approved) · destroy. Dogfooded: reproduced retrieval-parity 25/25 vs baseline and the cog-arch correspondence-loop known result (Brier 0.028648->0.000586, reboot-proven) and real-store reboot-prove at 10994-node scale, all inside a sandbox; prod untouched.
119 lines
7.1 KiB
Markdown
119 lines
7.1 KiB
Markdown
# nsbx — the Neuron Sandbox
|
|
|
|
**Dev environment as a primitive.** A reproducible way to run experiments *and code
|
|
changes* against the **real** engram runtime on an isolated snapshot of the live
|
|
mind — with a gated promote-to-prod path built on the proven rails.
|
|
|
|
Everyone (Tim, any team member, any agent) gets their own private, safe copy of the
|
|
mind to build against. **Prod — the live Neuron on `:8742` (engram) / `:7770`
|
|
(soul) — is untouchable from a sandbox.** A sandbox runs a *separate* engram
|
|
process, on a *separate* port, against a *separate* clone of the store. The only op
|
|
that can ever reach prod is `promote`, which is explicit, gated, and per-use
|
|
approved.
|
|
|
|
It **wraps the real engram binary** — it never reimplements any engram logic. It
|
|
generalises two proven proto-sandboxes into one primitive:
|
|
|
|
- the **cog-arch** build — isolated git worktree + build + clone of the live `.egm` + real C tests
|
|
- the **store-fix** cutover — secondary soul + launchctl `bootout → settle → bootstrap` rails
|
|
|
|
## Quickstart
|
|
|
|
```bash
|
|
export PATH="$PWD:$PATH" # or symlink nsbx onto your PATH
|
|
|
|
nsbx up # your private copy of the mind (auto-named <user>-dev)
|
|
nsbx run <name> api /api/stats # poke it
|
|
nsbx validate <name> # prove it: zero-loss, reboot, RSS, retrieval, keystones
|
|
nsbx destroy <name> # cheap teardown; live untouched
|
|
```
|
|
|
|
That is the whole loop. Sane defaults: stock prod binary, auto-allocated port
|
|
(`8900+`, never `8742`/`7770`), snapshot of the live store.
|
|
|
|
## The code-change dev loop (first-class)
|
|
|
|
Run *your changed runtime*, not just the stock binary, against a snapshot:
|
|
|
|
```bash
|
|
# build a runtime from a working tree, a git branch, or a prebuilt binary:
|
|
nsbx create feat --source /path/to/worktree # elc + cc build from source
|
|
nsbx create feat --branch feat/my-change --repo <r> # worktree the branch, then build
|
|
nsbx create feat --binary /path/to/engram # use a prebuilt binary
|
|
|
|
nsbx build feat --source /path/to/worktree # rebuild + hot-restart in place
|
|
nsbx validate feat # prove the change is safe
|
|
nsbx promote feat --i-approve-prod-cutover # gated rails cutover (see below)
|
|
```
|
|
|
|
The build replicates the engram release recipe exactly:
|
|
`elc engram/src/server.el > engram.c` then
|
|
`cc -std=c11 -O2 -I lang/runtime engram.c el_runtime.c engram_*.c -lcurl -lpthread`.
|
|
|
|
## Lifecycle
|
|
|
|
| op | what it does |
|
|
|----|--------------|
|
|
| `create <name> [--port N] [--source\|--branch\|--binary]` | consistent snapshot of the live store+WAL+config into an isolated dir; place or **build** the runtime; boot the real engram daemon on an isolated port. Named, versioned (binary sha + egm sha in `manifest.json`), reproducible. |
|
|
| `up [name]` | one command: create-if-missing then start; prints the URL. |
|
|
| `build <name> --source\|--branch` | rebuild the runtime from a code change and hot-restart on the same clone+port. |
|
|
| `run <name> <cmd…>` / `run <name> api <path> [json]` | run an experiment against the real runtime; capture output + before/after stats + wall time. Env: `$SBX_URL $SBX_PORT $SBX_KEY $SBX_DATA $SBX_BIN`. |
|
|
| `validate <name>` | the rails as first-class checks (below). |
|
|
| `promote <name> [--data] [--i-approve-prod-cutover]` | **the only prod-touching op.** Gated rails cutover. DRY-RUN plan unless approved. |
|
|
| `destroy <name>` | stop the isolated daemon, free the port, remove the clone. Live untouched. |
|
|
| `list` / `status <name>` | inspect. |
|
|
|
|
## `validate` — the rails as checks
|
|
|
|
- **zero-loss-under-load** — node/edge counts hold at/above baseline through ~15s of sustained tick+read load
|
|
- **reboot-prove** — counts survive a real stop→start of the daemon
|
|
- **rss-bound** — daemon RSS under `NSBX_RSS_BOUND_MB` (default 550 MB, from the store-fix reboot-proof)
|
|
- **retrieval-parity** — top-k node ids for a fixed probe set match the create-time baseline
|
|
- **keystone-integrity** — `kn-efeb4a5b…` and `kn-5b606390…` present and intact
|
|
|
|
A PASS writes `validate.json` stamped with the binary sha; `promote` refuses unless
|
|
the current binary has a fresh PASS on record.
|
|
|
|
## `promote` — gated cutover (rails only)
|
|
|
|
Default is a **dry-run plan**. With `--i-approve-prod-cutover` it, in order:
|
|
|
|
1. **snapshot-first** — back up live `egm`+`wal`+`plist` to `~/.neuron/backups/promote-<name>-<ts>/` with a `rollback.txt`
|
|
2. **additive** binary install — copy the validated binary to a *new* file, update the plist `ENGRAM_REAL_BIN` (old binary retained — additive/supersede, never destructive)
|
|
3. **rails cutover** — `launchctl bootout` → **settle-poll** (prints until the job is gone) → `launchctl bootstrap`. Never `pkill`, never `kickstart -k`.
|
|
4. **verify** — `/api/stats` returns, edges ≥ baseline, keystones intact
|
|
5. **auto-rollback armed** — any verify failure restores the plist (and data, if `--data`) and boots the prior binary back via the same rails
|
|
|
|
## Isolation guarantees
|
|
|
|
- separate **port** (`8900+`; refuses `8742`/`7770`), separate **store clone**, separate **process**
|
|
- a hard guard refuses to boot a sandbox daemon whose data dir resolves to the live store
|
|
- sandboxes are plain supervised background processes (not launchd), so teardown is a signal + settle-poll — it can never touch the prod launchd job
|
|
- prod is read exactly twice: once for the snapshot, and (only if you approve) during `promote`
|
|
|
|
## Layout
|
|
|
|
- tool: `tools/neuron-sandbox/nsbx` (this repo, branch `feat/neuron-sandbox`)
|
|
- runtime state: `~/.neuron/sandboxes/<name>/` — `data/` (clone), `bin/engram`, `build/`, `logs/`, `manifest.json`, `validate.json`, `baseline/`
|
|
|
|
## Validated (dogfood)
|
|
|
|
Standing up a sandbox from a live-store clone and reproducing a **known** result:
|
|
|
|
- **retrieval-parity 25/25** top-k id overlap vs baseline; sandbox boot-stats exactly matched the live baseline captured at snapshot time (10 672 nodes / 32 439 edges) — the wrapped real binary faithfully reloads the live mind
|
|
- reboot-prove + zero-loss PASS; RSS 379 MB < 550 MB; keystones intact
|
|
- the **cog-arch correspondence-loop** re-run *inside* the sandbox reproduced the known calibration numbers exactly: held-Brier **0.028648 → 0.000586** (98.0% reduction), monotone, **reboot bit-identical**, metastability holds; and the real-store Stance persistence reboot-proved at **10 994-node** scale (`think()` on real 768-dim embeddings) against a scratch copy of the sandbox's own clone — never live
|
|
- `promote` dry-run refused to touch prod; teardown freed the port; live `:8742`/`:7770` never perturbed (soul uptime unbroken)
|
|
|
|
## Migrating existing experiments
|
|
|
|
Each ad-hoc harness becomes `nsbx run <name> …` (or `--source` build) against a sandbox:
|
|
|
|
- **cog-arch** — `nsbx create x --source <worktree>` then `nsbx run x -- bash cogarch_dogfood.sh` (compiles + runs the real C cognition tests against `$SBX_DATA`)
|
|
- **codec / ingest / faculty** — `nsbx run x api /api/<endpoint> '<json>'` against the isolated daemon, or a script using `$SBX_URL`/`$SBX_KEY`; measure with the built-in before/after stats
|
|
|
|
## Env knobs
|
|
|
|
`NSBX_ROOT`, `NSBX_PORT_BASE`, `NSBX_RSS_BOUND_MB`, `NSBX_REMERGE_THRESHOLD`,
|
|
`EL_REPO` (for `elc` + runtime sources), `ENGRAM_LIVE_DATA_DIR`, `ENGRAM_LIVE_PLIST`.
|