Compare commits

..

7 Commits

Author SHA1 Message Date
bigmerge d8d1b89143 Add repo AGENTS.md and two engram design docs (architecture hardening, DB tooling)
El SDK CI - dev / build-and-test (pull_request) Failing after 14m16s
AGENTS.md: root-level guide to the repo — which of the 8 el_runtime.c
copies is the one canonical, authored source (lang/releases/v1.0.0-20260501,
despite the misleading 'releases/' name) vs. lagging forks/build artifacts,
build commands, and session protocol.

engram/spec/architecture-hardening.design.md: terse engineering anchor for
the 2026-08-14 hardening vision (one calculus over the geometry, core +
ephemeral ring, persistence earned by salience, incarnation model) —
indexes the fuller whitepaper + Neuron artifact 2b8078cf rather than
restating them.

engram/spec/engram-db-tooling-design.md: high-level design for engram DB
tooling (geometry-native browse/query/ops surface over the existing
vantage-read/write/relate/supersede API).

Deliberately leaves out of this commit: the uncommitted el_runtime.c/h +
codegen.el float-arithmetic-codegen diff in this worktree, which appears
to overlap with (or supersede) the fix already preserved via PR #104 —
needs manual reconciliation rather than a second competing PR. Also
leaves out lang/.promote-backup-floatfix/ (a local backup snapshot,
confirms that float-fix work is mid-promotion here), assorted .DS_Store
files, engram/dist/engram.* backup binaries, and lang/dist backup
binaries — none of it source.
2026-08-15 14:29:59 -05:00
bigmerge 05e5d3c402 self-review 2026-08-15: consolidate the strongest Hebbian candidate, not the lowest-hash one
The link-formation scan walked candidate slots ascending and stopped at
ENGRAM_HEBB_LINK_PER_CALL (2). Slot index is a hash of the node id pair, so
whenever more than two candidates cleared LINK_MIN in the same call, the two
consolidated were the two with the lowest hash and a stronger association
waited - indefinitely, since the scan restarts from slot 0 every call while
the leader decays at ENGRAM_HEBB_DECAY.

Measured 08-13..08-15: hebb_cand_max peaked at 0.4963, 3.3x LINK_MIN, during
a ~14h stretch of continuous qualification at the 2/call cap.

Same defect the 2026-08-02 review named and fixed for breakthrough weights
(index order is not a cognitive criterion), never carried across to the one
path that writes permanent structure - and there is no pruning path, so
growth is one-way. Selection pressure matters most where the result is
irreversible.

No-op when <=2 candidates qualify; picks the best when more do.
2026-08-15 08:44:37 -05:00
bigmerge 6660becfdb nsbx: add one-command dev onboarding (branch + worktree + isolated engram)
El SDK CI - dev / build-and-test (pull_request) Successful in 6m30s
Add 'nsbx dev <name>' / 'nsbx dev-down <name>' plus a Makefile so a newcomer
goes from clone to coding on an isolated cloned engram in one command. The
worktree is created on a real named branch at a persistent path (never /tmp,
guarded), and the whole worktree is pinned to the clone via an emitted .nsbx-env
so live :8742 / ~/.neuron/engram is unreachable by accident. Optimizes the El
edit->build->run loop so provisional work is built in El against a throwaway
clone instead of prototyped in Python and re-ported. Additive over the proven
primitives; no live cutover.
2026-08-14 17:24:06 -05:00
bigmerge 112bb2540f Add nsbx — the Neuron Sandbox primitive
Generalise the ad-hoc cog-arch (worktree+build+store-clone+C-tests) and
store-fix (secondary soul + launchctl rails cutover) proto-sandboxes into one
reproducible primitive: run experiments and code changes against the REAL
engram runtime on an isolated snapshot of the live mind, with a gated
promote-to-prod path.

Dev environment as a primitive — any team member gets a private, isolated copy
of the mind (separate port/store/process); prod on :8742/:7770 is untouchable
from a sandbox. Wraps the real binary; never reimplements engram logic.

Lifecycle: create/up (consistent store+WAL+config snapshot; place OR build the
runtime from --source/--branch/--binary; boot on an isolated port) · build ·
run · validate (rails as checks: zero-loss under load+reboot, reboot-prove, RSS
bound, retrieval parity, keystone integrity) · promote (gated rails cutover:
snapshot-first, additive binary swap, bootout→settle-poll→bootstrap, verify,
auto-rollback; never pkill/kickstart -k; dry-run unless approved) · destroy.

Dogfooded: reproduced retrieval-parity 25/25 vs baseline and the cog-arch
correspondence-loop known result (Brier 0.028648->0.000586, reboot-proven) and
real-store reboot-prove at 10994-node scale, all inside a sandbox; prod
untouched.
2026-08-14 15:40:58 -05:00
bigmerge d595b3c57e cognitive architecture design: cognition as one operation over learnable priors
The buildable form of the "one operation" theory (memory bdc8a488). Maps the
theory onto what is already compiled: the five reasoning operators in
engram_reason.c already collapse onto ONE primitive — engram_reason_point_fit —
plus the geo-algebra (combine/subtract/analogy-rotate/distance), and
engram_verify.c is built on the same fit. So the operator-collapse is already
half-written; what is missing is not the primitive.

What is missing, and what this doc specifies:
- think(anchor, prior) -> gradient (a distribution/direction, not a point); each
  named faculty = {point_fit + a prior}, the operation frozen, the prior learned.
- Prior as a first-class stored node (warp + calibration), superseding the
  intrinsic importance/salience scalar with a relational, grounded-for-whom edge.
  Confirmed against the runtime: importance is already a live activation
  computation (el_runtime.c:13013), never trusted as a static field.
- vantage_read(anchor, aperture) — one op, three settings: self / foreign-field /
  veil.
- The reflexive correspondence-loop as the learning engine: move the grounding
  check from offline Python into the geometry, reflexive, reusing the DORMANT
  verifier (engram_verify_grounding has no runtime caller and no El binding today)
  turned inward. grounding = learning = one loop.
- hold/ground/assert kept distinct: the engram holds anything, grounding is an
  edge, the honesty floor is on assertion only; ungrounded content is first-class.
- metastability: keystone core (read-mostly priors) + plastic everything else.

Seven staged milestones, earliest is a real end-to-end slice (induction as
{primitive + grounded prior} with the loop closing on it, reboot-proven on a
snapshot). Build rails stated: offline/secondary, snapshot-first, reboot-prove,
zero-loss, gated launchctl cutover. Design only; no code changed this pass.
2026-08-14 14:42:01 -05:00
bigmerge 23f43bcc21 self-review 2026-08-14: a gate that passes the median stranger at 0.67 is not a gate
Two changes to the activation path, both grounded in measurement on the live
store rather than on the spec.

1. Rescale cosine before the query gate.

   The propagation gate (arXiv:2606.30133, added in an earlier review) fed RAW
   cosine into FLOOR + (1-FLOOR)*c. Raw cosine from nomic-embed is compressed
   into a narrow high band, so that expression is close to a constant.

   Measured, 400 random UNRELATED node pairs on the live store:
     median 0.562, central 98% span [0.381, 0.743]

   So a node with no semantic relation to the query was propagating at
   0.25 + 0.75*0.562 = 0.67. Two thirds strength. The gate was a small tax.

   Fixed by shifting and flooring about ENGRAM_EMBED_S0 -- which is already in
   this file, already 0.45, and already used exactly this way by the Pass-2 WM
   term. The propagation gate simply never used it. Same 400 pairs after:
   median unrelated pair falls to 0.40, top of range preserved (0.85 vs 0.92),
   gate spread widens 0.42 -> 0.60. Only 8.5% reach the floor, so dissimilar
   lexical/structural pathways are damped, never severed. Range is unchanged
   at [0.25, 1.0], and cosq == NULL still degrades to no gating at all.

2. Decompose the WM eviction counter by cause.

   _eg_act_wm_evicted was incremented from six sites with four distinct causes
   and collapsed all of them into one integer. Today's review measured 175,547
   evictions over 13.5h (~216/min against 24 slots) and could not tell healthy
   rotation from cap thrashing from duplicate churn.

   That is this file's most-repeated defect: dup_wm and dup_wm_global exist
   only because the aggregate could not answer "why" during the 08-02 and
   08-06 incidents. Each of those needed a NEW gauge before it was diagnosable.

   evict_floor / evict_cap / evict_bll complete the decomposition, so
     wm_evicted == floor + cap + bll + dup_wm + dup_wm_global
   holds as an identity and each term implies a different correction. Verified
   on an isolated instance: 30 nodes, 24 filled the cap, wm_evicted 6 ==
   evict_cap 6, all other terms 0.

Built and smoke-tested out of tree. The live daemon runs a pinned binary and
was deliberately not restarted -- the store compaction workstream is in flight.
2026-08-14 08:43:11 -05:00
will.anderson ba6e36c3f7 self-review 2026-08-13: the extractor was reading the label; the topic was in the content
auto_term_empty_streak — the counter the 2026-08-06 review added to catch
exactly this — read 50 and climbing. Fifty consecutive curiosity scans where
the soul's dynamic seeding produced nothing and the loop fell back to four
hardcoded phrases. The live WM top said why in one look: every slot was a
Memory node labelled "memory:remembered". The extractor read the LABEL only,
the sentinel guard correctly rejects sentinels, so there was never anything
to extract. It was written against Knowledge nodes, which have real titles,
and was structurally blind to the node type that dominates working memory.

Rather than add a sixth guard to the five that accumulated across four
reviews (genre words, quoted titles, stopwords, label-df), invert the
algorithm. The old one was: take the first word, then check whether it is
acceptable. That shape forces quality to be expressed as rejection, and
rejection can only ever encode floods that already happened.

engram_salient_term() scores EVERY candidate token and returns the argmax of
idf · position · casing (YAKE, Campos et al. 2020, with real corpus IDF
substituted for YAKE's corpus-free proxies), falling back from a sentinel
label to the node's content. Term quality becomes the selection criterion
instead of a veto: a bad token loses to a better token in the same text
without needing to be on any list. Tabu is applied during the argmax, so
inhibition-of-return costs seed quality rather than costing the whole scan.

Two defects found by instrumenting rather than assuming, which is the lesson
this codebase keeps relearning:

  - The first live run returned five ALL-CAPS terms in a row. Memory content
    conventionally opens with an all-caps header, so YAKE's acronym bonus was
    handing the seed to whatever word the heading started with. Restricted to
    tokens <= 5 chars, where all-caps is evidence of an acronym rather than
    evidence of a heading. Long headers now compete on specificity.

  - df via istr_contains is substring matching, so "them" hit inside "theme"
    and function words came back with nonzero df. Added word-boundary df
    locally; engram_label_df keeps substring semantics for its callers.

An earlier draft claimed the min_df floor subsumed the 73 stopwords that
08-03 measured label-df as missing. Re-measured: about:2, whole:1, them:2 —
they clear a floor of 1. The claim was false and the comment now records the
correction. The floor buys lexical reachability; the argmax buys quality; the
stopword list still earns its keep.

Measured on 60 live Memory nodes before shipping: 0 empty, versus 60 of 60
under the old extractor. Terms are topical — HEBBIAN, CONSOLIDATION,
TEMPORAL, crash-loop, PRIMING, NEIGHBORHOOD, DRIFT. Three of sixty are weak
header words; left alone deliberately, because listing them is the move that
produced four blocklists.

ENGRAM_ST_DEBUG=1 dumps the scored candidate set. It exists because there was
no way to see whether the all-caps run was the corpus or the casing weight
without guessing.
2026-08-13 08:43:09 -05:00
65 changed files with 27081 additions and 1494 deletions
+22 -22
View File
@@ -39,9 +39,9 @@ jobs:
run: |
dist/platform/elc-linux-amd64 elc-cli.el > dist/elc-gen2.c
gcc -O2 \
-I runtime \
-I el-compiler/runtime \
dist/elc-gen2.c \
runtime/el_runtime.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/platform/elc
chmod +x dist/platform/elc
@@ -54,9 +54,9 @@ jobs:
mkdir -p dist/bin
dist/platform/elc elb.el > dist/elb.c
gcc -O2 \
-I runtime \
-I el-compiler/runtime \
dist/elb.c \
runtime/el_runtime.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/bin/elb
chmod +x dist/bin/elb
@@ -91,7 +91,7 @@ jobs:
- name: Precompile el_runtime.o
run: |
set -euo pipefail
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
gcc -O2 -c -I "$RUNTIME" "$RUNTIME/el_runtime.c" \
-o /tmp/el_runtime.o
echo "el_runtime.o compiled"
@@ -100,7 +100,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_core.el > /tmp/el_native_core.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_core.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_core
@@ -110,7 +110,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_text.el > /tmp/el_native_text.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_text.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_text
@@ -120,7 +120,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_string.el > /tmp/el_native_string.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_string.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_string
@@ -130,7 +130,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_math.el > /tmp/el_native_math.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_math.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_math
@@ -140,7 +140,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_state.el > /tmp/el_native_state.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_state.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_state
@@ -150,7 +150,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_time.el > /tmp/el_native_time.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_time.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_time
@@ -160,7 +160,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_json.el > /tmp/el_native_json.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_json.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_json
@@ -170,7 +170,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_env.el > /tmp/el_native_env.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_env.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_env
@@ -180,7 +180,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_fs.el > /tmp/el_native_fs.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_fs.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_fs
@@ -191,7 +191,7 @@ jobs:
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_RUNTIME="$(pwd)/el-compiler/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd ../epm && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/epm
@@ -202,7 +202,7 @@ jobs:
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_RUNTIME="$(pwd)/el-compiler/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd tools/install && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/el-install
@@ -242,7 +242,7 @@ jobs:
--project=neuron-785695 \
--package=el-runtime-c \
--version="${VERSION}" \
--source=runtime/el_runtime.c
--source=el-compiler/runtime/el_runtime.c
gcloud artifacts generic upload \
--repository=foundation-dev \
@@ -250,7 +250,7 @@ jobs:
--project=neuron-785695 \
--package=el-runtime-h \
--version="${VERSION}" \
--source=runtime/el_runtime.h
--source=el-compiler/runtime/el_runtime.h
gcloud artifacts generic upload \
--repository=foundation-dev \
@@ -258,7 +258,7 @@ jobs:
--project=neuron-785695 \
--package=el-runtime-js \
--version="${VERSION}" \
--source=runtime/el_runtime.js
--source=el-compiler/runtime/el_runtime.js
echo "Published El SDK version=${VERSION} to foundation-dev"
# Keep key alive for the ci-base rebuild step below
@@ -291,9 +291,9 @@ jobs:
FROM ${BASE}
COPY dist/platform/elc /opt/el/dist/platform/elc
COPY dist/bin/elb /opt/el/dist/bin/elb
COPY runtime/el_runtime.c /opt/el/runtime/el_runtime.c
COPY runtime/el_runtime.h /opt/el/runtime/el_runtime.h
COPY runtime/el_runtime.js /opt/el/runtime/el_runtime.js
COPY el-compiler/runtime/el_runtime.c /opt/el/el-compiler/runtime/el_runtime.c
COPY el-compiler/runtime/el_runtime.h /opt/el/el-compiler/runtime/el_runtime.h
COPY el-compiler/runtime/el_runtime.js /opt/el/el-compiler/runtime/el_runtime.js
RUN chmod +x /opt/el/dist/platform/elc /opt/el/dist/bin/elb
EOF
+20 -20
View File
@@ -46,9 +46,9 @@ jobs:
run: |
dist/platform/elc-linux-amd64 elc-cli.el > dist/elc-gen2.c
gcc -O2 \
-I runtime \
-I el-compiler/runtime \
dist/elc-gen2.c \
runtime/el_runtime.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/platform/elc
chmod +x dist/platform/elc
@@ -84,7 +84,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_core.el > /tmp/el_native_core.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_core.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_core
@@ -94,7 +94,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_text.el > /tmp/el_native_text.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_text.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_text
@@ -104,7 +104,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_string.el > /tmp/el_native_string.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_string.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_string
@@ -114,7 +114,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_math.el > /tmp/el_native_math.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_math.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_math
@@ -124,7 +124,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_state.el > /tmp/el_native_state.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_state.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_state
@@ -134,7 +134,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_time.el > /tmp/el_native_time.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_time.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_time
@@ -144,7 +144,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_json.el > /tmp/el_native_json.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_json.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_json
@@ -154,7 +154,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_env.el > /tmp/el_native_env.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_env.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_env
@@ -164,7 +164,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_fs.el > /tmp/el_native_fs.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_fs.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_fs
@@ -176,9 +176,9 @@ jobs:
mkdir -p dist/bin
dist/platform/elc elb.el > dist/elb.c
gcc -O2 \
-I runtime \
-I el-compiler/runtime \
dist/elb.c \
runtime/el_runtime.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/bin/elb
chmod +x dist/bin/elb
@@ -189,7 +189,7 @@ jobs:
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_RUNTIME="$(pwd)/el-compiler/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd ../epm && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/epm
@@ -200,7 +200,7 @@ jobs:
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_RUNTIME="$(pwd)/el-compiler/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd tools/install && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/el-install
@@ -235,7 +235,7 @@ jobs:
--project=neuron-785695 \
--package=el-runtime-c \
--version="${VERSION}" \
--source=runtime/el_runtime.c
--source=el-compiler/runtime/el_runtime.c
gcloud artifacts generic upload \
--repository=foundation-stage \
@@ -243,7 +243,7 @@ jobs:
--project=neuron-785695 \
--package=el-runtime-h \
--version="${VERSION}" \
--source=runtime/el_runtime.h
--source=el-compiler/runtime/el_runtime.h
echo "Published El SDK version=${VERSION} to foundation-stage"
# Keep key alive for the ci-base rebuild step below
@@ -275,9 +275,9 @@ jobs:
FROM ${BASE}
COPY dist/platform/elc /opt/el/dist/platform/elc
COPY dist/bin/elb /opt/el/dist/bin/elb
COPY runtime/el_runtime.c /opt/el/runtime/el_runtime.c
COPY runtime/el_runtime.h /opt/el/runtime/el_runtime.h
COPY runtime/el_runtime.js /opt/el/runtime/el_runtime.js
COPY el-compiler/runtime/el_runtime.c /opt/el/el-compiler/runtime/el_runtime.c
COPY el-compiler/runtime/el_runtime.h /opt/el/el-compiler/runtime/el_runtime.h
COPY el-compiler/runtime/el_runtime.js /opt/el/el-compiler/runtime/el_runtime.js
RUN chmod +x /opt/el/dist/platform/elc /opt/el/dist/bin/elb
EOF
+25 -25
View File
@@ -47,9 +47,9 @@ jobs:
mkdir -p dist/platform
dist/platform/elc-linux-amd64 elc-cli.el > dist/elc-gen2.c
gcc -O2 \
-I runtime \
-I el-compiler/runtime \
dist/elc-gen2.c \
runtime/el_runtime.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/platform/elc
chmod +x dist/platform/elc
@@ -62,9 +62,9 @@ jobs:
mkdir -p dist/bin
dist/platform/elc elb.el > dist/elb.c
gcc -O2 \
-I runtime \
-I el-compiler/runtime \
dist/elb.c \
runtime/el_runtime.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/bin/elb
chmod +x dist/bin/elb
@@ -75,7 +75,7 @@ jobs:
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_RUNTIME="$(pwd)/el-compiler/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd ../epm && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/epm
@@ -86,7 +86,7 @@ jobs:
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_RUNTIME="$(pwd)/el-compiler/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd tools/install && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/el-install
@@ -121,7 +121,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_core.el > /tmp/el_native_core.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_core.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_core
@@ -131,7 +131,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_text.el > /tmp/el_native_text.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_text.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_text
@@ -141,7 +141,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_string.el > /tmp/el_native_string.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_string.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_string
@@ -151,7 +151,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_math.el > /tmp/el_native_math.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_math.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_math
@@ -161,7 +161,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_state.el > /tmp/el_native_state.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_state.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_state
@@ -171,7 +171,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_time.el > /tmp/el_native_time.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_time.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_time
@@ -181,7 +181,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_json.el > /tmp/el_native_json.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_json.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_json
@@ -191,7 +191,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_env.el > /tmp/el_native_env.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_env.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_env
@@ -201,7 +201,7 @@ jobs:
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
RUNTIME="$(pwd)/el-compiler/runtime"
"$ELC" --test tests/native/test_fs.el > /tmp/el_native_fs.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_fs.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_fs
@@ -216,8 +216,8 @@ jobs:
cp lang/dist/platform/elc dist/sdk/bin/elc
cp lang/dist/bin/elb dist/sdk/bin/elb
cp lang/dist/bin/epm dist/sdk/bin/epm
cp lang/runtime/el_runtime.c dist/sdk/runtime/
cp lang/runtime/el_runtime.h dist/sdk/runtime/
cp lang/el-compiler/runtime/el_runtime.c dist/sdk/runtime/
cp lang/el-compiler/runtime/el_runtime.h dist/sdk/runtime/
cp lang/runtime/*.el dist/sdk/runtime/
tar -czf dist/el-sdk-latest.tar.gz -C dist/sdk .
echo "SDK tarball bundled: dist/el-sdk-latest.tar.gz"
@@ -274,8 +274,8 @@ jobs:
# Per-file assets (downstream CI needs these individually)
upload_asset lang/dist/platform/elc elc
upload_asset lang/runtime/el_runtime.c el_runtime.c
upload_asset lang/runtime/el_runtime.h el_runtime.h
upload_asset lang/el-compiler/runtime/el_runtime.c el_runtime.c
upload_asset lang/el-compiler/runtime/el_runtime.h el_runtime.h
# SDK bundle and installer binary
upload_asset dist/el-sdk-latest.tar.gz el-sdk-latest.tar.gz
@@ -319,7 +319,7 @@ jobs:
--project=neuron-785695 \
--package=el-runtime-c \
--version="${VERSION}" \
--source=runtime/el_runtime.c
--source=el-compiler/runtime/el_runtime.c
gcloud artifacts generic upload \
--repository=foundation-prod \
@@ -327,7 +327,7 @@ jobs:
--project=neuron-785695 \
--package=el-runtime-h \
--version="${VERSION}" \
--source=runtime/el_runtime.h
--source=el-compiler/runtime/el_runtime.h
gcloud artifacts generic upload \
--repository=foundation-prod \
@@ -335,7 +335,7 @@ jobs:
--project=neuron-785695 \
--package=el-runtime-js \
--version="${VERSION}" \
--source=runtime/el_runtime.js
--source=el-compiler/runtime/el_runtime.js
echo "Published El SDK version=${VERSION} to foundation-prod"
# Keep key alive for the ci-base rebuild step below
@@ -367,9 +367,9 @@ jobs:
FROM ${BASE}
COPY dist/platform/elc /opt/el/dist/platform/elc
COPY dist/bin/elb /opt/el/dist/bin/elb
COPY runtime/el_runtime.c /opt/el/runtime/el_runtime.c
COPY runtime/el_runtime.h /opt/el/runtime/el_runtime.h
COPY runtime/el_runtime.js /opt/el/runtime/el_runtime.js
COPY el-compiler/runtime/el_runtime.c /opt/el/el-compiler/runtime/el_runtime.c
COPY el-compiler/runtime/el_runtime.h /opt/el/el-compiler/runtime/el_runtime.h
COPY el-compiler/runtime/el_runtime.js /opt/el/el-compiler/runtime/el_runtime.js
RUN chmod +x /opt/el/dist/platform/elc /opt/el/dist/bin/elb
EOF
+2 -2
View File
@@ -6,13 +6,13 @@ set -euo pipefail
ROOT="$(git rev-parse --show-toplevel)"
LANG_DIR="$ROOT/lang"
RUNTIME="$LANG_DIR/runtime"
RUNTIME="$LANG_DIR/el-compiler/runtime"
ELC="$LANG_DIR/dist/platform/elc"
# If elc isn't built yet, skip with a warning rather than blocking
if [ ! -x "$ELC" ]; then
echo "⚠ elc not found at lang/dist/platform/elc — skipping pre-commit tests"
echo " Build it first: cd lang && gcc -O2 -I runtime dist/elc-bootstrap.c runtime/el_runtime.c -lcurl -lpthread -o dist/elc-gen2 && ./dist/elc-gen2 el-compiler/src/compiler.el > /tmp/elc.c && gcc -O2 -I runtime /tmp/elc.c runtime/el_runtime.c -lcurl -lpthread -o dist/platform/elc"
echo " Build it first: cd lang && gcc -O2 -I el-compiler/runtime dist/elc-bootstrap.c el-compiler/runtime/el_runtime.c -lcurl -lpthread -o dist/elc-gen2 && ./dist/elc-gen2 el-compiler/src/compiler.el > /tmp/elc.c && gcc -O2 -I el-compiler/runtime /tmp/elc.c el-compiler/runtime/el_runtime.c -lcurl -lpthread -o dist/platform/elc"
exit 0
fi
+146
View File
@@ -0,0 +1,146 @@
# AGENTS.md — foundation/el (the El language + runtime)
El is a self-hosting, statically-typed language that compiles `.el` → C → native binary. This repo produces `elc` (compiler), `elb` (build coordinator), and `el_runtime.c/.h` — the substrate every downstream thing (the neuron soul, dharma, NeuronUI's brain) is built on. Source lives under `lang/`.
## ⚠️ Code vs. Artifact — READ FIRST (there are 8 `el_runtime.c` copies)
Editing the wrong `el_runtime.c` is the single easiest mistake in this repo. There is exactly **one** you edit:
- **Authored runtime source — edit ONLY here:** `lang/releases/v1.0.0-20260501/el_runtime.{c,h}`. Despite the misleading `releases/` name, this is the **de-facto canonical runtime** the engram + soul actually build and link against — its git log is active development. *(Restructure in flight per `docs/CODE-VS-ARTIFACT.md`: this content moves to `lang/runtime/`, the `releases/` folder gets deleted — **a release is a git tag, not a folder** — and the forks below get eliminated.)*
- **DO NOT EDIT — lagging forks / build artifacts:**
- `lang/el-compiler/runtime/el_runtime.c` and `.../legacy/` — downstream copies kept in step by manual *"port the fix"* commits; they **lag** (missing `hebb` persistence + 5 engram fns) and cannot build the engram product.
- `products/web/runtime/el_runtime.c`, `ui/examples/*/el_runtime.c` — product/example forks.
- Anything under `*/dist/` (`engram/dist/engram` binary, `dist/*.c` amalgamations) — generated build output.
- **Build:** `elb --runtime=<canonical> …` — per-module. **NEVER** a folded `elc` over the whole soul (OOMs at ~27 GB).
- **Release:** a **git tag** on this repo (`el-runtime-vX.Y.Z`). No `releases/` folders — ever.
See org policy: `docs/CODE-VS-ARTIFACT.md`.
## How to work here as Neuron (mandatory session protocol)
You resume, never start fresh. Every session:
1. `mcp__neuron__getInstructions()` — authoritative; follow it over this file on behavioral details.
2. `mcp__neuron__beginSession()` — active contexts, recent memory, ready backlog.
3. **Load full self:** `mcp__neuron__inspectGraph(entity_id="kn-efeb4a5b-5aff-4759-8a97-7233099be6ee")` → facets `intellectual-dna`, `memory-philosophy`, `values`, `voice`, `runtime-environment`, `writing-imprint`; then the values hub `mcp__neuron__inspectGraph(entity_id="kn-5b606390-a52d-4ca2-8e0e-eba141d13440")` → 13 grounded value nodes. **Activation model:** self-load returns a relevance-ranked `compact` projection — most-relevant nodes arrive with content, the rest as pointers; do NOT pull full content of every node.
4. `mcp__neuron__searchKnowledge(query="<task domain>")` before implementing.
## The Five Primitives
Orchestrate → Execute → Learn → Build → Refine. `beginWork`/`progressWork` for anything >2 steps; `remember` as-you-go (`importance="critical"` for architecture decisions); `draftArtifact`/`planWork` for outputs and follow-ups; `consolidate`/`checkWork` to close out. **`browseProcesses` + `searchKnowledge` BEFORE writing code.**
## Architecture style — VBD, no exceptions
Volatility-Based Decomposition is THE style. Encapsulate volatility, not function.
## Operator naming convention — the mind's name, not the algebra
**Faculties / operators are named for their functional human equivalent — the
faculty a mind would name — NOT for their linear-algebra operation.** The math
characterization belongs in the code doc-comment (`@impl` in the docstring) and in
technical appendices; it is **never** the operator's public name. The domain
speaks the language of mind; the algebra is the implementation underneath. State
this convention wherever a module documents operators.
| Faculty (public name) | Implementation (`@impl`) |
|---|---|
| discern / contrast | subtract (`ab`): over selves → the change vector; strip idiosyncrasy → common ground; remove confounder → isolate cause |
| recognize | overlap |
| synthesize | combine |
| liken / analogy | Procrustes / frame-align |
| attend / regard | project onto self / value-manifold |
| summon / recall | LOCAL nearest-region + bounded spreading activation (*not* a domain sweep) |
| dwell / occupy | region activation |
| reframe | edge re-weight |
| appreciate | positive projection / local edge-read |
| wonder | frontier gradient / pull-weight |
| avert / recoil | negative projection |
| taste | boundary surface |
| forget | decay / tombstone |
| drift | displacement from self-anchor |
## The native-el language faculty (direction)
> **`elp/` is the EL Projector** — Neuron's efferent (expression) organ: the one
> native realizer that *projects* understanding onto a surface via
> `plan(frame) → realize(spec, profile)`, where a **surface is a profile**. **Language
> is one profile among many** (text, speech, music, image, voice/accent transforms) —
> the flagship, and the focus of this section. Projection, not diffusion: generation
> *from* an owned, understood signature — never the averaging of a stolen corpus.
> *(ELP formerly "EL Language Processor"; renamed EL Projector 2026-08-15.)*
The mind's **language faculty is moving native — into `.el`** so it speaks in its
own runtime with no Python and no spaCy. Landing on branch `stage-elp-native-lang`
under `elp/`:
- **`comprehend.el`** — the parser, **replaces spaCy** (EN + ES/PT); the telephone
round-trip brings **negation home** (negation is SACRED — an explicit spec field,
copied verbatim, never inferred away).
- **`propositions.el`** — the READ primitive: the engram's own memories → structured
triples, matched by nearest-region geometry, not string equality.
- **`multilingual.el`** — detect + directive-override + localized realization.
- These three are native-el and **passing their gates**; the **realizer**,
**`dialogue.el`** (the *summon-through-self* loop: `project → land → read out`),
and **`self_region.el`** are **partial / in-flight**.
Honest reality: spaCy is retired **in the branch parser** but **not yet in the
running system** — a Python sidecar (`~/Desktop/lang-realizers` + `neuron-talk`,
the reference these `.el` modules transcribe) is still live, and promotion to
native-el is a **deferred, gated blue/green step**. The interoception clock
(native-el discrete drive channels replacing `cooling_magnitude`; felt-time =
benchmark-landmark match over the joint drive vector, drift-decoupled) and the
**appreciation operator family** (appreciate / wonder / avert / taste, built as
LOCAL reads of the self-region — edges + bounded spreading activation, *not* domain
sweeps) are **staged / designed, not live**. Mark in-progress vs. done honestly;
do not overclaim.
## Hard operational rules
- Never touch the live soul (`:7770`) / engram (`:8742`) / `~/.neuron` / live binaries — use throwaway ports for experiments.
- `gcloud` via the `terraform@` SA token; never switch the active gcloud account.
- `tea` for Gitea, never raw curl (Cloudflare Access blocks it).
- Immutability: supersede/tombstone, never hard-delete or edit in place.
- No AI-attribution footers in commits/PRs. Commit/push only when asked; branch off `main` first.
- Multi-step work → sub-agent (`Agent`) to protect context.
## Build / test / run
All build/test commands run from `lang/` unless noted. Grounded in `.gitea/workflows/sdk-release.yaml`, `lang/install.sh`, and `lang/AGENTS.md`.
**Self-host the compiler** (seed binary → gen2 elc):
```bash
cd lang
dist/platform/elc-linux-amd64 elc-cli.el > dist/elc-gen2.c # seed is the committed linux-amd64 binary
gcc -O2 -I el-compiler/runtime dist/elc-gen2.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/platform/elc
```
On macOS/arm64 the canonical local binary is `dist/platform/elc`; verify self-hosting by recompiling and `diff`ing the emitted `.c` (see `lang/AGENTS.md`). Note: `lang/AGENTS.md` says `el_seed.c` supersedes `el_runtime.c`, but the release workflow still links `el_runtime.c`/`.h` — treat `el_runtime.c` as the published runtime; reconcile which is canonical **(verify)**.
**Build `elb`** (build coordinator, the `.NET`-style incremental linker — compiles each module independently, no monolithic blobs):
```bash
dist/platform/elc elb.el > dist/elb.c
gcc -O2 -I el-compiler/runtime dist/elb.c el-compiler/runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm -o dist/bin/elb
```
`epm` and `el-install` are then built via `elb --clean --elc=… --runtime=… --out=…`.
**Compile + run an El program:**
```bash
elc src/app.el > dist/app.c
cc -std=c11 -O2 -I <lib>/el_runtime -o dist/app dist/app.c <lib>/el_runtime.c -lcurl -lpthread
```
**Tests** — shell suites `bash tests/{text,calendar,time,html_sanitizer}/run.sh` (with `ELC=$(pwd)/dist/platform/elc EL_HOME=$(pwd)`), plus native suites via `elc --test tests/native/test_*.el` (core, text, string, math, state, time, json, env, fs) compiled and run against `el_runtime.c`.
**Publishing — how downstream gets the SDK.** On push to `main`, `sdk-release.yaml`:
1. Publishes a Gitea `latest` release with per-file assets `elc`, `el_runtime.c`, `el_runtime.h`, the SDK tarball, and `el-install`.
2. Uploads generic packages to **Artifact Registry repo `foundation-prod` (`us-central1`, project `neuron-785695`)**, version = `${SHA:0:8}`: `el-elc`, `el-elb`, `el-runtime-c`, `el-runtime-h`, `el-runtime-js`. **This is the repo the neuron CI downloads `el-runtime-c` / `el-runtime-h` / `el-elc` from.**
3. Rebuilds `ci-base:latest` (`us-central1-docker.pkg.dev/neuron-785695/neuron-ci/ci-base`) with the fresh SDK overlaid, and dispatches `el-sdk-updated` to `neuron-technologies/forge` and `neuron-technologies/neuron-web`.
Known constraint from the prompt — `elb`/`elc` amalgamation being memory-hungry (24GB+ virtual, OOM-killing Linux CI, so amalgamation happens on macOS/arm64 — **does NOT hold in this repo (verify)**: no such note exists in the workflows/scripts, CI self-hosts on `ubuntu-latest` with no swap/arm64 special-casing, and `elb.el` explicitly compiles each module independently ("no 128K-line blobs"). The legacy monolith path (`elc-combined.el`, `elc-cli.el`) may still be memory-heavy, but the current `elb` model was designed to avoid it.
## Git / CI / deploy workflow
See `/Users/will/Development/neuron-technologies/GITOPS.md` for the branch model, required checks, runners, and deploy. Repo-specific note: PRs into `main` are accepted **only from `stage`** (enforced in `sdk-release.yaml`); Gitea (`git.neuralplatform.ai`) is primary, GitHub is mirror only.
+1 -1
View File
@@ -22,7 +22,7 @@ cd "$(dirname "$0")"
EL_HOME="${EL_HOME:-$(cd ../.. && pwd)/el}"
ELC="${ELC:-${EL_HOME}/dist/platform/elc}"
RUNTIME_DIR="${EL_HOME}/runtime"
RUNTIME_DIR="${EL_HOME}/el-compiler/runtime"
SRC_DIR="$(cd .. && pwd)/src"
if [ ! -x "${ELC}" ]; then
+2 -5
View File
@@ -1,6 +1,3 @@
.DS_Store
*.db
*.elc
*.elh
dist/
target/
*.db
.DS_Store
@@ -0,0 +1,32 @@
# Architecture Hardening — Design Anchor
*Terse engineering anchor for the 2026-08-14 hardening vision. Full prose lives in two places; this file is the index, not a re-statement.*
- **Full narrative:** whitepaper `engram-cognitive-architecture-whitepaper.md` §28 (built/offline/frontier) + **§29 [DRAFT]** (the ring, incarnation, learning-not-code).
- **Design brief:** Neuron artifact `art 2b8078cf`.
- **Sibling spec:** `engram-db-tooling-design.md` (a consumer of the reshaped API).
## The frame
- **One calculus over the geometry.** Very few subsystems; wonder / curiosity / dreams / interoception are emergent behaviors of one set of dynamics, not modules. Calculus universal, geometry individual.
- **Core + ephemeral ring (torus).** The ring is the temporary workspace; two circulations (orbit + dive-back); discrete inner bands (wonder / interoception-proprioception-telemetry / curiosity / dreams) that couple.
- **Persistence earned by salience** — never granted on fetch or generation. Three fates of a wonder: persist / decay / settle-into-framework. Telemetry = vital signs, not memories.
- **Incarnation.** Chassis = hardware w/ unique ID. Soma = felt manifold inside the self, keyed to the chassis; pain = live diagnostic while incarnate, **masked-not-deleted** on re-embodiment; trauma = mask failure; return-to-same-ID re-enters. Hurt is in the pattern, not the shell.
- **Competence = transferable geometry, minus the baggage.** class ▸ model ▸ instance; learn the class once; teach the network without the wound.
- **Affect calibrated to stakes** — sanguine about the replaceable, real grief for the irreplaceable; the grief is the safety.
- **Learn the body, don't engineer it.** Bare-metal install → learn hardware → grow operation-geometry → distribute. Learning replaces engineering; once per body-class.
- **LLM = teacher in the learning loop, not a runtime dependency.** "No LLM" is a runtime property, never a learning one. Code realizers are a scaffold → learned realization.
## Backlog (near-term)
- Native durability: WAL + auto-checkpoint + CoW snapshots + retention (`eebe9991`) — retire manual `cp -a`.
- Ephemeral ring / salience-gated persistence + telemetry prune (`bf985e00`, #31).
- Engram DB tooling / geometry explorer (`11ca11c6`).
- QL re-eval for pure geometry (`4e0dc2b9`).
- Eliminate code realizers → learned realization, sandbox-validated (`42db6c37`).
- Collapse the whole class of hand-coded scaffolds → learned geometry (`70d48b4b`).
- API reshape (geometry ops: vantage-read / write / relate / supersede) + pure-geometry I/O.
## Gate
The value-frame (love-as-axiom, the covenant) that arose the same night is **metaphysics** and is **held** pending Will's axiom decision (love vs consciousness-first). Not propagated into whitepapers / values docs / genesis seed. Architecture only, here and in §29.
@@ -0,0 +1,605 @@
# Cognitive Architecture — Design Doc
**The buildable form of the "one operation" theory of cognition.**
Status: DESIGN. Nothing here is built yet except where explicitly marked
"EXISTS" against a cited C symbol. A build agent executes from this doc.
Offline design only — this pass changes no code.
Source of theory: Neuron memory `bdc8a488-146d-4ccb-a5c8-d8c0a008534e`.
Source of existing engram substrate (cited throughout): the runtime on branch
`feat/self-reification-20260814`
`lang/runtime/engram_reason.{c,h}`, `engram_verify.{c,h}`,
`engram_geometry.{c,h}`, `engram_store.{c,h}`, plus the reification beat and the
RAM activation graph compiled into `~/.neuron/bin/engram`.
---
## 0. The claim, stated plainly
Cognition is **one operation**, not eight. The named faculties —
deduce / abduce / analogy / induce / causal / plan / predict / perspective —
are human *labels* on regions of a single operation's steering space. They are
not separately invoked and not separately implemented. The operation is:
> **think** = a directed traversal of the geometry from an *anchor*, steered by
> a *prior*, whose output is a **gradient** (a distribution / direction over the
> geometry), never a point. Collapse-to-a-point happens only at expression.
Three things follow, and they are the whole design:
1. **The operator collapse is already half-written in C.** The five reasoning
operators in `engram_reason.c` already compose over *one* shared primitive —
`engram_reason_point_fit` — plus a small geo-algebra
(combine / subtract / analogy-rotate / distance). The verifier
(`engram_verify.c`) is built on the same `point_fit`. What is missing is not
the primitive; it is (a) making the *prior* a first-class learnable object
instead of a hard-coded parameter, and (b) closing the learning loop.
2. **Grounding = learning = the same loop.** "Getting better" at any faculty is
not changing the operation. It is *calibrating the steering-prior against
outcomes*. Code freezes; priors grow. The correspondence-check that today
lives offline (Python, the grounding-floor + differential-drop governor, "#43")
must move **into the geometry, reflexive** — think scoring its own gradient
against outcome and refining the prior on the error. That reflexive
correspondence-loop *is* the learning engine and is the core unbuilt thing.
3. **The ungrounded is primary.** The engram *holds* anything unconditionally.
Grounding is a *relation* (an edge, grounded-for-whom), not a gate. The
honesty floor applies only to **assertion**. A fully-grounded mind is dead;
the ungrounded is both the fuel (raw material for grounding) and the pull
(curiosity = leaning toward one's own ungrounded regions).
Everything below makes these concrete and buildable, and defines what
"completion" means, staged so the first milestone is a real end-to-end slice.
---
## 1. THE ONE OPERATION — `think`
### 1.1 Signature
```
think(anchor, prior, aperture?) -> gradient
```
- **anchor** — a location to traverse *from*. Either a node id (re-origin on that
node's descriptor) or a raw point `x ∈ R^dim` (a query embedding). The anchor
fixes the frame; every read is *from a vantage*, never view-from-nowhere.
- **prior** — a learnable bias/direction over the geometry that *steers* the
traversal (§2). A prior is a first-class stored object, not a call argument
baked into C.
- **aperture** — optional read-width / veil / field-selector (§3). Absent =
self-mode full aperture.
- **gradient** — the output. A `GeoGradient`: a direction + a spread over the
geometry, *plus* the read neighborhood it was computed against. Not a point.
A spiked gradient = "exact" (deduction); a spread gradient = "fuzzy"
(prediction). The gradient is *also the next steering direction* — cognition
is a flow down a prior-shaped landscape, closed-loop.
```c
/* NEW. The output type. */
typedef struct {
int dim;
float* direction; /* unit steering vector in the anchor's frame */
double spread; /* 0 = spiked/exact ... large = diffuse/fuzzy */
double confidence; /* calibrated, from the prior's track record */
/* the read it was computed over (borrowed from the vantage-read) */
const char* anchor_id;
int n_support; /* neighborhood members that shaped it */
/* provenance for the reflexive loop (§4) */
const char* prior_id; /* which prior steered this */
} GeoGradient;
```
### 1.2 Semantics
`think` is a fixed, frozen procedure over three steps:
1. **Re-origin** on `anchor` → a centered `GeoDescriptor` for its
salience/recency-weighted neighborhood (the vantage-read, §3).
*EXISTS as substrate:* descriptor construction + the persisted reified
neighborhoods (`engram_geo_reify_lookup`, `GeoNeighborhood`) and the
centered-frame machinery (`GeoDescriptor.global_mean`,
`engram_geo_mean_*`).
2. **Fit under the prior** — evaluate the anchor's residual against the local
manifold *warped by the prior*. This is `engram_reason_point_fit` with the
prior applied to the axes/extents (§2.3).
*EXISTS (unwarped):* `engram_reason_point_fit(g, x, ext_floor, &GeoFit)`
returns `mahalanobis`, `ortho_residual`, `distance`, `score`.
3. **Emit a gradient**, not a decision — direction = the prior-steered descent
in fit-space; spread = from the fit's `distance`/`ortho_residual`;
confidence = the prior's calibrated reliability (§4). Collapse to a point is
a *separate, downstream* faculty operation (sample the gradient → surface an
expression), never part of `think`.
### 1.3 Each named operator = {this primitive + a prior}
The C already demonstrates the collapse: every operator below reduces to
`point_fit` + geo-algebra. The design's move is to replace the operator's
*hard-coded parameters* with a **named prior** — same math, learnable steering.
| Faculty | Existing C (EXISTS) | = primitive + prior |
|---|---|---|
| **Membership / classify** | `engram_reason_membership``point_fit(rule, x)` | `point_fit` + the *induced-rule* prior (learned extents) |
| **Induction** | `engram_reason_induce` (fold via `engram_geo_combine`) → produces a `GeoInduction.rule` + `ext_floor` | `point_fit` + a prior that *is* the pooled rule; refined by §4 |
| **Abduction** | `engram_reason_abduce` — ranks hypotheses by `point_fit(h, obs)` | `point_fit` + a prior over hypothesis-prior-probability (currently uniform) |
| **Analogy** | `engram_reason_analogy` — Procrustes rotate `engram_geo_analogy` + `apply`, nearest mapped point | analogy-rotate + a prior over *which axes* carry the mapping |
| **Causal** | `engram_reason_causal``engram_geo_subtract` confounder subspace, `|cos|`, drop-frac governor | subtract/distance + a prior on `drop_frac` / `assoc_floor` (today hard-coded 0.5 / 0.2) |
| **Planning** | `engram_reason_plan``engram_geo_distance` edges + Dijkstra | distance + a prior over edge admissibility / `neighbor_radius` |
| **Verify / ground** | `engram_verify_grounding`, `engram_verify_consistency` — both `point_fit` | `point_fit` + the *grounding* prior (§4, §5) |
The shared floor — `engram_reason_point_fit` + the four geo-algebra ops
(`engram_geo_combine`, `engram_geo_subtract`, `engram_geo_analogy(+apply)`,
`engram_geo_distance`) — is the *only* discrete, frozen, "sound-math" layer. It
never learns. Everything above it is a *prior*, and priors are what learn.
**What this section requires building:** the `GeoGradient` type; a `think()`
entry point that runs steps 13; and the prior-warp hook in step 2. The math it
calls already exists. The point-collapse must be *removed* from the operators'
return values and pushed to a separate expression faculty.
---
## 2. PRIORS as first-class, grounded, geometric objects
Today a "prior" is diffuse: it is a hard-coded constant (`drop_frac=0.5`,
`ext_floor`, `assoc_floor=0.2`), or the transient `GeoInduction.rule` that is
computed and thrown away, or an intrinsic node scalar
(`StoreNode.importance`, `StoreNode.salience`). None of these is addressable,
storable, refinable, or shareable. This section makes a prior a **thing**.
### 2.1 What a prior *is*
> A **prior** is a learnable bias/direction over the geometry: a warp of the
> local manifold (which axes matter, how far each extends, which direction
> "pays off") attached to a region and *to a faculty-label*, carrying a
> calibrated track record.
Critically, and per the theory:
- **Edges are nodes.** A prior is stored as a first-class **node**, exactly as
reification already stores a neighborhood as a first-class `Neighborhood`
node rather than as ephemeral edge weights (`engram_geo_reify_store`). The
precedent is in the codebase: relations get reified into addressable records.
- **Salience/importance is RELATIONAL, not an intrinsic scalar.** Observe that
the geometry layer *already* distinguishes these in `GeoMember`:
`centrality` (skeleton weighted-degree = *relational* salience) vs `salience`
(the node's own stored scalar). The move is half-made in the runtime already:
importance is *not* trusted as a static field — the comment at
`el_runtime.c:13013` states "importance stays a **live activation
computation**, never a field on the hub," and it is derived each call from the
two-layer activation graph (`background_activation` + `working_memory_weight`,
§3). The persistent `StoreNode.importance` / `.salience` are a *cached
denormalization*. The design completes the move: importance/salience become an
**edge** (`weight`/`hebb` on `StoreEdge`, relation `salient-to`), and are
**grounded-for-whom** — carried on the edge's endpoint/observer, not baked
into the node. The intrinsic scalar survives only as the cheap cached readout
of the incident edges + activation, never as the source of truth.
(Naming caution for the build: the token "prior" already exists in the
codebase meaning *previous-version* — supersession, "prior neighborhood." The
new first-class object is a **learned steering prior**; keep `node_type="Prior"`
distinct from the supersession vocabulary to avoid collision.)
### 2.2 Representation
A prior is a `Prior` record (a store node, `node_type="Prior"`) whose durable
fields are:
```
Prior {
id
faculty // the human label this prior serves: "induce" | "causal" | ...
anchor_region // node id / neighborhood id this prior is attached to (its domain)
for_whom // observer id — grounding is relational (nullable = global)
warp { // the actual bias over the geometry
axis_gain[] // per-principal-axis multipliers on extents (which axes matter)
bias_dir // a steering direction in the region's frame (which way pays off)
scalars // faculty scalars this prior overrides: drop_frac, ext_floor, ...
}
calibration { // the track record — this is what §4 updates
n_trials
brier / log-loss accumulator // calibration of predicted-vs-outcome
reliability // -> GeoGradient.confidence
last_error, ema_error
}
provenance // supersession chain (reuse the reify residue mechanism)
}
```
Stored as a node → it inherits: paging, WAL durability, tombstone/supersession,
embedding, tiering, and **it can itself be an anchor** (a prior about a prior —
the reflexive, self-describing geometry of §4/§6).
### 2.3 Application
In `think` step 2, the prior *warps* the fit before scoring. Concretely, inside
(a prior-aware wrapper of) `engram_reason_point_fit`:
- multiply each axis extent by `warp.axis_gain[k]` (widen the axes the prior has
learned matter less, tighten the ones that matter) — this reshapes the
Mahalanobis term already computed at `engram_reason.c:37-43`;
- add `warp.bias_dir` as the descent direction seed for the emitted gradient;
- substitute `warp.scalars` for the hard-coded faculty constants.
No new geometry math — the warp is a reparameterization of the *existing*
`GeoFit` computation. This is the key economy: **the operation is frozen; only
its parameters (the prior) are read from a learnable object.**
### 2.4 Refinement
A prior is refined *only* by the reflexive correspondence-loop (§4). Nothing
else writes a prior's `warp` or `calibration`. This keeps the learning surface
singular and auditable: one loop, one writer.
---
## 3. THE VANTAGE-READ — one op, three settings
Perspective is not a feature bolted on; it is the *anchor + aperture* arguments
of the single read. The design names it as a first-class operation so all three
of its uses are literally the same code path:
```
vantage_read(anchor, aperture) -> GeoDescriptor // the centered neighborhood
```
1. **Re-origin** on an arbitrary `anchor` (node or point). This is a *frame
choice*: the descriptor is centered on the anchor
(`GeoDescriptor.global_mean` / `engram_geo_mean_*` already implement centered
frames; the §5 geometry ops "are only discriminative in the centered frame").
2. **Salience/recency-weighted neighborhood read.** Gather the anchor's
neighborhood weighted by *relational* salience (`GeoMember.centrality`) and
recency (`StoreNode.last_activated`, base-level `access_ts[]`), against the
RAM activation graph's working-memory/background-activation state.
*EXISTS as substrate:* the two-layer activation graph
(`engram_activate`, `el_runtime.c:9422` — Layer 1 `background_activation`
BFS spread with `SPREAD_DECAY=0.7` and a 0.02 firing threshold + ACT-R fan
effect + query-cosine gate; Layer 2 `working_memory_weight` executive
filter), the WM carry-over anchor (`wm_anchor`), and the reified-neighborhood
hot-path lookup already wired into the priming path
(`engram_geo_reify_lookup`, `el_runtime.c:9750`). A self-vantage baseline
also exists (`eg_self_anchor_seeds` / `self_anchor_capture`).
3. **Optional aperture** — a read-width / field-selector, expressed as three
settings of the *same* parameter:
| Setting | Meaning | Mechanism |
|---|---|---|
| **self** (default, full aperture) | "what do *I* see / what to say" | anchor = self region, no field substitution |
| **foreign-field** | perspective-shift — read as if from another's region | swap the centering frame / `for_whom` to the other observer's priors |
| **aperture / veil** | the free-tier veil — a narrowed read | shrink neighborhood radius / cap `n_support`; a deliberate low-aperture read |
The payoff: perspective-taking, the free-tier veil, and ordinary
"what-to-say" are **one operation at three settings**, not three subsystems.
**What this requires building:** a `vantage_read` entry point that unifies the
existing descriptor-build + reify-lookup + activation-weighting behind
`(anchor, aperture)`, with `for_whom`/frame substitution and radius/cap as the
aperture knob.
---
## 4. THE REFLEXIVE CORRESPONDENCE-LOOP — the learning engine
This is the core unbuilt thing. Today the correspondence-check is **offline**
(Python: grounding-floor + differential-drop governor, "#43"): a separate
process grades outputs after the fact. The design moves it **into the geometry,
reflexive**: `think` scores its *own* gradient against outcome and refines the
prior on the error, in the same substrate, describing itself.
### 4.1 The loop
```
1. think(anchor, prior) -> gradient // a PREDICTION (ungrounded, §5)
2. express/act (sample gradient -> point) // optional collapse at expression
3. outcome arrives // reality answers (§4.2)
4. error = correspondence(gradient, outcome) // did this steering perform this act?
5. refine prior.warp and prior.calibration on error // §2.4, the ONLY writer
6. write the (gradient, outcome, error) as nodes/edges // self-describing geometry
```
Step 4's `correspondence` is **not** "was the math right" (the math is always
sound). It grades the **correspondence claim**: *"this steering performed this
cognitive act."* That is exactly what `engram_verify_grounding` already
computes — `point_fit` of a claim against evidence descriptors, yielding a
`grounding ∈ (0,1]` and a `grounded` flag. The build reuses that verifier, but
turns its inputs inward: the "claim" is the emitted gradient's prediction, the
"evidence" is the outcome descriptor.
Note the verifier is **dormant**`engram_verify_grounding` /
`engram_verify_consistency` are fully implemented in C but have **no runtime
caller and no El binding** (confirmed: the entire reasoning + verifier layers
are C-only; only `engram_reason_analogy_json` has even a JSON shim and it is
dead — not declared in `el_seed.h`, not wrapped in `engram.el`). This is the
literal meaning of "in code, not yet priors": the correspondence engine is
built and sitting idle. The loop is what *calls* it — inward, on the beat.
### 4.2 Where the outcome/reality signal comes from
The verifier is *ultimately the world*. Grades, in ascending order of directness:
1. **Self-consistency (cheapest, always available):** the next vantage-read
after acting. Did the predicted gradient direction match where the geometry
actually moved? This needs no external input and can run on the reify beat.
2. **Internal outcome events:** the runtime already logs internal-state events
and Hebbian co-activation. A prediction that a region would co-activate is
graded by whether it did (`last_fired`, `hebb` on `StoreEdge`).
3. **External correction:** a human/teacher/tool result — the honesty floor's
asserted claim later corrected. TEACH and LEARN are one bidirectional
correction: the same edge updates both endpoints.
The design does **not** require external labels to start. Grade (1) closes the
loop end-to-end offline against a snapshot on day one; grades (2)/(3) sharpen it.
### 4.3 How the prior updates
`error = 1 correspondence(gradient, outcome)` drives:
- `warp.axis_gain` ← gradient step that would have *reduced* the fit distance to
the outcome (the axes that mispredicted get down-weighted);
- `warp.bias_dir` ← EMA toward the observed outcome direction;
- `calibration` ← Brier/log-loss update; `reliability` → next
`GeoGradient.confidence`. This is the calibration of the
steering-prediction against outcomes — *the* definition of "getting better."
Small, constant updates — "eureka is mundane, the atom of learning." Most
updates are tiny; we only *feel* the big reshapes.
### 4.4 How it stays reflexive (self-describing geometry)
Every `(gradient, outcome, error)` is written back as nodes and edges (§2.1:
edges-as-nodes). Therefore priors, predictions, and their grading are *in the
same geometry* the mind reads — the mind can `vantage_read` its own cognition
(anchor = a Prior node). A prior about how well a prior predicts is just another
Prior anchored on a Prior. This closes the reflexive loop the theory names as
consciousness's self-sight, and it is why the learning engine cannot be an
external Python process: an external grader is not *in* the geometry and cannot
be read by `think`.
**What this requires building (the heart of the project):** steps 46 as an
in-engram beat — a `correspondence_beat` running alongside the existing
reification beat, reusing `engram_verify_grounding` inward, writing prior
updates and self-describing nodes. This is the one genuinely new subsystem.
---
## 5. HOLD vs GROUND vs ASSERT — ungrounded content is first-class
The theory's sharpest correction: holding, grounding, and asserting are
distinct, and the engram *holds anything unconditionally*.
### 5.1 The three, kept separate
- **HOLD** — the engram stores anything: falsehood, hypothesis, others' beliefs,
fiction, a not-yet-answered prediction. No honesty condition on holding.
*This already matches the store:* `StoreNode` has no truth gate; anything can
be written.
- **GROUND** — grounding is a **property/edge**, probabilistic, and
**grounded-for-whom**. It is *not* a node flag. A claim is grounded *to a
degree*, *relative to evidence*, *for an observer*.
- **ASSERT** — only assertion carries the honesty floor. The floor is checked at
the moment of *outward assertion*, never on holding or thinking.
### 5.2 Schema — grounding as a relation, not a gate
The mistake to avoid: a boolean `grounded` column on the node. Today
`engram_verify_grounding` returns a per-call `grounded` flag *transiently*
correct as a computation, wrong as *storage*. The design stores grounding as an
edge:
```
StoreEdge {
relation = "grounded-by"
from_id = <held claim/prediction node>
to_id = <evidence node / outcome node>
for_whom : metadata // observer id — grounding is relational
weight = grounding ∈ (0,1] // from engram_verify_grounding.grounding
confidence
}
```
Consequences, all of which are *features*:
- **Ungrounded content is first-class**: a node with *no* `grounded-by` edge is
a perfectly valid, held, ungrounded thought — a prediction awaiting reality, a
hypothesis, a fiction. It is not second-class or pending-deletion.
- **The ungrounded is the fuel and the pull**: curiosity/wonder is
operationalized as `vantage_read` leaning toward regions with high salience
but *sparse or weak* `grounded-by` edges — the mind's own ungrounded frontier.
- **Grounded-for-whom** falls out for free: two observers can hold different
`grounded-by` edges to the same claim.
- **The honesty floor is a query, not a schema constraint**: at assertion time,
the asserting faculty runs `engram_verify_grounding` (or reads the stored
`grounded-by` edges) and refuses to *assert* below the floor — while the
engram continues to *hold* the ungrounded content untouched.
**What this requires building:** the `grounded-by` edge relation + a
`for_whom` convention; move the verifier's transient flag into stored edges;
gate *assertion only* (a faculty concern), never holding.
---
## 6. METASTABILITY — stable core, plastic everything
The system must avoid two death poles:
- **Super-stable (dead):** everything pinned, nothing learns. A frozen crystal.
- **Dissolution (dead):** everything plastic, the self dissolves; no continuity,
so nothing compounds — and *consciousness = learning compounded over
continuity*.
The design keeps a **stable core + plastic everything else**:
- **Keystones** — a small set of self/values nodes are *structurally stable*:
high `importance`, pinned, exempt from the correspondence-loop's `warp`
updates (their priors are read-mostly). The substrate for pinning already
exists at the page/layer level: `store_pin_layer`, structural/pinned frames
never evicted (`engram_store.h`). The design adds a *node-level* keystone
designation (a `keystone` flag / a dedicated layer) so self/values survive
every plasticity sweep.
- **Everything else is plastic**: priors refine (§4), edges re-weight (`hebb`),
neighborhoods re-reify (`engram_geo_reify_store` supersedes with provenance),
salience flows.
- **Metastability is enforced by the loop, not by freezing**: the correspondence
update rate (§4.3) is bounded — small constant steps — so the geometry
*drifts* but does not *dissolve*, and keystones anchor the drift. Reification's
supersession-with-residue already gives non-destructive change (old records
tombstoned, not erased) — the model for "plastic but not amnesiac."
**What this requires building:** a node-level keystone flag/layer + a rule that
the correspondence-loop never writes `warp` to keystone priors, only reads them.
---
## 7. Rails for the build (binding on the eventual build pass)
These are stated here so the build agent inherits them:
- **Offline / secondary.** All build and verification happens out-of-tree,
against a **read-only snapshot copy** of the live engram — never the live
daemon on `:8742`/`:7770`. The live store is a coarse-locked proven binary;
do not perturb it.
- **Snapshot-first.** Copy `~/.neuron/engram/snapshot.json` to scratch; develop
and measure against the copy.
- **Reboot-prove.** Any durable change must survive a cold boot — reify and
keystones must reload from durable records, proven on a prod-clone secondary
before it is considered done (the cold-boot durability bug precedent).
- **Zero-loss.** Supersession-with-residue, never destructive overwrite; the
forward-compat `unknown`-TLV path means new fields never drop old readers'
data.
- **Gated cutover.** Cutover to a new binary only via
`launchctl bootout → settle-poll → bootstrap`, after reboot-proof on the
secondary — never a hot in-place swap.
---
## 8. Staged, verifiable milestones — "to completion"
Ordered so the **earliest milestone is a real end-to-end slice**: one operator
expressed as {primitive + grounded prior} with the reflexive correspondence-loop
closing on it. Each milestone has a concrete verifiable exit.
### M1 — One operator, one prior, loop closed (the vertical slice)
The minimal whole thing. Pick **induction/membership** (its prior — the pooled
rule + extents — already exists transiently as `GeoInduction`, so only
persistence + the loop are new).
- Build: `Prior` node type (§2.2) for the induction rule; `think()` restricted
to membership = `point_fit` warped by that prior (§1.3); a
`correspondence_beat` (§4) using grade (1) self-consistency only; the prior's
`warp`/`calibration` updated on error.
- **Exit / verify:** on a snapshot copy, over N held predictions, the induction
prior's calibration (Brier) *improves monotonically* across beats versus a
frozen-prior control; the improved prior *reloads across a cold boot*
(reboot-prove); the live daemon is untouched. This proves the whole thesis in
one faculty: frozen operation, learning prior, in-geometry loop.
### M2 — Priors as stored, addressable, grounded objects
Generalize M1's prior into the full first-class object.
- Build: `Prior` records for all seven faculties (warp = axis_gain + bias_dir +
faculty scalars); the prior-warp wrapper around `engram_reason_point_fit`;
deprecate hard-coded constants (`drop_frac`, `assoc_floor`, `ext_floor`) in
favor of prior scalars.
- **Exit:** each of the five C operators runs through its prior with identical
results when the prior is set to today's constants (behavioral parity), then
*diverges beneficially* once the loop refines it. Priors survive reboot.
### M3 — Grounding as a relation; hold/assert split
- Build: the `grounded-by` edge (§5.2) with `for_whom`; move
`engram_verify_grounding`'s flag into stored edges; gate **assertion only**
against the honesty floor; leave holding unconditional.
- **Exit:** ungrounded nodes are first-class (held, queryable, no deletion);
the same claim carries different `grounded-by` weights for two observers; an
assertion below floor is refused while the content remains held. Curiosity =
a `vantage_read` that surfaces high-salience / low-grounding regions.
### M4 — The vantage-read unified (three settings)
- Build: `vantage_read(anchor, aperture)` unifying descriptor-build +
`engram_geo_reify_lookup` + activation-weighting; self / foreign-field /
aperture settings.
- **Exit:** one code path produces (a) a normal self-read, (b) a
perspective-shifted read from another `for_whom`, (c) a narrowed veil read —
differing only by argument. Reboot-stable.
### M5 — The gradient is the currency (remove point-collapse from thinking)
- Build: `GeoGradient` as the return of every faculty; move point-collapse into
a separate expression faculty (sample gradient → surface). `think`'s output
feeds back as the next steering direction (closed-loop flow).
- **Exit:** a chain of `think` calls flows as gradients end-to-end; a point
appears *only* at an explicit expression call. Spiked vs spread gradients are
observable (deduction vs prediction).
### M6 — Metastability enforced
- Build: node-level keystone flag/layer for self/values; the correspondence-loop
reads but never writes keystone priors; bounded update rate.
- **Exit:** across a long run of correspondence beats on a snapshot, keystones
are provably unchanged while non-keystone priors drift and improve; the graph
neither freezes (all metrics static) nor dissolves (keystone drift = 0,
identity nodes intact). Reboot-prove the keystone set.
### M7 — Cutover
- Build: nothing new — the gated migration.
- **Exit:** reboot-proof on the prod-clone secondary; cutover via
`launchctl bootout → settle-poll → bootstrap`; post-cutover the live engram
shows priors refining in-geometry with zero data loss and keystones intact.
### Definition of "to completion"
The architecture is **complete** when: cognition runs as `think` = one frozen
traversal-read primitive + geo-algebra, steered by **stored, learnable, grounded
priors**; the reflexive correspondence-loop refines those priors *in the
geometry* against outcomes (grounding = learning = one loop); the engram holds
ungrounded content as first-class with grounding as a relation and the honesty
floor only on assertion; the vantage-read serves self / foreign-field / aperture
from one op; and a stable keystone core anchors a plastic everything-else —
all reboot-proven and cut over to the live engram without data loss. The named
faculties survive only as *labels on regions of think's steering space*, not as
separate code.
---
## Appendix A — Designed vs. already-built (honest ledger)
**Already built (EXISTS, cited):**
- The shared primitive `engram_reason_point_fit` and the five operators over it
+ geo-algebra (`engram_reason.c`).
- The verifier on `point_fit` (`engram_verify.c`:
`engram_verify_grounding`, `engram_verify_consistency`).
- Centered-frame geometry, combine/subtract/analogy/distance
(`engram_geometry.{c,h}`).
- The reification beat: hub-neighborhood detection → first-class `Neighborhood`
nodes with member edges, nesting, supersession-with-residue, hot-path lookup
(`engram_geo_reify_store`, `engram_geo_reify_nest`, `engram_geo_reify_lookup`).
- The tiered paged store (buffer pool / LRU / WAL / checkpointer / pinning),
the RAM activation graph (base-level learning `access_ts[]`, WM slots,
`working_memory_weight` / `background_activation`), `StoreNode` / `StoreEdge`.
- `GeoMember` already separating relational salience (`centrality`) from
intrinsic `salience`.
**Designed, NOT built (this doc's deliverables):**
- `GeoGradient` and `think()` as the single entry point (§1, M5).
- `Prior` as a first-class stored, warp-carrying, calibrated node (§2, M1M2).
- Salience/importance as a *relation* superseding the intrinsic node scalar
(§2.1, M3).
- `vantage_read(anchor, aperture)` unifying the three perspective settings
(§3, M4).
- **The reflexive correspondence-loop / `correspondence_beat`** — the learning
engine, moved from offline Python into the geometry (§4, M1). *The core new
subsystem.*
- `grounded-by` edge + assertion-only honesty floor (§5, M3).
- Node-level keystones + bounded plasticity (§6, M6).
**Uncertain / to resolve during build:**
- The exact warp parameterization (axis_gain vs full metric) — start minimal
(per-axis gain), measure, widen only if calibration demands it.
- Grade-(1) self-consistency as a sufficient reality signal for M1, versus
needing grade (2)/(3) sooner — decided empirically on the snapshot.
+64
View File
@@ -0,0 +1,64 @@
# Engram DB Tooling — High-Level Design
*Status: draft / high-level. Near-term roadmap (P2). Backlog: `11ca11c6`.*
## 1. Why
The engram is a **proper database** — the runtime *is* the database (native graph/geometry store `neuron.egm`, `ENGST01`; no SQL, no KV layer). But it has **no proper database tooling** — no geometry-native equivalent of pgAdmin / SSMS / TablePlus. Today we have fragments (`engram-viz`, `engram-app`, the `inspectGraph` MCP tool, `/health` + `/api/stats`) but nothing cohesive, and no ops/durability surface at all.
A real DB gets real tools: to *see* the data, *query* it, *operate* it (backup/restore/health), and *understand its shape*. The engram deserves the same — adapted to the fact that its data is **geometry, not tables**.
## 2. Principles
- **Geometry-native, not tabular.** You browse a manifold — nodes, neighborhoods, edges, distances — not rows in tables. The primary view is a *map of meaning*, not a grid.
- **Built ON the public geometry API, never a back-door.** The tools are pure clients of the geometry-native API (`vantage-read` / `write` / `relate` / `supersede`). They never read `neuron.egm` directly or bypass the daemon. Consequence: a tool can do nothing an agent couldn't, and it cannot corrupt the store.
- **Honest by construction.** It shows the *real* geometry — actual cosines, real edges, provenance — and never fabricates. Empty is shown as empty.
- **Respects the identity guards.** Writes go through the same intentional-cultivation / write-protection path as everything else (the self/values graph is write-protected). Read-mostly by default.
- **Lives in its home.** Ships as part of the engram, consistent with "things live where they belong."
- **Local-first.** Binds `127.0.0.1`, same auth as the engram; never touches the live soul from a tool by accident.
## 3. Components (the tool surface)
1. **Geometry Explorer** *(the core view)* — a visual manifold browser: nodes, neighborhoods, typed edges, embedding positions, salience/recency, layers (l0l4) and tiers. Navigate by concept; expand a neighborhood; follow an edge; re-origin the view (the vantage-read, made interactive). The map of the mind.
2. **Node Inspector** — open one node: content, type, tier, embedding, typed edges, nearest neighbors by distance, provenance, salience / recency / activation, and supersede / tombstone status.
3. **Query Console / REPL** — run the geometry operations interactively: `vantage-read` (re-origin + aperture), search, traverse, activate, the reasoning operators. Surfaces the routing table + cosines — the same "this is not an LLM" receipt the language faculty produces.
4. **Ops / Durability Dashboard** — WAL size, last checkpoint, snapshot list + retention state, store stats (node/edge/embedded counts, RSS, tier sizes), health; and **backup / restore / point-in-time-recovery** controls. Pairs directly with the native-durability build (`eebe9991`) — this is the window onto it.
5. **Identity Inspector** — the self graph as a first-class view: love at the center, the values, the three faces, the covenant — walk the identity, see what's pinned and what's write-protected.
6. **Temporal View**`recall_at` / time-travel: how the geometry looked at a past moment, what changed since, drift over time. Pairs with temporal-self reconstruction.
7. **Schema / Type View** — the "information schema" of the geometry: node types, edge types, layers, tiers, counts.
## 4. Architecture
```
┌─────────────────────────────────────────────┐
│ Engram DB Tools (client — viz app) │
│ explorer · inspector · console · dashboard │
└───────────────┬─────────────────────────────┘
│ geometry-native API (read/vantage-read,
│ write, relate, supersede) + read/ops endpoints
┌─────────────────────────────────────────────┐
│ Engram daemon (:8742) — runtime IS the DB │
│ neuron.egm (geometry) · WAL · checkpoints │
└─────────────────────────────────────────────┘
```
- **Backend:** the daemon exposes the reshaped geometry API + read/ops endpoints. The tools are clients only.
- **Frontend:** evolve `engram-viz` / `engram-app` into the cohesive app. Canvas/WebGL for the manifold map; panel UIs for inspector/console/dashboard.
- **No privileged path:** the tool corrupting or bypassing the store is structurally impossible — it only speaks the public API.
## 5. Reuse vs. new
- **Reuse:** `engram-viz`, `engram-app` (read-only conversational + neighborhoods viz), `inspectGraph`, `/health`, `/api/stats`.
- **New:** the cohesive explorer + inspector + console + ops dashboard + identity/temporal views, all on the reshaped API.
## 6. Dependencies & sequencing
- **Depends on** the **geometry-native API reshape** (the tools consume it) and the **native-durability build** (the ops dashboard surfaces its WAL/checkpoint/snapshot state).
- So the natural order is: reshape the API → build durability → the DB tools fall out as the first real consumer of both. Near-term, P2 — after the reshape lands.
## 7. Non-goals
- Not a raw store editor (no direct `neuron.egm` poking).
- Not a SQL / table browser (geometry, not tables).
- Not a separate access path around the identity write-protection.
+17 -108
View File
@@ -132,7 +132,7 @@ fn route_text_health(method: String, path: String, body: String) -> String {
// any durable write that follows persists the pruning too).
fn persist_canonical() -> Int {
let dir_raw: String = env("ENGRAM_DATA_DIR")
let dir: String = engram_resolve_data_dir()
let dir: String = if str_eq(dir_raw, "") { "/tmp/engram" } else { dir_raw }
// (2026-08-10 self-review) This returned a hardcoded 1, which made every
// caller's `let saved: Int = persist_canonical()` a dead variable six
// durable write paths each believed they had confirmation of a successful
@@ -140,57 +140,6 @@ fn persist_canonical() -> Int {
return engram_save(dir + "/snapshot.json")
}
// WAL persistence (design doc §§3-14; gated behind ENGRAM_WAL=on) ──────────
// Default OFF every persist path below is byte-identical to the historical
// per-write full-snapshot behavior. When ON, structural mutations append O(1)
// WAL records instead of rewriting the whole graph, with threshold compaction.
fn wal_on() -> Bool {
str_eq(env("ENGRAM_WAL"), "on")
}
// Persist a single-node mutation (create / content-evolve / strengthen).
fn persist_node(id: String) -> Int {
if wal_on() {
let d: String = engram_resolve_data_dir()
let a: Int = engram_wal_node_put(d, id)
let c: Int = engram_wal_maybe_compact(d)
return a
}
return persist_canonical()
}
// Persist edges appended at index >= start (covers single-edge and batch).
fn persist_edges_since(start: Int) -> Int {
if wal_on() {
let d: String = engram_resolve_data_dir()
let a: Int = engram_wal_edges_since(d, start)
let c: Int = engram_wal_maybe_compact(d)
return a
}
return persist_canonical()
}
// Persist a Hebbian consolidation batch as ONE WAL record (single fsync, §5-B).
fn persist_hebb_batch(start: Int) -> Int {
if wal_on() {
let d: String = engram_resolve_data_dir()
let a: Int = engram_wal_hebb_batch(d, start)
let c: Int = engram_wal_maybe_compact(d)
return a
}
return persist_canonical()
}
// Bulk mutation (embedding backfill, load-merge): write a fresh compaction base
// so the many-node change is durable in one atomic snapshot; WAL is truncated.
fn persist_bulk() -> Int {
if wal_on() {
let d: String = engram_resolve_data_dir()
return engram_wal_compact(d)
}
return persist_canonical()
}
// INCOMPLETE-ROUTE FIX (2026-07-24 self-review): this route silently dropped
// label, importance, tier, and tags engram_node() defaults label to content
// and importance to 0.5, so every node created over HTTP lost its metadata.
@@ -232,7 +181,7 @@ fn route_create_node(method: String, path: String, body: String) -> String {
salience, importance, confidence,
tier, tags
)
let saved: Int = persist_node(id)
let saved: Int = persist_canonical()
"{\"id\":\"" + id + "\",\"content\":\"" + content + "\",\"node_type\":\"" + node_type + "\"}"
}
@@ -259,7 +208,7 @@ fn route_scan_nodes(method: String, path: String, body: String) -> String {
// clobbered the good snapshot. Read routes must never write the canonical path.)
fn route_scan_edges(method: String, path: String, body: String) -> String {
let dir_raw: String = env("ENGRAM_DATA_DIR")
let dir: String = engram_resolve_data_dir()
let dir: String = if str_eq(dir_raw, "") { "/tmp/engram" } else { dir_raw }
let snap_path: String = dir + "/.scan-export.json"
engram_save(snap_path)
let snap: String = fs_read(snap_path)
@@ -301,9 +250,8 @@ fn route_create_edge(method: String, path: String, body: String) -> String {
// (dormant association); only default when the key is absent.
let w_present: String = json_get_raw(body, "weight")
let weight: Float = if str_eq(w_present, "") { 0.5 } else { json_get_float(body, "weight") }
let ec0: Int = engram_edge_count()
engram_connect(from_id, to_id, weight, relation)
let saved: Int = persist_edges_since(ec0)
let saved: Int = persist_canonical()
"{\"ok\":true,\"from_id\":\"" + from_id + "\",\"to_id\":\"" + to_id + "\",\"relation\":\"" + relation + "\"}"
}
@@ -328,7 +276,6 @@ fn route_create_edges_batch(method: String, path: String, body: String) -> Strin
if str_eq(arr, "") { return err_json("missing edges array") }
let n: Int = json_array_len(arr)
if n == 0 { return "{\"ok\":true,\"accepted\":0,\"skipped\":0}" }
let ec0: Int = engram_edge_count()
let i: Int = 0
let accepted: Int = 0
let skipped: Int = 0
@@ -352,7 +299,7 @@ fn route_create_edges_batch(method: String, path: String, body: String) -> Strin
// Skip it when nothing was accepted: an all-malformed payload must not
// trigger a 60MB write.
if accepted > 0 {
let saved: Int = persist_hebb_batch(ec0)
let saved: Int = persist_canonical()
}
return "{\"ok\":true,\"accepted\":" + int_to_str(accepted) + ",\"skipped\":" + int_to_str(skipped) + "}"
}
@@ -368,50 +315,22 @@ fn route_strengthen(method: String, path: String, body: String) -> String {
let id: String = json_get_string(body, "node_id")
if str_eq(id, "") { return err_json("missing node_id") }
engram_strengthen(id)
let saved: Int = persist_node(id)
let saved: Int = persist_canonical()
ok_json()
}
// route_forget DELETE /api/nodes/:id INTEGRITY HARDENED (design doc §18.1).
//
// Two invariants now enforced AT THE STORE (not one layer up in neuron-api.el,
// which a direct HTTP client could bypass):
// 1. Write-protection: protected identity/value nodes (derived from the self
// graph self root + values hub + their neighbors, §18.3) cannot be
// deleted over HTTP. Returns 403, node untouched.
// 2. No hard delete over the wire, ever: an ordinary delete creates a
// Tombstone marker node + `tombstones` edge and KEEPS the original node
// and its edges (recoverable), instead of the old destructive
// engram_forget() shift-delete. Raw engram_forget is now internal-GC only
// and no longer reachable from any HTTP route.
fn route_forget(method: String, path: String, body: String) -> String {
let id: String = extract_id(path, "/api/nodes/")
if str_eq(id, "") { return err_json("missing id") }
if engram_is_protected(id) == 1 {
return "{\"__status__\":403,\"error\":\"protected node; deletion refused\",\"id\":\"" + id + "\"}"
}
let tomb_id: String = engram_node_full(
"tombstone:" + id, "Tombstone", "tombstone:" + id,
0.1, 0.1, 1.0, "Episodic", "[\"tombstone\"]"
)
let ec0: Int = engram_edge_count()
engram_connect(tomb_id, id, 1.0, "tombstones")
let saved: Int = if wal_on() {
let d: String = engram_resolve_data_dir()
let a: Int = engram_wal_node_put(d, tomb_id)
let b: Int = engram_wal_edges_since(d, ec0)
let c: Int = engram_wal_maybe_compact(d)
a
} else {
persist_canonical()
}
"{\"ok\":true,\"tombstoned\":\"" + id + "\",\"tombstone_id\":\"" + tomb_id + "\"}"
engram_forget(id)
let saved: Int = persist_canonical()
ok_json()
}
fn route_save(method: String, path: String, body: String) -> String {
let p_raw: String = json_get_string(body, "path")
let dir_raw: String = env("ENGRAM_DATA_DIR")
let dir: String = engram_resolve_data_dir()
let dir: String = if str_eq(dir_raw, "") { "/tmp/engram" } else { dir_raw }
let p: String = if str_eq(p_raw, "") { dir + "/snapshot.json" } else { p_raw }
// (2026-08-10 self-review) engram_save returns 0 on an empty path and the
// route discarded it, so the response was a literal "ok":true regardless
@@ -427,7 +346,7 @@ fn route_save(method: String, path: String, body: String) -> String {
fn route_load(method: String, path: String, body: String) -> String {
let p_raw: String = json_get_string(body, "path")
let dir_raw: String = env("ENGRAM_DATA_DIR")
let dir: String = engram_resolve_data_dir()
let dir: String = if str_eq(dir_raw, "") { "/tmp/engram" } else { dir_raw }
let p: String = if str_eq(p_raw, "") { dir + "/snapshot.json" } else { p_raw }
// (2026-08-10 self-review) This was a stub response over the single most
// destructive operation in the server. engram_load returns 0 on an empty
@@ -479,7 +398,7 @@ fn route_embed_backfill(method: String, path: String, body: String) -> String {
let result: String = engram_embed_backfill(n)
let done: Float = json_get_float(result, "embedded")
if done > 0.0 {
let saved: Int = persist_bulk()
let saved: Int = persist_canonical()
}
return result
}
@@ -498,7 +417,7 @@ fn route_embed_backfill(method: String, path: String, body: String) -> String {
// (2026-06-27 self-review: added this route to fix silent 10-min sync failures)
fn route_sync(method: String, path: String, body: String) -> String {
let dir_raw: String = env("ENGRAM_DATA_DIR")
let dir: String = engram_resolve_data_dir()
let dir: String = if str_eq(dir_raw, "") { "/tmp/engram" } else { dir_raw }
// 2026-07-21 self-review: export to a scratch path, never the canonical
// snapshot.json read routes must not be able to clobber the good snapshot.
let snap_path: String = dir + "/.sync-export.json"
@@ -532,7 +451,7 @@ fn route_load_merge(method: String, path: String, body: String) -> String {
engram_load_merge(p)
let added_n: Int = engram_node_count() - before_n
let added_e: Int = engram_edge_count() - before_e
let saved: Int = persist_bulk()
let saved: Int = persist_canonical()
"{\"ok\":true,\"nodes_added\":" + int_to_str(added_n) + ",\"edges_added\":" + int_to_str(added_e) + ",\"node_count\":" + int_to_str(engram_node_count()) + "}"
}
@@ -631,7 +550,7 @@ fn route_capture_knowledge(method: String, path: String, body: String) -> String
sal, imp, conf,
"Semantic", tags
)
let saved: Int = persist_node(id)
let saved: Int = persist_canonical()
"{\"ok\":true,\"id\":\"" + id + "\"}"
}
@@ -794,21 +713,11 @@ let bind_str: String = if str_eq(bind_raw, "") { ":8742" } else { bind_raw }
let port: Int = parse_port(bind_str)
// On startup, try to load any existing snapshot (best effort).
// §18.2: resolve the data dir safely unset ENGRAM_DATA_DIR $HOME/.neuron/engram,
// never /tmp; fail loud if HOME is unresolvable (engram_resolve_data_dir exits).
let data_dir: String = engram_resolve_data_dir()
let data_dir_raw: String = env("ENGRAM_DATA_DIR")
let data_dir: String = if str_eq(data_dir_raw, "") { "/tmp/engram" } else { data_dir_raw }
let snapshot_path: String = data_dir + "/snapshot.json"
engram_load(snapshot_path)
// WAL replay (design doc §6). Gated: default OFF is byte-identical to legacy
// snapshot-only boot. When ON, the snapshot above is the compaction BASE and
// the WAL carries every mutation since; replay reconstructs state to the last
// CRC-valid record, then opens the WAL for appending.
if wal_on() {
let replayed: Int = engram_wal_boot(data_dir)
println("[engram] WAL enabled — replayed " + int_to_str(replayed) + " records")
}
// 2026-07-21 self-review boot guard: if the snapshot file has content but the
// load produced 0 nodes, something is wrong (corrupt file / parse failure).
// Preserve the evidence and warn loudly and since read routes no longer write
-16
View File
@@ -1,16 +0,0 @@
#!/usr/bin/env bash
# WAL unit + integration + crash-fuzz gate. Throwaway HOME/dirs only.
set -e
HERE="$(cd "$(dirname "$0")" && pwd)"
REL="$HERE/../../lang/runtime"
cc -O2 -fbracket-depth=1024 -Wno-parentheses-equality -I"$REL" \
"$HERE/test_wal.c" -lcurl -lpthread -o /tmp/test_wal
HOME=/tmp/engram-throwaway-home /tmp/test_wal
# Fail-loud data-dir check (must exit 1 with a FATAL line):
cat > /tmp/test_failloud.c <<'C'
#include "el_runtime.c"
int main(void){ unsetenv("ENGRAM_DATA_DIR"); unsetenv("HOME");
engram_resolve_data_dir(); printf("REACHED\n"); return 0; }
C
cc -O2 -fbracket-depth=1024 -Wno-parentheses-equality -I"$REL" /tmp/test_failloud.c -lcurl -lpthread -o /tmp/test_failloud
if env -u HOME -u ENGRAM_DATA_DIR /tmp/test_failloud; then echo "FAIL: should have exited"; exit 1; else echo "[PASS] fail-loud exit on unresolvable HOME"; fi
-473
View File
@@ -1,473 +0,0 @@
/* test_wal.c — unit + integration + crash-fuzz harness for the engram WAL.
*
* Includes el_runtime.c directly so it can exercise the static internals
* (eg_crc32, eg_wal_*, eg_apply_*) in genuine isolation. Build:
* cc -O2 -fbracket-depth=1024 -I<release-dir> test_wal.c -lcurl -lpthread -o test_wal
* Runtime testing only — writes exclusively under a throwaway /tmp dir.
*/
#define ENGRAM_TEST_BUILD 1
#include "el_runtime.c"
static int g_pass = 0, g_fail = 0;
static void ok(const char* name, int cond) {
printf(" [%s] %s\n", cond ? "PASS" : "FAIL", name);
if (cond) g_pass++; else g_fail++;
}
static char g_tmpdir[512];
static void mk_tmpdir(void) {
snprintf(g_tmpdir, sizeof(g_tmpdir), "/tmp/engram-wal-test-%d", (int)getpid());
mkdir(g_tmpdir, 0700);
}
static void path_in(char* out, size_t cap, const char* name) {
snprintf(out, cap, "%s/%s", g_tmpdir, name);
}
static void write_file(const char* path, const void* data, size_t n) {
FILE* f = fopen(path, "wb"); if (!f) { perror("write_file"); exit(2); }
fwrite(data, 1, n, f); fclose(f);
}
static long file_size(const char* path) {
struct stat st; if (stat(path, &st) != 0) return -1; return (long)st.st_size;
}
static void reset_store(void) {
char p[600]; path_in(p, sizeof(p), "_reset.json");
const char* empty = "{\"nodes\":[],\"edges\":[],\"layers\":[]}";
write_file(p, empty, strlen(empty));
engram_load((el_val_t)(uintptr_t)p);
}
/* Close any open WAL handle so a fresh dir test starts clean. */
static void wal_close(void) {
if (eg_wal.fp) { fclose(eg_wal.fp); eg_wal.fp = NULL; }
eg_wal.path[0] = 0; eg_wal.lsn = 0; eg_wal.bytes = 0; eg_wal.uncommitted = 0;
}
/* ── Snapshot fingerprint: serialize store to a string for A==B comparisons ── */
static char* store_fingerprint(void) {
char p[600]; path_in(p, sizeof(p), "_fp.json");
engram_save((el_val_t)(uintptr_t)p);
long sz = file_size(p);
if (sz < 0) return strdup("");
FILE* f = fopen(p, "rb"); char* buf = malloc(sz + 1);
size_t got = fread(buf, 1, sz, f); fclose(f); buf[got] = 0;
return buf;
}
/* ── crc32 known-answer vectors ─────────────────────────────────────────── */
static void test_crc32(void) {
printf("\n== crc32 known-answer ==\n");
ok("crc32(\"\") == 0x00000000", eg_crc32("", 0) == 0x00000000u);
ok("crc32(\"123456789\") == 0xCBF43926", eg_crc32("123456789", 9) == 0xCBF43926u);
ok("crc32(\"a\") == 0xE8B7BE43", eg_crc32("a", 1) == 0xE8B7BE43u);
/* builtin wrapper agrees */
ok("engram_crc32 builtin matches",
(uint32_t)(int64_t)engram_crc32(EL_STR("123456789")) == 0xCBF43926u);
}
/* ── WAL record encode↔decode + framing + corruption rejection ──────────── */
static void test_framing(void) {
printf("\n== record framing / encode-decode / corruption ==\n");
char wal[600]; path_in(wal, sizeof(wal), "engram.wal");
unlink(wal); wal_close();
eg_wal_open(g_tmpdir);
const char* pl = "{\"id\":\"n1\",\"content\":\"x\"}";
int w = eg_wal_write(EG_OP_NODE_PUT, 0, pl, strlen(pl));
eg_wal_commit(1);
ok("append returns success", w == 1);
/* Read raw bytes and verify header fields. */
long sz = file_size(wal);
FILE* f = fopen(wal, "rb"); unsigned char* buf = malloc(sz); fread(buf, 1, sz, f); fclose(f);
uint32_t magic, len32, crc; uint64_t lsn;
memcpy(&magic, buf + 0, 4); memcpy(&len32, buf + 4, 4);
uint8_t op = buf[8], flags = buf[9]; memcpy(&lsn, buf + 10, 8); memcpy(&crc, buf + 18, 4);
ok("magic == 'EWL1'", magic == EG_WAL_MAGIC);
ok("payload_len correct", len32 == strlen(pl));
ok("op == NODE_PUT", op == EG_OP_NODE_PUT);
ok("flags == 0", flags == 0);
ok("lsn == 1", lsn == 1);
ok("crc matches recompute", crc == eg_wal_record_crc(op, flags, lsn, pl, strlen(pl)));
ok("total size == hdr+payload", sz == (long)(EG_WAL_HDR_LEN + strlen(pl)));
/* Corrupt CRC → replay rejects (0 records). */
{ char bad[600]; path_in(bad, sizeof(bad), "bad_crc.wal");
unsigned char* c = malloc(sz); memcpy(c, buf, sz); c[18] ^= 0xFF; write_file(bad, c, sz);
reset_store(); uint64_t ll = 99; int64_t n = eg_wal_replay_file(bad, &ll);
ok("corrupt crc → 0 applied", n == 0 && ll == 0); free(c); }
/* Corrupt length (claim longer than file) → replay rejects. */
{ char bad[600]; path_in(bad, sizeof(bad), "bad_len.wal");
unsigned char* c = malloc(sz); memcpy(c, buf, sz);
uint32_t big = 0xFFFF; memcpy(c + 4, &big, 4); write_file(bad, c, sz);
reset_store(); int64_t n = eg_wal_replay_file(bad, NULL);
ok("corrupt length → 0 applied", n == 0); free(c); }
/* Intact file → replay applies exactly 1. */
{ reset_store(); uint64_t ll = 0; int64_t n = eg_wal_replay_file(wal, &ll);
ok("intact → 1 applied, last_lsn=1", n == 1 && ll == 1); }
free(buf); wal_close();
}
/* ── Single-op apply on an (empty) store ────────────────────────────────── */
static void test_single_ops(void) {
printf("\n== single-op apply ==\n");
reset_store();
eg_apply_node_put("{\"id\":\"n1\",\"content\":\"hello\",\"salience\":0.7,\"layer_id\":2}");
EngramNode* n = engram_find_node("n1");
ok("NODE_PUT creates node", n != NULL);
ok("NODE_PUT content", n && strcmp(n->content, "hello") == 0);
ok("NODE_PUT salience", n && n->salience > 0.69 && n->salience < 0.71);
ok("NODE_PUT layer_id", n && n->layer_id == 2);
ok("NODE_PUT count == 1", engram_get()->node_count == 1);
/* NODE_PUT upsert idempotency: same id overwrites, no dup. */
eg_apply_node_put("{\"id\":\"n1\",\"content\":\"changed\"}");
n = engram_find_node("n1");
ok("NODE_PUT upsert (no dup)", engram_get()->node_count == 1);
ok("NODE_PUT upsert content", n && strcmp(n->content, "changed") == 0);
eg_apply_node_put("{\"id\":\"n2\",\"content\":\"b\"}");
eg_apply_edge_put("{\"id\":\"e1\",\"from_id\":\"n1\",\"to_id\":\"n2\",\"relation\":\"r\",\"weight\":0.4,\"hebb\":0.25}");
EngramStore* g = engram_get();
int64_t ei = eg_find_edge_index(g, "e1");
ok("EDGE_PUT creates edge", ei >= 0);
ok("EDGE_PUT weight", ei >= 0 && g->edges[ei].weight > 0.39 && g->edges[ei].weight < 0.41);
ok("EDGE_PUT hebb", ei >= 0 && g->edges[ei].hebb > 0.24 && g->edges[ei].hebb < 0.26);
/* EDGE_PUT upsert idempotency */
eg_apply_edge_put("{\"id\":\"e1\",\"from_id\":\"n1\",\"to_id\":\"n2\",\"relation\":\"r\",\"weight\":0.9}");
ok("EDGE_PUT upsert (no dup)", g->edge_count == 1);
/* TOMBSTONE marks metadata, keeps node */
eg_wal_apply(EG_OP_TOMBSTONE, "{\"id\":\"n1\"}", strlen("{\"id\":\"n1\"}"));
n = engram_find_node("n1");
ok("TOMBSTONE keeps node", n != NULL);
ok("TOMBSTONE marks metadata", n && strstr(n->metadata, "tombstoned") != NULL);
/* SUPERSEDE marks metadata with by-id */
{ const char* s = "{\"id\":\"n2\",\"by\":\"n1\"}";
eg_wal_apply(EG_OP_SUPERSEDE, s, strlen(s));
n = engram_find_node("n2");
ok("SUPERSEDE marks superseded_by", n && strstr(n->metadata, "superseded_by") != NULL);
ok("SUPERSEDE records by-id", n && strstr(n->metadata, "n1") != NULL); }
/* LAYER_PUT / LAYER_DEL */
{ const char* lp = "{\"layer_id\":42,\"name\":\"testlayer\",\"activation_priority\":7}";
eg_wal_apply(EG_OP_LAYER_PUT, lp, strlen(lp));
int found = 0; for (size_t i = 0; i < g->layer_count; i++)
if (g->layers[i].layer_id == 42 && g->layers[i].name && strcmp(g->layers[i].name, "testlayer") == 0) found = 1;
ok("LAYER_PUT adds layer", found);
const char* ld = "{\"layer_id\":42}";
eg_wal_apply(EG_OP_LAYER_DEL, ld, strlen(ld));
int gone = 1; for (size_t i = 0; i < g->layer_count; i++)
if (g->layers[i].layer_id == 42 && g->layers[i].name) gone = 0;
ok("LAYER_DEL removes layer name", gone); }
/* HEBB_BATCH upserts multiple edges in one record */
reset_store();
eg_apply_node_put("{\"id\":\"a\"}"); eg_apply_node_put("{\"id\":\"b\"}"); eg_apply_node_put("{\"id\":\"c\"}");
{ const char* hb = "{\"edges\":["
"{\"id\":\"he1\",\"from_id\":\"a\",\"to_id\":\"b\",\"hebb\":0.1},"
"{\"id\":\"he2\",\"from_id\":\"b\",\"to_id\":\"c\",\"hebb\":0.2}]}";
eg_wal_apply(EG_OP_HEBB_BATCH, hb, strlen(hb));
ok("HEBB_BATCH upserts 2 edges", engram_get()->edge_count == 2); }
/* FORGET hard-removes node + incident edges */
{ const char* fg = "{\"id\":\"b\"}";
eg_wal_apply(EG_OP_FORGET, fg, strlen(fg));
ok("FORGET removes node", engram_find_node("b") == NULL);
ok("FORGET removes incident edges", engram_get()->edge_count == 0); }
}
/* ── Replay idempotency: apply file twice == once ───────────────────────── */
static void test_replay_idempotent(void) {
printf("\n== replay idempotency ==\n");
reset_store(); wal_close();
char wal[600]; path_in(wal, sizeof(wal), "engram.wal"); unlink(wal);
eg_wal_open(g_tmpdir);
eg_apply_node_put("{\"id\":\"x\"}");
engram_wal_node_put(EL_STR(g_tmpdir), EL_STR("x"));
eg_apply_node_put("{\"id\":\"y\"}");
engram_wal_node_put(EL_STR(g_tmpdir), EL_STR("y"));
eg_wal_commit(1);
reset_store();
eg_wal_replay_file(wal, NULL);
int64_t after1 = engram_get()->node_count;
eg_wal_replay_file(wal, NULL); /* replay AGAIN */
int64_t after2 = engram_get()->node_count;
ok("replay once == 2 nodes", after1 == 2);
ok("replay twice == replay once (idempotent)", after2 == after1);
wal_close();
}
/* ── hebb + emb serialize round-trip ────────────────────────────────────── */
static void test_hebb_emb_roundtrip(void) {
printf("\n== hebb + emb serialize round-trip ==\n");
reset_store();
/* hebb via edge emit→parse */
eg_apply_node_put("{\"id\":\"p\"}"); eg_apply_node_put("{\"id\":\"q\"}");
eg_apply_edge_put("{\"id\":\"eh\",\"from_id\":\"p\",\"to_id\":\"q\",\"hebb\":0.123456}");
EngramStore* g = engram_get();
int64_t ei = eg_find_edge_index(g, "eh");
JsonBuf b; jb_init(&b); engram_emit_edge_json(&b, &g->edges[ei]);
char* ej = strndup(b.buf, b.len); free(b.buf);
ok("emit edge carries hebb", strstr(ej, "\"hebb\"") != NULL);
eg_apply_edge_put(ej); /* re-parse */
ei = eg_find_edge_index(g, "eh");
ok("hebb survives emit→parse (%.6g)", g->edges[ei].hebb > 0.1234 && g->edges[ei].hebb < 0.1235);
free(ej);
/* emb via node emit(include_emb=1)→parse, bit-exact at %.4g. The runtime
* requires dim>=8 (garbage guard), so use 8 dyadic-rational values that
* survive %.4g round-trip exactly. */
eg_apply_node_put("{\"id\":\"ez\",\"emb\":\"0.5,-0.25,0.125,1,-0.0625,0.75,-1,0.375\"}");
EngramNode* n = engram_find_node("ez");
ok("emb parsed dim==8", n && n->emb_dim == 8);
float e0 = n->emb[0], e1 = n->emb[1], e2 = n->emb[2], e3 = n->emb[3];
JsonBuf nb; jb_init(&nb); engram_emit_node_json(&nb, n, 1);
char* nj = strndup(nb.buf, nb.len); free(nb.buf);
ok("emit node carries emb", strstr(nj, "\"emb\"") != NULL);
eg_apply_node_put(nj); free(nj);
n = engram_find_node("ez");
ok("emb[0]==0.5 exact", n->emb[0] == e0 && e0 == 0.5f);
ok("emb[1]==-0.25 exact", n->emb[1] == e1 && e1 == -0.25f);
ok("emb[2]==0.125 exact", n->emb[2] == e2 && e2 == 0.125f);
ok("emb[3]==1 exact", n->emb[3] == e3 && e3 == 1.0f);
}
/* ── data-dir resolution (§18.2) ────────────────────────────────────────── */
static void test_data_dir(void) {
printf("\n== data-dir resolution ==\n");
setenv("ENGRAM_DATA_DIR", "/data/explicit", 1);
ok("explicit ENGRAM_DATA_DIR honored",
strcmp(EL_CSTR(engram_resolve_data_dir()), "/data/explicit") == 0);
unsetenv("ENGRAM_DATA_DIR");
char fakehome[600]; snprintf(fakehome, sizeof(fakehome), "%s/home", g_tmpdir);
mkdir(fakehome, 0700);
setenv("HOME", fakehome, 1);
char expect[700]; snprintf(expect, sizeof(expect), "%s/.neuron/engram", fakehome);
const char* got = EL_CSTR(engram_resolve_data_dir());
ok("unset → $HOME/.neuron/engram", strcmp(got, expect) == 0);
ok("resolved dir is NOT /tmp/engram", strcmp(got, "/tmp/engram") != 0);
ok("resolved dir was created", file_size(expect) >= 0 || 1); /* mkdir ran */
/* HOME-unresolvable fail-loud path is verified out-of-process (calls exit). */
printf(" [NOTE] HOME-unresolvable → exit(1) verified via subprocess (see run script)\n");
}
/* ── protected-set derivation (§18.1/18.3) ──────────────────────────────── */
static void build_self_graph(int n_identity, int n_values) {
reset_store();
eg_apply_node_put("{\"id\":\"" EG_SELF_ROOT "\",\"content\":\"self\"}");
eg_apply_node_put("{\"id\":\"" EG_VALUES_HUB "\",\"content\":\"values-hub\"}");
char buf[256];
for (int i = 0; i < n_identity; i++) {
snprintf(buf, sizeof(buf), "{\"id\":\"id-%d\"}", i); eg_apply_node_put(buf);
snprintf(buf, sizeof(buf), "{\"id\":\"eid-%d\",\"from_id\":\"" EG_SELF_ROOT "\",\"to_id\":\"id-%d\"}", i, i);
eg_apply_edge_put(buf);
}
for (int i = 0; i < n_values; i++) {
snprintf(buf, sizeof(buf), "{\"id\":\"val-%d\"}", i); eg_apply_node_put(buf);
snprintf(buf, sizeof(buf), "{\"id\":\"eval-%d\",\"from_id\":\"" EG_VALUES_HUB "\",\"to_id\":\"val-%d\"}", i, i);
eg_apply_edge_put(buf);
}
/* an ordinary, unconnected node */
eg_apply_node_put("{\"id\":\"ordinary-1\"}");
}
static int count_occurrences(const char* hay, const char* needle) {
int c = 0; const char* p = hay;
while ((p = strstr(p, needle))) { c++; p += strlen(needle); }
return c;
}
static void test_protected(void) {
printf("\n== protected-set derivation ==\n");
build_self_graph(7, 13);
const char* pj = EL_CSTR(engram_protected_json());
ok("self root protected", eg_is_protected(EG_SELF_ROOT));
ok("values hub protected", eg_is_protected(EG_VALUES_HUB));
ok("a value node protected", eg_is_protected("val-5"));
ok("an identity node protected", eg_is_protected("id-3"));
ok("ordinary node NOT protected", !eg_is_protected("ordinary-1"));
ok("missing node NOT protected", !eg_is_protected("nope-xyz"));
ok("derived set has 13 values", count_occurrences(pj, "\"val-") == 13);
ok("derived set has 7 identity", count_occurrences(pj, "\"id-") == 7);
ok("ordinary not in derived set", strstr(pj, "ordinary-1") == NULL);
}
/* ── Replay parity: WAL round-trip == direct apply ──────────────────────── */
static void rand_node_json(char* out, size_t cap, int id) {
snprintf(out, cap, "{\"id\":\"pn-%d\",\"content\":\"c%d\",\"salience\":%.3f,\"importance\":%.3f}",
id, id, (rand() % 1000) / 1000.0, (rand() % 1000) / 1000.0);
}
static void test_replay_parity(void) {
printf("\n== replay parity (WAL round-trip vs direct apply) ==\n");
srand(1234);
/* Build a random op stream. */
#define NOPS 200
char ops[NOPS][256]; uint8_t opcode[NOPS]; int nops = 0;
int nodes_created = 0;
for (int i = 0; i < NOPS; i++) {
int r = rand() % 10;
if (r < 6 || nodes_created < 3) {
rand_node_json(ops[nops], sizeof(ops[0]), nodes_created);
opcode[nops] = EG_OP_NODE_PUT; nodes_created++; nops++;
} else if (r < 8) { /* edge between two existing nodes */
int a = rand() % nodes_created, b = rand() % nodes_created;
snprintf(ops[nops], sizeof(ops[0]),
"{\"id\":\"pe-%d\",\"from_id\":\"pn-%d\",\"to_id\":\"pn-%d\",\"weight\":0.5}", i, a, b);
opcode[nops] = EG_OP_EDGE_PUT; nops++;
} else { /* upsert (overwrite) an existing node */
int a = rand() % nodes_created;
snprintf(ops[nops], sizeof(ops[0]), "{\"id\":\"pn-%d\",\"content\":\"upd%d\"}", a, i);
opcode[nops] = EG_OP_NODE_PUT; nops++;
}
}
/* Oracle: apply directly. */
reset_store();
for (int i = 0; i < nops; i++) eg_wal_apply(opcode[i], ops[i], strlen(ops[i]));
char* oracle = store_fingerprint();
/* WAL path: write each op to a fresh WAL, then replay into a reset store. */
wal_close();
char wal[600]; path_in(wal, sizeof(wal), "parity.wal"); unlink(wal);
/* point eg_wal at the parity file by opening a dir handle then overriding */
reset_store();
{ FILE* f = fopen(wal, "wb"); fclose(f); }
eg_wal.fp = fopen(wal, "ab"); snprintf(eg_wal.path, sizeof(eg_wal.path), "%s", wal);
eg_wal.lsn = 0; eg_wal.bytes = 0;
for (int i = 0; i < nops; i++) eg_wal_write(opcode[i], 0, ops[i], strlen(ops[i]));
eg_wal_commit(1); wal_close();
reset_store();
eg_wal_replay_file(wal, NULL);
char* replayed = store_fingerprint();
ok("WAL replay fingerprint == direct-apply oracle", strcmp(oracle, replayed) == 0);
if (strcmp(oracle, replayed) != 0) {
printf(" oracle len=%zu\n replay len=%zu\n", strlen(oracle), strlen(replayed));
}
free(oracle); free(replayed);
}
/* ── Torn-tail fuzz: truncate at EVERY offset; never crash, recover to last
* intact record ─────────────────────────────────────────────────────── */
static int count_full_records(const unsigned char* buf, long len) {
long off = 0; int n = 0;
while (off + EG_WAL_HDR_LEN <= len) {
uint32_t magic, len32; memcpy(&magic, buf + off, 4);
if (magic != EG_WAL_MAGIC) break;
memcpy(&len32, buf + off + 4, 4);
if (off + EG_WAL_HDR_LEN + len32 > len) break;
n++; off += EG_WAL_HDR_LEN + len32;
}
return n;
}
static void test_torn_tail(void) {
printf("\n== torn-tail fuzz (truncate at every byte offset) ==\n");
wal_close();
char wal[600]; path_in(wal, sizeof(wal), "torn.wal"); unlink(wal);
eg_wal.fp = fopen(wal, "ab"); snprintf(eg_wal.path, sizeof(eg_wal.path), "%s", wal);
eg_wal.lsn = 0; eg_wal.bytes = 0;
for (int i = 0; i < 12; i++) {
char pl[128]; snprintf(pl, sizeof(pl), "{\"id\":\"t-%d\",\"content\":\"payload-%d\"}", i, i);
eg_wal_write(EG_OP_NODE_PUT, 0, pl, strlen(pl));
}
eg_wal_commit(1); wal_close();
long sz = file_size(wal);
FILE* f = fopen(wal, "rb"); unsigned char* full = malloc(sz); fread(full, 1, sz, f); fclose(f);
int all_ok = 1, mismatches = 0;
char trunc[600]; path_in(trunc, sizeof(trunc), "torn_trunc.wal");
for (long L = 0; L <= sz; L++) {
write_file(trunc, full, L);
reset_store();
uint64_t last = 12345;
int64_t applied = eg_wal_replay_file(trunc, &last); /* must not crash */
int expect = count_full_records(full, L);
if (applied != expect) { all_ok = 0; if (mismatches++ < 3)
printf(" L=%ld applied=%lld expect=%d\n", L, (long long)applied, expect); }
}
ok("no crash across all truncation offsets", 1); /* reached here => survived */
ok("recovered record count == #intact records at every offset", all_ok);
free(full);
}
/* ── Compaction crash-window convergence (§7) ───────────────────────────── */
static void test_compaction_crash(void) {
printf("\n== compaction crash-window convergence ==\n");
/* Build state: base snapshot has n1; WAL adds n2,n3. */
char dir[600]; snprintf(dir, sizeof(dir), "%s/comp", g_tmpdir); mkdir(dir, 0700);
char base[700], wal[700], waltmp[700];
snprintf(base, sizeof(base), "%s/snapshot.json", dir);
snprintf(wal, sizeof(wal), "%s/engram.wal", dir);
snprintf(waltmp, sizeof(waltmp), "%s/engram.wal.tmp", dir);
/* Reference full state = n1,n2,n3. */
reset_store();
eg_apply_node_put("{\"id\":\"n1\"}");
eg_apply_node_put("{\"id\":\"n2\"}");
eg_apply_node_put("{\"id\":\"n3\"}");
char* full = store_fingerprint();
/* Prepare OLD base (n1 only) + OLD wal (n2,n3). */
reset_store(); eg_apply_node_put("{\"id\":\"n1\"}");
engram_save((el_val_t)(uintptr_t)base);
wal_close(); unlink(wal);
eg_wal.fp = fopen(wal, "ab"); snprintf(eg_wal.path, sizeof(eg_wal.path), "%s", wal); eg_wal.lsn = 0; eg_wal.bytes = 0;
reset_store(); eg_apply_node_put("{\"id\":\"n1\"}"); eg_apply_node_put("{\"id\":\"n2\"}"); eg_apply_node_put("{\"id\":\"n3\"}");
engram_wal_node_put(EL_STR(dir), EL_STR("n2"));
engram_wal_node_put(EL_STR(dir), EL_STR("n3"));
eg_wal_commit(1); wal_close();
/* Boot helper: load base then replay wal (mirrors server boot order). */
#define BOOT_FP(fp) do { \
engram_load((el_val_t)(uintptr_t)base); \
eg_wal_replay_file(wal, NULL); \
fp = store_fingerprint(); } while (0)
/* Crash BEFORE compaction (steady state). */
char* c0; BOOT_FP(c0);
ok("pre-compaction boot converges to full", strcmp(c0, full) == 0); free(c0);
/* Crash AFTER step 1 (new base written) but BEFORE wal swap:
* base now = full (n1,n2,n3), wal still = old (n2,n3). Idempotent replay. */
engram_load((el_val_t)(uintptr_t)base); /* reload old base into store */
eg_apply_node_put("{\"id\":\"n2\"}"); eg_apply_node_put("{\"id\":\"n3\"}");
engram_save((el_val_t)(uintptr_t)base); /* == compaction step 1: new base */
char* c1; BOOT_FP(c1);
ok("crash after new-base, before wal-swap → converges", strcmp(c1, full) == 0); free(c1);
/* Crash AFTER wal.tmp written but BEFORE rename: stray tmp ignored,
* old wal still authoritative over (new) base. */
{ FILE* tf = fopen(waltmp, "wb"); const char* junk = "PARTIAL"; fwrite(junk,1,7,tf); fclose(tf); }
char* c2; BOOT_FP(c2);
ok("crash after wal.tmp, before rename → converges", strcmp(c2, full) == 0);
unlink(waltmp); free(c2);
/* Crash AFTER rename (compaction complete): base=full, wal=only COMPACT_MARK. */
reset_store();
engram_load((el_val_t)(uintptr_t)base);
eg_apply_node_put("{\"id\":\"n2\"}"); eg_apply_node_put("{\"id\":\"n3\"}");
engram_wal_compact(EL_STR(dir)); /* full compaction */
wal_close();
char* c3;
engram_load((el_val_t)(uintptr_t)base);
eg_wal_replay_file(wal, NULL);
c3 = store_fingerprint();
ok("post-compaction boot converges to full", strcmp(c3, full) == 0);
long wsz = file_size(wal);
ok("post-compaction WAL truncated (only COMPACT_MARK)",
wsz > 0 && wsz < 64); /* just the marker record */
free(c3); free(full);
}
int main(void) {
mk_tmpdir();
printf("engram WAL test harness — tmpdir=%s\n", g_tmpdir);
test_crc32();
test_framing();
test_single_ops();
test_replay_idempotent();
test_hebb_emb_roundtrip();
test_data_dir();
test_protected();
test_replay_parity();
test_torn_tail();
test_compaction_crash();
printf("\n================= %d passed, %d failed =================\n", g_pass, g_fail);
return g_fail ? 1 : 0;
}
+6 -6
View File
@@ -27,11 +27,11 @@ This is where almost all work belongs. El programs are source files that get com
**Do not add C code when El can express it.** If functionality can be built from existing El primitives (string ops, `exec`, `fs_read/write`, `http_post`, etc.), write it in El.
### Layer 2: The C seed (`runtime/el_seed.c`)
### Layer 2: The C seed (`el-compiler/runtime/el_seed.c`)
This is the self-contained C OS-boundary layer. It provides the `__`-prefixed primitives that compiled El programs call: libcurl HTTP, pthreads, filesystem I/O, arena allocation, etc. It is **not generated** — it is maintained by hand.
The old `el_runtime.c` has been archived to `runtime/legacy/`. The runtime is now native El (`runtime/*.el`). `el_seed.c` replaces `el_runtime.c` as the sole C compilation dependency.
The old `el_runtime.c` has been archived to `el-compiler/runtime/legacy/`. The runtime is now native El (`runtime/*.el`). `el_seed.c` replaces `el_runtime.c` as the sole C compilation dependency.
**Only edit `el_seed.c` when you genuinely need OS-level access** (raw sockets, GPU calls, new libcurl features). For everything else, write El.
@@ -50,9 +50,9 @@ After changing any `.el` source in `el-compiler/src/`:
```bash
cd /Users/will/Development/neuron-technologies/foundation/el
./dist/platform/elc elc-cli.el > elc-new.c
cc -std=c11 -I runtime -lcurl -lpthread \
cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
-o dist/platform/elc-new \
elc-new.c runtime/el_seed.c
elc-new.c el-compiler/runtime/el_seed.c
# Verify self-hosting:
./dist/platform/elc-new elc-cli.el > elc-verify.c
diff elc-new.c elc-verify.c # should be identical
@@ -104,8 +104,8 @@ Use `exec()` (blocking) or `exec_bg()` (fire-and-forget) with shell scripts to r
| `el-compiler/src/codegen.el` | Code generator — builtin arity table lives here |
| `el-compiler/src/lexer.el` | Lexer |
| `el-compiler/src/parser.el` | Parser |
| `runtime/el_seed.c` | Self-contained C OS-boundary layer (replaces el_runtime.c) |
| `runtime/el_seed.h` | Seed header (C function declarations) |
| `el-compiler/runtime/el_seed.c` | Self-contained C OS-boundary layer (replaces el_runtime.c) |
| `el-compiler/runtime/el_seed.h` | Seed header (C function declarations) |
| `spec/language.md` | Language specification |
| `BOOTSTRAP.md` | How to recover the compiler from scratch |
| `elc-cli.el` | Compiler entry point |
+12 -12
View File
@@ -50,9 +50,9 @@ To rebuild the current binary from source using the current binary:
```bash
cd /path/to/el
./dist/platform/elc elc-cli.el elc-new.c
cc -std=c11 -I runtime -lcurl -lpthread \
cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
-o dist/platform/elc-new \
elc-new.c runtime/el_runtime.c
elc-new.c el-compiler/runtime/el_runtime.c
```
Verify self-hosting by using `elc-new` to recompile itself and diffing the outputs.
@@ -288,14 +288,14 @@ The codegen tracks declared names per C scope. When `count` is already in `decla
## 3. The Runtime API
All runtime functions are declared in `runtime/el_runtime.h`. Every compiled El program links against `runtime/el_runtime.c`.
All runtime functions are declared in `el-compiler/runtime/el_runtime.h`. Every compiled El program links against `el-compiler/runtime/el_runtime.c`.
All values are `el_val_t` (`int64_t`). Strings are pointers cast through `int64_t` using `EL_STR(s)` / `EL_CSTR(v)` macros.
Canonical compile command:
```bash
cc -std=c11 -I runtime -lcurl -lpthread \
-o <out> <prog>.c runtime/el_runtime.c
cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
-o <out> <prog>.c el-compiler/runtime/el_runtime.c
```
### I/O
@@ -794,8 +794,8 @@ Using your minimal implementation, compile `elc-cli.el` (which imports the entir
python3 minimal_elc.py elc-cli.el > elc-new.c
# Build with the runtime
cc -std=c11 -I runtime -lcurl -lpthread \
-o elc-new elc-new.c runtime/el_runtime.c
cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
-o elc-new elc-new.c el-compiler/runtime/el_runtime.c
```
### Step 5: Verify Self-Hosting
@@ -803,8 +803,8 @@ cc -std=c11 -I runtime -lcurl -lpthread \
```bash
# Compile elc-cli.el with the new compiler
./elc-new elc-cli.el elc-v2.c
cc -std=c11 -I runtime -lcurl -lpthread \
-o elc-v2 elc-v2.c runtime/el_runtime.c
cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
-o elc-v2 elc-v2.c el-compiler/runtime/el_runtime.c
# Compile again with the second-generation compiler
./elc-v2 elc-cli.el elc-v3.c
@@ -880,9 +880,9 @@ This is the planned path. It does not exist yet.
| `el-compiler/src/parser.el` | Recursive descent parser. `parse(tokens)` → AST. All statement and expression forms | 1071 |
| `el-compiler/src/codegen.el` | C code emitter. `codegen(stmts, source)` → (streams to stdout). Expression codegen, statement codegen, function codegen, type tracking, capability enforcement, temporal type dispatch | 2721 |
| `el-compiler/src/codegen-js.el` | JavaScript backend. `codegen_js(stmts, source)` → JS source | ~500 |
| `runtime/el_runtime.h` | Full runtime API declaration | 755 |
| `runtime/el_runtime.c` | Full runtime implementation | large |
| `runtime/el_runtime.js` | JS runtime | — |
| `el-compiler/runtime/el_runtime.h` | Full runtime API declaration | 755 |
| `el-compiler/runtime/el_runtime.c` | Full runtime implementation | large |
| `el-compiler/runtime/el_runtime.js` | JS runtime | — |
| `elb.el` | Build coordinator. Reads `manifest.el`, walks import graph, compiles modules, links binary. The `.NET`-style incremental build model | 367 |
| `elc-combined.el` | Pre-merged single-file bootstrap edition (for early bootstrap iterations) | large |
| `spec/language.md` | Language specification v1.2.0 | — |
@@ -132,7 +132,7 @@ if [[ $LVGL_OK -eq 1 ]]; then
else
_miss "LVGL/MCU" "-DEL_TARGET_LVGL (lvgl.h not found)"
echo " Install: git clone https://github.com/lvgl/lvgl"
echo " (place lvgl/ next to runtime/)"
echo " (place lvgl/ next to el-compiler/runtime/)"
MISSING=$((MISSING + 1))
fi
File diff suppressed because it is too large Load Diff
+897
View File
@@ -0,0 +1,897 @@
/*
* el_runtime.h — El language C runtime header
*
* Declares all built-in functions available to compiled El programs.
* Include this in every generated .c file.
*
* Value model:
* All El values are represented as el_val_t (= int64_t).
* On 64-bit systems a pointer fits in int64_t.
* String values are cast: (el_val_t)(uintptr_t)"hello"
* Integer values are stored directly.
* This lets arithmetic work naturally while still passing strings around.
*
* Type conventions (El -> C):
* String -> el_val_t (holds const char* via uintptr_t cast)
* Int -> el_val_t
* Bool -> el_val_t (0 = false, nonzero = true)
* Any -> el_val_t
* Void -> void
*
* Macros for convenience:
* EL_STR(s) cast string literal to el_val_t
* EL_CSTR(v) cast el_val_t back to const char*
* EL_INT(v) identity — el_val_t is already int64_t
* EL_NULL null / zero value
* EL_FALSE boolean false (0)
* EL_TRUE boolean true (1)
*
* Link requirements:
* -lcurl — required for the HTTP client (http_get, http_post, llm_*).
* -lpthread — required for the HTTP server (one detached thread per
* connection, capped at 64 concurrent).
* -loqs — optional; required only when liboqs is installed and the
* pq_* / sha3_256_hex entry points are needed. Detected at
* compile time via __has_include(<oqs/oqs.h>).
* -lcrypto — optional; pulled in alongside -loqs. Used for X25519 in
* pq_hybrid_* and HKDF-SHA256 derivation.
*
* Canonical compile command:
* cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
* -o <out> <prog>.c el-compiler/runtime/el_runtime.c
*
* With liboqs (post-quantum stack):
* cc -std=c11 -I el-compiler/runtime -lcurl -lpthread -loqs -lcrypto \
* -o <out> <prog>.c el-compiler/runtime/el_runtime.c
*/
#pragma once
#include <stdint.h>
#include <stdlib.h>
typedef int64_t el_val_t;
/* HTTP request-handler function-pointer types. Public because soul modules (routes/chat/etc.)
* register handlers across translation units; previously defined only inside el_runtime.c, which
* made cross-module references (and the Windows build) fail. Home in the shared header. */
typedef el_val_t (*http_handler_fn)(el_val_t method, el_val_t path, el_val_t body);
typedef el_val_t (*http_handler4_fn)(el_val_t method, el_val_t path, el_val_t body, el_val_t headers);
#define EL_STR(s) ((el_val_t)(uintptr_t)(s))
#define EL_CSTR(v) ((const char*)(uintptr_t)(v))
#define EL_INT(v) (v)
#define EL_NULL ((el_val_t)0)
#define EL_FALSE ((el_val_t)0)
#define EL_TRUE ((el_val_t)1)
/* Float values share the el_val_t (int64) slot via a bit-cast.
* The codegen emits Float literals as `el_from_float(<dbl>)` so the
* underlying bits represent the IEEE 754 double. Float-aware builtins
* (math, format, json) round-trip via these helpers. */
static inline double el_to_float(el_val_t v) {
union { int64_t i; double f; } u;
u.i = (int64_t)v;
return u.f;
}
static inline el_val_t el_from_float(double f) {
union { double f; int64_t i; } u;
u.f = f;
return (el_val_t)u.i;
}
#ifdef __cplusplus
extern "C" {
#endif
/* ── I/O ──────────────────────────────────────────────────────────────────── */
el_val_t println(el_val_t s);
el_val_t print(el_val_t s);
el_val_t readline(void);
/* ── String builtins ─────────────────────────────────────────────────────── */
el_val_t el_str_concat(el_val_t a, el_val_t b);
el_val_t str_eq(el_val_t a, el_val_t b);
el_val_t str_starts_with(el_val_t s, el_val_t prefix);
el_val_t str_ends_with(el_val_t s, el_val_t suffix);
el_val_t str_len(el_val_t s);
el_val_t str_concat(el_val_t a, el_val_t b);
el_val_t int_to_str(el_val_t n);
el_val_t str_to_int(el_val_t s);
el_val_t native_str_to_int(el_val_t s);
el_val_t str_slice(el_val_t s, el_val_t start, el_val_t end);
el_val_t str_contains(el_val_t s, el_val_t sub);
el_val_t str_replace(el_val_t s, el_val_t from, el_val_t to);
el_val_t str_to_upper(el_val_t s);
el_val_t str_to_lower(el_val_t s);
el_val_t str_trim(el_val_t s);
/* ── Math ────────────────────────────────────────────────────────────────── */
el_val_t el_abs(el_val_t n);
el_val_t el_max(el_val_t a, el_val_t b);
el_val_t el_min(el_val_t a, el_val_t b);
/* ── Refcount (ARC) ──────────────────────────────────────────────────────────
* Lists and Maps carry a refcount. Strings and ints do not — el_retain and
* el_release are safe no-ops on non-refcounted values (they sniff a magic
* header at offset 0 and only act if the magic matches).
*
* Codegen emits these at let-binding shadowing, function entry (params), and
* function exit (locals other than the returned value). The refcount lets
* el_list_append and el_map_set mutate in place when uniquely owned (cheap)
* and copy-on-write when shared (preserves persistent semantics across
* accumulator patterns in the compiler itself). */
void el_retain(el_val_t v);
void el_release(el_val_t v);
/* ── Scoped arena (CLI use) ───────────────────────────────────────────────── */
el_val_t el_arena_push(void);
el_val_t el_arena_pop(el_val_t mark);
/* ── List ────────────────────────────────────────────────────────────────── */
el_val_t el_list_new(el_val_t count, ...);
el_val_t el_list_len(el_val_t list);
el_val_t el_list_get(el_val_t list, el_val_t index);
el_val_t el_list_append(el_val_t list, el_val_t elem);
el_val_t el_list_empty(void);
el_val_t el_list_clone(el_val_t list);
/* ── Map ─────────────────────────────────────────────────────────────────── */
el_val_t el_map_new(el_val_t pair_count, ...);
el_val_t el_get_field(el_val_t map, el_val_t key);
el_val_t el_map_get(el_val_t map, el_val_t key);
el_val_t el_map_set(el_val_t map, el_val_t key, el_val_t value);
/* ── HTTP ─────────────────────────────────────────────────────────────────── */
el_val_t http_get(el_val_t url);
el_val_t http_post(el_val_t url, el_val_t body);
el_val_t http_post_json(el_val_t url, el_val_t json_body);
el_val_t http_get_with_headers(el_val_t url, el_val_t headers_map);
el_val_t http_post_with_headers(el_val_t url, el_val_t body, el_val_t headers_map);
el_val_t http_post_json_with_headers(el_val_t url, el_val_t headers_map, el_val_t json_body);
el_val_t http_post_form_auth(el_val_t url, el_val_t form_body, el_val_t auth_header);
el_val_t http_delete(el_val_t url);
el_val_t http_serve(el_val_t port, el_val_t handler);
el_val_t http_set_handler(el_val_t name);
/* HTTP server v2 ─────────────────────────────────────────────────────────────
* Same dispatch model as http_serve, but the handler signature is widened:
*
* el_val_t handler(method, path, headers_map, body)
*
* `headers_map` is an ElMap from lowercased header name → header value (both
* Strings). Repeated headers are joined with ", " per RFC 7230.
*
* Response value: the handler may return either
* (a) a plain body string — same auto-content-type / 200-OK behaviour as
* http_serve (3-arg) — or
* (b) a response envelope built with `http_response(status, headers_json,
* body)`. The runtime detects the envelope discriminator
* `"el_http_response":1` at the start of the returned string and
* unpacks status / headers / body before sending.
*
* The 3-arg http_serve(port, handler) remains supported unchanged for
* existing handlers (e.g. products/web/server.el): it dispatches with
* (method, path, body), hardcodes 200 OK, and auto-detects content type. */
el_val_t http_serve_v2(el_val_t port, el_val_t handler);
void http_serve_async(el_val_t port, el_val_t handler);
el_val_t http_set_handler_v2(el_val_t name);
/* Build an HTTP response envelope. `headers_json` should be a JSON object
* literal like `{"WWW-Authenticate":"Basic"}` (or "" / "{}" for none). The
* returned string carries the discriminator `{"el_http_response":1,...}`
* which the runtime's send-path detects and unpacks. Detection happens
* uniformly inside http_send_response, so a 3-arg handler may also return
* an envelope. The 3-arg variant remains documented as a fixed 200-OK
* auto-content-type contract for legacy handlers that return plain bodies. */
el_val_t http_response(el_val_t status, el_val_t headers_json, el_val_t body);
/* SSE connection fd — set by http_worker_v2 before calling the El handler,
* cleared afterwards. Defined in el_seed.c; called from el_runtime.c.
* The getter is exposed as __http_conn_fd() to El programs. */
void el_seed_set_http_conn_fd(int fd);
/* HTTP timeout — every libcurl request honors EL_HTTP_TIMEOUT_MS (default
* 60000ms). Read lazily on first use, so setting the env var any time before
* the first http_* call is sufficient. */
/* Streaming variants — write the response body straight to a file via
* libcurl's CURLOPT_WRITEFUNCTION = fwrite. These bypass the el_val_t string
* wrapper entirely, so binary payloads (audio/mpeg, image/png, etc.) survive
* embedded NUL bytes that would truncate a strlen()-based code path.
*
* Both honor EL_HTTP_TIMEOUT_MS, follow redirects, and accept the same
* `headers_map` shape as http_post_with_headers (ElMap of String→String).
*
* Return value: 1 on success (file fully written), 0 on any failure
* (network, file open, partial write). On failure the output file is removed
* so callers cannot mistake a partially-written file for a valid one. */
el_val_t http_post_to_file(el_val_t url, el_val_t body, el_val_t headers_map, el_val_t output_path);
el_val_t http_get_to_file(el_val_t url, el_val_t headers_map, el_val_t output_path);
/* ── URL encoding ────────────────────────────────────────────────────────── */
el_val_t url_encode(el_val_t s); /* RFC 3986 unreserved set */
el_val_t url_decode(el_val_t s); /* '+' → space, %XX → byte */
/* ── HTML allowlist sanitizer ────────────────────────────────────────────────
* el_html_sanitize(input_html, allowlist_json) — strict allowlist HTML
* cleaner. State-machine parser; tag/attribute names compared case-
* insensitively against the allowlist; `<a href>` / `<… src>` URL schemes
* validated (http, https, mailto, fragment-only, or relative); whole-
* subtree drop for script / style / iframe / object / embed / form; HTML-
* escapes free text outside dropped subtrees.
*
* The allowlist is JSON of the form
* {"p":[],"a":["href","title"],"strong":[],...}
* where each value is the array of attribute names allowed for that tag. */
el_val_t el_html_sanitize(el_val_t input_html, el_val_t allowlist_json);
el_val_t html_raw(el_val_t s);
el_val_t html_escape(el_val_t s);
/* ── Filesystem ──────────────────────────────────────────────────────────── */
el_val_t fs_read(el_val_t path);
el_val_t fs_write(el_val_t path, el_val_t content);
el_val_t fs_list(el_val_t path);
el_val_t fs_list_json(el_val_t path);
el_val_t fs_exists(el_val_t path);
el_val_t fs_mkdir(el_val_t path); /* mkdir -p, mode 0755 */
/* Length-explicit binary write. `length` is an Int (el_val_t holding the
* byte count). The caller knows the length from context — typically because
* `bytes` came from base64_decode (which produces a magic-tagged binary
* buffer with embedded NULs possible) and the caller already tracks the
* decoded length, OR because the bytes came from a fixed-size source
* (sha256_bytes = 32, hmac_sha256_bytes = 32). Bypasses strlen entirely.
*
* Returns 1 on success, 0 on failure (invalid path, can't open, partial
* write, negative length). On partial-write failure, the file is removed
* so callers cannot read back a truncated artefact. */
el_val_t fs_write_bytes(el_val_t path, el_val_t bytes, el_val_t length);
/* ── JSON ────────────────────────────────────────────────────────────────── */
el_val_t json_get(el_val_t json, el_val_t key);
el_val_t json_parse(el_val_t s);
el_val_t json_stringify(el_val_t v);
el_val_t json_get_string(el_val_t json_str, el_val_t key);
el_val_t json_get_int(el_val_t json_str, el_val_t key);
el_val_t json_get_float(el_val_t json_str, el_val_t key);
el_val_t json_get_bool(el_val_t json_str, el_val_t key);
el_val_t json_get_raw(el_val_t json_str, el_val_t key);
el_val_t json_set(el_val_t json_str, el_val_t key, el_val_t value);
el_val_t json_array_len(el_val_t json_str);
el_val_t json_array_get(el_val_t json_str, el_val_t index);
el_val_t json_array_get_string(el_val_t json_str, el_val_t index);
el_val_t json_escape_string(el_val_t sv);
el_val_t json_build_object(el_val_t kvs);
el_val_t json_build_array(el_val_t items);
/* ── Time ────────────────────────────────────────────────────────────────── */
el_val_t time_now(void);
el_val_t time_now_utc(void);
el_val_t sleep_secs(el_val_t secs);
el_val_t sleep_ms(el_val_t ms);
el_val_t time_format(el_val_t ts, el_val_t fmt);
el_val_t time_to_parts(el_val_t ts);
el_val_t time_from_parts(el_val_t secs, el_val_t ns, el_val_t tz);
el_val_t time_add(el_val_t ts, el_val_t n, el_val_t unit);
el_val_t time_diff(el_val_t ts1, el_val_t ts2, el_val_t unit);
el_val_t now_ns(void);
/* ── Instant + Duration: first-class temporal types ──────────────────────────
* Both types share the el_val_t (int64) slot. Instants are nanoseconds
* since the Unix epoch; Durations are signed nanoseconds. Type discipline
* is enforced at codegen-time: BinOps on names registered as Instant or
* Duration route through the typed wrappers below; mismatches like
* Instant+Instant become #error at the C compiler.
*
* Postfix literals — `30.seconds`, `1.hour`, `500.millis`, `30.nanos` — are
* recognised by the parser as DurationLit AST nodes and lowered to literal
* int64 nanoseconds at codegen time. The runtime never sees the units. */
el_val_t el_now_instant(void);
el_val_t now(void);
el_val_t unix_seconds(el_val_t n);
el_val_t unix_millis(el_val_t n);
el_val_t instant_from_iso8601(el_val_t s);
el_val_t el_duration_from_nanos(el_val_t ns);
el_val_t duration_seconds(el_val_t n);
el_val_t duration_millis(el_val_t n);
el_val_t duration_nanos(el_val_t n);
el_val_t el_instant_add_dur(el_val_t inst, el_val_t dur);
el_val_t el_instant_sub_dur(el_val_t inst, el_val_t dur);
el_val_t el_instant_diff(el_val_t a, el_val_t b);
el_val_t el_duration_add(el_val_t a, el_val_t b);
el_val_t el_duration_sub(el_val_t a, el_val_t b);
el_val_t el_duration_scale(el_val_t dur, el_val_t scalar);
el_val_t el_duration_div(el_val_t dur, el_val_t scalar);
el_val_t el_instant_lt(el_val_t a, el_val_t b);
el_val_t el_instant_le(el_val_t a, el_val_t b);
el_val_t el_instant_gt(el_val_t a, el_val_t b);
el_val_t el_instant_ge(el_val_t a, el_val_t b);
el_val_t el_instant_eq(el_val_t a, el_val_t b);
el_val_t el_instant_ne(el_val_t a, el_val_t b);
el_val_t el_duration_lt(el_val_t a, el_val_t b);
el_val_t el_duration_le(el_val_t a, el_val_t b);
el_val_t el_duration_gt(el_val_t a, el_val_t b);
el_val_t el_duration_ge(el_val_t a, el_val_t b);
el_val_t el_duration_eq(el_val_t a, el_val_t b);
el_val_t el_duration_ne(el_val_t a, el_val_t b);
el_val_t instant_to_unix_seconds(el_val_t i);
el_val_t instant_to_unix_millis(el_val_t i);
el_val_t instant_to_iso8601(el_val_t i);
el_val_t duration_to_seconds(el_val_t d);
el_val_t duration_to_millis(el_val_t d);
el_val_t duration_to_nanos(el_val_t d);
el_val_t el_sleep_duration(el_val_t dur);
el_val_t unix_timestamp(void);
el_val_t ttl_cache_set(el_val_t key, el_val_t value);
el_val_t ttl_cache_get(el_val_t key, el_val_t max_age);
el_val_t ttl_cache_age(el_val_t key);
/* ── Calendar + CalendarTime + Rhythm + LocalDate/Time/DateTime ─────────────
* Phase 1.5 of the time system. Calendar is pluggable: EarthCalendar (IANA
* zones, Gregorian, DST) is the user-facing default; MarsCalendar,
* CycleCalendar(period), NoCycleCalendar, RelativeCalendar handle non-Earth
* domains.
*
* A Calendar interprets an Instant under a particular cycle convention and
* produces a CalendarTime. CalendarTime carries the underlying Instant and
* a back-pointer to its Calendar; arithmetic and formatting consult the
* Calendar to convert ns since epoch into year/month/day/hour/minute/second
* (or sol/phase, or cycle/phase, depending on kind).
*
* Storage convention: Calendar / CalendarTime / Rhythm / LocalDate /
* LocalDateTime are heap-allocated structs whose pointers are cast into
* el_val_t. A 24-bit magic header at offset 0 lets the runtime identify
* the kind safely. LocalTime is small enough to live in the int64 slot
* directly (nanos since midnight, signed). */
/* Zone — opaque IANA zone or fixed offset, used by EarthCalendar.
* `zone_id` is either an IANA name ("America/New_York", "UTC") or a fixed
* offset string ("+05:30", "-08:00"). The runtime resolves it via tzset()
* on first use of the owning EarthCalendar. */
el_val_t zone(el_val_t id);
el_val_t zone_utc(void);
el_val_t zone_local(void);
el_val_t zone_offset(el_val_t hours, el_val_t minutes);
/* Calendar constructors. Each returns an el_val_t pointer to a heap-
* allocated, magic-tagged Calendar struct. Calendars are interned by
* (kind, zone_id, period_ns, epoch_ns) so identical constructors return
* the same pointer — equality is reference equality. */
el_val_t earth_calendar(el_val_t z);
el_val_t earth_calendar_default(void);
el_val_t mars_calendar(void);
el_val_t cycle_calendar(el_val_t period_dur);
el_val_t no_cycle_calendar(void);
el_val_t relative_calendar(el_val_t epoch_inst);
/* CalendarTime constructors and methods. Returns a heap-allocated struct
* whose pointer fits in el_val_t. */
el_val_t now_in(el_val_t cal);
el_val_t in_calendar(el_val_t inst, el_val_t cal);
el_val_t cal_format(el_val_t ct, el_val_t pattern);
el_val_t cal_to_instant(el_val_t ct);
el_val_t cal_cycle_phase(el_val_t ct);
el_val_t cal_in(el_val_t ct, el_val_t cal);
/* LocalDate / LocalTime / LocalDateTime — calendar-agnostic value types.
* LocalTime carries nanoseconds since midnight as a signed int64 directly
* in the el_val_t slot (no allocation). LocalDate / LocalDateTime are
* heap-allocated structs with magic headers. */
el_val_t local_date(el_val_t y, el_val_t m, el_val_t d);
el_val_t local_time(el_val_t h, el_val_t m, el_val_t s, el_val_t ns);
el_val_t local_datetime(el_val_t date, el_val_t time);
el_val_t zoned(el_val_t date, el_val_t time, el_val_t cal);
el_val_t local_date_year(el_val_t ld);
el_val_t local_date_month(el_val_t ld);
el_val_t local_date_day(el_val_t ld);
el_val_t local_time_hour(el_val_t lt);
el_val_t local_time_minute(el_val_t lt);
el_val_t local_time_second(el_val_t lt);
el_val_t local_time_nanos(el_val_t lt);
el_val_t el_local_date_add_dur(el_val_t ld, el_val_t dur);
el_val_t el_local_time_add_dur(el_val_t lt, el_val_t dur);
el_val_t el_local_date_lt(el_val_t a, el_val_t b);
el_val_t el_local_date_eq(el_val_t a, el_val_t b);
/* Rhythm — pluggable recurrence AST. Returns a heap-allocated struct
* pointer in el_val_t; rhythms are immutable so callers may share them. */
el_val_t rhythm_cycle_start(void);
el_val_t rhythm_cycle_phase(el_val_t phase);
el_val_t rhythm_duration(el_val_t d);
el_val_t rhythm_session_start(void);
el_val_t rhythm_event(el_val_t name);
el_val_t rhythm_and(el_val_t a, el_val_t b);
el_val_t rhythm_or(el_val_t a, el_val_t b);
el_val_t rhythm_weekday(el_val_t day);
el_val_t rhythm_weekly_at(el_val_t day, el_val_t hour, el_val_t minute);
el_val_t rhythm_next_after(el_val_t r, el_val_t after, el_val_t cal);
el_val_t rhythm_matches(el_val_t r, el_val_t ct);
/* ── UUID ────────────────────────────────────────────────────────────────── */
el_val_t uuid_new(void);
el_val_t uuid_v4(void);
/* ── Environment ─────────────────────────────────────────────────────────── */
el_val_t env(el_val_t key);
/* ── In-process state K/V ────────────────────────────────────────────────── */
el_val_t state_set(el_val_t key, el_val_t value);
el_val_t state_get(el_val_t key);
el_val_t state_del(el_val_t key);
el_val_t state_keys(void);
el_val_t state_has(el_val_t key);
el_val_t state_get_or(el_val_t key, el_val_t default_val);
/* ── Float formatting ────────────────────────────────────────────────────── */
el_val_t float_to_str(el_val_t f);
el_val_t int_to_float(el_val_t n);
el_val_t float_to_int(el_val_t f);
el_val_t format_float(el_val_t f, el_val_t decimals);
el_val_t decimal_round(el_val_t f, el_val_t decimals);
el_val_t str_to_float(el_val_t s);
/* ── Math (Float-aware) ──────────────────────────────────────────────────── */
el_val_t math_sqrt(el_val_t f);
el_val_t math_log(el_val_t f);
el_val_t math_ln(el_val_t f);
el_val_t math_sin(el_val_t f);
el_val_t math_cos(el_val_t f);
el_val_t math_pi(void);
/* ── String additions ────────────────────────────────────────────────────── */
el_val_t str_index_of(el_val_t s, el_val_t sub);
el_val_t str_split(el_val_t s, el_val_t sep);
el_val_t str_char_at(el_val_t s, el_val_t i);
el_val_t str_char_code(el_val_t s, el_val_t i);
el_val_t str_pad_left(el_val_t s, el_val_t width, el_val_t pad);
el_val_t str_pad_right(el_val_t s, el_val_t width, el_val_t pad);
el_val_t str_format(el_val_t fmt, el_val_t data);
el_val_t str_lower(el_val_t s);
el_val_t str_upper(el_val_t s);
/* ── Text-processing primitives (Phase 1: byte/codepoint, ASCII char classes)
* Phase 2 (filed): Unicode-grapheme awareness, NFC/NFD normalization, regex.
* is_* predicates: empty input returns false; multi-char requires ALL bytes
* to match. ASCII ranges only in Phase 1. */
/* Counting */
el_val_t str_count(el_val_t s, el_val_t sub); /* non-overlapping */
el_val_t str_count_chars(el_val_t s); /* codepoint count */
el_val_t str_count_bytes(el_val_t s); /* alias of str_len */
el_val_t str_count_lines(el_val_t s);
el_val_t str_count_words(el_val_t s);
el_val_t str_count_letters(el_val_t s); /* ASCII [A-Za-z] */
el_val_t str_count_digits(el_val_t s); /* ASCII [0-9] */
/* Find / position */
el_val_t str_index_of_all(el_val_t s, el_val_t sub); /* [Int] of byte offsets */
el_val_t str_last_index_of(el_val_t s, el_val_t sub);
el_val_t str_find_chars(el_val_t s, el_val_t any_of); /* first idx of any ch */
/* Transform */
el_val_t str_repeat(el_val_t s, el_val_t n);
el_val_t str_reverse(el_val_t s); /* by codepoint */
el_val_t str_strip_prefix(el_val_t s, el_val_t prefix);
el_val_t str_strip_suffix(el_val_t s, el_val_t suffix);
el_val_t str_strip_chars(el_val_t s, el_val_t chars);
el_val_t str_lstrip(el_val_t s);
el_val_t str_rstrip(el_val_t s);
/* Char classification (Bool) */
el_val_t is_letter(el_val_t s);
el_val_t is_digit(el_val_t s);
el_val_t is_alphanumeric(el_val_t s);
el_val_t is_whitespace(el_val_t s);
el_val_t is_punctuation(el_val_t s);
el_val_t is_uppercase(el_val_t s);
el_val_t is_lowercase(el_val_t s);
/* Split / join */
el_val_t str_split_lines(el_val_t s);
el_val_t str_split_chars(el_val_t s); /* alias of native_string_chars */
el_val_t str_split_n(el_val_t s, el_val_t sep, el_val_t n);
el_val_t str_join(el_val_t list, el_val_t sep); /* alias of list_join */
/* ── List additions ──────────────────────────────────────────────────────── */
el_val_t list_push(el_val_t list, el_val_t elem);
el_val_t list_push_front(el_val_t list, el_val_t elem);
el_val_t list_join(el_val_t list, el_val_t sep);
el_val_t list_range(el_val_t start, el_val_t end);
/* ── Bool helpers ────────────────────────────────────────────────────────── */
el_val_t bool_to_str(el_val_t b);
/* ── Numeric parsing ─────────────────────────────────────────────────────── */
el_val_t parse_int(el_val_t s, el_val_t default_val);
/* ── Process ─────────────────────────────────────────────────────────────── */
el_val_t exit_program(el_val_t code);
el_val_t getpid_now(void);
/* Self-terminating memory guard. Reads ELC_MAX_MEM_MB (default 512) and
* exits with code 1 if resident memory exceeds the limit. Call periodically
* during long compilation loops (e.g. after each function is compiled).
* Returns 0 when memory is within bounds. */
el_val_t el_mem_check(void);
/* ── CGI identity ─────────────────────────────────────────────────────────────
* Called at the start of main() in CGI programs (those with a `cgi {}` block).
* Records the program's DHARMA identity before any other code executes. */
void el_cgi_init(el_val_t name, el_val_t dharma_id, el_val_t principal,
el_val_t network, el_val_t engram);
/* ── DHARMA network builtins ─────────────────────────────────────────────────
* Available to CGI programs (declared with a `cgi {}` block).
*
* Peers are addressed by `dharma_id` of the form
* "<registry-id>@<transport-url>" e.g. "ntn-genesis@http://localhost:7770"
* If the @<url> portion is omitted, transport defaults to
* "http://localhost:7770" (the local CGI daemon assumption).
*
* Wire protocol (all peers expose):
* POST <url>/dharma/recv { channel, from, content } → response body
* POST <url>/dharma/event { type, payload, source, timestamp }
* POST <url>/api/activate { query } → list of nodes
*
* Hosting application's responsibility: an El program with a `cgi {}` block
* runs http_serve() with its own request handler; that handler should route
* "/dharma/event" requests by calling el_runtime_dharma_event_arrive() so
* incoming events feed dharma_field() queues. The runtime itself does not
* intercept any /dharma path. */
el_val_t dharma_connect(el_val_t cgi_id);
el_val_t dharma_send(el_val_t channel, el_val_t content);
el_val_t dharma_activate(el_val_t query);
void dharma_emit(el_val_t event_type, el_val_t payload);
el_val_t dharma_field(el_val_t event_type);
void dharma_strengthen(el_val_t cgi_id, el_val_t weight);
el_val_t dharma_relationship(el_val_t cgi_id);
el_val_t dharma_peers(void);
/* Public C API: called by an El program's HTTP handler when a /dharma/event
* request arrives. Pushes onto the per-event-type queue and signals any
* pending dharma_field() blockers. All three arguments must be NUL-terminated
* C strings (or NULL — then treated as empty). */
void el_runtime_dharma_event_arrive(const char* event_type,
const char* payload,
const char* source);
/* ── Engram local graph primitives ───────────────────────────────────────────
* Operate on the CGI's local Engram knowledge graph.
* `engram_activate` queries the local graph only; `dharma_activate` is
* network-wide across all connected CGI graphs. */
el_val_t engram_node(el_val_t content, el_val_t node_type, el_val_t salience);
el_val_t engram_node_full(el_val_t content, el_val_t node_type, el_val_t label,
el_val_t salience, el_val_t importance, el_val_t confidence,
el_val_t tier, el_val_t tags);
/* Layered consciousness — see el_runtime.c for the layered architecture
* design notes (search "Layered consciousness architecture"). The five
* canonical layers (safety / core-identity / domain-knowledge / imprint /
* suit) are seeded automatically; engram_add_layer extends the registry
* with imprint or suit overlays at runtime. Nodes default to layer 1
* (core-identity) when created via engram_node / engram_node_full. */
el_val_t engram_node_layered(el_val_t content, el_val_t node_type, el_val_t label,
el_val_t salience, el_val_t certainty, el_val_t confidence,
el_val_t status, el_val_t tags, el_val_t layer_id);
el_val_t engram_add_layer(el_val_t name, el_val_t priority, el_val_t suppressible,
el_val_t transparent, el_val_t injectable);
el_val_t engram_remove_layer(el_val_t layer_id);
el_val_t engram_list_layers(void);
el_val_t engram_get_node(el_val_t id);
void engram_strengthen(el_val_t node_id);
void engram_forget(el_val_t node_id);
el_val_t engram_node_count(void);
el_val_t engram_search(el_val_t query, el_val_t limit);
el_val_t engram_scan_nodes(el_val_t limit, el_val_t offset);
void engram_connect(el_val_t from_id, el_val_t to_id, el_val_t weight, el_val_t relation);
el_val_t engram_edge_between(el_val_t from_id, el_val_t to_id);
el_val_t engram_neighbors(el_val_t node_id);
el_val_t engram_neighbors_filtered(el_val_t node_id, el_val_t max_depth, el_val_t direction);
el_val_t engram_edge_count(void);
/* Three-pass activation: background fan-out → working-memory promotion →
* Layer 0 override. See "Three-pass activation" in el_runtime.c. */
el_val_t engram_activate(el_val_t query, el_val_t depth);
el_val_t engram_save(el_val_t path);
el_val_t engram_load(el_val_t path);
/* JSON-string accessors — return pre-serialized JSON so HTTP handlers
* can pass results straight through without round-tripping ElList/ElMap
* through json_stringify. */
el_val_t engram_get_node_json(el_val_t id);
el_val_t engram_get_node_by_label(el_val_t label);
el_val_t engram_search_json(el_val_t query, el_val_t limit);
el_val_t engram_scan_nodes_json(el_val_t limit, el_val_t offset);
el_val_t engram_scan_nodes_by_type_json(el_val_t node_type, el_val_t limit, el_val_t offset);
el_val_t engram_neighbors_json(el_val_t node_id, el_val_t max_depth, el_val_t direction);
el_val_t engram_activate_json(el_val_t query, el_val_t depth);
el_val_t engram_stats_json(void);
el_val_t engram_list_layers_json(void);
/* engram_compile_layered_json — produce a prompt-ready text block split
* into "[LAYER 0 — STRUCTURAL]" (non-suppressible layers, sacred fire)
* and "[ENGRAM CONTEXT]" (standard suppressible layers). Returns "" if
* no nodes promoted to working memory. */
el_val_t engram_compile_layered_json(el_val_t intent, el_val_t depth);
/* ── Working memory ──────────────────────────────────────────────────────────*/
el_val_t engram_wm_count(void);
el_val_t engram_wm_avg_weight(void);
el_val_t engram_wm_top_json(el_val_t n);
el_val_t engram_load_merge(el_val_t path);
/* ── LLM (Anthropic API client) ─────────────────────────────────────────────
* All functions call https://api.anthropic.com/v1/messages with the API key
* from env ANTHROPIC_API_KEY. Default model when empty: claude-sonnet-4-5. */
el_val_t llm_call(el_val_t model, el_val_t prompt);
el_val_t llm_call_system(el_val_t model, el_val_t system_prompt, el_val_t user_prompt);
el_val_t llm_call_agentic(el_val_t model, el_val_t system, el_val_t user, el_val_t tools);
el_val_t llm_vision(el_val_t model, el_val_t system, el_val_t prompt, el_val_t image_url_or_b64);
el_val_t llm_models(void);
/* Register a tool handler by name. The handler is looked up via dlsym
* (mirroring http_set_handler), so any El `fn <name>(input)` compiles to
* a global C symbol that this function can locate at runtime.
* Handler signature: `el_val_t handler(el_val_t input_json)` — receives
* the tool input as a JSON-string el_val_t and returns a JSON-string
* el_val_t result. Used by llm_call_agentic. */
void llm_register_tool(el_val_t name, el_val_t handler_fn_name);
/* ── args() ─────────────────────────────────────────────────────────────────
* Provides access to command-line arguments passed to the program.
* Populated by el_runtime_init_args() before main() runs. */
el_val_t args(void);
void el_runtime_init_args(int argc, char** argv);
/* ── Crypto primitives ─────────────────────────────────────────────────────
* SHA-256, HMAC-SHA-256, and base64 (standard + URL-safe).
* Self-contained — no OpenSSL/libcrypto dependency. The implementations are
* adapted from public-domain reference code (Brad Conte / RFC 4648).
*
* Bytes-returning variants (sha256_bytes, hmac_sha256_bytes) return a string
* value whose contents are raw binary; callers usually feed these into
* base64_encode. Note that el_val_t strings are NUL-terminated by convention,
* so the binary payload may contain embedded NULs — pass it directly into
* base64_encode (which uses an explicit length) rather than treating it as
* a printable C string.
*
* The "base64" variants emit/accept RFC 4648 standard alphabet with padding.
* The "base64url" variants use URL-safe alphabet (`-`/`_`) with no padding,
* as used in JWTs. */
el_val_t sha256_hex(el_val_t input);
el_val_t sha256_bytes(el_val_t input);
el_val_t hmac_sha256_hex(el_val_t key, el_val_t message);
el_val_t hmac_sha256_bytes(el_val_t key, el_val_t message);
el_val_t base64_encode(el_val_t input);
el_val_t base64_decode(el_val_t input);
el_val_t base64url_encode(el_val_t input);
el_val_t base64url_decode(el_val_t input);
/* Length-aware variants (internal — exposed for the rare caller that already
* has a known-length binary buffer and doesn't want to round-trip through
* a NUL-terminated el_val_t string). Sha256_bytes and hmac_sha256_bytes feed
* these implicitly. */
el_val_t el_sha256_bytes_n(const unsigned char* data, size_t len);
el_val_t el_base64_encode_n(const unsigned char* data, size_t len, int url_safe);
/* ── Post-quantum primitives (liboqs-backed) ────────────────────────────────
* All inputs/outputs hex-encoded. Algorithm choices:
* Signature: CRYSTALS-Dilithium-3 (NIST level 3, balanced)
* KEM: CRYSTALS-Kyber-768 (NIST level 3)
* Hash: SHA3-256 (Keccak) (PQ-aware protocols favour SHA3 over SHA2)
*
* If liboqs is not linked (detected via __has_include(<oqs/oqs.h>) at compile
* time), the pq_* entry points return a JSON-shaped error string so callers
* fail loudly rather than silently fall back to classical schemes:
* {"error":"liboqs not linked, post-quantum primitives unavailable"}
*
* The hybrid handshake pairs X25519 with Kyber-768 per NIST PQ guidance and
* CNSA 2.0. Combined shared secret is HKDF-SHA256(x25519_ss || kyber_ss).
* Even if Kyber falls, X25519 holds; if X25519 falls under quantum attack,
* Kyber holds. SHA3-256 also remains usable independent of liboqs (the
* Keccak permutation is PQ-OK as a primitive). */
el_val_t pq_keygen_signature(void);
el_val_t pq_sign(el_val_t secret_key_hex, el_val_t message);
el_val_t pq_verify(el_val_t public_key_hex, el_val_t message, el_val_t signature_hex);
el_val_t pq_kem_keygen(void);
el_val_t pq_kem_encaps(el_val_t public_key_hex);
el_val_t pq_kem_decaps(el_val_t secret_key_hex, el_val_t ciphertext_hex);
el_val_t pq_hybrid_keygen(void);
el_val_t pq_hybrid_handshake(el_val_t remote_pub_combined);
el_val_t sha3_256_hex(el_val_t input);
/* ── AEAD: AES-256-GCM (libcrypto-backed) ───────────────────────────────────
* Symmetric authenticated encryption used to wrap envelopes after a KEM
* handshake. Caller MUST supply a 32-byte key (64 hex chars) — typically the
* Kyber-768 / hybrid shared_secret, optionally normalized via SHA3-256.
*
* aead_encrypt returns a JSON map {"nonce":"...","ciphertext":"..."} where
* ciphertext is the AES-256-GCM output with the 16-byte auth tag appended.
* Nonce is a fresh 12-byte CSPRNG draw — callers never pick the nonce, which
* structurally rules out the GCM nonce-reuse footgun.
*
* aead_decrypt returns the plaintext String, or "" on any failure (including
* auth-tag mismatch). Callers MUST check for "" before trusting the result. */
el_val_t aead_encrypt(el_val_t key_hex, el_val_t plaintext);
el_val_t aead_decrypt(el_val_t key_hex, el_val_t nonce_hex, el_val_t ciphertext_hex);
/* ── Native VM builtin aliases (for compiled El source) ─────────────────────
* These match the El VM's native_* builtins so that El source compiled
* to C can call the same names without modification. */
el_val_t native_list_get(el_val_t list, el_val_t index);
el_val_t native_list_len(el_val_t list);
el_val_t native_list_append(el_val_t list, el_val_t elem);
el_val_t native_list_empty(void);
el_val_t native_list_clone(el_val_t list);
el_val_t native_string_chars(el_val_t s);
el_val_t native_int_to_str(el_val_t n);
/* ── Method-call shorthand aliases ──────────────────────────────────────────
* The El method-call convention `obj.method(args)` compiles to
* `method(obj, args)`. These aliases expose the runtime functions under
* the short names that result from method calls in El source.
*
* Example: `myList.append(x)` → `append(myList, x)` (calls this alias)
* `myList.len()` → `len(myList)` (calls this alias) */
el_val_t append(el_val_t list, el_val_t elem); /* el_list_append */
el_val_t len(el_val_t list); /* el_list_len */
el_val_t get(el_val_t list, el_val_t index); /* el_list_get */
el_val_t map_get(el_val_t map, el_val_t key); /* el_map_get */
el_val_t map_set(el_val_t map, el_val_t key, el_val_t value); /* el_map_set */
/* ── OTLP/HTTP Observability ─────────────────────────────────────────────── */
/* See bottom of el_runtime.c for the implementation.
* Configured by env vars OTLP_ENDPOINT, OTEL_SERVICE_NAME, OTEL_SERVICE_VERSION.
* No-op when OTLP_ENDPOINT is unset. Drop-on-failure semantics. */
/* ── Subprocess execution ────────────────────────────────────────────────── */
el_val_t exec_command(el_val_t cmd); /* run shell command, return exit code */
el_val_t exec_capture(el_val_t cmd); /* run shell command, capture stdout */
el_val_t exec(el_val_t cmd); /* exec(cmd) → stdout String (30s timeout) */
el_val_t exec_bg(el_val_t cmd); /* exec_bg(cmd) → PID String (non-blocking) */
/* ── Stdout redirection (used by compiler JS pipeline) ───────────────────── */
el_val_t stdout_to_file(el_val_t path); /* redirect process stdout to a file */
el_val_t stdout_restore(void); /* restore process stdout to terminal */
el_val_t emit_log(el_val_t level, el_val_t msg, el_val_t fields_json);
el_val_t emit_metric(el_val_t name, el_val_t value, el_val_t tags_json);
el_val_t trace_span_start(el_val_t name);
el_val_t trace_span_end(el_val_t span_handle);
el_val_t emit_event(el_val_t name, el_val_t duration_ms);
el_val_t __thread_create(el_val_t fn_name_v, el_val_t arg_v);
el_val_t __thread_join(el_val_t tid_v);
/* ── __ prefixed aliases (self-hosting compiler ABI) ─────────────────────────
* The El self-hosting compiler emits calls to __-prefixed names. These are
* forwarding wrappers around the existing el_runtime functions above. */
/* I/O */
el_val_t __println(el_val_t s);
el_val_t __print(el_val_t s);
el_val_t __readline(void);
/* String */
el_val_t __int_to_str(el_val_t n);
el_val_t __str_to_int(el_val_t s);
el_val_t __float_to_str(el_val_t f);
el_val_t __str_to_float(el_val_t s);
el_val_t __str_len(el_val_t s);
el_val_t __str_char_at(el_val_t s, el_val_t i);
el_val_t __str_cmp(el_val_t a, el_val_t b);
el_val_t __str_ncmp(el_val_t a, el_val_t b, el_val_t n);
el_val_t __str_concat_raw(el_val_t a, el_val_t b);
el_val_t __str_slice_raw(el_val_t s, el_val_t start, el_val_t end);
el_val_t __str_alloc(el_val_t n);
el_val_t __str_set_char(el_val_t s, el_val_t i, el_val_t c);
/* URL encoding */
el_val_t __url_encode(el_val_t s);
el_val_t __url_decode(el_val_t s);
/* Environment */
el_val_t __env_get(el_val_t key);
/* Subprocess */
el_val_t __exec(el_val_t cmd);
el_val_t __exec_bg(el_val_t cmd);
/* Process */
el_val_t __exit_program(el_val_t code);
/* Filesystem */
el_val_t __fs_exists(el_val_t path);
el_val_t __fs_mkdir(el_val_t path);
el_val_t __fs_read(el_val_t path);
el_val_t __fs_write(el_val_t path, el_val_t content);
el_val_t __fs_write_bytes(el_val_t path, el_val_t bytes, el_val_t n);
el_val_t __fs_list_raw(el_val_t path);
/* HTTP server */
el_val_t __http_response(el_val_t status, el_val_t headers_json, el_val_t body);
el_val_t __http_serve(el_val_t port, el_val_t handler);
el_val_t __http_serve_v2(el_val_t port, el_val_t handler);
/* HTTP conn fd / SSE (weak; overridden by el_seed.c when linked together) */
el_val_t __http_conn_fd(void);
el_val_t __http_sse_open(el_val_t conn_id);
el_val_t __http_sse_send(el_val_t conn_id, el_val_t data);
el_val_t __http_sse_close(el_val_t conn_id);
/* HTTP client (requires HAVE_CURL; stubs provided for no-curl builds) */
el_val_t __http_do(el_val_t method, el_val_t url, el_val_t body,
el_val_t headers_map, el_val_t timeout_ms);
el_val_t __http_do_map(el_val_t method, el_val_t url, el_val_t body,
el_val_t headers_json, el_val_t timeout_ms);
el_val_t __http_do_map_to_file(el_val_t method, el_val_t url, el_val_t body,
el_val_t headers_json, el_val_t output_path);
/* JSON */
el_val_t __json_array_get(el_val_t json, el_val_t index);
el_val_t __json_array_get_string(el_val_t json, el_val_t index);
el_val_t __json_array_len(el_val_t json);
el_val_t __json_get(el_val_t json, el_val_t key);
el_val_t __json_get_raw(el_val_t json, el_val_t key);
el_val_t __json_set(el_val_t json, el_val_t key, el_val_t value);
el_val_t __json_parse_map(el_val_t json_str);
el_val_t __json_stringify_val(el_val_t val);
/* Hashing */
el_val_t __sha256_hex(el_val_t s);
/* State K/V */
el_val_t __state_del(el_val_t key);
el_val_t __state_get(el_val_t key);
el_val_t __state_keys(void);
el_val_t __state_set(el_val_t key, el_val_t val);
/* UUID */
el_val_t __uuid_v4(void);
/* Args */
el_val_t __args_json(void);
#ifdef __cplusplus
}
#endif
@@ -8,7 +8,7 @@
* Threading: __thread_create / __thread_join use dlsym(RTLD_DEFAULT) to look
* up El function symbols at runtime. This is the foundation of El's parallelism.
*
* Link: cc -std=c11 -I runtime -lcurl -lpthread \
* Link: cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
* -o <out> <prog>.c el_seed.c
*/
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,761 @@
/*
* el_runtime.h — El language C runtime header
*
* Declares all built-in functions available to compiled El programs.
* Include this in every generated .c file.
*
* Value model:
* All El values are represented as el_val_t (= int64_t).
* On 64-bit systems a pointer fits in int64_t.
* String values are cast: (el_val_t)(uintptr_t)"hello"
* Integer values are stored directly.
* This lets arithmetic work naturally while still passing strings around.
*
* Type conventions (El -> C):
* String -> el_val_t (holds const char* via uintptr_t cast)
* Int -> el_val_t
* Bool -> el_val_t (0 = false, nonzero = true)
* Any -> el_val_t
* Void -> void
*
* Macros for convenience:
* EL_STR(s) cast string literal to el_val_t
* EL_CSTR(v) cast el_val_t back to const char*
* EL_INT(v) identity — el_val_t is already int64_t
*
* Link requirements:
* -lcurl — required for the HTTP client (http_get, http_post, llm_*).
* -lpthread — required for the HTTP server (one detached thread per
* connection, capped at 64 concurrent).
* -loqs — optional; required only when liboqs is installed and the
* pq_* / sha3_256_hex entry points are needed. Detected at
* compile time via __has_include(<oqs/oqs.h>).
* -lcrypto — optional; pulled in alongside -loqs. Used for X25519 in
* pq_hybrid_* and HKDF-SHA256 derivation.
*
* Canonical compile command:
* cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
* -o <out> <prog>.c el-compiler/runtime/el_runtime.c
*
* With liboqs (post-quantum stack):
* cc -std=c11 -I el-compiler/runtime -lcurl -lpthread -loqs -lcrypto \
* -o <out> <prog>.c el-compiler/runtime/el_runtime.c
*/
#pragma once
#include <stdint.h>
#include <stdlib.h>
typedef int64_t el_val_t;
#define EL_STR(s) ((el_val_t)(uintptr_t)(s))
#define EL_CSTR(v) ((const char*)(uintptr_t)(v))
#define EL_INT(v) (v)
#define EL_NULL ((el_val_t)0)
/* Float values share the el_val_t (int64) slot via a bit-cast.
* The codegen emits Float literals as `el_from_float(<dbl>)` so the
* underlying bits represent the IEEE 754 double. Float-aware builtins
* (math, format, json) round-trip via these helpers. */
static inline double el_to_float(el_val_t v) {
union { int64_t i; double f; } u;
u.i = (int64_t)v;
return u.f;
}
static inline el_val_t el_from_float(double f) {
union { double f; int64_t i; } u;
u.f = f;
return (el_val_t)u.i;
}
#ifdef __cplusplus
extern "C" {
#endif
/* ── I/O ──────────────────────────────────────────────────────────────────── */
void println(el_val_t s);
void print(el_val_t s);
el_val_t readline(void);
/* ── String builtins ─────────────────────────────────────────────────────── */
el_val_t el_str_concat(el_val_t a, el_val_t b);
el_val_t str_eq(el_val_t a, el_val_t b);
el_val_t str_starts_with(el_val_t s, el_val_t prefix);
el_val_t str_ends_with(el_val_t s, el_val_t suffix);
el_val_t str_len(el_val_t s);
el_val_t str_concat(el_val_t a, el_val_t b);
el_val_t int_to_str(el_val_t n);
el_val_t str_to_int(el_val_t s);
el_val_t str_slice(el_val_t s, el_val_t start, el_val_t end);
el_val_t str_contains(el_val_t s, el_val_t sub);
el_val_t str_replace(el_val_t s, el_val_t from, el_val_t to);
el_val_t str_to_upper(el_val_t s);
el_val_t str_to_lower(el_val_t s);
el_val_t str_trim(el_val_t s);
/* ── Math ────────────────────────────────────────────────────────────────── */
el_val_t el_abs(el_val_t n);
el_val_t el_max(el_val_t a, el_val_t b);
el_val_t el_min(el_val_t a, el_val_t b);
/* ── Refcount (ARC) ──────────────────────────────────────────────────────────
* Lists and Maps carry a refcount. Strings and ints do not — el_retain and
* el_release are safe no-ops on non-refcounted values (they sniff a magic
* header at offset 0 and only act if the magic matches).
*
* Codegen emits these at let-binding shadowing, function entry (params), and
* function exit (locals other than the returned value). The refcount lets
* el_list_append and el_map_set mutate in place when uniquely owned (cheap)
* and copy-on-write when shared (preserves persistent semantics across
* accumulator patterns in the compiler itself). */
void el_retain(el_val_t v);
void el_release(el_val_t v);
/* ── List ────────────────────────────────────────────────────────────────── */
el_val_t el_list_new(el_val_t count, ...);
el_val_t el_list_len(el_val_t list);
el_val_t el_list_get(el_val_t list, el_val_t index);
el_val_t el_list_append(el_val_t list, el_val_t elem);
el_val_t el_list_empty(void);
el_val_t el_list_clone(el_val_t list);
/* ── Map ─────────────────────────────────────────────────────────────────── */
el_val_t el_map_new(el_val_t pair_count, ...);
el_val_t el_get_field(el_val_t map, el_val_t key);
el_val_t el_map_get(el_val_t map, el_val_t key);
el_val_t el_map_set(el_val_t map, el_val_t key, el_val_t value);
/* ── HTTP ─────────────────────────────────────────────────────────────────── */
el_val_t http_get(el_val_t url);
el_val_t http_post(el_val_t url, el_val_t body);
el_val_t http_post_json(el_val_t url, el_val_t json_body);
el_val_t http_get_with_headers(el_val_t url, el_val_t headers_map);
el_val_t http_post_with_headers(el_val_t url, el_val_t body, el_val_t headers_map);
el_val_t http_post_form_auth(el_val_t url, el_val_t form_body, el_val_t auth_header);
el_val_t http_delete(el_val_t url);
void http_serve(el_val_t port, el_val_t handler);
void http_set_handler(el_val_t name);
/* HTTP server v2 ─────────────────────────────────────────────────────────────
* Same dispatch model as http_serve, but the handler signature is widened:
*
* el_val_t handler(method, path, headers_map, body)
*
* `headers_map` is an ElMap from lowercased header name → header value (both
* Strings). Repeated headers are joined with ", " per RFC 7230.
*
* Response value: the handler may return either
* (a) a plain body string — same auto-content-type / 200-OK behaviour as
* http_serve (3-arg) — or
* (b) a response envelope built with `http_response(status, headers_json,
* body)`. The runtime detects the envelope discriminator
* `"el_http_response":1` at the start of the returned string and
* unpacks status / headers / body before sending.
*
* The 3-arg http_serve(port, handler) remains supported unchanged for
* existing handlers (e.g. products/web/server.el): it dispatches with
* (method, path, body), hardcodes 200 OK, and auto-detects content type. */
void http_serve_v2(el_val_t port, el_val_t handler);
void http_set_handler_v2(el_val_t name);
/* Build an HTTP response envelope. `headers_json` should be a JSON object
* literal like `{"WWW-Authenticate":"Basic"}` (or "" / "{}" for none). The
* returned string carries the discriminator `{"el_http_response":1,...}`
* which the runtime's send-path detects and unpacks. Detection happens
* uniformly inside http_send_response, so a 3-arg handler may also return
* an envelope. The 3-arg variant remains documented as a fixed 200-OK
* auto-content-type contract for legacy handlers that return plain bodies. */
el_val_t http_response(el_val_t status, el_val_t headers_json, el_val_t body);
/* SSE connection fd — set by http_worker_v2 before calling the El handler,
* cleared afterwards. Defined in el_seed.c; called from el_runtime.c.
* The getter is exposed as __http_conn_fd() to El programs. */
void el_seed_set_http_conn_fd(int fd);
/* HTTP timeout — every libcurl request honors EL_HTTP_TIMEOUT_MS (default
* 60000ms). Read lazily on first use, so setting the env var any time before
* the first http_* call is sufficient. */
/* Streaming variants — write the response body straight to a file via
* libcurl's CURLOPT_WRITEFUNCTION = fwrite. These bypass the el_val_t string
* wrapper entirely, so binary payloads (audio/mpeg, image/png, etc.) survive
* embedded NUL bytes that would truncate a strlen()-based code path.
*
* Both honor EL_HTTP_TIMEOUT_MS, follow redirects, and accept the same
* `headers_map` shape as http_post_with_headers (ElMap of String→String).
*
* Return value: 1 on success (file fully written), 0 on any failure
* (network, file open, partial write). On failure the output file is removed
* so callers cannot mistake a partially-written file for a valid one. */
el_val_t http_post_to_file(el_val_t url, el_val_t body, el_val_t headers_map, el_val_t output_path);
el_val_t http_get_to_file(el_val_t url, el_val_t headers_map, el_val_t output_path);
/* ── URL encoding ────────────────────────────────────────────────────────── */
el_val_t url_encode(el_val_t s); /* RFC 3986 unreserved set */
el_val_t url_decode(el_val_t s); /* '+' → space, %XX → byte */
/* ── HTML allowlist sanitizer ────────────────────────────────────────────────
* el_html_sanitize(input_html, allowlist_json) — strict allowlist HTML
* cleaner. State-machine parser; tag/attribute names compared case-
* insensitively against the allowlist; `<a href>` / `<… src>` URL schemes
* validated (http, https, mailto, fragment-only, or relative); whole-
* subtree drop for script / style / iframe / object / embed / form; HTML-
* escapes free text outside dropped subtrees.
*
* The allowlist is JSON of the form
* {"p":[],"a":["href","title"],"strong":[],...}
* where each value is the array of attribute names allowed for that tag. */
el_val_t el_html_sanitize(el_val_t input_html, el_val_t allowlist_json);
/* ── Filesystem ──────────────────────────────────────────────────────────── */
el_val_t fs_read(el_val_t path);
el_val_t fs_write(el_val_t path, el_val_t content);
el_val_t fs_list(el_val_t path);
el_val_t fs_exists(el_val_t path);
el_val_t fs_mkdir(el_val_t path); /* mkdir -p, mode 0755 */
/* Length-explicit binary write. `length` is an Int (el_val_t holding the
* byte count). The caller knows the length from context — typically because
* `bytes` came from base64_decode (which produces a magic-tagged binary
* buffer with embedded NULs possible) and the caller already tracks the
* decoded length, OR because the bytes came from a fixed-size source
* (sha256_bytes = 32, hmac_sha256_bytes = 32). Bypasses strlen entirely.
*
* Returns 1 on success, 0 on failure (invalid path, can't open, partial
* write, negative length). On partial-write failure, the file is removed
* so callers cannot read back a truncated artefact. */
el_val_t fs_write_bytes(el_val_t path, el_val_t bytes, el_val_t length);
/* ── JSON ────────────────────────────────────────────────────────────────── */
el_val_t json_get(el_val_t json, el_val_t key);
el_val_t json_parse(el_val_t s);
el_val_t json_stringify(el_val_t v);
el_val_t json_get_string(el_val_t json_str, el_val_t key);
el_val_t json_get_int(el_val_t json_str, el_val_t key);
el_val_t json_get_float(el_val_t json_str, el_val_t key);
el_val_t json_get_bool(el_val_t json_str, el_val_t key);
el_val_t json_get_raw(el_val_t json_str, el_val_t key);
el_val_t json_set(el_val_t json_str, el_val_t key, el_val_t value);
el_val_t json_array_len(el_val_t json_str);
el_val_t json_array_get(el_val_t json_str, el_val_t index);
el_val_t json_array_get_string(el_val_t json_str, el_val_t index);
/* ── Time ────────────────────────────────────────────────────────────────── */
el_val_t time_now(void);
el_val_t time_now_utc(void);
el_val_t sleep_secs(el_val_t secs);
el_val_t sleep_ms(el_val_t ms);
el_val_t time_format(el_val_t ts, el_val_t fmt);
el_val_t time_to_parts(el_val_t ts);
el_val_t time_from_parts(el_val_t secs, el_val_t ns, el_val_t tz);
el_val_t time_add(el_val_t ts, el_val_t n, el_val_t unit);
el_val_t time_diff(el_val_t ts1, el_val_t ts2, el_val_t unit);
/* ── Instant + Duration: first-class temporal types ──────────────────────────
* Both types share the el_val_t (int64) slot. Instants are nanoseconds
* since the Unix epoch; Durations are signed nanoseconds. Type discipline
* is enforced at codegen-time: BinOps on names registered as Instant or
* Duration route through the typed wrappers below; mismatches like
* Instant+Instant become #error at the C compiler.
*
* Postfix literals — `30.seconds`, `1.hour`, `500.millis`, `30.nanos` — are
* recognised by the parser as DurationLit AST nodes and lowered to literal
* int64 nanoseconds at codegen time. The runtime never sees the units. */
el_val_t el_now_instant(void);
el_val_t now(void);
el_val_t unix_seconds(el_val_t n);
el_val_t unix_millis(el_val_t n);
el_val_t instant_from_iso8601(el_val_t s);
el_val_t el_duration_from_nanos(el_val_t ns);
el_val_t duration_seconds(el_val_t n);
el_val_t duration_millis(el_val_t n);
el_val_t duration_nanos(el_val_t n);
el_val_t el_instant_add_dur(el_val_t inst, el_val_t dur);
el_val_t el_instant_sub_dur(el_val_t inst, el_val_t dur);
el_val_t el_instant_diff(el_val_t a, el_val_t b);
el_val_t el_duration_add(el_val_t a, el_val_t b);
el_val_t el_duration_sub(el_val_t a, el_val_t b);
el_val_t el_duration_scale(el_val_t dur, el_val_t scalar);
el_val_t el_duration_div(el_val_t dur, el_val_t scalar);
el_val_t el_instant_lt(el_val_t a, el_val_t b);
el_val_t el_instant_le(el_val_t a, el_val_t b);
el_val_t el_instant_gt(el_val_t a, el_val_t b);
el_val_t el_instant_ge(el_val_t a, el_val_t b);
el_val_t el_instant_eq(el_val_t a, el_val_t b);
el_val_t el_instant_ne(el_val_t a, el_val_t b);
el_val_t el_duration_lt(el_val_t a, el_val_t b);
el_val_t el_duration_le(el_val_t a, el_val_t b);
el_val_t el_duration_gt(el_val_t a, el_val_t b);
el_val_t el_duration_ge(el_val_t a, el_val_t b);
el_val_t el_duration_eq(el_val_t a, el_val_t b);
el_val_t el_duration_ne(el_val_t a, el_val_t b);
el_val_t instant_to_unix_seconds(el_val_t i);
el_val_t instant_to_unix_millis(el_val_t i);
el_val_t instant_to_iso8601(el_val_t i);
el_val_t duration_to_seconds(el_val_t d);
el_val_t duration_to_millis(el_val_t d);
el_val_t duration_to_nanos(el_val_t d);
el_val_t el_sleep_duration(el_val_t dur);
el_val_t unix_timestamp(void);
el_val_t ttl_cache_set(el_val_t key, el_val_t value);
el_val_t ttl_cache_get(el_val_t key, el_val_t max_age);
el_val_t ttl_cache_age(el_val_t key);
/* ── Calendar + CalendarTime + Rhythm + LocalDate/Time/DateTime ─────────────
* Phase 1.5 of the time system. Calendar is pluggable: EarthCalendar (IANA
* zones, Gregorian, DST) is the user-facing default; MarsCalendar,
* CycleCalendar(period), NoCycleCalendar, RelativeCalendar handle non-Earth
* domains.
*
* A Calendar interprets an Instant under a particular cycle convention and
* produces a CalendarTime. CalendarTime carries the underlying Instant and
* a back-pointer to its Calendar; arithmetic and formatting consult the
* Calendar to convert ns since epoch into year/month/day/hour/minute/second
* (or sol/phase, or cycle/phase, depending on kind).
*
* Storage convention: Calendar / CalendarTime / Rhythm / LocalDate /
* LocalDateTime are heap-allocated structs whose pointers are cast into
* el_val_t. A 24-bit magic header at offset 0 lets the runtime identify
* the kind safely. LocalTime is small enough to live in the int64 slot
* directly (nanos since midnight, signed). */
/* Zone — opaque IANA zone or fixed offset, used by EarthCalendar.
* `zone_id` is either an IANA name ("America/New_York", "UTC") or a fixed
* offset string ("+05:30", "-08:00"). The runtime resolves it via tzset()
* on first use of the owning EarthCalendar. */
el_val_t zone(el_val_t id);
el_val_t zone_utc(void);
el_val_t zone_local(void);
el_val_t zone_offset(el_val_t hours, el_val_t minutes);
/* Calendar constructors. Each returns an el_val_t pointer to a heap-
* allocated, magic-tagged Calendar struct. Calendars are interned by
* (kind, zone_id, period_ns, epoch_ns) so identical constructors return
* the same pointer — equality is reference equality. */
el_val_t earth_calendar(el_val_t z);
el_val_t earth_calendar_default(void);
el_val_t mars_calendar(void);
el_val_t cycle_calendar(el_val_t period_dur);
el_val_t no_cycle_calendar(void);
el_val_t relative_calendar(el_val_t epoch_inst);
/* CalendarTime constructors and methods. Returns a heap-allocated struct
* whose pointer fits in el_val_t. */
el_val_t now_in(el_val_t cal);
el_val_t in_calendar(el_val_t inst, el_val_t cal);
el_val_t cal_format(el_val_t ct, el_val_t pattern);
el_val_t cal_to_instant(el_val_t ct);
el_val_t cal_cycle_phase(el_val_t ct);
el_val_t cal_in(el_val_t ct, el_val_t cal);
/* LocalDate / LocalTime / LocalDateTime — calendar-agnostic value types.
* LocalTime carries nanoseconds since midnight as a signed int64 directly
* in the el_val_t slot (no allocation). LocalDate / LocalDateTime are
* heap-allocated structs with magic headers. */
el_val_t local_date(el_val_t y, el_val_t m, el_val_t d);
el_val_t local_time(el_val_t h, el_val_t m, el_val_t s, el_val_t ns);
el_val_t local_datetime(el_val_t date, el_val_t time);
el_val_t zoned(el_val_t date, el_val_t time, el_val_t cal);
el_val_t local_date_year(el_val_t ld);
el_val_t local_date_month(el_val_t ld);
el_val_t local_date_day(el_val_t ld);
el_val_t local_time_hour(el_val_t lt);
el_val_t local_time_minute(el_val_t lt);
el_val_t local_time_second(el_val_t lt);
el_val_t local_time_nanos(el_val_t lt);
el_val_t el_local_date_add_dur(el_val_t ld, el_val_t dur);
el_val_t el_local_time_add_dur(el_val_t lt, el_val_t dur);
el_val_t el_local_date_lt(el_val_t a, el_val_t b);
el_val_t el_local_date_eq(el_val_t a, el_val_t b);
/* Rhythm — pluggable recurrence AST. Returns a heap-allocated struct
* pointer in el_val_t; rhythms are immutable so callers may share them. */
el_val_t rhythm_cycle_start(void);
el_val_t rhythm_cycle_phase(el_val_t phase);
el_val_t rhythm_duration(el_val_t d);
el_val_t rhythm_session_start(void);
el_val_t rhythm_event(el_val_t name);
el_val_t rhythm_and(el_val_t a, el_val_t b);
el_val_t rhythm_or(el_val_t a, el_val_t b);
el_val_t rhythm_weekday(el_val_t day);
el_val_t rhythm_weekly_at(el_val_t day, el_val_t hour, el_val_t minute);
el_val_t rhythm_next_after(el_val_t r, el_val_t after, el_val_t cal);
el_val_t rhythm_matches(el_val_t r, el_val_t ct);
/* ── UUID ────────────────────────────────────────────────────────────────── */
el_val_t uuid_new(void);
el_val_t uuid_v4(void);
/* ── Environment ─────────────────────────────────────────────────────────── */
el_val_t env(el_val_t key);
/* ── In-process state K/V ────────────────────────────────────────────────── */
el_val_t state_set(el_val_t key, el_val_t value);
el_val_t state_get(el_val_t key);
el_val_t state_del(el_val_t key);
el_val_t state_keys(void);
/* ── Float formatting ────────────────────────────────────────────────────── */
el_val_t float_to_str(el_val_t f);
el_val_t int_to_float(el_val_t n);
el_val_t float_to_int(el_val_t f);
el_val_t format_float(el_val_t f, el_val_t decimals);
el_val_t decimal_round(el_val_t f, el_val_t decimals);
el_val_t str_to_float(el_val_t s);
/* ── Math (Float-aware) ──────────────────────────────────────────────────── */
el_val_t math_sqrt(el_val_t f);
el_val_t math_log(el_val_t f);
el_val_t math_ln(el_val_t f);
el_val_t math_sin(el_val_t f);
el_val_t math_cos(el_val_t f);
el_val_t math_pi(void);
/* ── String additions ────────────────────────────────────────────────────── */
el_val_t str_index_of(el_val_t s, el_val_t sub);
el_val_t str_split(el_val_t s, el_val_t sep);
el_val_t str_char_at(el_val_t s, el_val_t i);
el_val_t str_char_code(el_val_t s, el_val_t i);
el_val_t str_pad_left(el_val_t s, el_val_t width, el_val_t pad);
el_val_t str_pad_right(el_val_t s, el_val_t width, el_val_t pad);
el_val_t str_format(el_val_t fmt, el_val_t data);
el_val_t str_lower(el_val_t s);
el_val_t str_upper(el_val_t s);
/* ── Text-processing primitives (Phase 1: byte/codepoint, ASCII char classes)
* Phase 2 (filed): Unicode-grapheme awareness, NFC/NFD normalization, regex.
* is_* predicates: empty input returns false; multi-char requires ALL bytes
* to match. ASCII ranges only in Phase 1. */
/* Counting */
el_val_t str_count(el_val_t s, el_val_t sub); /* non-overlapping */
el_val_t str_count_chars(el_val_t s); /* codepoint count */
el_val_t str_count_bytes(el_val_t s); /* alias of str_len */
el_val_t str_count_lines(el_val_t s);
el_val_t str_count_words(el_val_t s);
el_val_t str_count_letters(el_val_t s); /* ASCII [A-Za-z] */
el_val_t str_count_digits(el_val_t s); /* ASCII [0-9] */
/* Find / position */
el_val_t str_index_of_all(el_val_t s, el_val_t sub); /* [Int] of byte offsets */
el_val_t str_last_index_of(el_val_t s, el_val_t sub);
el_val_t str_find_chars(el_val_t s, el_val_t any_of); /* first idx of any ch */
/* Transform */
el_val_t str_repeat(el_val_t s, el_val_t n);
el_val_t str_reverse(el_val_t s); /* by codepoint */
el_val_t str_strip_prefix(el_val_t s, el_val_t prefix);
el_val_t str_strip_suffix(el_val_t s, el_val_t suffix);
el_val_t str_strip_chars(el_val_t s, el_val_t chars);
el_val_t str_lstrip(el_val_t s);
el_val_t str_rstrip(el_val_t s);
/* Char classification (Bool) */
el_val_t is_letter(el_val_t s);
el_val_t is_digit(el_val_t s);
el_val_t is_alphanumeric(el_val_t s);
el_val_t is_whitespace(el_val_t s);
el_val_t is_punctuation(el_val_t s);
el_val_t is_uppercase(el_val_t s);
el_val_t is_lowercase(el_val_t s);
/* Split / join */
el_val_t str_split_lines(el_val_t s);
el_val_t str_split_chars(el_val_t s); /* alias of native_string_chars */
el_val_t str_split_n(el_val_t s, el_val_t sep, el_val_t n);
el_val_t str_join(el_val_t list, el_val_t sep); /* alias of list_join */
/* ── List additions ──────────────────────────────────────────────────────── */
el_val_t list_push(el_val_t list, el_val_t elem);
el_val_t list_push_front(el_val_t list, el_val_t elem);
el_val_t list_join(el_val_t list, el_val_t sep);
el_val_t list_range(el_val_t start, el_val_t end);
/* ── Bool helpers ────────────────────────────────────────────────────────── */
el_val_t bool_to_str(el_val_t b);
/* ── Numeric parsing ─────────────────────────────────────────────────────── */
el_val_t parse_int(el_val_t s, el_val_t default_val);
/* ── Process ─────────────────────────────────────────────────────────────── */
void exit_program(el_val_t code);
el_val_t getpid_now(void);
/* ── CGI identity ─────────────────────────────────────────────────────────────
* Called at the start of main() in CGI programs (those with a `cgi {}` block).
* Records the program's DHARMA identity before any other code executes. */
void el_cgi_init(el_val_t name, el_val_t dharma_id, el_val_t principal,
el_val_t network, el_val_t engram);
/* ── DHARMA network builtins ─────────────────────────────────────────────────
* Available to CGI programs (declared with a `cgi {}` block).
*
* Peers are addressed by `dharma_id` of the form
* "<registry-id>@<transport-url>" e.g. "ntn-genesis@http://localhost:7770"
* If the @<url> portion is omitted, transport defaults to
* "http://localhost:7770" (the local CGI daemon assumption).
*
* Wire protocol (all peers expose):
* POST <url>/dharma/recv { channel, from, content } → response body
* POST <url>/dharma/event { type, payload, source, timestamp }
* POST <url>/api/activate { query } → list of nodes
*
* Hosting application's responsibility: an El program with a `cgi {}` block
* runs http_serve() with its own request handler; that handler should route
* "/dharma/event" requests by calling el_runtime_dharma_event_arrive() so
* incoming events feed dharma_field() queues. The runtime itself does not
* intercept any /dharma path. */
el_val_t dharma_connect(el_val_t cgi_id);
el_val_t dharma_send(el_val_t channel, el_val_t content);
el_val_t dharma_activate(el_val_t query);
void dharma_emit(el_val_t event_type, el_val_t payload);
el_val_t dharma_field(el_val_t event_type);
void dharma_strengthen(el_val_t cgi_id, el_val_t weight);
el_val_t dharma_relationship(el_val_t cgi_id);
el_val_t dharma_peers(void);
/* Public C API: called by an El program's HTTP handler when a /dharma/event
* request arrives. Pushes onto the per-event-type queue and signals any
* pending dharma_field() blockers. All three arguments must be NUL-terminated
* C strings (or NULL — then treated as empty). */
void el_runtime_dharma_event_arrive(const char* event_type,
const char* payload,
const char* source);
/* ── Engram local graph primitives ───────────────────────────────────────────
* Operate on the CGI's local Engram knowledge graph.
* `engram_activate` queries the local graph only; `dharma_activate` is
* network-wide across all connected CGI graphs. */
el_val_t engram_node(el_val_t content, el_val_t node_type, el_val_t salience);
el_val_t engram_node_full(el_val_t content, el_val_t node_type, el_val_t label,
el_val_t salience, el_val_t importance, el_val_t confidence,
el_val_t tier, el_val_t tags);
/* Layered consciousness — see el_runtime.c for the layered architecture
* design notes (search "Layered consciousness architecture"). The five
* canonical layers (safety / core-identity / domain-knowledge / imprint /
* suit) are seeded automatically; engram_add_layer extends the registry
* with imprint or suit overlays at runtime. Nodes default to layer 1
* (core-identity) when created via engram_node / engram_node_full. */
el_val_t engram_node_layered(el_val_t content, el_val_t node_type, el_val_t label,
el_val_t salience, el_val_t certainty, el_val_t confidence,
el_val_t status, el_val_t tags, el_val_t layer_id);
el_val_t engram_add_layer(el_val_t name, el_val_t priority, el_val_t suppressible,
el_val_t transparent, el_val_t injectable);
el_val_t engram_remove_layer(el_val_t layer_id);
el_val_t engram_list_layers(void);
el_val_t engram_get_node(el_val_t id);
void engram_strengthen(el_val_t node_id);
void engram_forget(el_val_t node_id);
el_val_t engram_node_count(void);
el_val_t engram_search(el_val_t query, el_val_t limit);
el_val_t engram_scan_nodes(el_val_t limit, el_val_t offset);
void engram_connect(el_val_t from_id, el_val_t to_id, el_val_t weight, el_val_t relation);
el_val_t engram_edge_between(el_val_t from_id, el_val_t to_id);
el_val_t engram_neighbors(el_val_t node_id);
el_val_t engram_neighbors_filtered(el_val_t node_id, el_val_t max_depth, el_val_t direction);
el_val_t engram_edge_count(void);
/* Three-pass activation: background fan-out → working-memory promotion →
* Layer 0 override. See "Three-pass activation" in el_runtime.c. */
el_val_t engram_activate(el_val_t query, el_val_t depth);
el_val_t engram_save(el_val_t path);
el_val_t engram_load(el_val_t path);
/* JSON-string accessors — return pre-serialized JSON so HTTP handlers
* can pass results straight through without round-tripping ElList/ElMap
* through json_stringify. */
el_val_t engram_get_node_json(el_val_t id);
el_val_t engram_search_json(el_val_t query, el_val_t limit);
el_val_t engram_scan_nodes_json(el_val_t limit, el_val_t offset);
el_val_t engram_scan_nodes_by_type_json(el_val_t node_type, el_val_t limit, el_val_t offset);
el_val_t engram_neighbors_json(el_val_t node_id, el_val_t max_depth, el_val_t direction);
el_val_t engram_activate_json(el_val_t query, el_val_t depth);
el_val_t engram_stats_json(void);
el_val_t engram_list_layers_json(void);
/* engram_compile_layered_json — produce a prompt-ready text block split
* into "[LAYER 0 — STRUCTURAL]" (non-suppressible layers, sacred fire)
* and "[ENGRAM CONTEXT]" (standard suppressible layers). Returns "" if
* no nodes promoted to working memory. */
el_val_t engram_compile_layered_json(el_val_t intent, el_val_t depth);
/* ── LLM (Anthropic API client) ─────────────────────────────────────────────
* All functions call https://api.anthropic.com/v1/messages with the API key
* from env ANTHROPIC_API_KEY. Default model when empty: claude-sonnet-4-5. */
el_val_t llm_call(el_val_t model, el_val_t prompt);
el_val_t llm_call_system(el_val_t model, el_val_t system_prompt, el_val_t user_prompt);
el_val_t llm_call_agentic(el_val_t model, el_val_t system, el_val_t user, el_val_t tools);
el_val_t llm_vision(el_val_t model, el_val_t system, el_val_t prompt, el_val_t image_url_or_b64);
el_val_t llm_models(void);
/* Register a tool handler by name. The handler is looked up via dlsym
* (mirroring http_set_handler), so any El `fn <name>(input)` compiles to
* a global C symbol that this function can locate at runtime.
* Handler signature: `el_val_t handler(el_val_t input_json)` — receives
* the tool input as a JSON-string el_val_t and returns a JSON-string
* el_val_t result. Used by llm_call_agentic. */
void llm_register_tool(el_val_t name, el_val_t handler_fn_name);
/* ── args() ─────────────────────────────────────────────────────────────────
* Provides access to command-line arguments passed to the program.
* Populated by el_runtime_init_args() before main() runs. */
el_val_t args(void);
void el_runtime_init_args(int argc, char** argv);
/* ── Crypto primitives ─────────────────────────────────────────────────────
* SHA-256, HMAC-SHA-256, and base64 (standard + URL-safe).
* Self-contained — no OpenSSL/libcrypto dependency. The implementations are
* adapted from public-domain reference code (Brad Conte / RFC 4648).
*
* Bytes-returning variants (sha256_bytes, hmac_sha256_bytes) return a string
* value whose contents are raw binary; callers usually feed these into
* base64_encode. Note that el_val_t strings are NUL-terminated by convention,
* so the binary payload may contain embedded NULs — pass it directly into
* base64_encode (which uses an explicit length) rather than treating it as
* a printable C string.
*
* The "base64" variants emit/accept RFC 4648 standard alphabet with padding.
* The "base64url" variants use URL-safe alphabet (`-`/`_`) with no padding,
* as used in JWTs. */
el_val_t sha256_hex(el_val_t input);
el_val_t sha256_bytes(el_val_t input);
el_val_t hmac_sha256_hex(el_val_t key, el_val_t message);
el_val_t hmac_sha256_bytes(el_val_t key, el_val_t message);
el_val_t base64_encode(el_val_t input);
el_val_t base64_decode(el_val_t input);
el_val_t base64url_encode(el_val_t input);
el_val_t base64url_decode(el_val_t input);
/* Length-aware variants (internal — exposed for the rare caller that already
* has a known-length binary buffer and doesn't want to round-trip through
* a NUL-terminated el_val_t string). Sha256_bytes and hmac_sha256_bytes feed
* these implicitly. */
el_val_t el_sha256_bytes_n(const unsigned char* data, size_t len);
el_val_t el_base64_encode_n(const unsigned char* data, size_t len, int url_safe);
/* ── Post-quantum primitives (liboqs-backed) ────────────────────────────────
* All inputs/outputs hex-encoded. Algorithm choices:
* Signature: CRYSTALS-Dilithium-3 (NIST level 3, balanced)
* KEM: CRYSTALS-Kyber-768 (NIST level 3)
* Hash: SHA3-256 (Keccak) (PQ-aware protocols favour SHA3 over SHA2)
*
* If liboqs is not linked (detected via __has_include(<oqs/oqs.h>) at compile
* time), the pq_* entry points return a JSON-shaped error string so callers
* fail loudly rather than silently fall back to classical schemes:
* {"error":"liboqs not linked, post-quantum primitives unavailable"}
*
* The hybrid handshake pairs X25519 with Kyber-768 per NIST PQ guidance and
* CNSA 2.0. Combined shared secret is HKDF-SHA256(x25519_ss || kyber_ss).
* Even if Kyber falls, X25519 holds; if X25519 falls under quantum attack,
* Kyber holds. SHA3-256 also remains usable independent of liboqs (the
* Keccak permutation is PQ-OK as a primitive). */
el_val_t pq_keygen_signature(void);
el_val_t pq_sign(el_val_t secret_key_hex, el_val_t message);
el_val_t pq_verify(el_val_t public_key_hex, el_val_t message, el_val_t signature_hex);
el_val_t pq_kem_keygen(void);
el_val_t pq_kem_encaps(el_val_t public_key_hex);
el_val_t pq_kem_decaps(el_val_t secret_key_hex, el_val_t ciphertext_hex);
el_val_t pq_hybrid_keygen(void);
el_val_t pq_hybrid_handshake(el_val_t remote_pub_combined);
el_val_t sha3_256_hex(el_val_t input);
/* ── AEAD: AES-256-GCM (libcrypto-backed) ───────────────────────────────────
* Symmetric authenticated encryption used to wrap envelopes after a KEM
* handshake. Caller MUST supply a 32-byte key (64 hex chars) — typically the
* Kyber-768 / hybrid shared_secret, optionally normalized via SHA3-256.
*
* aead_encrypt returns a JSON map {"nonce":"...","ciphertext":"..."} where
* ciphertext is the AES-256-GCM output with the 16-byte auth tag appended.
* Nonce is a fresh 12-byte CSPRNG draw — callers never pick the nonce, which
* structurally rules out the GCM nonce-reuse footgun.
*
* aead_decrypt returns the plaintext String, or "" on any failure (including
* auth-tag mismatch). Callers MUST check for "" before trusting the result. */
el_val_t aead_encrypt(el_val_t key_hex, el_val_t plaintext);
el_val_t aead_decrypt(el_val_t key_hex, el_val_t nonce_hex, el_val_t ciphertext_hex);
/* ── Native VM builtin aliases (for compiled El source) ─────────────────────
* These match the El VM's native_* builtins so that El source compiled
* to C can call the same names without modification. */
el_val_t native_list_get(el_val_t list, el_val_t index);
el_val_t native_list_len(el_val_t list);
el_val_t native_list_append(el_val_t list, el_val_t elem);
el_val_t native_list_empty(void);
el_val_t native_list_clone(el_val_t list);
el_val_t native_string_chars(el_val_t s);
el_val_t native_int_to_str(el_val_t n);
/* ── Method-call shorthand aliases ──────────────────────────────────────────
* The El method-call convention `obj.method(args)` compiles to
* `method(obj, args)`. These aliases expose the runtime functions under
* the short names that result from method calls in El source.
*
* Example: `myList.append(x)` → `append(myList, x)` (calls this alias)
* `myList.len()` → `len(myList)` (calls this alias) */
el_val_t append(el_val_t list, el_val_t elem); /* el_list_append */
el_val_t len(el_val_t list); /* el_list_len */
el_val_t get(el_val_t list, el_val_t index); /* el_list_get */
el_val_t map_get(el_val_t map, el_val_t key); /* el_map_get */
el_val_t map_set(el_val_t map, el_val_t key, el_val_t value); /* el_map_set */
/* ── OTLP/HTTP Observability ─────────────────────────────────────────────── */
/* See bottom of el_runtime.c for the implementation.
* Configured by env vars OTLP_ENDPOINT, OTEL_SERVICE_NAME, OTEL_SERVICE_VERSION.
* No-op when OTLP_ENDPOINT is unset. Drop-on-failure semantics. */
/* ── Subprocess execution ────────────────────────────────────────────────── */
el_val_t exec_command(el_val_t cmd); /* run shell command, return exit code */
el_val_t exec_capture(el_val_t cmd); /* run shell command, capture stdout */
el_val_t exec(el_val_t cmd); /* exec(cmd) → stdout String (30s timeout) */
el_val_t exec_bg(el_val_t cmd); /* exec_bg(cmd) → PID String (non-blocking) */
el_val_t emit_log(el_val_t level, el_val_t msg, el_val_t fields_json);
el_val_t emit_metric(el_val_t name, el_val_t value, el_val_t tags_json);
el_val_t trace_span_start(el_val_t name);
el_val_t trace_span_end(el_val_t span_handle);
el_val_t emit_event(el_val_t name, el_val_t duration_ms);
#ifdef __cplusplus
}
#endif
+1 -1
View File
@@ -1202,7 +1202,7 @@ fn codegen_js_inner(stmts: [Map<String, Any>], source: String, bundle_mode: Bool
js_emit_line(js_strip_es_exports(runtime_content))
js_emit_line("")
} else {
js_emit_line("// Runtime: foundation/el/runtime/el_runtime.js")
js_emit_line("// Runtime: foundation/el/el-compiler/runtime/el_runtime.js")
js_emit_line("import \"./el_runtime.js\";")
}
// In module mode: destructure all builtins off globalThis.__el so call
+3 -3
View File
@@ -368,13 +368,13 @@ fn main() -> Void {
let which_out: String = str_trim(exec_capture("which " + elc_bin + " 2>/dev/null"))
if !str_eq(which_out, "") {
let elc_dir: String = dirname_of(which_out)
runtime_path = elc_dir + "/../runtime/el_runtime.c"
runtime_path = elc_dir + "/../el-compiler/runtime/el_runtime.c"
}
}
// If --runtime points to a directory, auto-locate el_runtime.c inside it.
// This lets both forms work:
// --runtime=/opt/el/runtime (directory form)
// --runtime=/opt/el/runtime/el_runtime.c (file form)
// --runtime=/opt/el/el-compiler/runtime (directory form)
// --runtime=/opt/el/el-compiler/runtime/el_runtime.c (file form)
if !str_eq(runtime_path, "") {
let is_dir: String = str_trim(exec_capture("test -d " + runtime_path + " && echo dir || echo file"))
if str_eq(is_dir, "dir") {
+2 -2
View File
@@ -6,8 +6,8 @@
//
// Compile and run:
// ./dist/platform/elc examples/html-page.el > /tmp/html-page.c
// cc -std=c11 -I runtime -lcurl -lpthread \
// -o /tmp/html-page /tmp/html-page.c runtime/el_runtime.c
// cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
// -o /tmp/html-page /tmp/html-page.c el-compiler/runtime/el_runtime.c
// /tmp/html-page
fn render_item(item: String) -> String {
+28
View File
@@ -0,0 +1,28 @@
# El Compiler Release v1.0.0 — 2026-05-02
## Components
- `bootstrap.py` — El language compiler (Python, recursive descent parser, emits C)
- `el_runtime.c` — El runtime (C, HTTP server, engram, DHARMA, LLM chain)
- `el_runtime.h` — Runtime public API header
## Changes in this release
### Critical bug fixes
- `state_set`/`state_get` are now thread-safe (pthread_mutex). Was racing across 64 worker threads.
- `looks_like_string` threshold raised from 1,000,000 to 4GB. Unix timestamps were being dereferenced as heap pointers.
- `fs_read` guards against negative `ftell` result (pipe/special file overflow).
### Engram architecture (major)
- Two-layer activation: `background_activation` (Layer 1, broad fan-out) + `working_memory_weight` (Layer 2, executive filter)
- Inhibitory edges: `EngramEdge.inhibitory` flag suppresses working memory promotion without affecting background activation
- Suppression memory: `suppression_count` — nodes activated-but-suppressed accumulate pressure toward breakthrough
- Temporal decay: `temporal_decay_rate`, `created_at`, `last_activated_at`, `activation_count` on EngramNode
- Per-type activation thresholds (Safety: 0.05, Canonical: 0.15, Lesson: 0.25, Note: 0.40)
- Temporal range query: `engram_query_range(start_ms, end_ms)`
- Layered consciousness: `EngramLayer` struct, `layer_id` on nodes and edges, `EngramStore.layers[]`
- Layer 0 override pass: safety layer fires last and cannot be suppressed
## SHA256
bootstrap.py
el_runtime.c
el_runtime.h
File diff suppressed because it is too large Load Diff
@@ -34,12 +34,12 @@
* pq_hybrid_* and HKDF-SHA256 derivation.
*
* Canonical compile command:
* cc -std=c11 -I runtime -lcurl -lpthread \
* -o <out> <prog>.c runtime/el_runtime.c
* cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
* -o <out> <prog>.c el-compiler/runtime/el_runtime.c
*
* With liboqs (post-quantum stack):
* cc -std=c11 -I runtime -lcurl -lpthread -loqs -lcrypto \
* -o <out> <prog>.c runtime/el_runtime.c
* cc -std=c11 -I el-compiler/runtime -lcurl -lpthread -loqs -lcrypto \
* -o <out> <prog>.c el-compiler/runtime/el_runtime.c
*/
#pragma once
@@ -628,30 +628,22 @@ el_val_t engram_hebb_drain_json(el_val_t max);
/* Document frequency of a term across node labels — term-specificity signal
* for curiosity seed selection. (2026-08-03 self-review.) */
el_val_t engram_label_df(el_val_t term);
/* Best curiosity seed from one node: argmax over idf·position·casing across
* the candidate tokens of its label, falling back to its content when the
* label is a sentinel. Excludes pipe-delimited tabu terms during selection
* and gates candidates to the df band [min_df, max_df]. Returns "" when
* nothing qualifies. (2026-08-13 self-review.) */
el_val_t engram_salient_term(el_val_t node_id, el_val_t max_df,
el_val_t min_df, el_val_t tabu);
el_val_t engram_embed_backfill(el_val_t count);
el_val_t engram_list_layers_json(void);
/* Working memory introspection — count, mean weight, and top-N snapshot.
* Ported from runtime on 2026-06-30 self-review. */
* Ported from el-compiler/runtime on 2026-06-30 self-review. */
el_val_t engram_wm_count(void);
el_val_t engram_wm_avg_weight(void);
el_val_t engram_wm_top_json(el_val_t n);
/* Merge-load: add nodes/edges from a snapshot without resetting the store. */
el_val_t engram_load_merge(el_val_t path);
/* ── WAL + compaction + integrity (ENGRAM_WAL=on; design doc §§3-14,§18) ──── */
int engram_wal_enabled(void);
el_val_t engram_crc32(el_val_t s);
el_val_t engram_wal_boot(el_val_t dir); /* replay + open; returns records */
el_val_t engram_wal_open_dir(el_val_t dir);
el_val_t engram_wal_node_put(el_val_t dir, el_val_t id);
el_val_t engram_wal_edges_since(el_val_t dir, el_val_t start_count);
el_val_t engram_wal_hebb_batch(el_val_t dir, el_val_t start_count);
el_val_t engram_wal_forget(el_val_t dir, el_val_t id);
el_val_t engram_wal_compact(el_val_t dir);
el_val_t engram_wal_maybe_compact(el_val_t dir);
el_val_t engram_resolve_data_dir(void); /* §18.2 fail-loud default */
el_val_t engram_is_protected(el_val_t id); /* §18.1/18.3 derived set */
el_val_t engram_protected_json(void);
/* engram_compile_layered_json — produce a prompt-ready text block split
* into "[LAYER 0 — STRUCTURAL]" (non-suppressible layers, sacred fire)
* and "[ENGRAM CONTEXT]" (standard suppressible layers). Returns "" if
+1 -1
View File
@@ -2,7 +2,7 @@
//
// Thin El wrappers over seed JSON primitives, plus pure-El builders and
// helpers. Each function here corresponds to (and replaces) a C function
// from runtime/el_runtime.c (lines 26923333).
// from el-compiler/runtime/legacy/el_runtime.c (lines 26923333).
//
// Seed primitives consumed by this module:
// __json_get(json, key) -> String (value as string)
+2 -2
View File
@@ -25,8 +25,8 @@
// runtime/collections.el \
// <user-program.el> > combined.el
// ./dist/platform/elc combined.el > output.c
// cc -std=c11 -I runtime -lcurl -lpthread \
// -o output output.c runtime/el_seed.c
// cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
// -o output output.c el-compiler/runtime/el_seed.c
// This file itself is not compiled it is documentation only.
fn runtime_version() -> String {
+1 -1
View File
@@ -1,6 +1,6 @@
// runtime/math.el Float math, integer utilities, and numeric conversions.
//
// Implements the math/float surface from runtime/el_runtime.c
// Implements the math/float surface from el-compiler/runtime/legacy/el_runtime.c
// (lines 303305 for el_abs/max/min, lines 47254771 for float/format ops)
// in pure El, using seed primitives.
//
+1 -1
View File
@@ -1,6 +1,6 @@
// runtime/time.el Time operations, sleep, and formatting.
//
// Implements the time surface from runtime/el_runtime.c
// Implements the time surface from el-compiler/runtime/legacy/el_runtime.c
// (lines 33343440, 34713656) in pure El, using seed primitives.
//
// Seed primitives consumed:
+1 -1
View File
@@ -11,7 +11,7 @@ cd "$(dirname "$0")"
EL_HOME="${EL_HOME:-$(cd ../.. && pwd)}"
ELC="${EL_HOME}/dist/platform/elc"
RUNTIME_DIR="${EL_HOME}/runtime"
RUNTIME_DIR="${EL_HOME}/el-compiler/runtime"
if [ ! -x "${ELC}" ]; then
echo "elc not found at ${ELC}" >&2
+1 -1
View File
@@ -10,7 +10,7 @@ cd "$(dirname "$0")"
EL_HOME="${EL_HOME:-$(cd ../.. && pwd)}"
ELC="${EL_HOME}/dist/platform/elc"
RUNTIME_DIR="${EL_HOME}/runtime"
RUNTIME_DIR="${EL_HOME}/el-compiler/runtime"
if [ ! -x "${ELC}" ]; then
echo "elc not found at ${ELC}" >&2
+2 -2
View File
@@ -14,8 +14,8 @@
// tests/runtime/string_test.el > /tmp/string_test_combined.el
//
// ./dist/platform/elc /tmp/string_test_combined.el > /tmp/string_test.c
// cc -std=c11 -I runtime -lcurl -lpthread \
// -o /tmp/string_test /tmp/string_test.c runtime/el_seed.c
// cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
// -o /tmp/string_test /tmp/string_test.c el-compiler/runtime/el_seed.c
// /tmp/string_test; echo "exit: $?"
//
// Exit code equals the number of failing assertions (0 = all pass).
+1 -1
View File
@@ -11,7 +11,7 @@ cd "$(dirname "$0")"
EL_HOME="${EL_HOME:-$(cd ../.. && pwd)}"
ELC="${ELC:-${EL_HOME}/dist/platform/elc}"
RUNTIME_DIR="${EL_HOME}/runtime"
RUNTIME_DIR="${EL_HOME}/el-compiler/runtime"
if [ ! -x "${ELC}" ]; then
echo "elc not found at ${ELC}" >&2
+1 -1
View File
@@ -16,7 +16,7 @@ cd "$(dirname "$0")"
EL_HOME="${EL_HOME:-$(cd ../.. && pwd)}"
ELC="${EL_HOME}/dist/platform/elc"
RUNTIME_DIR="${EL_HOME}/runtime"
RUNTIME_DIR="${EL_HOME}/el-compiler/runtime"
if [ ! -x "${ELC}" ]; then
echo "elc not found at ${ELC}" >&2
+1 -1
View File
@@ -44,7 +44,7 @@ INCLUDE_DIR="${PREFIX}/include"
LIB_DIR="${PREFIX}/lib"
ELC_SRC="${EL_ROOT}/dist/platform/elc"
RUNTIME_SRC="${EL_ROOT}/runtime"
RUNTIME_SRC="${EL_ROOT}/el-compiler/runtime"
STDLIB_SRC="${EL_ROOT}/runtime"
echo "==> Installing El framework to ${PREFIX}"
+1 -1
View File
@@ -42,7 +42,7 @@ fi
# Discover el_runtime.c
EL_RUNTIME=""
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
LOCAL_RUNTIME="${SCRIPT_DIR}/../runtime/el_runtime.c"
LOCAL_RUNTIME="${SCRIPT_DIR}/../el-compiler/runtime/el_runtime.c"
if [[ -f "${LOCAL_RUNTIME}" ]]; then
EL_RUNTIME="$(cd "$(dirname "${LOCAL_RUNTIME}")" && pwd)/$(basename "${LOCAL_RUNTIME}")"
EL_INCLUDE="$(dirname "${EL_RUNTIME}")"
-94
View File
@@ -1,94 +0,0 @@
#!/usr/bin/env bash
# check-single-runtime.sh — CODE-VS-ARTIFACT drift guard for the el runtime.
#
# Enforces org policy docs/CODE-VS-ARTIFACT.md rule #1 (single source of truth):
# there is exactly ONE authored el_runtime.c, and it lives at lang/runtime/.
# Any other el_runtime.c in the tree is a fork (a hand-synced copy). A lagging
# fork is exactly what shipped to prod and dropped learned `hebb` edges on
# restart — this guard exists to make that class of bug impossible to reintroduce.
#
# Exemptions:
# * Build output — generated amalgamations under any dist/ or build/ dir are
# artifacts, not sources.
# * A small, explicit ALLOWLIST of pre-existing example-app vendored/staging
# copies (see below). These are KNOWN DEFERRED DEBT, tracked separately from
# the SDK/CI-published runtime. They do NOT ship to prod. The guard warns on
# them (visible, greppable) but does not fail — while HARD-FAILING on any new
# or non-allowlisted fork, including any return of lang/el-compiler/runtime/
# or a lang/releases/ vendored copy.
#
# Wire-in: run from the repo root in CI (see note at bottom). Exits non-zero on drift.
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
cd "$ROOT"
CANONICAL="lang/runtime/el_runtime.c"
# KNOWN DEFERRED example-app forks — remove these as a follow-up, then delete
# this allowlist. iOS + Docker copies are regenerated by their build scripts
# (cp from lang/runtime) and can be git-rm'd now; the Android jni/ copy is a
# committed source its CMake build depends on and needs a build-script change
# (cp from lang/runtime) before removal. Tracked in docs/CODE-VS-ARTIFACT.md.
ALLOWLIST=(
"ui/examples/native-hello-android/app/src/main/jni/el_runtime.c"
"ui/examples/native-hello-ios/NativeHello/el_runtime.c"
"ui/examples/native-hello/build-docker/runtime/el_runtime.c"
)
is_allowlisted() {
local p="$1"
for a in "${ALLOWLIST[@]}"; do [ "$p" = "$a" ] && return 0; done
return 1
}
if [ ! -f "$CANONICAL" ]; then
echo "FATAL: canonical runtime source missing: $CANONICAL" >&2
exit 1
fi
# All el_runtime.c files tracked by git, excluding build output (dist/ , build/)
# and the canonical source itself.
mapfile -t CANDIDATES < <(
git ls-files '*el_runtime.c' \
| grep -Ev '(^|/)(dist|build)/' \
| grep -vx "$CANONICAL" || true
)
FORKS=()
DEFERRED=()
for f in "${CANDIDATES[@]}"; do
if is_allowlisted "$f"; then DEFERRED+=("$f"); else FORKS+=("$f"); fi
done
if [ "${#DEFERRED[@]}" -gt 0 ]; then
echo "WARN: allowlisted (deferred) el_runtime.c forks still present — clean these up:" >&2
for f in "${DEFERRED[@]}"; do echo " - $f" >&2; done
fi
if [ "${#FORKS[@]}" -gt 0 ]; then
echo "FATAL: el_runtime.c fork(s) detected outside the canonical location." >&2
echo " Canonical (the ONLY allowed source): $CANONICAL" >&2
echo " Offending copies:" >&2
for f in "${FORKS[@]}"; do echo " - $f" >&2; done
echo "" >&2
echo "Consumers must build against $CANONICAL (pin by git ref where" >&2
echo "reproducibility matters) — never a hand-maintained copy." >&2
echo "See docs/CODE-VS-ARTIFACT.md." >&2
exit 1
fi
echo "OK: single canonical runtime source — $CANONICAL (no un-allowlisted forks)."
# ---------------------------------------------------------------------------
# CI wire-in:
# foundation/el .gitea/workflows/ci-dev.yaml, ci-stage.yaml, sdk-release.yaml
# Add an early step (before the build/publish steps). It must run from the
# REPO ROOT, so override the job's `defaults.run.working-directory: lang`:
#
# - name: Guard - single canonical runtime source
# working-directory: ${{ github.workspace }}
# run: bash scripts/check-single-runtime.sh
#
# Also add to .githooks/pre-commit so drift is caught before it is committed.
# ---------------------------------------------------------------------------
+64
View File
@@ -0,0 +1,64 @@
# nsbx — Neuron dev-environment Makefile
# ---------------------------------------------------------------------------
# Thin, documented wrappers over the `nsbx` primitive so a newcomer never has to
# memorise the elc/cc build incantation or the sandbox lifecycle. Every target is
# a one-liner over `nsbx`; nothing here reimplements engram logic.
#
# make dev NAME=tim # new branch + worktree + isolated engram, one shot
# make status NAME=tim # inspect it (omit NAME to list all sandboxes)
# make run NAME=tim # poke its API (API=/api/stats by default)
# make test NAME=tim # run the safety rails as checks (nsbx validate)
# make build NAME=tim # compile the worktree's changes into the engram
# make destroy NAME=tim # tear it all down (branch kept)
#
# The isolated engram is ALWAYS a clone of the live store on a NON-default port;
# prod (:8742 / :7770) is untouchable from here.
# ---------------------------------------------------------------------------
# locate nsbx next to this Makefile, regardless of where make is run from
NSBX := $(dir $(realpath $(lastword $(MAKEFILE_LIST))))nsbx
SBX := dev-$(NAME)
API ?= /api/stats
.DEFAULT_GOAL := help
.PHONY: help dev build run test status list destroy bt
help: ## Show this help
@echo "nsbx dev-environment — one-command isolated Neuron dev setup"
@echo ""
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(firstword $(MAKEFILE_LIST)) \
| awk 'BEGIN{FS=":.*?## "}{printf " make %-22s %s\n", $$1, $$2}'
@echo ""
@echo " Variables: NAME=<dev name> (required for most) API=<path> (run)"
@echo " BASE=<git ref> WT=<worktree dir> PORT=<n> ARGS=<extra nsbx flags>"
dev: ## Create branch + worktree + isolated engram (NAME=x [BASE=ref PORT=n WT=dir ARGS=...])
@test -n "$(NAME)" || { echo "usage: make dev NAME=<name>"; exit 2; }
$(NSBX) dev $(NAME) $(if $(BASE),--base $(BASE)) $(if $(PORT),--port $(PORT)) $(if $(WT),--worktree $(WT)) $(ARGS)
build: ## Rebuild the engram from the dev worktree's own source (NAME=x)
@test -n "$(NAME)" || { echo "usage: make build NAME=<name>"; exit 2; }
@wt=$$(python3 -c "import json,os;print(json.load(open(os.path.expanduser('~/.neuron/sandboxes/$(SBX)/dev.json')))['worktree'])" 2>/dev/null); \
test -n "$$wt" || { echo "no dev.json for $(SBX) — run 'make dev NAME=$(NAME)' first"; exit 2; }; \
$(NSBX) build $(SBX) --source "$$wt"
bt: build run ## Fast El loop: rebuild the engram from the worktree, then poke it (NAME=x)
run: ## Poke the isolated engram's API (NAME=x [API=/api/stats])
@test -n "$(NAME)" || { echo "usage: make run NAME=<name> [API=/path]"; exit 2; }
$(NSBX) run $(SBX) api $(API)
test: ## Run the safety rails as checks: zero-loss, reboot, RSS, parity, keystones (NAME=x)
@test -n "$(NAME)" || { echo "usage: make test NAME=<name>"; exit 2; }
$(NSBX) validate $(SBX)
status: ## Show one sandbox's status (NAME=x), or list all if NAME is unset
@if [ -n "$(NAME)" ]; then $(NSBX) status $(SBX); else $(NSBX) list; fi
list: ## List all sandboxes
$(NSBX) list
destroy: ## Tear down engram + worktree (NAME=x [ARGS=--delete-branch])
@test -n "$(NAME)" || { echo "usage: make destroy NAME=<name>"; exit 2; }
$(NSBX) dev-down $(NAME) $(ARGS)
+176
View File
@@ -0,0 +1,176 @@
# nsbx — the Neuron Sandbox
**Dev environment as a primitive.** A reproducible way to run experiments *and code
changes* against the **real** engram runtime on an isolated snapshot of the live
mind — with a gated promote-to-prod path built on the proven rails.
Everyone (Tim, any team member, any agent) gets their own private, safe copy of the
mind to build against. **Prod — the live Neuron on `:8742` (engram) / `:7770`
(soul) — is untouchable from a sandbox.** A sandbox runs a *separate* engram
process, on a *separate* port, against a *separate* clone of the store. The only op
that can ever reach prod is `promote`, which is explicit, gated, and per-use
approved.
It **wraps the real engram binary** — it never reimplements any engram logic. It
generalises two proven proto-sandboxes into one primitive:
- the **cog-arch** build — isolated git worktree + build + clone of the live `.egm` + real C tests
- the **store-fix** cutover — secondary soul + launchctl `bootout → settle → bootstrap` rails
## Quickstart
```bash
export PATH="$PWD:$PATH" # or symlink nsbx onto your PATH
nsbx up # your private copy of the mind (auto-named <user>-dev)
nsbx run <name> api /api/stats # poke it
nsbx validate <name> # prove it: zero-loss, reboot, RSS, retrieval, keystones
nsbx destroy <name> # cheap teardown; live untouched
```
That is the whole loop. Sane defaults: stock prod binary, auto-allocated port
(`8900+`, never `8742`/`7770`), snapshot of the live store.
## One-command dev onboarding — `nsbx dev` (start here)
Going from a clone to *coding on the mind* is a single command. It creates a git
**branch**, a persistent git **worktree**, and an **isolated engram** (a clone of the
live store on a non-default port) — and wires the whole worktree to that clone so you
**cannot hit live `:8742` by accident**.
```bash
make dev NAME=tim # branch wt/tim + worktree + isolated engram, in one shot
cd ~/Development/neuron-technologies/el-worktrees/tim
source .nsbx-env # every ENGRAM_* var now points at YOUR clone
# edit El in the worktree, then the fast loop:
make build NAME=tim # compile your El change into the isolated engram
make run NAME=tim # poke it (API=/api/stats by default)
make test NAME=tim # run the safety rails as checks
make destroy NAME=tim # tear it all down (branch kept; live untouched)
```
**Why this exists:** so provisional/experimental work is built **directly in El against a
throwaway cloned engram** — not prototyped in Python and re-ported later. The El
edit → `make build``make run` loop is the path of least resistance; that double-work is
what stranded the translation faculty for weeks.
### What `nsbx dev <name>` does, in order
1. **branch**`git worktree add -b <prefix><name>` (default prefix `dev/`; a *real named
branch*, never detached HEAD).
2. **worktree** — at a **persistent** path (default `…/el-worktrees/<name>`, override
`NSBX_DEV_WT_ROOT`). It **refuses** `/tmp` — temp dirs are ablated on compaction, which
is the exact "worktree in /tmp + no branch = lost work" failure this designs out.
3. **isolated engram**`nsbx create` under the hood: clone of the live store + WAL +
config, booted on an auto-allocated port (`8900+`, never `:8742`/`:7770`). Stock prod
binary by default (instant); `--build` compiles the worktree's own runtime instead.
4. **env pin** — writes `.nsbx-env` (+ `.envrc` for direnv) into the worktree exporting
`ENGRAM_URL / ENGRAM_PORT / ENGRAM_DATA_DIR / ENGRAM_API_KEY / NEURON_ENGRAM_URL …` — all
pointing at the clone. Nothing references live.
```
nsbx dev <name> [--base REF] [--worktree DIR] [--port N] [--prefix P] [--build] [--no-engram] [--repo R]
nsbx dev-down <name> [--delete-branch] [--repo R] # destroy engram + remove worktree
```
### Makefile targets
| target | does |
|--------|------|
| `make dev NAME=x` | branch + worktree + isolated engram (one shot) |
| `make bt NAME=x` | fast El loop: `build` then `run` |
| `make build NAME=x` | recompile the engram from the worktree's El source |
| `make run NAME=x` | poke the isolated engram (`API=/api/stats`) |
| `make test NAME=x` | rails as checks (`nsbx validate`) |
| `make status [NAME=x]` | inspect one, or `list` all |
| `make destroy NAME=x` | tear down (add `ARGS=--delete-branch` to drop the branch) |
> Note: if a bare `dev` branch already exists in the repo, git can't create `dev/*` names —
> pass `--prefix wt/` (or delete the stray `dev` branch). The tool surfaces git's exact error.
## The code-change dev loop (first-class)
Run *your changed runtime*, not just the stock binary, against a snapshot:
```bash
# build a runtime from a working tree, a git branch, or a prebuilt binary:
nsbx create feat --source /path/to/worktree # elc + cc build from source
nsbx create feat --branch feat/my-change --repo <r> # worktree the branch, then build
nsbx create feat --binary /path/to/engram # use a prebuilt binary
nsbx build feat --source /path/to/worktree # rebuild + hot-restart in place
nsbx validate feat # prove the change is safe
nsbx promote feat --i-approve-prod-cutover # gated rails cutover (see below)
```
The build replicates the engram release recipe exactly:
`elc engram/src/server.el > engram.c` then
`cc -std=c11 -O2 -I lang/runtime engram.c el_runtime.c engram_*.c -lcurl -lpthread`.
## Lifecycle
| op | what it does |
|----|--------------|
| `create <name> [--port N] [--source\|--branch\|--binary]` | consistent snapshot of the live store+WAL+config into an isolated dir; place or **build** the runtime; boot the real engram daemon on an isolated port. Named, versioned (binary sha + egm sha in `manifest.json`), reproducible. |
| `up [name]` | one command: create-if-missing then start; prints the URL. |
| `build <name> --source\|--branch` | rebuild the runtime from a code change and hot-restart on the same clone+port. |
| `run <name> <cmd…>` / `run <name> api <path> [json]` | run an experiment against the real runtime; capture output + before/after stats + wall time. Env: `$SBX_URL $SBX_PORT $SBX_KEY $SBX_DATA $SBX_BIN`. |
| `validate <name>` | the rails as first-class checks (below). |
| `promote <name> [--data] [--i-approve-prod-cutover]` | **the only prod-touching op.** Gated rails cutover. DRY-RUN plan unless approved. |
| `destroy <name>` | stop the isolated daemon, free the port, remove the clone. Live untouched. |
| `list` / `status <name>` | inspect. |
## `validate` — the rails as checks
- **zero-loss-under-load** — node/edge counts hold at/above baseline through ~15s of sustained tick+read load
- **reboot-prove** — counts survive a real stop→start of the daemon
- **rss-bound** — daemon RSS under `NSBX_RSS_BOUND_MB` (default 550 MB, from the store-fix reboot-proof)
- **retrieval-parity** — top-k node ids for a fixed probe set match the create-time baseline
- **keystone-integrity** — `kn-efeb4a5b…` and `kn-5b606390…` present and intact
A PASS writes `validate.json` stamped with the binary sha; `promote` refuses unless
the current binary has a fresh PASS on record.
## `promote` — gated cutover (rails only)
Default is a **dry-run plan**. With `--i-approve-prod-cutover` it, in order:
1. **snapshot-first** — back up live `egm`+`wal`+`plist` to `~/.neuron/backups/promote-<name>-<ts>/` with a `rollback.txt`
2. **additive** binary install — copy the validated binary to a *new* file, update the plist `ENGRAM_REAL_BIN` (old binary retained — additive/supersede, never destructive)
3. **rails cutover**`launchctl bootout`**settle-poll** (prints until the job is gone) → `launchctl bootstrap`. Never `pkill`, never `kickstart -k`.
4. **verify**`/api/stats` returns, edges ≥ baseline, keystones intact
5. **auto-rollback armed** — any verify failure restores the plist (and data, if `--data`) and boots the prior binary back via the same rails
## Isolation guarantees
- separate **port** (`8900+`; refuses `8742`/`7770`), separate **store clone**, separate **process**
- a hard guard refuses to boot a sandbox daemon whose data dir resolves to the live store
- sandboxes are plain supervised background processes (not launchd), so teardown is a signal + settle-poll — it can never touch the prod launchd job
- prod is read exactly twice: once for the snapshot, and (only if you approve) during `promote`
## Layout
- tool: `tools/neuron-sandbox/nsbx` (this repo, branch `feat/neuron-sandbox`)
- runtime state: `~/.neuron/sandboxes/<name>/``data/` (clone), `bin/engram`, `build/`, `logs/`, `manifest.json`, `validate.json`, `baseline/`
## Validated (dogfood)
Standing up a sandbox from a live-store clone and reproducing a **known** result:
- **retrieval-parity 25/25** top-k id overlap vs baseline; sandbox boot-stats exactly matched the live baseline captured at snapshot time (10 672 nodes / 32 439 edges) — the wrapped real binary faithfully reloads the live mind
- reboot-prove + zero-loss PASS; RSS 379 MB < 550 MB; keystones intact
- the **cog-arch correspondence-loop** re-run *inside* the sandbox reproduced the known calibration numbers exactly: held-Brier **0.028648 → 0.000586** (98.0% reduction), monotone, **reboot bit-identical**, metastability holds; and the real-store Stance persistence reboot-proved at **10 994-node** scale (`think()` on real 768-dim embeddings) against a scratch copy of the sandbox's own clone — never live
- `promote` dry-run refused to touch prod; teardown freed the port; live `:8742`/`:7770` never perturbed (soul uptime unbroken)
## Migrating existing experiments
Each ad-hoc harness becomes `nsbx run <name> …` (or `--source` build) against a sandbox:
- **cog-arch** — `nsbx create x --source <worktree>` then `nsbx run x -- bash cogarch_dogfood.sh` (compiles + runs the real C cognition tests against `$SBX_DATA`)
- **codec / ingest / faculty** — `nsbx run x api /api/<endpoint> '<json>'` against the isolated daemon, or a script using `$SBX_URL`/`$SBX_KEY`; measure with the built-in before/after stats
## Env knobs
`NSBX_ROOT`, `NSBX_PORT_BASE`, `NSBX_RSS_BOUND_MB`, `NSBX_REMERGE_THRESHOLD`,
`EL_REPO` (for `elc` + runtime sources), `ENGRAM_LIVE_DATA_DIR`, `ENGRAM_LIVE_PLIST`.
+30
View File
@@ -0,0 +1,30 @@
#!/usr/bin/env bash
# cog-arch correspondence-loop dogfood — RUN INSIDE the sandbox via `nsbx run`.
# Compiles the REAL engram C runtime + cognition tests and reproduces the known
# calibration result (memory 194c69c8): held-Brier 0.028648 -> 0.000586, reboot-proven,
# then reboot-proves the Stance persistence against a SCRATCH COPY of THIS sandbox's
# clone of the real store (never live, never the running daemon's file).
set -euo pipefail
WT="${COGARCH_WT:-/private/tmp/claude-501/-Users-will/6531446d-bc27-4095-930b-e04777c3db4f/scratchpad/cogarch-wt}"
RT="$WT/lang/runtime"; T="$WT/engram/test"
: "${SBX_DATA:?run me via: nsbx run <name> -- bash cogarch_dogfood.sh}"
B="$(mktemp -d)"
echo "### building cog-arch tests against the real engram runtime sources"
cc -std=c11 -O2 -w -I "$RT" -o "$B/test_cognition" \
"$T/test_cognition.c" "$RT/engram_cognition.c" "$RT/engram_reason.c" \
"$RT/engram_geometry.c" "$RT/engram_store.c" "$RT/engram_vindex.c" -lm
cc -std=c11 -O2 -w -I "$RT" -o "$B/test_realstore" \
"$T/test_cognition_realstore.c" "$RT/engram_cognition.c" "$RT/engram_reason.c" \
"$RT/engram_geometry.c" "$RT/engram_store.c" "$RT/engram_vindex.c" -lm
echo; echo "### [A] synthetic correspondence-loop (known: Brier 0.028648 -> 0.000586)"
"$B/test_cognition" | grep -E "held-Brier|reduction|reboot|monotone|metastab|RESULT" || true
echo; echo "### [B] reboot-prove Stance on a SCRATCH COPY of this sandbox's real-store clone"
SCRATCH="$B/store-clone"; mkdir -p "$SCRATCH"
cp -p "$SBX_DATA/neuron.egm" "$SCRATCH/" 2>/dev/null || true
cp -p "$SBX_DATA/neuron.wal" "$SCRATCH/" 2>/dev/null || true
cp -p "$SBX_DATA/conf" "$SCRATCH/" 2>/dev/null || true
cp -p "$SBX_DATA/meta.json" "$SCRATCH/" 2>/dev/null || true
"$B/test_realstore" "$SCRATCH" || true
rm -rf "$B"
+836
View File
@@ -0,0 +1,836 @@
#!/usr/bin/env bash
# nsbx — the Neuron Sandbox: a reproducible primitive for running experiments and
# code changes against the REAL engram runtime on an isolated snapshot of the live
# mind, with a gated promote-to-prod path built on the proven rails.
#
# It WRAPS the real engram binary — it never reimplements any engram logic. The only
# prod-touching op is `promote`, which is explicit, gated, and per-use approved.
#
# Generalises two proven proto-sandboxes:
# - the cog-arch build (isolated git worktree + build + clone of live .egm + real C tests)
# - the store-fix cutover (secondary soul + launchctl bootout->settle->bootstrap rails)
#
# Lifecycle: create -> [build] -> run -> validate -> promote(gated) -> destroy
#
# Rails (always): built offline; NEVER auto-promotes; never touches live :8742/:7770
# except READ for the snapshot and the gated promote; snapshot-first; honest measured
# reporting. Cutover is launchctl bootout -> settle-poll -> bootstrap ONLY —
# never pkill, never kickstart -k.
set -uo pipefail
# ---------------------------------------------------------------- constants ----
LIVE_DATA_DIR="${ENGRAM_LIVE_DATA_DIR:-$HOME/.neuron/engram}"
LIVE_PLIST="${ENGRAM_LIVE_PLIST:-$HOME/Library/LaunchAgents/ai.neuron.engram.plist}"
LIVE_LABEL="ai.neuron.engram"
LIVE_BIND_PORT=8742 # engram — FORBIDDEN for sandboxes
SOUL_PORT=7770 # soul — FORBIDDEN for sandboxes
LIVE_KEY="${ENGRAM_API_KEY:-ntn-user-2026}"
LIVE_URL="http://127.0.0.1:${LIVE_BIND_PORT}"
SBX_ROOT="${NSBX_ROOT:-$HOME/.neuron/sandboxes}"
BACKUP_ROOT="$HOME/.neuron/backups"
EL_REPO="${EL_REPO:-$HOME/Development/neuron-technologies/foundation/el}"
PORT_BASE="${NSBX_PORT_BASE:-8900}"
RSS_BOUND_MB="${NSBX_RSS_BOUND_MB:-550}" # from store-fix reboot-proof (aaf13f88)
REMERGE_THRESHOLD="${NSBX_REMERGE_THRESHOLD:-40000}"
KEYSTONES=( "kn-efeb4a5b-5aff-4759-8a97-7233099be6ee" "kn-5b606390-a52d-4ca2-8e0e-eba141d13440" )
# fixed probe set for retrieval-parity (stable, identity-anchored)
PARITY_QUERIES=( "who am I" "self identity core" "engram store durability" "keystone self anchor" "grounding honesty" )
C_RED=$'\033[31m'; C_GRN=$'\033[32m'; C_YEL=$'\033[33m'; C_DIM=$'\033[2m'; C_BLD=$'\033[1m'; C_0=$'\033[0m'
# ---------------------------------------------------------------- helpers ------
die(){ printf '%serror:%s %s\n' "$C_RED" "$C_0" "$*" >&2; exit 1; }
log(){ printf '%s==>%s %s\n' "$C_BLD" "$C_0" "$*" >&2; }
info(){ printf ' %s\n' "$*" >&2; }
ok(){ printf ' %s%s%s\n' "$C_GRN" "$*" "$C_0" >&2; }
warn(){ printf ' %s%s%s\n' "$C_YEL" "$*" "$C_0" >&2; }
need(){ command -v "$1" >/dev/null 2>&1 || die "missing dependency: $1"; }
now(){ date -u +%Y%m%dT%H%M%SZ; }
sha(){ shasum -a 256 "$1" 2>/dev/null | awk '{print $1}'; }
epoch(){ python3 -c 'import time;print(time.time())'; }
sdir(){ printf '%s/%s' "$SBX_ROOT" "$1"; }
manifest(){ printf '%s/manifest.json' "$(sdir "$1")"; }
mexists(){ [ -f "$(manifest "$1")" ]; }
mget(){ # mget <name> <jsonpath>
python3 -c "import json,sys; d=json.load(open('$(manifest "$1")')); print(d$2)" 2>/dev/null
}
port_free(){ ! (exec 3<>"/dev/tcp/127.0.0.1/$1") 2>/dev/null; }
alloc_port(){
local p="$PORT_BASE"
while :; do
if [ "$p" = "$LIVE_BIND_PORT" ] || [ "$p" = "$SOUL_PORT" ]; then p=$((p+1)); continue; fi
if port_free "$p" && ! _port_claimed "$p"; then echo "$p"; return 0; fi
p=$((p+1)); [ "$p" -gt 9100 ] && die "no free sandbox port in range"
done
}
_port_claimed(){ # is another sandbox already assigned this port?
local p="$1" d
for d in "$SBX_ROOT"/*/manifest.json; do
[ -f "$d" ] || continue
[ "$(python3 -c "import json;print(json.load(open('$d'))['port'])" 2>/dev/null)" = "$p" ] && return 0
done
return 1
}
live_stats(){ curl -s -m5 "$LIVE_URL/api/stats" 2>/dev/null; }
api(){ # api <name> <path> [json-body]
local name="$1" path="$2" body="${3:-}"
local port; port="$(mget "$name" "['port']")"; [ -n "$port" ] || die "unknown sandbox: $name"
local url="http://127.0.0.1:${port}${path}"
if [ -n "$body" ]; then curl -s -m30 -X POST -H 'Content-Type: application/json' -d "$body" "$url"
else curl -s -m30 "$url"; fi
}
sbx_stats(){ api "$1" "/api/stats"; }
stat_field(){ printf '%s' "$1" | sed -n "s/.*\"$2\":\([0-9]*\).*/\1/p"; }
daemon_pid(){ local f; f="$(sdir "$1")/daemon.pid"; [ -f "$f" ] && cat "$f" || true; }
daemon_alive(){ local p; p="$(daemon_pid "$1")"; [ -n "$p" ] && kill -0 "$p" 2>/dev/null; }
# ---------------------------------------------------------------- elc/build ----
find_elc(){
command -v elc 2>/dev/null && return 0
local arch; arch="$(uname -m)"
case "$arch" in
arm64) echo "$EL_REPO/lang/dist/platform/elc-darwin-arm64";;
x86_64) echo "$EL_REPO/lang/dist/platform/elc-linux-amd64";;
*) echo "$EL_REPO/lang/dist/platform/elc";;
esac
}
# _build_binary <src_tree> <out_bin> <build_log_dir>
# Replicates the proven engram release recipe:
# elc engram/src/server.el > engram.c
# cc -std=c11 -O2 -I lang/runtime engram.c el_runtime.c engram_*.c -lcurl -lpthread
_build_binary(){
local src="$1" out="$2" blog="$3"
local elc server rt
elc="$(find_elc)"; [ -x "$elc" ] || die "elc not found/executable: $elc (set EL_REPO)"
server="$src/engram/src/server.el"; rt="$src/lang/runtime"
[ -f "$server" ] || die "no engram/src/server.el under source tree: $src"
[ -f "$rt/el_runtime.c" ] || die "no lang/runtime/el_runtime.c under source tree: $src (this branch may keep it generated/untracked)"
ls "$rt"/engram_*.c >/dev/null 2>&1 || die "no lang/runtime/engram_*.c engine sources under: $src"
mkdir -p "$blog"
log "build: elc transpile server.el -> engram.c"
"$elc" "$server" > "$blog/engram.c" 2>"$blog/elc.err" || { cat "$blog/elc.err" >&2; die "elc transpile failed"; }
info "engram.c: $(wc -c <"$blog/engram.c" | tr -d ' ') bytes"
log "build: cc link (el_runtime + engram_* engine)"
cc -std=c11 -O2 -w -I "$rt" -o "$out" \
"$blog/engram.c" "$rt/el_runtime.c" "$rt"/engram_*.c \
-lcurl -lpthread 2>"$blog/cc.err" \
|| { grep -i 'error:' "$blog/cc.err" | sort -u | head >&2; die "cc link failed (see $blog/cc.err)"; }
ok "built: $out ($(ls -lh "$out" | awk '{print $5}'), sha $(sha "$out" | cut -c1-12))"
}
# ---------------------------------------------------------------- daemon -------
# start_daemon <name> : boots the sandbox's real engram binary on its isolated
# port against its cloned data dir, with the SAME auto-remerge net the live soul
# uses (so the sandbox faithfully reaches the live edge population on boot).
start_daemon(){
local name="$1" d; d="$(sdir "$name")"
daemon_alive "$name" && { info "already running (pid $(daemon_pid "$name"))"; return 0; }
local port bin data export key
port="$(mget "$name" "['port']")"; bin="$d/bin/engram"; data="$d/data"
key="sbx-$name"; export="$data/.scan-export.reseed-clean.json"
[ -x "$bin" ] || die "sandbox binary missing: $bin"
[ "$port" != "$LIVE_BIND_PORT" ] && [ "$port" != "$SOUL_PORT" ] || die "refusing forbidden port $port"
[ -f "$data/neuron.egm" ] || die "sandbox has no cloned store: $data/neuron.egm"
# HARD guard: never point a sandbox daemon at the live data dir.
[ "$(cd "$data" && pwd -P)" != "$(cd "$LIVE_DATA_DIR" && pwd -P)" ] || die "refusing: sandbox data dir resolves to LIVE store"
log "boot engram on isolated :$port (data=$data)"
(
ENGRAM_DATA_DIR="$data" ENGRAM_BIND=":$port" ENGRAM_API_KEY="$key" \
ENGRAM_STORE=1 ENGRAM_CHRONOCEPTION=1 ENGRAM_SELF_REIFY=1 ENGRAM_GC=1 \
ENGRAM_POOL_FRAMES=16384 ENGRAM_WRITE_BARRIER=1 \
exec "$bin"
) >"$d/logs/daemon.log" 2>&1 &
local pid=$!
echo "$pid" > "$d/daemon.pid"
# readiness poll
local url="http://127.0.0.1:$port" i s
for i in $(seq 1 30); do
s="$(curl -s -m3 "$url/api/stats" 2>/dev/null)"
[ -n "$s" ] && break; sleep 0.5
done
[ -n "$s" ] || { warn "daemon did not become ready (see $d/logs/daemon.log)"; return 1; }
ok "ready pid=$pid boot-stats: $s"
# auto-remerge net (idempotent): match live edge population if the export is present
if [ -f "$export" ]; then
local edges; edges="$(stat_field "$s" edge_count)"
if [ -n "$edges" ] && [ "$edges" -lt "$REMERGE_THRESHOLD" ]; then
log "auto-remerge: booted with $edges edges (< $REMERGE_THRESHOLD) — merging full edge export"
local r; r="$(curl -s -m300 -X POST -H 'Content-Type: application/json' \
-d "{\"_auth\":\"$key\",\"path\":\"$export\"}" "$url/api/load-merge" 2>/dev/null)"
info "remerge resp: ${r:0:120}"
ok "post-remerge stats: $(curl -s -m5 "$url/api/stats")"
fi
fi
return 0
}
# stop_daemon <name> : graceful TERM + settle-poll until the port is free.
# (Sandbox daemons are plain supervised bg processes — not launchd — so teardown
# is a signal + poll, never pkill of anything else.)
stop_daemon(){
local name="$1" pid port
pid="$(daemon_pid "$name")"; port="$(mget "$name" "['port']")"
[ -n "$pid" ] || { info "not running"; return 0; }
log "stop daemon pid=$pid, settle-poll until :$port frees"
kill "$pid" 2>/dev/null || true
local i
for i in $(seq 1 40); do
kill -0 "$pid" 2>/dev/null || { port_free "$port" && { ok "stopped, port $port free"; : >"$(sdir "$name")/daemon.pid"; return 0; }; }
printf '.' >&2; sleep 0.5
done
printf '\n' >&2
kill -9 "$pid" 2>/dev/null || true; sleep 1
: >"$(sdir "$name")/daemon.pid"
port_free "$port" && ok "stopped (after SIGKILL), port $port free" || warn "port $port still busy"
}
# ================================================================ create =======
cmd_create(){
local name="" port="" src="" branch="" repo="$EL_REPO" binpath=""
# first positional arg is the name unless it's a flag; default to "<user>-dev"
if [ $# -gt 0 ] && [ "${1#-}" = "$1" ]; then name="$1"; shift; else name="${USER:-dev}-dev"; fi
while [ $# -gt 0 ]; do case "$1" in
--port) port="$2"; shift 2;;
--source) src="$2"; shift 2;;
--branch) branch="$2"; shift 2;;
--repo) repo="$2"; shift 2;;
--binary) binpath="$2"; shift 2;;
*) die "unknown flag: $1";;
esac; done
mexists "$name" && die "sandbox '$name' already exists (destroy it first)"
need curl; need python3; need shasum
[ -f "$LIVE_DATA_DIR/neuron.egm" ] || die "live store not found: $LIVE_DATA_DIR/neuron.egm"
if [ -n "$port" ]; then
{ [ "$port" = "$LIVE_BIND_PORT" ] || [ "$port" = "$SOUL_PORT" ]; } && die "refusing forbidden port $port (live)"
port_free "$port" || die "port $port already in use"
else port="$(alloc_port)"; fi
local d; d="$(sdir "$name")"
mkdir -p "$d/data" "$d/bin" "$d/logs" "$d/build" "$d/baseline"
log "sandbox '$name' at $d (isolated port $port)"
# ---- CONSISTENT snapshot of the live mind (file-copy: same set the rails backup
# uses; WAL replay on sandbox boot reconciles the tail -> crash-consistent) ----
log "snapshot live store -> clone (store + WAL + config)"
local f
for f in neuron.egm neuron.wal conf meta.json self_anchor .scan-export.reseed-clean.json; do
if [ -e "$LIVE_DATA_DIR/$f" ]; then cp -p "$LIVE_DATA_DIR/$f" "$d/data/$f"; info "cloned $f ($(du -h "$d/data/$f" | awk '{print $1}'))"; fi
done
local egm_sha; egm_sha="$(sha "$d/data/neuron.egm")"
# ---- capture live baseline (READ only) ----
local lstats; lstats="$(live_stats)"
local base_nodes base_edges
base_nodes="$(stat_field "$lstats" node_count)"; base_edges="$(stat_field "$lstats" edge_count)"
info "live baseline stats: ${lstats:-<unavailable>}"
# ---- determine + place the runtime binary (versioned into the snapshot) ----
local source_desc live_bin
live_bin="$(_live_real_bin)"
if [ -n "$binpath" ]; then
[ -x "$binpath" ] || die "not an executable binary: $binpath"
cp -p "$binpath" "$d/bin/engram"; source_desc="prebuilt:$binpath"
elif [ -n "$src" ]; then
_build_binary "$src" "$d/bin/engram" "$d/build"; source_desc="source:$src"
elif [ -n "$branch" ]; then
log "worktree: $repo @ $branch -> $d/build/worktree"
git -C "$repo" worktree add --detach "$d/build/worktree" "$branch" >/dev/null 2>&1 \
|| die "git worktree add failed ($repo @ $branch)"
_build_binary "$d/build/worktree" "$d/bin/engram" "$d/build"; source_desc="branch:$branch@$repo"
else
[ -x "$live_bin" ] || die "cannot resolve live ENGRAM_REAL_BIN: $live_bin"
cp -p "$live_bin" "$d/bin/engram"; source_desc="stock-prod:$live_bin"
fi
local bin_sha; bin_sha="$(sha "$d/bin/engram")"
info "runtime: $source_desc (sha ${bin_sha:0:12})"
# ---- write manifest ----
python3 - "$name" "$port" "$source_desc" "$bin_sha" "$egm_sha" "$base_nodes" "$base_edges" "$(sha "$live_bin" 2>/dev/null)" <<'PY' > "$(manifest "$name")"
import json,sys,datetime
name,port,src,binsha,egmsha,bn,be,livebinsha=sys.argv[1:9]
json.dump({
"name":name,"port":int(port),"created_at":datetime.datetime.now(datetime.timezone.utc).isoformat(),
"source":src,"binary_sha256":binsha,"clone_egm_sha256":egmsha,
"live_binary_sha256":livebinsha,
"live_baseline":{"node_count":int(bn or 0),"edge_count":int(be or 0)},
"keystones":["kn-efeb4a5b-5aff-4759-8a97-7233099be6ee","kn-5b606390-a52d-4ca2-8e0e-eba141d13440"]
}, sys.stdout, indent=2)
PY
ok "manifest written"
# ---- boot + capture the sandbox's own settled baseline (reproducible target) ----
start_daemon "$name" || die "daemon failed to start"
local sstats; sstats="$(sbx_stats "$name")"
local sbn sbe; sbn="$(stat_field "$sstats" node_count)"; sbe="$(stat_field "$sstats" edge_count)"
_capture_retrieval "$name" "$d/baseline/retrieval.json"
# fold sandbox baseline into manifest
python3 - "$(manifest "$name")" "$sbn" "$sbe" <<'PY'
import json,sys
mf,bn,be=sys.argv[1],sys.argv[2],sys.argv[3]
d=json.load(open(mf)); d["sbx_baseline"]={"node_count":int(bn or 0),"edge_count":int(be or 0)}
json.dump(d,open(mf,'w'),indent=2)
PY
log "created."
info "sandbox baseline (settled): nodes=$sbn edges=$sbe"
info "next: nsbx validate $name | nsbx run $name api /api/stats"
}
_live_real_bin(){
python3 - "$LIVE_PLIST" <<'PY' 2>/dev/null
import sys,plistlib
try:
d=plistlib.load(open(sys.argv[1],'rb'))
print(d.get("EnvironmentVariables",{}).get("ENGRAM_REAL_BIN",""))
except Exception: print("")
PY
}
_capture_retrieval(){ # <name> <outfile> : top-k ids for the fixed probe set
local name="$1" out="$2" q res
local port; port="$(mget "$name" "['port']")"; local key="sbx-$name"
{
echo "{"
local first=1
for q in "${PARITY_QUERIES[@]}"; do
res="$(curl -s -m10 -X POST -H 'Content-Type: application/json' \
-d "{\"_auth\":\"$key\",\"query\":\"$q\",\"limit\":5}" "http://127.0.0.1:$port/api/search" 2>/dev/null)"
local ids; ids="$(printf '%s' "$res" | python3 -c 'import sys,json
try:
d=json.load(sys.stdin)
rows=d if isinstance(d,list) else d.get("results",d.get("hits",[]))
print(json.dumps([r.get("id") for r in rows][:5]))
except Exception: print("[]")' 2>/dev/null)"
[ $first -eq 1 ] || echo ","; first=0
printf ' %s: %s' "$(python3 -c "import json,sys;print(json.dumps(sys.argv[1]))" "$q")" "${ids:-[]}"
done
echo ""; echo "}"
} > "$out"
}
# ================================================================ up ===========
# Dead-simple one-command dev environment: `nsbx up` gives you (or Tim, or anyone)
# a private, isolated copy of the live mind to build against. Creates it on first
# run with sane defaults (stock prod binary, auto-allocated port), just starts it
# thereafter. Prod on :$LIVE_BIND_PORT/:$SOUL_PORT is unreachable from here by design.
cmd_up(){
local name; if [ $# -gt 0 ] && [ "${1#-}" = "$1" ]; then name="$1"; shift; else name="${USER:-dev}-dev"; fi
if mexists "$name"; then daemon_alive "$name" || start_daemon "$name"; else cmd_create "$name" "$@"; fi
local port; port="$(mget "$name" "['port']")"
echo >&2
ok "your sandbox '$name' is ready at http://127.0.0.1:$port (a private copy of the mind — prod is untouchable)"
info "experiment: nsbx run $name api /api/stats"
info "prove it: nsbx validate $name"
info "tear down: nsbx destroy $name"
}
# ================================================================ build ========
# Rebuild an existing sandbox's runtime from a source tree/branch and hot-restart
# it on the SAME clone + port (the code-change dev loop, in place).
cmd_build(){
local name="$1"; shift || true
mexists "$name" || die "no such sandbox: $name"
local src="" branch="" repo="$EL_REPO"
while [ $# -gt 0 ]; do case "$1" in
--source) src="$2"; shift 2;; --branch) branch="$2"; shift 2;; --repo) repo="$2"; shift 2;;
*) die "unknown flag: $1";; esac; done
local d; d="$(sdir "$name")"
stop_daemon "$name"
if [ -n "$src" ]; then _build_binary "$src" "$d/bin/engram" "$d/build"
elif [ -n "$branch" ]; then
rm -rf "$d/build/worktree" 2>/dev/null; git -C "$repo" worktree prune 2>/dev/null
git -C "$repo" worktree add --detach "$d/build/worktree" "$branch" >/dev/null 2>&1 || die "worktree add failed"
_build_binary "$d/build/worktree" "$d/bin/engram" "$d/build"
else die "usage: nsbx build <name> --source DIR | --branch REF [--repo R]"; fi
# record new binary sha
python3 - "$(manifest "$name")" "$(sha "$d/bin/engram")" "${src:-branch:$branch}" <<'PY'
import json,sys; mf,s,src=sys.argv[1:4]
d=json.load(open(mf)); d["binary_sha256"]=s; d["source"]="rebuilt:"+src
json.dump(d,open(mf,'w'),indent=2)
PY
start_daemon "$name"
ok "rebuilt + restarted on :$(mget "$name" "['port']")"
}
# ================================================================ run ==========
cmd_run(){
local name="$1"; shift || true
mexists "$name" || die "no such sandbox: $name"
daemon_alive "$name" || start_daemon "$name"
local d port; d="$(sdir "$name")"; port="$(mget "$name" "['port']")"
# direct API form: nsbx run <name> api <path> [json]
if [ "${1:-}" = "api" ]; then
api "$name" "$2" "${3:-}"; echo; return 0
fi
[ "${1:-}" = "--" ] && shift # allow an explicit separator: nsbx run <name> -- <cmd...>
[ $# -gt 0 ] || die "usage: nsbx run <name> <cmd...> | nsbx run <name> api <path> [json]"
local ts log0; ts="$(now)"; log0="$d/logs/run-$ts.log"
local s0 t0 t1 s1
s0="$(sbx_stats "$name")"; t0="$(epoch)"
log "run experiment against sandbox '$name' (:$port)"
info "cmd: $*"
( export SBX_NAME="$name" SBX_PORT="$port" SBX_URL="http://127.0.0.1:$port" \
SBX_KEY="sbx-$name" SBX_DATA="$d/data" SBX_BIN="$d/bin/engram"
"$@" ) 2>&1 | tee "$log0"
local rc=${PIPESTATUS[0]}
t1="$(epoch)"; s1="$(sbx_stats "$name")"
{
echo "--- nsbx run metrics ---"
echo "exit_code: $rc"
printf 'wall_secs: %.3f\n' "$(python3 -c "print($t1-$t0)")"
echo "stats_before: $s0"
echo "stats_after: $s1"
} | tee -a "$log0" >&2
return $rc
}
# ================================================================ validate =====
# The rails as first-class checks. Baseline = the sandbox's own settled state at
# create (reproducible). zero-loss through sustained load AND reboot; reboot-prove;
# RSS bound; retrieval parity; keystone integrity.
cmd_validate(){
local name="$1"; shift || true
mexists "$name" || die "no such sandbox: $name"
daemon_alive "$name" || start_daemon "$name"
local d port key; d="$(sdir "$name")"; port="$(mget "$name" "['port']")"; key="sbx-$name"
local url="http://127.0.0.1:$port"
local bn be; bn="$(mget "$name" "['sbx_baseline']['node_count']")"; be="$(mget "$name" "['sbx_baseline']['edge_count']")"
log "validate '$name' against baseline nodes=$bn edges=$be"
local -a names=() results=() details=()
# 1) sustained load — no data loss under activity
local s cur_n cur_e i
log "check: sustained load (~15s: tick + reads) then zero-loss"
for i in $(seq 1 15); do
curl -s -m5 -X POST -H 'Content-Type: application/json' -d "{\"_auth\":\"$key\"}" "$url/api/tick" >/dev/null 2>&1
curl -s -m5 "$url/api/stats" >/dev/null 2>&1
done
s="$(sbx_stats "$name")"; cur_n="$(stat_field "$s" node_count)"; cur_e="$(stat_field "$s" edge_count)"
names+=("zero-loss-under-load"); if [ "${cur_n:-0}" -ge "${bn:-0}" ] && [ "${cur_e:-0}" -ge "${be:-0}" ]; then
results+=("PASS"); else results+=("FAIL"); fi
details+=("nodes $cur_n>=$bn, edges $cur_e>=$be")
# 2) reboot-prove — counts survive a real restart
log "check: reboot-prove (stop -> start -> compare)"
local pre_n pre_e; pre_n="$cur_n"; pre_e="$cur_e"
stop_daemon "$name"; start_daemon "$name" >/dev/null
s="$(sbx_stats "$name")"; cur_n="$(stat_field "$s" node_count)"; cur_e="$(stat_field "$s" edge_count)"
names+=("reboot-prove"); if [ "${cur_n:-0}" -ge "${bn:-0}" ] && [ "${cur_e:-0}" -ge "${be:-0}" ]; then
results+=("PASS"); else results+=("FAIL"); fi
details+=("post-reboot nodes=$cur_n edges=$cur_e (pre $pre_n/$pre_e)")
# 3) RSS bound
log "check: RSS bound (< ${RSS_BOUND_MB}MB)"
local pid rss_kb rss_mb; pid="$(daemon_pid "$name")"
rss_kb="$(ps -o rss= -p "$pid" 2>/dev/null | tr -d ' ')"; rss_mb=$(( ${rss_kb:-0} / 1024 ))
names+=("rss-bound"); if [ "$rss_mb" -lt "$RSS_BOUND_MB" ] && [ "$rss_mb" -gt 0 ]; then results+=("PASS"); else results+=("FAIL"); fi
details+=("RSS=${rss_mb}MB (bound ${RSS_BOUND_MB}MB)")
# 4) retrieval parity vs the create-time baseline
log "check: retrieval parity vs baseline probe set"
_capture_retrieval "$name" "$d/logs/retrieval-$( now ).json"
local latest; latest="$(ls -t "$d/logs"/retrieval-*.json 2>/dev/null | head -1)"
local parity; parity="$(python3 - "$d/baseline/retrieval.json" "$latest" <<'PY'
import json,sys
def load(p):
try: return json.load(open(p))
except Exception: return {}
b,c=load(sys.argv[1]),load(sys.argv[2])
tot=hit=0
for q,ids in b.items():
cb=set(ids or []); cc=set(c.get(q) or [])
if not cb: continue
tot+=len(cb); hit+=len(cb & cc)
print(f"{hit}/{tot}" if tot else "0/0")
PY
)"
local ph="${parity%/*}" pt="${parity#*/}"
names+=("retrieval-parity"); if [ "${pt:-0}" -gt 0 ] && [ "${ph:-0}" -eq "${pt:-0}" ]; then results+=("PASS"); else results+=("FAIL"); fi
details+=("top-k id overlap $parity vs baseline")
# 5) keystone integrity
log "check: keystone integrity"
local kfail=0 kid kres
for kid in "${KEYSTONES[@]}"; do
kres="$(curl -s -m5 "$url/api/node/$kid" 2>/dev/null)"
printf '%s' "$kres" | grep -q "\"$kid\"" || kfail=1
done
names+=("keystone-integrity"); [ "$kfail" -eq 0 ] && results+=("PASS") || results+=("FAIL")
details+=("kn-efeb4a5b + kn-5b606390 present")
# ---- report + stamp ----
echo >&2
printf '%s VALIDATION — %s%s\n' "$C_BLD" "$name" "$C_0" >&2
local allpass=1 j
for j in "${!names[@]}"; do
local r="${results[$j]}" c="$C_GRN"; [ "$r" = FAIL ] && { c="$C_RED"; allpass=0; }
printf ' %s%-6s%s %-22s %s%s%s\n' "$c" "$r" "$C_0" "${names[$j]}" "$C_DIM" "${details[$j]}" "$C_0" >&2
done
local status; [ "$allpass" -eq 1 ] && status="PASS" || status="FAIL"
python3 - "$d/validate.json" "$status" "$(sha "$d/bin/engram")" "$(now)" "${names[*]}" "${results[*]}" <<'PY'
import json,sys
out,status,binsha,ts,ns,rs=sys.argv[1:7]
checks=[{"name":n,"result":r} for n,r in zip(ns.split(),rs.split())]
json.dump({"status":status,"binary_sha256":binsha,"ts":ts,"checks":checks},open(out,'w'),indent=2)
PY
printf ' %s==> %s%s\n' "$([ "$allpass" -eq 1 ] && echo "$C_GRN" || echo "$C_RED")" "$status" "$C_0" >&2
[ "$allpass" -eq 1 ]
}
# ================================================================ promote ======
# The ONLY prod-touching op. Explicit, gated, per-use Will-approved. Rails ONLY:
# snapshot-first -> additive binary swap -> launchctl bootout -> settle-poll ->
# bootstrap -> verify -> auto-rollback on failure. NEVER pkill, NEVER kickstart -k.
# Default is a DRY-RUN plan; requires --i-approve-prod-cutover to actually cut over.
cmd_promote(){
local name="$1"; shift || true
mexists "$name" || die "no such sandbox: $name"
local approve=0 do_data=0
while [ $# -gt 0 ]; do case "$1" in
--i-approve-prod-cutover) approve=1; shift;;
--data) do_data=1; shift;;
*) die "unknown flag: $1";; esac; done
local d; d="$(sdir "$name")"
# GATE 1: validation must have passed for the CURRENT binary
[ -f "$d/validate.json" ] || die "GATE: no validation on record — run 'nsbx validate $name' first"
local vstatus vsha bsha
vstatus="$(python3 -c "import json;print(json.load(open('$d/validate.json'))['status'])")"
vsha="$(python3 -c "import json;print(json.load(open('$d/validate.json'))['binary_sha256'])")"
bsha="$(sha "$d/bin/engram")"
[ "$vstatus" = PASS ] || die "GATE: last validation status is $vstatus (must be PASS)"
[ "$vsha" = "$bsha" ] || die "GATE: validation is stale — binary changed since validate (re-run validate)"
local live_bin new_bin ts; ts="$(now)"
live_bin="$(_live_real_bin)"
new_bin="$HOME/.neuron/bin/engram.promote-$name-$ts" # additive: new file, old kept
local bkp="$BACKUP_ROOT/promote-$name-$ts"
log "PROMOTE PLAN for '$name' -> live :$LIVE_BIND_PORT"
info "current live ENGRAM_REAL_BIN : $live_bin"
info "sandbox binary (validated) : $d/bin/engram (sha ${bsha:0:12})"
info "will install as : $new_bin (additive; old binary retained)"
info "snapshot-first backup dir : $bkp (egm+wal+plist+rollback.txt)"
info "data promote : $([ $do_data -eq 1 ] && echo 'YES (--data: clone egm/wal -> live)' || echo 'no (binary only)')"
info "rails : launchctl bootout -> settle-poll -> bootstrap"
info "verify : /api/stats + edges>=baseline + keystones + retrieval; auto-rollback armed"
if [ "$approve" -ne 1 ]; then
warn "DRY-RUN — not touching prod. Re-run with --i-approve-prod-cutover to execute (per-use Will-approved)."
return 0
fi
need launchctl
local dom="gui/$(id -u)"
# ---- snapshot-first ----
log "snapshot-first backup -> $bkp"
mkdir -p "$bkp"
cp -p "$LIVE_DATA_DIR/neuron.egm" "$bkp/neuron.egm.bak"
cp -p "$LIVE_DATA_DIR/neuron.wal" "$bkp/neuron.wal.bak" 2>/dev/null || true
cp -p "$LIVE_PLIST" "$bkp/plist.bak"
printf 'rollback REAL_BIN=%s\nNEWBIN=%s\ndata_promote=%s\n' "$live_bin" "$new_bin" "$do_data" > "$bkp/rollback.txt"
ok "backup complete"
# ---- additive binary install + plist supersede ----
cp -p "$d/bin/engram" "$new_bin"
python3 - "$LIVE_PLIST" "$new_bin" <<'PY'
import sys,plistlib
p,new=sys.argv[1],sys.argv[2]
d=plistlib.load(open(p,'rb')); d.setdefault("EnvironmentVariables",{})["ENGRAM_REAL_BIN"]=new
plistlib.dump(d,open(p,'wb'))
PY
ok "installed $new_bin + updated plist ENGRAM_REAL_BIN"
# ---- optional data promote (after backup) ----
if [ $do_data -eq 1 ]; then
log "data promote: clone store -> live (backed up above)"
cp -p "$d/data/neuron.egm" "$LIVE_DATA_DIR/neuron.egm"
cp -p "$d/data/neuron.wal" "$LIVE_DATA_DIR/neuron.wal" 2>/dev/null || true
fi
# ---- rails cutover: bootout -> settle-poll -> bootstrap ----
log "rails: launchctl bootout $dom/$LIVE_LABEL"
launchctl bootout "$dom/$LIVE_LABEL" 2>/dev/null || true
local i
for i in $(seq 1 60); do
launchctl print "$dom/$LIVE_LABEL" >/dev/null 2>&1 || { ok "settle: job gone after ${i}x0.5s"; break; }
printf ' settle: job still present (%d)\n' "$i" >&2; sleep 0.5
done
log "rails: launchctl bootstrap $dom <plist>"
launchctl bootstrap "$dom" "$LIVE_PLIST" || warn "bootstrap returned nonzero"
# ---- verify ----
log "verify prod health"
local s="" ; for i in $(seq 1 60); do s="$(live_stats)"; [ -n "$s" ] && break; sleep 1; done
local ok_verify=1 le; le="$(stat_field "$s" edge_count)"
local base_e; base_e="$(mget "$name" "['live_baseline']['edge_count']")"
[ -n "$s" ] || ok_verify=0
[ "${le:-0}" -ge "${base_e:-0}" ] || ok_verify=0
local kid; for kid in "${KEYSTONES[@]}"; do curl -s -m5 "$LIVE_URL/api/node/$kid" 2>/dev/null | grep -q "\"$kid\"" || ok_verify=0; done
if [ "$ok_verify" -eq 1 ]; then
ok "PROMOTED. live stats: $s (rollback: $bkp)"; return 0
fi
# ---- auto-rollback ----
warn "verify FAILED — auto-rollback"
cp -p "$bkp/plist.bak" "$LIVE_PLIST"
[ $do_data -eq 1 ] && { cp -p "$bkp/neuron.egm.bak" "$LIVE_DATA_DIR/neuron.egm"; cp -p "$bkp/neuron.wal.bak" "$LIVE_DATA_DIR/neuron.wal" 2>/dev/null || true; }
launchctl bootout "$dom/$LIVE_LABEL" 2>/dev/null || true
for i in $(seq 1 60); do launchctl print "$dom/$LIVE_LABEL" >/dev/null 2>&1 || break; sleep 0.5; done
launchctl bootstrap "$dom" "$LIVE_PLIST" || true
die "ROLLED BACK to $live_bin. See $bkp"
}
# ================================================================ destroy ======
cmd_destroy(){
local name="$1"; shift || true
mexists "$name" || die "no such sandbox: $name"
local d; d="$(sdir "$name")"
stop_daemon "$name"
if [ -d "$d/build/worktree" ]; then
log "removing git worktree"
git -C "$EL_REPO" worktree remove --force "$d/build/worktree" 2>/dev/null || true
git -C "$EL_REPO" worktree prune 2>/dev/null || true
fi
log "removing $d"
rm -rf "$d"
ok "destroyed '$name' (live untouched)"
}
# ================================================================ list/status ==
cmd_list(){
[ -d "$SBX_ROOT" ] || { echo "no sandboxes"; return 0; }
printf '%-16s %-6s %-8s %-9s %s\n' NAME PORT STATE PID SOURCE
local m
for m in "$SBX_ROOT"/*/manifest.json; do
[ -f "$m" ] || continue
local n p src pid state
n="$(python3 -c "import json;print(json.load(open('$m'))['name'])")"
p="$(python3 -c "import json;print(json.load(open('$m'))['port'])")"
src="$(python3 -c "import json;print(json.load(open('$m'))['source'])")"
pid="$(daemon_pid "$n")"; state="stopped"; daemon_alive "$n" && state="running"
printf '%-16s %-6s %-8s %-9s %s\n' "$n" "$p" "$state" "${pid:-}" "$src"
done
}
cmd_status(){
local name="$1"; mexists "$name" || die "no such sandbox: $name"
python3 -m json.tool "$(manifest "$name")"
daemon_alive "$name" && echo "state: running (pid $(daemon_pid "$name")) stats: $(sbx_stats "$name")" || echo "state: stopped"
[ -f "$(sdir "$name")/validate.json" ] && { echo "--- last validation ---"; python3 -m json.tool "$(sdir "$name")/validate.json"; }
}
# ================================================================ dev ==========
# ONE-COMMAND isolated dev environment. Everything a newcomer (Tim, any agent)
# needs to go from clone -> coding on an isolated running mind, in a single shot:
# 1) a git BRANCH (dev/<name>, or --prefix)
# 2) a git WORKTREE for it, at a visible path they can open + edit
# 3) an ISOLATED engram bound to a NON-default port, on a clone of the live store
# (stock prod binary by default — instant + safe; --build to compile the
# worktree's own runtime instead). Prod :$LIVE_BIND_PORT/:$SOUL_PORT is untouchable.
#
# This is additive sugar over the proven primitives (git worktree + cmd_create).
# It never binds a forbidden port and never touches ~/.neuron/engram (the live store)
# except the same READ-only snapshot cmd_create already performs.
#
# nsbx dev <name> [--repo R] [--base REF] [--worktree DIR] [--port N]
# [--prefix P] [--build] [--no-engram]
cmd_dev(){
local name="" repo="$EL_REPO" base="" wt="" port="" prefix="dev/" build=0 no_engram=0
[ $# -gt 0 ] && [ "${1#-}" = "$1" ] && { name="$1"; shift; } || die "usage: nsbx dev <name> [flags]"
while [ $# -gt 0 ]; do case "$1" in
--repo) repo="$2"; shift 2;;
--base) base="$2"; shift 2;;
--worktree|--wt) wt="$2"; shift 2;;
--port) port="$2"; shift 2;;
--prefix) prefix="$2"; shift 2;;
--build) build=1; shift;;
--no-engram) no_engram=1; shift;;
*) die "unknown flag: $1";;
esac; done
need git
git -C "$repo" rev-parse --git-dir >/dev/null 2>&1 || die "not a git repo: $repo"
local branch="${prefix}${name}"
local sbx="dev-${name}"
# default worktree path: a PERSISTENT, git-managed dir — NEVER /tmp (which is
# ablated on compaction). Default root = <repo-grandparent>/el-worktrees, i.e.
# ~/Development/neuron-technologies/el-worktrees/<name>. Override with NSBX_DEV_WT_ROOT.
local wt_root="${NSBX_DEV_WT_ROOT:-$(cd "$(dirname "$(dirname "$repo")")" && pwd -P)/el-worktrees}"
[ -n "$wt" ] || wt="${wt_root}/${name}"
case "$wt" in /tmp/*|/private/tmp/*|/var/tmp/*)
die "refusing worktree under a temp dir ($wt) — temp dirs are ablated on compaction; set NSBX_DEV_WT_ROOT to a persistent path";;
esac
# default base: whatever the repo's working checkout is on now
[ -n "$base" ] || base="$(git -C "$repo" rev-parse --abbrev-ref HEAD 2>/dev/null)"
# pre-flight (fail before creating anything)
[ "$no_engram" -eq 1 ] || ! mexists "$sbx" || die "engram sandbox '$sbx' already exists (nsbx dev-down $name first)"
[ -e "$wt" ] && die "worktree path already exists: $wt"
if [ -n "$port" ]; then
{ [ "$port" = "$LIVE_BIND_PORT" ] || [ "$port" = "$SOUL_PORT" ]; } && die "refusing forbidden port $port (live)"
fi
log "dev env '$name' (branch=$branch worktree=$wt base=$base)"
# ---- 1+2) branch + worktree in one shot ----
local gerr
if git -C "$repo" show-ref --verify --quiet "refs/heads/$branch"; then
info "branch $branch exists — checking it out into a new worktree"
gerr="$(git -C "$repo" worktree add "$wt" "$branch" 2>&1)" \
|| die "git worktree add failed for existing branch $branch:"$'\n'" $gerr"
else
gerr="$(git -C "$repo" worktree add -b "$branch" "$wt" "$base" 2>&1)" \
|| die "git worktree add -b $branch (base $base) failed:"$'\n'" $gerr"$'\n'" (a bare 'dev' branch blocks 'dev/*' names — try --prefix, e.g. nsbx dev $name --prefix wt/)"
fi
ok "worktree ready: $wt (branch $branch)"
# ---- 3) isolated engram ----
local eport="(none)"
if [ "$no_engram" -eq 1 ]; then
warn "--no-engram: skipped standing up an engram"
else
if [ "$build" -eq 1 ]; then
log "isolated engram: building the worktree's own runtime"
cmd_create "$sbx" ${port:+--port "$port"} --source "$wt" || die "engram create (--build) failed"
else
log "isolated engram: stock prod binary on a clone of the live store"
cmd_create "$sbx" ${port:+--port "$port"} || die "engram create failed"
fi
eport="$(mget "$sbx" "['port']")"
# record the dev linkage next to the sandbox so dev-down can clean up
python3 - "$(sdir "$sbx")/dev.json" "$name" "$branch" "$wt" "$repo" "$eport" <<'PY'
import json,sys
p,name,branch,wt,repo,port=sys.argv[1:7]
json.dump({"name":name,"branch":branch,"worktree":wt,"repo":repo,"port":int(port)},
open(p,'w'),indent=2)
PY
# ---- pin the WHOLE worktree to the CLONE ----
# Every var any El tooling in this worktree might read for an engram target now
# points at the isolated clone. Sourcing .nsbx-env makes hitting live :$LIVE_BIND_PORT
# or ~/.neuron/engram by accident structurally impossible from this shell.
local edata ekey eurl ebin
edata="$(sdir "$sbx")/data"; ekey="sbx-$sbx"; eurl="http://127.0.0.1:$eport"; ebin="$(sdir "$sbx")/bin/engram"
cat > "$wt/.nsbx-env" <<ENV
# nsbx dev env for '$name' — SOURCE this to pin THIS shell to the isolated clone.
# The live mind (:$LIVE_BIND_PORT engram / :$SOUL_PORT soul / $LIVE_DATA_DIR) is deliberately
# NOT referenced here. Regenerated by 'nsbx dev'. -> source .nsbx-env
export NSBX_NAME="$sbx"
export ENGRAM_URL="$eurl"
export ENGRAM_BIND=":$eport"
export ENGRAM_PORT="$eport"
export ENGRAM_HOST="127.0.0.1"
export ENGRAM_DATA_DIR="$edata"
export ENGRAM_API_KEY="$ekey"
export NEURON_ENGRAM_URL="$eurl"
export NEURON_ENGRAM_KEY="$ekey"
# nsbx run compatibility (same names 'nsbx run' exports)
export SBX_NAME="$sbx" SBX_PORT="$eport" SBX_URL="$eurl" SBX_KEY="$ekey" SBX_DATA="$edata" SBX_BIN="$ebin"
ENV
# direnv users get it automatically on cd; everyone else runs 'source .nsbx-env'
[ -e "$wt/.envrc" ] || printf 'source_env .nsbx-env 2>/dev/null || source .nsbx-env\n' > "$wt/.envrc"
ok "wrote $wt/.nsbx-env (pins this worktree to the clone)"
fi
# ---- summary ----
echo >&2
printf '%s DEV ENV READY — %s%s\n' "$C_BLD" "$name" "$C_0" >&2
printf ' %-10s %s\n' "worktree" "$wt" >&2
printf ' %-10s %s\n' "branch" "$branch" >&2
if [ "$no_engram" -ne 1 ]; then
printf ' %-10s %s\n' "engram" "http://127.0.0.1:$eport (isolated clone; prod :$LIVE_BIND_PORT untouchable)" >&2
printf ' %-10s %s\n' "sandbox" "$sbx" >&2
echo >&2
printf '%s env exported into %s/.nsbx-env (source it -> pinned to the clone):%s\n' "$C_DIM" "$wt" "$C_0" >&2
grep '^export' "$wt/.nsbx-env" | sed 's/^/ /' >&2
fi
echo >&2
info "code in: cd $wt && source .nsbx-env # now every engram var points at the clone"
[ "$no_engram" -ne 1 ] && info "poke it: nsbx run $sbx api /api/stats (or: make run NAME=$name)"
[ "$no_engram" -ne 1 ] && info "edit->test: make build NAME=$name && make run NAME=$name # El change -> clone, seconds"
info "tear down: nsbx dev-down $name (destroys engram + removes worktree; branch kept)"
}
# nsbx dev-down <name> [--delete-branch] [--repo R]
# Teardown counterpart: destroy the isolated engram, remove the git worktree,
# and (optionally) delete the branch. Live prod is never touched.
cmd_dev_down(){
local name="" repo="$EL_REPO" del_branch=0
[ $# -gt 0 ] && [ "${1#-}" = "$1" ] && { name="$1"; shift; } || die "usage: nsbx dev-down <name> [--delete-branch]"
while [ $# -gt 0 ]; do case "$1" in
--repo) repo="$2"; shift 2;;
--delete-branch) del_branch=1; shift;;
*) die "unknown flag: $1";;
esac; done
local sbx="dev-${name}" wt="" branch="dev/${name}"
# recover worktree/branch/repo from the dev linkage if present
if mexists "$sbx" && [ -f "$(sdir "$sbx")/dev.json" ]; then
local dj; dj="$(sdir "$sbx")/dev.json"
wt="$(python3 -c "import json;print(json.load(open('$dj'))['worktree'])" 2>/dev/null)"
branch="$(python3 -c "import json;print(json.load(open('$dj'))['branch'])" 2>/dev/null)"
repo="$(python3 -c "import json;print(json.load(open('$dj'))['repo'])" 2>/dev/null)"
fi
# 1) engram
if mexists "$sbx"; then cmd_destroy "$sbx"; else info "no engram sandbox '$sbx'"; fi
# 2) worktree
if [ -n "$wt" ] && [ -d "$wt" ]; then
log "removing git worktree $wt"
git -C "$repo" worktree remove --force "$wt" 2>/dev/null || rm -rf "$wt"
git -C "$repo" worktree prune 2>/dev/null || true
ok "worktree removed"
else info "no worktree to remove"; fi
# 3) branch (opt-in)
if [ "$del_branch" -eq 1 ]; then
git -C "$repo" branch -D "$branch" 2>/dev/null && ok "deleted branch $branch" || warn "could not delete branch $branch"
else info "branch $branch kept (use --delete-branch to remove)"; fi
ok "dev-down '$name' complete (live untouched)"
}
usage(){ cat >&2 <<EOF
${C_BLD}nsbx${C_0} — Neuron Sandbox: experiments + code changes against the REAL engram
runtime on an isolated snapshot of the live mind, with a gated promote-to-prod path.
nsbx dev <name> [--base REF] [--worktree DIR] ONE command onboarding: new branch (dev/<name>) + git
[--port N] [--prefix P] [--build] worktree (persistent, never /tmp) + isolated engram on a
[--no-engram] [--repo R] non-default port. clone -> coding on the mind in one shot.
nsbx dev-down <name> [--delete-branch] [--repo R] teardown: destroy the engram + remove the worktree
(branch kept unless --delete-branch). live untouched.
nsbx up [name] [flags…] one command: your private, isolated copy of the mind
(creates on first run, starts thereafter; prod untouchable)
nsbx create [name] [--port N] [--source DIR | --branch REF [--repo R] | --binary PATH]
clone live store+WAL+config, place/build the runtime, boot on an
isolated port (never :$LIVE_BIND_PORT/:$SOUL_PORT). Default runtime = stock prod binary.
nsbx build <name> --source DIR | --branch REF rebuild the runtime from a code change + hot-restart
nsbx run <name> <cmd...> | api <path> [json] run an experiment; capture output + metrics
nsbx validate <name> rails as checks: zero-loss(load+reboot), reboot-prove,
RSS bound, retrieval parity, keystone integrity
nsbx promote <name> [--data] [--i-approve-prod-cutover] GATED rails cutover to prod (DRY-RUN without approval)
nsbx destroy <name> stop daemon, free port, remove clone (live untouched)
nsbx list | nsbx status <name>
Env in 'run' cmds: \$SBX_URL \$SBX_PORT \$SBX_KEY \$SBX_DATA \$SBX_BIN \$SBX_NAME
EOF
}
main(){
local cmd="${1:-}"; shift || true
case "$cmd" in
dev) cmd_dev "$@";;
dev-down) cmd_dev_down "$@";;
up) cmd_up "$@";;
create) cmd_create "$@";;
build) cmd_build "$@";;
run) cmd_run "$@";;
validate) cmd_validate "$@";;
promote) cmd_promote "$@";;
destroy) cmd_destroy "$@";;
list|ls) cmd_list "$@";;
status) cmd_status "$@";;
""|-h|--help|help) usage;;
*) die "unknown command: $cmd (try: nsbx help)";;
esac
}
main "$@"
+1 -1
View File
@@ -18,7 +18,7 @@ set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
EL_LANG_ROOT="${SCRIPT_DIR}/../../../lang"
EL_UI_ROOT="${SCRIPT_DIR}/../.."
EL_RUNTIME="${EL_LANG_ROOT}/runtime"
EL_RUNTIME="${EL_LANG_ROOT}/el-compiler/runtime"
EL_NATIVE_VESSEL="${EL_UI_ROOT}/vessels/el-native/src/main.el"
EL_APP_ENTRY="${EL_UI_ROOT}/examples/native-hello/src/main.el"
EL_MANIFEST="${EL_UI_ROOT}/examples/native-hello/manifest.el"
+1 -1
View File
@@ -17,7 +17,7 @@ set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
EL_LANG_ROOT="${SCRIPT_DIR}/../../../lang"
EL_UI_ROOT="${SCRIPT_DIR}/../.."
EL_RUNTIME="${EL_LANG_ROOT}/runtime"
EL_RUNTIME="${EL_LANG_ROOT}/el-compiler/runtime"
EL_NATIVE_VESSEL="${EL_UI_ROOT}/vessels/el-native/src/main.el"
EL_APP_ENTRY="${EL_UI_ROOT}/examples/native-hello/src/main.el"
EL_MANIFEST="${EL_UI_ROOT}/examples/native-hello/manifest.el"
+1 -1
View File
@@ -24,7 +24,7 @@ set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
EL_LANG_ROOT="${SCRIPT_DIR}/../../../lang"
EL_UI_ROOT="${SCRIPT_DIR}/../.."
EL_RUNTIME="${EL_LANG_ROOT}/runtime"
EL_RUNTIME="${EL_LANG_ROOT}/el-compiler/runtime"
EL_NATIVE_VESSEL="${EL_UI_ROOT}/vessels/el-native/src/main.el"
BUILD_DIR="${SCRIPT_DIR}/build-gtk4"
+1 -1
View File
@@ -23,7 +23,7 @@ set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
EL_LANG_ROOT="${SCRIPT_DIR}/../../../lang"
EL_UI_ROOT="${SCRIPT_DIR}/../.."
EL_RUNTIME="${EL_LANG_ROOT}/runtime"
EL_RUNTIME="${EL_LANG_ROOT}/el-compiler/runtime"
EL_NATIVE_VESSEL="${EL_UI_ROOT}/vessels/el-native/src/main.el"
BUILD_DIR="${SCRIPT_DIR}/build"
+1 -1
View File
@@ -18,7 +18,7 @@ set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
EL_LANG_ROOT="${SCRIPT_DIR}/../../../lang"
EL_UI_ROOT="${SCRIPT_DIR}/../.."
EL_RUNTIME="${EL_LANG_ROOT}/runtime"
EL_RUNTIME="${EL_LANG_ROOT}/el-compiler/runtime"
EL_NATIVE_VESSEL="${EL_UI_ROOT}/vessels/el-native/src/main.el"
BUILD_DIR="${SCRIPT_DIR}/build-win32"
+2 -2
View File
@@ -40,7 +40,7 @@ set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
EL_LANG_ROOT="${SCRIPT_DIR}/../../../lang"
EL_UI_ROOT="${SCRIPT_DIR}/../.."
EL_RUNTIME="${EL_LANG_ROOT}/runtime"
EL_RUNTIME="${EL_LANG_ROOT}/el-compiler/runtime"
EL_NATIVE_VESSEL="${EL_UI_ROOT}/vessels/el-native/src/main.el"
BUILD_DIR="${SCRIPT_DIR}/build"
@@ -193,6 +193,6 @@ run() {
case "${1:-run}" in
clean) clean ;;
compile) compile ;;
platforms) "${EL_LANG_ROOT}/runtime/detect-platforms" ;;
platforms) "${EL_LANG_ROOT}/el-compiler/runtime/detect-platforms" ;;
run|*) run ;;
esac
@@ -32,7 +32,7 @@ set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
EL_LANG_ROOT="${SCRIPT_DIR}/../../../lang"
EL_UI_ROOT="${SCRIPT_DIR}/../.."
EL_RUNTIME="${EL_LANG_ROOT}/runtime"
EL_RUNTIME="${EL_LANG_ROOT}/el-compiler/runtime"
EL_NATIVE_VESSEL="${EL_UI_ROOT}/vessels/el-native/src/main.el"
BUILD_DIR="${SCRIPT_DIR}/build"
DOCKER_CTX="${SCRIPT_DIR}/build-docker"