Tim Lingo
d5319d2849
tests/ has held 14 test programs for months with no way to run them. CI does
not run them. The convention printed in their own headers
(`elc soul.el && ./soul --test tests/x.el`) refers to a --test flag the El
runtime does not implement. So the tests were documentation, not gates — which
is how a P0 safety regression shipped with a test directory sitting right
there.
scripts/run-el-test.sh compiles and runs one test program. It reuses the
gen-soul-amalgam.sh discovery: elc emits only an extern prototype for a module
that has a .elh beside it, and inlines the bodies when it does not, so a test
importing ../chat.el must be compiled in a scratch tree with the headers
removed. Scratch copy on purpose — the worktree is shared. It runs the binary
under a throwaway HOME so a test can never reach the live engram.
Exit status is the gate: the El tests print failures and still exit 0, so the
runner greps for FAIL lines and for a zero assertion count as well.
tests/test_history_amplification.el pins the invariant #129 violated: the
window the safety screen READS must be the window conv_history_record WRITES.
Not "must be called conv_history" — must AGREE.
THIS COMMIT IS RED BY DESIGN. On this tree the test fails one assertion:
3. REGRESSION #129 — agentic screen reads the session's own window
FAIL: distress history escalates the agentic screen to hard_bell
got: soft_bell
expected: hard_bell
history amplification tests: 8 passed, 1 failed (runner exit 1)
The next commit turns it green by changing one line. Two legs, one variable —
that is the whole point of committing the test first.
Two flaws in the older harness that this one does not copy: the idiom
`let pass_count = pass_count + 1` inside an assert function declares a local
that dies with the call, so every existing suite prints "0 passed, 0 failed"
regardless of outcome; and a test program without a `cgi` block compiles as a
'utility', which may not reference the self-formation primitives chat.el's
agentic loop calls — it fails to build on a capability violation it never
triggers at runtime.
Refs #129
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit b842e82f77)
109 lines
4.6 KiB
Bash
Executable File
109 lines
4.6 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# run-el-test.sh — compile and run one El test program from tests/.
|
|
#
|
|
# WHY THIS EXISTS (2026-08-07, issue #129):
|
|
# tests/ has held 14 test programs for months with no way to run them. CI does
|
|
# not run them. The convention printed in their own headers
|
|
# (`elc soul.el && ./soul --test tests/x.el`) refers to a --test flag the El
|
|
# runtime does not implement. So the tests were documentation, not gates —
|
|
# which is how a P0 safety regression shipped with a test directory present.
|
|
#
|
|
# THE RECIPE, AND WHY IT IS THIS SHAPE:
|
|
# Same discovery as gen-soul-amalgam.sh — `elc --target=c` emits only an extern
|
|
# prototype for any module that has a .elh header next to it, and inlines the
|
|
# module's bodies when it does not. A test that imports ../chat.el therefore
|
|
# compiles to a 18 KB unit full of unresolved externs unless the headers are
|
|
# out of the way. So: copy the sources into a scratch tree, delete every .elh
|
|
# on the import chain, and compile the test there.
|
|
#
|
|
# Scratch copy on purpose: the worktree is shared with other terminals and
|
|
# deleting headers in place would be a shared-tree mutation with no owner.
|
|
#
|
|
# EXIT STATUS IS THE GATE: non-zero if the binary fails to build, crashes, or if
|
|
# its output contains a FAIL line or reports a non-zero failed count. Do not
|
|
# "improve" this into something that only checks the exit code of the test
|
|
# binary — these El tests print failures and still exit 0.
|
|
#
|
|
# usage: scripts/run-el-test.sh tests/test_history_amplification.el
|
|
set -euo pipefail
|
|
|
|
TEST_REL="${1:?usage: run-el-test.sh tests/<test>.el}"
|
|
SRC="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
|
TEST_NAME="$(basename "$TEST_REL" .el)"
|
|
|
|
ELC="${ELC:-$HOME/neuron-dev-stack/src/el/lang/dist/platform/elc}"
|
|
[ -x "$ELC" ] || ELC="$HOME/el-sdk/elc"
|
|
[ -x "$ELC" ] || { echo "[run-el-test] FAIL: no elc found (set ELC=)"; exit 1; }
|
|
|
|
RTC="${RTC:-$SRC/vendor/el-runtime/v1.0.0-20260501/el_runtime.c}"
|
|
[ -f "$RTC" ] || RTC="$HOME/el-sdk/el_runtime.c"
|
|
[ -f "$RTC" ] || { echo "[run-el-test] FAIL: no el_runtime.c found (set RTC=)"; exit 1; }
|
|
RTDIR="$(dirname "$RTC")"
|
|
|
|
EL_REPO="${EL_REPO:-$HOME/Development/neuron-technologies/el}"
|
|
SSL="${SSL_PREFIX:-/opt/homebrew/opt/openssl@3}"
|
|
|
|
GEN="$(mktemp -d "${TMPDIR:-/tmp}/el-test.XXXXXX")"
|
|
trap 'rm -rf "$GEN"' EXIT
|
|
|
|
mkdir -p "$GEN/neuron/tests" "$GEN/foundation/el/elp/src"
|
|
cp "$SRC"/*.el "$GEN/neuron/"
|
|
cp "$SRC"/tests/*.el "$GEN/neuron/tests/" 2>/dev/null || true
|
|
[ -d "$EL_REPO/elp/src" ] && cp "$EL_REPO"/elp/src/*.el "$GEN/foundation/el/elp/src/" 2>/dev/null || true
|
|
# The whole recipe depends on there being no headers to short-circuit inlining.
|
|
find "$GEN" -name '*.elh' -delete
|
|
|
|
echo "[run-el-test] compiling $TEST_REL"
|
|
( cd "$GEN/neuron" && "$ELC" --target=c "tests/${TEST_NAME}.el" ) > "$GEN/${TEST_NAME}.c"
|
|
|
|
BODIES=$(grep -c '^el_val_t .*) {$' "$GEN/${TEST_NAME}.c" || true)
|
|
echo "[run-el-test] $(wc -c < "$GEN/${TEST_NAME}.c" | tr -d ' ') bytes, ${BODIES} inlined function bodies"
|
|
# A test that imports ../chat.el pulls in the bulk of the engine. A tiny body
|
|
# count means an import was read from a header instead of inlined, and the test
|
|
# would be exercising extern stubs rather than the real code.
|
|
if [ "$BODIES" -lt 100 ]; then
|
|
echo "[run-el-test] FAIL: only $BODIES inlined bodies — an import was not inlined"
|
|
exit 1
|
|
fi
|
|
|
|
cc -O2 -DHAVE_CURL \
|
|
-I"$RTDIR" -I"$SSL/include" -L"$SSL/lib" \
|
|
"$GEN/${TEST_NAME}.c" "$RTC" \
|
|
-lssl -lcrypto -lcurl -lpthread -lm \
|
|
-o "$GEN/${TEST_NAME}" 2> "$GEN/cc.log" || {
|
|
echo "[run-el-test] FAIL: compile error"; tail -30 "$GEN/cc.log"; exit 1; }
|
|
|
|
# arm64 pointer-truncation guard (cc-brain.sh's rule): an implicit declaration of
|
|
# a runtime symbol truncates its returned pointer to 32 bits.
|
|
if grep -E 'implicit.*(engram_|el_)' "$GEN/cc.log"; then
|
|
echo "[run-el-test] FAIL: implicit declarations of runtime symbols"; exit 1; fi
|
|
|
|
# Throwaway HOME so a test can never read or write the live engram at ~/.neuron.
|
|
TEST_HOME="$GEN/home"
|
|
mkdir -p "$TEST_HOME"
|
|
|
|
echo "[run-el-test] running $TEST_NAME"
|
|
set +e
|
|
HOME="$TEST_HOME" NEURON_HOME="$TEST_HOME/.neuron" "$GEN/${TEST_NAME}" 2>&1 | tee "$GEN/out.txt"
|
|
RC=${PIPESTATUS[0]}
|
|
set -e
|
|
|
|
if [ "$RC" -ne 0 ]; then
|
|
echo "[run-el-test] FAIL: $TEST_NAME exited $RC (crash or abort)"
|
|
exit 1
|
|
fi
|
|
if grep -q " FAIL:" "$GEN/out.txt"; then
|
|
echo "[run-el-test] FAIL: $TEST_NAME reported failing assertions"
|
|
exit 1
|
|
fi
|
|
if grep -qE '[1-9][0-9]* failed' "$GEN/out.txt"; then
|
|
echo "[run-el-test] FAIL: $TEST_NAME reported a non-zero failed count"
|
|
exit 1
|
|
fi
|
|
if ! grep -q "PASS:" "$GEN/out.txt"; then
|
|
echo "[run-el-test] FAIL: $TEST_NAME produced no assertions at all"
|
|
exit 1
|
|
fi
|
|
|
|
echo "[run-el-test] PASS: $TEST_NAME"
|