engram: WAL persistence + integrity hardening + single canonical runtime

Establish lang/runtime/ as the ONE canonical el runtime (from the active runtime that carries hebb/emb persistence + the new WAL); repoint the el CI publish, engram build, elb default, and in-repo build scripts to it; delete the el-compiler/runtime + lang/releases/ forks; add scripts/check-single-runtime.sh drift guard. Fixes a live prod bug: the el CI published el-runtime-c/-h from the LAGGING el-compiler fork (0 hebb refs), so the shipped soul never persisted Hebbian edge weights — learned co-activation was wiped on every restart. Publishing from canonical ships the stranded 'learning that cannot outlive the process' fix. WAL storage engine + integrity fixes (DELETE->tombstone + store-layer protection, safe data-dir default) ride in behind ENGRAM_WAL (default off = byte-identical to today). Verified: engram elb per-module build clean, WAL gate 66/66, native smoke ok, drift-guard green.
2026-08-11 21:31:10 -05:00
parent 791b0880b7
commit 0a72fced28
57 changed files with 1481 additions and 24485 deletions
@@ -0,0 +1,94 @@
+#!/usr/bin/env bash
+# check-single-runtime.sh — CODE-VS-ARTIFACT drift guard for the el runtime.
+#
+# Enforces org policy docs/CODE-VS-ARTIFACT.md rule #1 (single source of truth):
+# there is exactly ONE authored el_runtime.c, and it lives at lang/runtime/.
+# Any other el_runtime.c in the tree is a fork (a hand-synced copy). A lagging
+# fork is exactly what shipped to prod and dropped learned `hebb` edges on
+# restart — this guard exists to make that class of bug impossible to reintroduce.
+#
+# Exemptions:
+#   * Build output — generated amalgamations under any dist/ or build/ dir are
+#     artifacts, not sources.
+#   * A small, explicit ALLOWLIST of pre-existing example-app vendored/staging
+#     copies (see below). These are KNOWN DEFERRED DEBT, tracked separately from
+#     the SDK/CI-published runtime. They do NOT ship to prod. The guard warns on
+#     them (visible, greppable) but does not fail — while HARD-FAILING on any new
+#     or non-allowlisted fork, including any return of lang/el-compiler/runtime/
+#     or a lang/releases/ vendored copy.
+#
+# Wire-in: run from the repo root in CI (see note at bottom). Exits non-zero on drift.
+set -euo pipefail
+
+ROOT="$(cd "$(dirname "$0")/.." && pwd)"
+cd "$ROOT"
+
+CANONICAL="lang/runtime/el_runtime.c"
+
+# KNOWN DEFERRED example-app forks — remove these as a follow-up, then delete
+# this allowlist. iOS + Docker copies are regenerated by their build scripts
+# (cp from lang/runtime) and can be git-rm'd now; the Android jni/ copy is a
+# committed source its CMake build depends on and needs a build-script change
+# (cp from lang/runtime) before removal. Tracked in docs/CODE-VS-ARTIFACT.md.
+ALLOWLIST=(
+  "ui/examples/native-hello-android/app/src/main/jni/el_runtime.c"
+  "ui/examples/native-hello-ios/NativeHello/el_runtime.c"
+  "ui/examples/native-hello/build-docker/runtime/el_runtime.c"
+)
+
+is_allowlisted() {
+  local p="$1"
+  for a in "${ALLOWLIST[@]}"; do [ "$p" = "$a" ] && return 0; done
+  return 1
+}
+
+if [ ! -f "$CANONICAL" ]; then
+  echo "FATAL: canonical runtime source missing: $CANONICAL" >&2
+  exit 1
+fi
+
+# All el_runtime.c files tracked by git, excluding build output (dist/ , build/)
+# and the canonical source itself.
+mapfile -t CANDIDATES < <(
+  git ls-files '*el_runtime.c' \
+    | grep -Ev '(^|/)(dist|build)/' \
+    | grep -vx "$CANONICAL" || true
+)
+
+FORKS=()
+DEFERRED=()
+for f in "${CANDIDATES[@]}"; do
+  if is_allowlisted "$f"; then DEFERRED+=("$f"); else FORKS+=("$f"); fi
+done
+
+if [ "${#DEFERRED[@]}" -gt 0 ]; then
+  echo "WARN: allowlisted (deferred) el_runtime.c forks still present — clean these up:" >&2
+  for f in "${DEFERRED[@]}"; do echo "        - $f" >&2; done
+fi
+
+if [ "${#FORKS[@]}" -gt 0 ]; then
+  echo "FATAL: el_runtime.c fork(s) detected outside the canonical location." >&2
+  echo "       Canonical (the ONLY allowed source): $CANONICAL" >&2
+  echo "       Offending copies:" >&2
+  for f in "${FORKS[@]}"; do echo "         - $f" >&2; done
+  echo "" >&2
+  echo "Consumers must build against $CANONICAL (pin by git ref where" >&2
+  echo "reproducibility matters) — never a hand-maintained copy." >&2
+  echo "See docs/CODE-VS-ARTIFACT.md." >&2
+  exit 1
+fi
+
+echo "OK: single canonical runtime source — $CANONICAL (no un-allowlisted forks)."
+
+# ---------------------------------------------------------------------------
+# CI wire-in:
+#   foundation/el .gitea/workflows/ci-dev.yaml, ci-stage.yaml, sdk-release.yaml
+#   Add an early step (before the build/publish steps). It must run from the
+#   REPO ROOT, so override the job's `defaults.run.working-directory: lang`:
+#
+#     - name: Guard - single canonical runtime source
+#       working-directory: ${{ github.workspace }}
+#       run: bash scripts/check-single-runtime.sh
+#
+#   Also add to .githooks/pre-commit so drift is caught before it is committed.
+# ---------------------------------------------------------------------------