From 027a573d89e2f0a9010c5fed7b8e0073d93ac686 Mon Sep 17 00:00:00 2001 From: Tim Lingo <1timlingo@gmail.com> Date: Fri, 7 Aug 2026 10:30:27 -0500 Subject: [PATCH] feat(gate): make a state_get with no producer a build error, not a silence MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Retires the defect class behind #129. The engine's state store returns "" for a key nothing writes — no error, no warning, no log. That is how the agentic path's crisis-escalation input scored 0 on every real conversation for two days after ff421d3 moved conversation history behind conv_hist_key(session_id) and left one consumer reading the old "conv_history" bucket by hand. scripts/verify-state-keys.sh is the gate; scripts/state-key-audit.py is the El reader behind it. Two checks: DEAD-READ a state_get whose key resolves to something no state_set in the tree produces. HAND-ROLLED a literal that belongs to a namespace a helper owns, accessed without the helper. This is #129's actual shape, and DEAD-READ alone does NOT catch it: the dead handle_chat() still writes "conv_history" through conv_hist_key(""). Stating that plainly because a gate that only appears to work is worse than none. WHY IT DOES NOT CRY WOLF. Keys are usually computed, so a literal-matching script would flood and be switched off in a day. The resolver handles concatenation (matched on the static prefix), helper functions (resolved to their possible returns, with guard conditions folded so conv_hist_key("") does not falsely claim to produce the session_hist_ namespace), keys built into a local, and keys arriving as a parameter (resolved through the call sites). 278 of 278 sites on this tree resolve: UNRESOLVED 0, FINDINGS 0. Unresolvable keys would be listed and would NOT fail the build. TWO-LEG PROOF, one variable — agentic_safety_screen's single line: pre-fix scripts/verify-state-keys.sh --root chat.el:2536 state_get("conv_history") conv_hist_key() owns this key namespace (EXACT 'conv_history') FAIL: 1 state-key finding(s) exit 1 as-is scripts/verify-state-keys.sh FINDINGS (0) ... PASS exit 0 INDEPENDENT CONFIRMATION: run read-only against origin/feat/soul-openai-tools-v2, which carries the same defect on its own, the gate reported chat.el:2937 — the exact line 43d0449's message had named by hand, with no prior knowledge. Against origin/fix/129-on-openai-tools: PASS. PRODUCER-MOVED CONTROLS: renaming the sole writer of an EXACT key (soul_model) orphans 3 readers across 3 files; renaming the sole writer of a PREFIX namespace (agent_workspace_root_*) orphans 3 readers, including when the producer moves to a NARROWER namespace — a case an earlier, more permissive prefix rule let through. That rule is now directional, with the reason written next to it. FOUND ON ITS FIRST RUN, unprompted: soul.el's state_set("soul_identity", ...) was deleted 2026-05-13 in b163fa6 (a commit about awareness/ISE writes) and five readers in chat.el were left behind — build_system_prompt, the vision handler, the agentic system prompt and two council handlers have prefixed "" for ~3 months. studio.el:57 emits "principal":"" and never had a producer. Both are recorded in state-key-baseline.txt with dates and causes so the gate can be turned on today; they are DEBT, not false positives, and every run prints them. Baseline signatures carry no line number (an unrelated edit must not un-mute an accepted finding) but do carry a count, so a GROWTH in a baselined finding still fails the build. Engine behaviour unchanged: this commit adds scripts only, no .el is touched. CI is deliberately NOT wired here — .gitea/workflows/ci.yaml has changes in flight from someone else, and turning the gate on would immediately red feat/soul-openai-tools-v2 (correctly). That flip should be deliberate. Rung reached: RUNS — the gate executes (0.15s), discriminates on four independent test pairs, and its verdicts are quoted above. Not wired to CI, and no engine binary was built from this branch. Co-Authored-By: Claude Opus 5 (1M context) --- scripts/state-key-audit.py | 937 +++++++++++++++++++++++++++++++++ scripts/state-key-baseline.txt | 28 + scripts/state-key-external.txt | 16 + scripts/verify-state-keys.sh | 118 +++++ 4 files changed, 1099 insertions(+) create mode 100755 scripts/state-key-audit.py create mode 100644 scripts/state-key-baseline.txt create mode 100644 scripts/state-key-external.txt create mode 100755 scripts/verify-state-keys.sh diff --git a/scripts/state-key-audit.py b/scripts/state-key-audit.py new file mode 100755 index 0000000..f9a41c6 --- /dev/null +++ b/scripts/state-key-audit.py @@ -0,0 +1,937 @@ +#!/usr/bin/env python3 +"""state-key-audit.py — the analyzer behind scripts/verify-state-keys.sh. + +Read that script's header for WHY this exists (issue #129). This file is the +HOW: a small El reader that resolves the key expression at every state_get / +state_set site, including keys that are computed. + +WHAT IT PARSES + El as this engine writes it: `fn f(a: T, b: T) -> T { ... }`, `let x: T = e`, + `return e`, `if c { a } else { b }` as an expression, `+` concatenation, + `"..."` with backslash escapes, `//` line comments. No block comments, no + const/match/struct exist in this dialect (verified over the whole tree). + +KEY PATTERNS — the only two things a key expression can resolve to + EXACT "soul_model" the whole key is known + PREFIX "session_hist_" a known head, then runtime text + (plus UNRESOLVED, which is a report line and never a failure) + +RESOLUTION — resolve_expr() returns a SET of patterns; unions are how branches, +multiple returns, and multiple bindings of one name are represented. + literal "k" -> {EXACT k} + concat A + B -> fold left; all-static -> EXACT, + static head + dynamic tail -> PREFIX + if-expression if c {A} else {B} -> resolve(A) | resolve(B), except that + str_eq(X,"") with X statically "" + folds to the taken branch only + call f(args) -> union over f's return expressions, + with f's params bound to THIS call + site's actual argument expressions + local var let k = e; state_get(k)-> union over every `let k =` in the + enclosing function + parameter fn g(k) { state_get(k) }-> union over the argument at that + position across every call site of g + anything else json_get(...), env(...)-> UNRESOLVED +Recursion is depth- and cycle-guarded; a guard trip yields UNRESOLVED, never a +failure. + +COVERAGE — a read is satisfied when some write can produce the same key: + read EXACT k <- write EXACT k, or write PREFIX p where k starts with p + read PREFIX p <- write EXACT k where k starts with p, or write PREFIX q + where p and q are prefixes of each other +Deliberately permissive at the boundaries: a gate that cries wolf gets deleted. +""" + +import os +import re +import sys + +MAX_DEPTH = 12 + +# ── patterns ──────────────────────────────────────────────────────────────── +EXACT = "exact" +PREFIX = "prefix" + + +def pat_exact(s): + return (EXACT, s) + + +def pat_prefix(s): + # A prefix with no static text at all carries no information; that is the + # UNRESOLVED case, not a pattern. + return (PREFIX, s) if s else None + + +def covers(write, read): + """Can a write of pattern `write` produce a key that `read` reads? + + The prefix rule is DIRECTIONAL, and that direction is the whole point. A + write namespace that is the same or BROADER than the read namespace covers + it (write "rl:" covers read "rl:x"). A write namespace that is NARROWER does + NOT (write "session_histv2_" does not cover read "session_hist_") — being + permissive there re-opens the exact hole this gate exists to close: rename + the producer, leave the readers, stay green. Verified with a control run + that renames sessions.el's writer and leaves its four readers behind.""" + wk, wv = write + rk, rv = read + if rk == EXACT: + return rv == wv if wk == EXACT else rv.startswith(wv) + # read is a PREFIX: some key starting with rv is read + if wk == EXACT: + return wv.startswith(rv) # that one written key is in range + return rv.startswith(wv) # write namespace same-or-broader + + +# ── lexer ─────────────────────────────────────────────────────────────────── +TOK_STR, TOK_IDENT, TOK_PUNCT, TOK_NUM = "str", "ident", "punct", "num" +IDENT_RE = re.compile(r"[A-Za-z_][A-Za-z0-9_]*") +NUM_RE = re.compile(r"[0-9]+(\.[0-9]+)?") + + +class Tok: + __slots__ = ("kind", "val", "line") + + def __init__(self, kind, val, line): + self.kind, self.val, self.line = kind, val, line + + def __repr__(self): + return "%s(%r)@%d" % (self.kind, self.val, self.line) + + +def lex(src): + toks, i, n, line = [], 0, len(src), 1 + while i < n: + c = src[i] + if c == "\n": + line += 1 + i += 1 + continue + if c in " \t\r": + i += 1 + continue + if c == "/" and i + 1 < n and src[i + 1] == "/": + while i < n and src[i] != "\n": + i += 1 + continue + if c == '"': + j, buf = i + 1, [] + while j < n: + if src[j] == "\\" and j + 1 < n: + esc = src[j + 1] + buf.append({"n": "\n", "t": "\t", "r": "\r"}.get(esc, esc)) + j += 2 + continue + if src[j] == '"': + break + if src[j] == "\n": + line += 1 + buf.append(src[j]) + j += 1 + toks.append(Tok(TOK_STR, "".join(buf), line)) + i = j + 1 + continue + m = IDENT_RE.match(src, i) + if m: + toks.append(Tok(TOK_IDENT, m.group(0), line)) + i = m.end() + continue + m = NUM_RE.match(src, i) + if m: + toks.append(Tok(TOK_NUM, m.group(0), line)) + i = m.end() + continue + toks.append(Tok(TOK_PUNCT, c, line)) + i += 1 + return toks + + +def match_close(toks, i, open_ch, close_ch): + """toks[i] is open_ch; return index of its matching close_ch.""" + depth = 0 + while i < len(toks): + if toks[i].kind == TOK_PUNCT: + if toks[i].val == open_ch: + depth += 1 + elif toks[i].val == close_ch: + depth -= 1 + if depth == 0: + return i + i += 1 + return len(toks) - 1 + + +# ── program model ─────────────────────────────────────────────────────────── +class Func: + def __init__(self, name, path, line, params, toks, start, end): + self.name, self.path, self.line = name, path, line + self.params = params # [param name] + self.toks = toks # the whole file's token list + self.start, self.end = start, end # body token range, exclusive of braces + self.lets = None # name -> [expr token ranges], lazily built + + +class Site: + def __init__(self, kind, path, line, func, arg_range, text): + self.kind = kind # "get" | "set" + self.path, self.line = path, line + self.func = func + self.arg_range = arg_range + self.text = text # source text of the key expression + self.pats = set() + self.unresolved = False + self.literal = None # set when the key expression is a bare literal + + +class Program: + def __init__(self): + self.files = {} # path -> toks + self.funcs = {} # name -> [Func] (El allows no overloads, but be safe) + self.toplevel = [] # [Func] one per file, params=[] + self.sites = [] # [Site] + self.calls = {} # callee name -> [(Func caller, [arg ranges])] + + # -- loading ------------------------------------------------------------ + def load(self, path, rel): + with open(path, "r", encoding="utf-8", errors="replace") as fh: + src = fh.read() + toks = lex(src) + self.files[rel] = toks + self._scan_funcs(rel, toks) + + def _scan_funcs(self, rel, toks): + covered = [] + i = 0 + while i < len(toks): + t = toks[i] + if t.kind == TOK_IDENT and t.val == "fn" and i + 2 < len(toks) \ + and toks[i + 1].kind == TOK_IDENT and toks[i + 2].val == "(": + name = toks[i + 1].val + pclose = match_close(toks, i + 2, "(", ")") + params = self._params(toks, i + 3, pclose) + bopen = pclose + 1 + while bopen < len(toks) and toks[bopen].val != "{": + bopen += 1 + bclose = match_close(toks, bopen, "{", "}") + f = Func(name, rel, t.line, params, toks, bopen + 1, bclose) + self.funcs.setdefault(name, []).append(f) + covered.append((i, bclose)) + i = bclose + 1 + continue + i += 1 + # everything outside a fn is the file's top-level "function" + tl = Func("" % rel, rel, 1, [], toks, 0, len(toks)) + tl.covered = covered + self.toplevel.append(tl) + + @staticmethod + def _params(toks, i, end): + """`a: T, b: T` -> ['a','b'] (top-level commas only).""" + names, depth, expect = [], 0, True + while i < end: + t = toks[i] + if t.kind == TOK_PUNCT and t.val in "([{": + depth += 1 + elif t.kind == TOK_PUNCT and t.val in ")]}": + depth -= 1 + elif depth == 0 and t.kind == TOK_PUNCT and t.val == ",": + expect = True + elif depth == 0 and expect and t.kind == TOK_IDENT: + names.append(t.val) + expect = False + i += 1 + return names + + def func_at(self, rel, tok_index): + for f in self.funcs_in(rel): + if f.start <= tok_index < f.end: + return f + for f in self.toplevel: + if f.path == rel: + return f + return None + + def funcs_in(self, rel): + for fl in self.funcs.values(): + for f in fl: + if f.path == rel: + yield f + + # -- indexing ----------------------------------------------------------- + def index(self): + for rel, toks in self.files.items(): + i = 0 + while i < len(toks): + t = toks[i] + if t.kind == TOK_IDENT and i + 1 < len(toks) and toks[i + 1].val == "(" \ + and t.val not in KEYWORDS \ + and not (i > 0 and toks[i - 1].kind == TOK_IDENT + and toks[i - 1].val == "fn"): + # ^ the `fn f(a: T)` declaration is not a call site; counting + # it as one makes every parameter resolve to its own name + # and reports the whole function UNRESOLVED. + close = match_close(toks, i + 1, "(", ")") + args = split_args(toks, i + 2, close) + self.calls.setdefault(t.val, []).append( + (self.func_at(rel, i), args, rel, t.line)) + if t.val in ("state_get", "state_set") and args: + self.sites.append(Site( + "get" if t.val == "state_get" else "set", + rel, t.line, self.func_at(rel, i), args[0], + render(toks, *args[0]))) + i += 1 + + # -- resolution --------------------------------------------------------- + def lets_of(self, f): + if f.lets is not None: + return f.lets + f.lets = {} + toks = f.toks + skip = getattr(f, "covered", []) + i = f.start + while i < f.end: + if any(a <= i <= b for a, b in skip): + i = max(b for a, b in skip if a <= i <= b) + 1 + continue + t = toks[i] + if t.kind == TOK_IDENT and t.val == "let" and i + 1 < f.end \ + and toks[i + 1].kind == TOK_IDENT: + name = toks[i + 1].val + j = i + 2 + if j < f.end and toks[j].val == ":": # skip the type + while j < f.end and toks[j].val != "=": + j += 1 + if j < f.end and toks[j].val == "=": + s = j + 1 + e = stmt_end(toks, s, f.end) + f.lets.setdefault(name, []).append((s, e)) + i = e + continue + i += 1 + return f.lets + + def returns_of(self, ctx, depth=0, seen=None): + """The value expressions of a function, in the context it was CALLED in. + + Context-sensitive on purpose. `conv_hist_key` is written as a guard: + + if str_eq(session_id, "") { return "conv_history" } + return "session_hist_" + session_id + + Collecting both returns flat would make state_set(conv_hist_key("")) — the + dead handle_chat() write — claim to produce the session_hist_ namespace + too. That is a producer this engine does not actually have, and claiming + it would let the gate stay green if sessions.el's real writer vanished: + a masking hole in the exact namespace #129 lives in. So a guard whose + condition folds is honoured, and the branch not taken is dropped.""" + out = [] + self._values(ctx.toks, ctx.start, ctx.end, ctx, depth, + seen if seen is not None else set(), out) + return out + + def _values(self, toks, s, e, ctx, depth, seen, out): + """Append the value expressions of a statement sequence. + Returns True when the sequence definitely returns (rest unreachable).""" + if depth > MAX_DEPTH: + return False + i = s + while i < e: + t = toks[i] + if t.kind == TOK_IDENT and t.val == "return": + j = stmt_end(toks, i + 1, e) + if j > i + 1: + out.append((i + 1, j)) + return True + if t.kind == TOK_IDENT and t.val == "let": + i = stmt_end(toks, i + 2, e) + continue + if t.kind == TOK_IDENT and t.val == "if": + i = self._if_stmt(toks, i, e, ctx, depth, seen, out) + if i is True: + return True + continue + if t.kind == TOK_PUNCT and t.val in "([{": + i = match_close(toks, i, t.val, + {"(": ")", "[": "]", "{": "}"}[t.val]) + 1 + continue + en = stmt_end(toks, i, e) + if en <= i: + i += 1 + continue + if en >= e: # trailing expression = the value + out.append((i, en)) + i = en + return False + + def _if_stmt(self, toks, i, e, ctx, depth, seen, out): + """Walk one if / else-if / else chain. Returns the next index, or True + if the chain definitely returns on every reachable branch.""" + bopen = i + 1 + while bopen < e and toks[bopen].val != "{": + bopen += 1 + if bopen >= e: + return e + bclose = match_close(toks, bopen, "{", "}") + fold = self._fold_cond(toks, i + 1, bopen, ctx, depth, seen) + + j = bclose + 1 + else_s = else_e = None + if j < e and toks[j].kind == TOK_IDENT and toks[j].val == "else": + if j + 1 < e and toks[j + 1].val == "{": + ec = match_close(toks, j + 1, "{", "}") + else_s, else_e = j + 2, ec + j = ec + 1 + else: # `else if ...` — the rest of the chain + else_s = j + 1 + else_e = stmt_end(toks, j + 1, e) + j = else_e + + then_ret = else_ret = False + if fold is not False: + then_ret = self._values(toks, bopen + 1, bclose, ctx, depth + 1, seen, out) + if fold is not True and else_s is not None: + else_ret = self._values(toks, else_s, else_e, ctx, depth + 1, seen, out) + + if fold is True and then_ret: + return True + if fold is False and else_s is not None and else_ret: + return True + if fold is None and else_s is not None and then_ret and else_ret: + return True + return j + + def resolve(self, rng, func, depth=0, seen=None): + """-> (set of patterns, unresolved_flag)""" + if seen is None: + seen = set() + if depth > MAX_DEPTH: + return set(), True + return self._expr(func.toks, rng[0], rng[1], func, depth, seen) + + # -- expression walker -------------------------------------------------- + def _expr(self, toks, s, e, func, depth, seen): + parts, cur, d = [], s, 0 + i = s + while i < e: # split on top-level '+' + v = toks[i].val + if toks[i].kind == TOK_PUNCT and v in "([{": + d += 1 + elif toks[i].kind == TOK_PUNCT and v in ")]}": + d -= 1 + elif d == 0 and toks[i].kind == TOK_PUNCT and v == "+" and i > s: + parts.append((cur, i)) + cur = i + 1 + i += 1 + parts.append((cur, e)) + if len(parts) == 1: + return self._primary(toks, s, e, func, depth, seen) + + # concatenation: keep folding while every operand so far is EXACT + head, unres = "", False + static = True + for (ps, pe) in parts: + pats, u = self._primary(toks, ps, pe, func, depth, seen) + exacts = {p[1] for p in pats if p[0] == EXACT} + if static and len(exacts) == 1 and not u and len(pats) == 1: + head += exacts.pop() + continue + if static and pats and all(p[0] == EXACT for p in pats) and len(pats) > 1: + # a branchy static operand: keep the shared head only + static = False + head += os.path.commonprefix(sorted({p[1] for p in pats})) + break + static = False + # first non-static operand: everything after it is runtime text + if (ps, pe) == parts[0]: + for p in pats: + if p[0] == PREFIX: + head = p[1] + break + if not head: + unres = True + break + if static: + return {pat_exact(head)}, False + p = pat_prefix(head) + return ({p} if p else set()), (unres or not p) + + def _primary(self, toks, s, e, func, depth, seen): + while s < e and toks[s].kind == TOK_PUNCT and toks[s].val == "(" \ + and match_close(toks, s, "(", ")") == e - 1: + s, e = s + 1, e - 1 + if s >= e: + return set(), True + t = toks[s] + + if t.kind == TOK_STR and e == s + 1: + return {pat_exact(t.val)}, False + + if t.kind == TOK_IDENT and t.val == "if": + return self._if_expr(toks, s, e, func, depth, seen) + + if t.kind == TOK_IDENT and s + 1 < e and toks[s + 1].val == "(": + close = match_close(toks, s + 1, "(", ")") + if close == e - 1: + return self._call(toks, t.val, split_args(toks, s + 2, close), + func, depth, seen) + + if t.kind == TOK_IDENT and e == s + 1: + return self._var(t.val, func, depth, seen) + + return set(), True + + def _if_expr(self, toks, s, e, func, depth, seen): + bopen = s + 1 + while bopen < e and toks[bopen].val != "{": + bopen += 1 + cond = (s + 1, bopen) + bclose = match_close(toks, bopen, "{", "}") + then_rng = block_tail(toks, bopen + 1, bclose) or (bopen + 1, bclose) + + else_rng = None + j = bclose + 1 + if j < e and toks[j].kind == TOK_IDENT and toks[j].val == "else": + if j + 1 < e and toks[j + 1].val == "{": + ec = match_close(toks, j + 1, "{", "}") + else_rng = block_tail(toks, j + 2, ec) or (j + 2, ec) + else: + else_rng = (j + 1, e) # `else if ...` + + taken = self._fold_cond(toks, cond[0], cond[1], func, depth, seen) + rngs = [] + if taken is not False: + rngs.append(then_rng) + if taken is not True and else_rng: + rngs.append(else_rng) + + pats, unres = set(), False + for r in rngs: + p, u = self._expr(toks, r[0], r[1], func, depth + 1, seen) + pats |= p + unres = unres or u + return pats, unres + + def _fold_cond(self, toks, s, e, func, depth, seen): + """Constant-fold `str_eq(X, "")` / `!str_eq(X, "")` so a helper called with + a literal (conv_hist_key("")) yields only the branch it really takes. + Returns True / False / None(unknown).""" + neg = False + if s < e and toks[s].kind == TOK_PUNCT and toks[s].val == "!": + neg, s = True, s + 1 + if not (s < e and toks[s].kind == TOK_IDENT and toks[s].val == "str_eq" + and s + 1 < e and toks[s + 1].val == "("): + return None + close = match_close(toks, s + 1, "(", ")") + if close != e - 1: + return None + args = split_args(toks, s + 2, close) + if len(args) != 2: + return None + va, ua = self._expr(toks, args[0][0], args[0][1], func, depth + 1, seen) + vb, ub = self._expr(toks, args[1][0], args[1][1], func, depth + 1, seen) + if ua or ub or len(va) != 1 or len(vb) != 1: + return None + (ka, sa), (kb, sb) = va.pop(), vb.pop() + if ka != EXACT or kb != EXACT: + return None + r = (sa == sb) + return (not r) if neg else r + + def _call(self, toks, name, args, func, depth, seen): + cands = self.funcs.get(name) + if not cands: + return set(), True # builtin: json_get, env, ... + pats, unres = set(), False + for callee in cands: + key = ("fn", callee.path, callee.name, tuple(args)) + if key in seen: + unres = True + continue + seen = seen | {key} + # bind the callee's params to THIS call site's argument expressions + binding = {} + for idx, pname in enumerate(callee.params): + if idx < len(args): + binding[pname] = (args[idx], func) + callee_ctx = _Bound(callee, binding) + for r in self.returns_of(callee_ctx, depth + 1, seen): + p, u = self._expr(callee.toks, r[0], r[1], callee_ctx, + depth + 1, seen) + pats |= p + unres = unres or u + return pats, unres + + def _var(self, name, func, depth, seen): + real = func.func if isinstance(func, _Bound) else func + + # 1. a parameter bound by the call site we came through + if isinstance(func, _Bound) and name in func.binding: + rng, caller_ctx = func.binding[name] + return self._expr(caller_ctx.toks, rng[0], rng[1], caller_ctx, + depth + 1, seen) + + # 2. a local `let` in the enclosing function + lets = self.lets_of(real) + if name in lets: + key = ("let", real.path, real.name, name) + if key in seen: + return set(), True + seen = seen | {key} + pats, unres = set(), False + for rng in lets[name]: + p, u = self._expr(real.toks, rng[0], rng[1], real, depth + 1, seen) + pats |= p + unres = unres or u + return pats, unres + + # 3. an unbound parameter -> look at every call site of the enclosing fn + if name in real.params: + key = ("param", real.path, real.name, name) + if key in seen: + return set(), True + seen = seen | {key} + idx = real.params.index(name) + pats, unres = set(), False + sites = self.calls.get(real.name, []) + if not sites: + return set(), True + for caller, args, _rel, _line in sites: + if caller is None or idx >= len(args): + unres = True + continue + p, u = self._expr(caller.toks, args[idx][0], args[idx][1], + caller, depth + 1, seen) + pats |= p + unres = unres or u + return pats, unres + + # 4. a file-level / cross-file top-level `let` + for tl in self.toplevel: + lets = self.lets_of(tl) + if name in lets: + key = ("let", tl.path, tl.name, name) + if key in seen: + return set(), True + seen2 = seen | {key} + pats, unres = set(), False + for rng in lets[name]: + p, u = self._expr(tl.toks, rng[0], rng[1], tl, depth + 1, seen2) + pats |= p + unres = unres or u + return pats, unres + + return set(), True + + +class _Bound: + """A callee view that also knows what its params were called with.""" + + def __init__(self, func, binding): + self.func, self.binding = func, binding + self.toks, self.start, self.end = func.toks, func.start, func.end + self.params, self.path, self.name = func.params, func.path, func.name + + def __getattr__(self, k): + return getattr(self.func, k) + + +# ── token helpers ─────────────────────────────────────────────────────────── +def split_args(toks, s, e): + out, cur, d = [], s, 0 + i = s + while i < e: + v = toks[i].val + if toks[i].kind == TOK_PUNCT and v in "([{": + d += 1 + elif toks[i].kind == TOK_PUNCT and v in ")]}": + d -= 1 + elif d == 0 and toks[i].kind == TOK_PUNCT and v == ",": + out.append((cur, i)) + cur = i + 1 + i += 1 + if cur < e: + out.append((cur, e)) + return out + + +STMT_START = {"let", "return", "if", "while", "for"} +KEYWORDS = {"if", "while", "for", "return", "fn", "let", "else", "match"} + + +def stmt_end(toks, s, limit): + """End of the expression starting at s: the next top-level statement + boundary. El has no semicolons, so a newline that starts a new statement + ends this one.""" + d, i = 0, s + while i < limit: + t = toks[i] + if t.kind == TOK_PUNCT and t.val in "([": + d += 1 + elif t.kind == TOK_PUNCT and t.val in ")]": + d -= 1 + if d < 0: + return i + elif t.kind == TOK_PUNCT and t.val == "{": + # a brace at depth 0 belongs to this expression only when it is an + # if/else block that is part of it + d += 1 + elif t.kind == TOK_PUNCT and t.val == "}": + d -= 1 + if d < 0: + return i + elif d == 0 and t.kind == TOK_PUNCT and t.val == ",": + return i + elif d == 0 and i > s and t.kind == TOK_IDENT and t.val in STMT_START: + if t.val == "if" and toks[i - 1].kind == TOK_IDENT and toks[i - 1].val == "else": + i += 1 + continue + return i + elif d == 0 and i > s and t.kind == TOK_IDENT and t.val == "fn": + return i + i += 1 + return limit + + +def block_tail(toks, s, e): + """The trailing expression of a block, if the block ends in one.""" + i, last = s, None + while i < e: + t = toks[i] + if t.kind == TOK_IDENT and t.val in ("let", "return"): + i = stmt_end(toks, i + 1, e) + last = None + continue + if t.kind == TOK_PUNCT and t.val in "([{": + i = match_close(toks, i, t.val, {"(": ")", "[": "]", "{": "}"}[t.val]) + 1 + continue + st = i + en = stmt_end(toks, i, e) + if en <= st: + i = st + 1 + continue + last = (st, en) + i = en + return last + + +def render(toks, s, e): + out = [] + for t in toks[s:e]: + out.append('"%s"' % t.val if t.kind == TOK_STR else t.val) + return " ".join(out) + + +# ── the gate ──────────────────────────────────────────────────────────────── +def collect(root, include_tests): + files = [] + for dirpath, dirnames, filenames in os.walk(root): + dirnames[:] = [d for d in dirnames + if d not in ("dist", "vendor", ".git", "node_modules")] + rel_dir = os.path.relpath(dirpath, root) + if not include_tests and rel_dir.split(os.sep)[0] == "tests": + continue + for fn in sorted(filenames): + if fn.endswith(".el"): + rel = os.path.normpath(os.path.join(rel_dir, fn)) + files.append((os.path.join(dirpath, fn), rel)) + return sorted(files, key=lambda x: x[1]) + + +def is_bare_literal(prog, site): + toks = prog.files[site.path] + s, e = site.arg_range + return e == s + 1 and toks[s].kind == TOK_STR + + +def read_decl(path): + """A declaration file: one entry per line, `# ...` comments stripped.""" + out = [] + if not path or not os.path.exists(path): + return out + with open(path) as fh: + for ln in fh: + ln = ln.split("#", 1)[0].strip() + if ln: + out.append(ln) + return out + + +def opt(argv, name, default=None): + for i, a in enumerate(argv): + if a == name and i + 1 < len(argv): + return argv[i + 1] + return default + + +def main(argv): + root = os.path.abspath(argv[1]) if len(argv) > 1 and not argv[1].startswith("-") else "." + include_tests = "--include-tests" in argv + verbose = "--verbose" in argv + baseline_path = opt(argv, "--baseline") + external_path = opt(argv, "--external") + + prog = Program() + for path, rel in collect(root, include_tests): + prog.load(path, rel) + prog.index() + for site in prog.sites: + pats, unres = prog.resolve(site.arg_range, site.func) + site.pats, site.unresolved = {p for p in pats if p}, unres + if is_bare_literal(prog, site): + site.literal = prog.files[site.path][site.arg_range[0]].val + + writes = [s for s in prog.sites if s.kind == "set"] + reads = [s for s in prog.sites if s.kind == "get"] + write_pats = set() + for w in writes: + write_pats |= w.pats + + # Declared host-set keys: written by something outside the El tree (an + # operator, the installer, a host process). Each entry must carry a reason. + external = [] + for ln in read_decl(external_path): + parts = ln.split(None, 1) + if len(parts) != 2 or parts[0] not in (EXACT, PREFIX): + print("bad --external line (want `exact|prefix `): %r" % ln, + file=sys.stderr) + return 2 + external.append((parts[0], parts[1])) + write_pats |= set(external) + + # F1 — a read of a key no write in the tree produces. + f1 = [] + for r in reads: + for p in sorted(r.pats): + if not any(covers(w, p) for w in write_pats): + f1.append((r, p)) + + # F2 — a key namespace owned by a helper, accessed by a hand-rolled literal. + # This is the #129 shape: the producer moved behind conv_hist_key() and + # one consumer kept spelling the old key out by hand. + owners = {} # helper fn name -> its value set + for s in prog.sites: + toks = prog.files[s.path] + a, b = s.arg_range + if toks[a].kind == TOK_IDENT and a + 1 < b and toks[a + 1].val == "(" \ + and match_close(toks, a + 1, "(", ")") == b - 1 \ + and toks[a].val in prog.funcs: + name = toks[a].val + if name not in owners: + vals = set() + for callee in prog.funcs[name]: + # No call context here on purpose: the OWNED namespace is + # every key the helper can ever produce, over all call sites. + for rng in prog.returns_of(callee): + p, _ = prog._expr(callee.toks, rng[0], rng[1], callee, 0, set()) + vals |= {x for x in p if x} + owners[name] = vals + f2 = [] + for s in prog.sites: + if s.literal is None: + continue + for owner, vals in sorted(owners.items()): + for v in sorted(vals): + if covers(v, pat_exact(s.literal)): + f2.append((s, owner, v)) + break + else: + continue + break + + unresolved = [s for s in prog.sites if s.unresolved or not s.pats] + + # Baseline signatures carry NO line number on purpose: an unrelated edit that + # shifts a line must not un-mute an accepted finding (that is crying wolf), + # but a GROWTH in count must not hide either. So a baseline entry is + # ` [xN]` and only the first N matches are muted. + baseline, bad_baseline = {}, [] + for ln in read_decl(baseline_path): + n, key = 1, ln + parts = ln.rsplit(" x", 1) + if len(parts) == 2 and parts[1].isdigit(): + key, n = parts[0].strip(), int(parts[1]) + baseline[key] = n + + def sig(path, code, detail): + return "%s %s %s" % (path, code, detail) + + findings = [] + for r, p in f1: + findings.append((sig(r.path, "DEAD-READ", "%s:%s" % p), r.line, + " %s:%d state_get(%s)\n resolves to %s %r — no state_set in the tree produces it" + % (r.path, r.line, r.text, p[0].upper(), p[1]))) + for s, owner, v in f2: + findings.append((sig(s.path, "HAND-ROLLED", "%s<-%s()" % (s.literal, owner)), s.line, + " %s:%d state_%s(\"%s\")\n %s() owns this key namespace (%s %r) — go through the helper, " + "or a rename orphans this site silently" % (s.path, s.line, s.kind, s.literal, owner, v[0].upper(), v[1]))) + findings.sort(key=lambda f: (f[0], f[1])) + + live, muted, budget = [], [], dict(baseline) + for f in findings: + if budget.get(f[0], 0) > 0: + budget[f[0]] -= 1 + muted.append(f) + else: + live.append(f) + stale = sorted(k for k, v in budget.items() if v > 0) + + print("── state-key audit ─────────────────────────────────────────────") + print("scanned %d .el files%s" % (len(prog.files), + "" if include_tests else " (tests/ excluded)")) + print("sites %d state_set, %d state_get" % (len(writes), len(reads))) + print("keys %d distinct write patterns" % len(write_pats)) + print("") + + if verbose: + print("WRITE PATTERNS") + for k, v in sorted(write_pats): + print(" %-6s %s" % (k, v)) + print("") + + if external: + print("DECLARED HOST-SET (%d) — %s" % (len(external), external_path)) + for k, v in sorted(external): + print(" %-6s %s" % (k, v)) + print("") + + print("UNRESOLVED (%d) — reported, never fails the build" % len(unresolved)) + if not unresolved: + print(" (none)") + for s in sorted(unresolved, key=lambda x: (x.path, x.line)): + print(" %s:%d state_%s(%s)%s" + % (s.path, s.line, s.kind, s.text, + " [partial: %s]" % ", ".join("%s %r" % p for p in sorted(s.pats)) + if s.pats else "")) + print("") + + if muted: + print("BASELINED (%d) — pre-existing debt accepted in %s. NOT clean; fix these." + % (len(muted), baseline_path)) + for sg, line, _ in muted: + print(" %s (line %d)" % (sg, line)) + print("") + if stale: + print("STALE BASELINE (%d) — entries that no longer match anything; delete them:" + % len(stale)) + for sg in stale: + print(" %s" % sg) + print("") + + print("FINDINGS (%d)" % len(live)) + if not live: + print(" (none)") + for _, _, body in live: + print(body) + print("") + + if live: + print("FAIL: %d state-key finding(s). See scripts/verify-state-keys.sh " + "for why this gate exists (issue #129)." % len(live)) + return 1 + print("PASS: every resolvable state_get key has a producer, and no key " + "namespace is spelled two ways.") + return 0 + + +if __name__ == "__main__": + sys.exit(main(sys.argv)) diff --git a/scripts/state-key-baseline.txt b/scripts/state-key-baseline.txt new file mode 100644 index 0000000..36408dd --- /dev/null +++ b/scripts/state-key-baseline.txt @@ -0,0 +1,28 @@ +# state-key-baseline.txt — findings that already existed when this gate landed +# (2026-08-07). Each one is a REAL defect of the #129 class, not a false +# positive. They are muted only so the gate can be turned on today instead of +# being deferred until the debt is paid; every run still prints them under +# BASELINED with the word "debt". +# +# THIS FILE SHOULD ONLY EVER SHRINK. Adding a line means you are shipping a +# known silent-"" read. If you must, date it and say why in the comment. +# +# format: [xN] # N = how many sites are accepted +# No line numbers on purpose: an unrelated edit must not un-mute an accepted +# finding, but a GROWTH in count is NOT muted — the extra site fails the build. +# +chat.el DEAD-READ exact:soul_identity x5 +# ^ soul.el used to run `state_set("soul_identity", soul_identity)`. It was +# deleted on 2026-05-13 in b163fa6 ("feat(awareness): route ISE writes to HTTP +# Engram ..."), a commit about something else entirely, and the five readers in +# chat.el were left behind. Since that date build_system_prompt (737), the +# vision handler (1745), the agentic system prompt (2620), the council +# transcript handler (3425) and 3480 have all been prefixing "" — exactly the +# #129 shape, found by this gate on its first run. Sites: 737, 1745, 2620, +# 3425, 3480. Fix = restore the boot-time write or delete the reads; not done +# here because this branch must not change engine behaviour. + +studio.el DEAD-READ exact:soul_principal x1 +# ^ studio.el:57 dharma_registry() emits "principal":"" on every call — no +# producer has ever existed in the tree's history (git log -S finds none). +# Never-wired rather than orphaned, same silent-"" result. diff --git a/scripts/state-key-external.txt b/scripts/state-key-external.txt new file mode 100644 index 0000000..c39acf5 --- /dev/null +++ b/scripts/state-key-external.txt @@ -0,0 +1,16 @@ +# state-key-external.txt — state keys the engine READS but deliberately never +# WRITES, because a host outside the El tree sets them (an operator, the +# installer, a deployment env). Read scripts/verify-state-keys.sh for why this +# list has to exist and why it has to stay short. +# +# THE RULE FOR ADDING A LINE: the read site must already treat "" as a defined +# default (`if str_eq(x, "") { }`) AND the source must say so in a +# comment. "I could not find the writer" is NOT a reason — that is the #129 +# defect, and it belongs in state-key-baseline.txt with a date, not here. +# +# format: exact|prefix # why, and where the source says so +# +exact soul_rate_limit # routes.el:59-61 — "configurable via soul state key ... Falls back to 60 req/min if not set." +exact web_search_tool_version # chat.el:1884-1910 — version lives in state "so a future bump is a config write, not a recompile"; defaults to web_search_20250305 +exact platform_auth # stewardship.el:92 — host-set capability flag; fail-CLOSED (anything but "true" denies the platform tool) +exact security_research_authorized # awareness.el:991-996 — state override for env SECURITY_RESEARCH_TOKEN; fail-closed, defaults false diff --git a/scripts/verify-state-keys.sh b/scripts/verify-state-keys.sh new file mode 100755 index 0000000..04dcd3e --- /dev/null +++ b/scripts/verify-state-keys.sh @@ -0,0 +1,118 @@ +#!/usr/bin/env bash +# verify-state-keys.sh — the state-key gate. Retires a defect class at build time. +# +# ── WHY THIS EXISTS. DO NOT DELETE IT AS NOISE. ────────────────────────────── +# +# The engine keeps runtime values in a key-value store: state_set("k", v) writes, +# state_get("k") reads. A read of a key that NOTHING writes returns an empty +# string. Silently. No error, no warning, no log line. The El compiler cannot see +# it, no test sees it, and the product keeps running — just with a hole in it. +# +# That is how issue #129 happened. ff421d3 (2026-08-05) correctly moved +# conversation history to a per-session key behind conv_hist_key(session_id). One +# consumer did not move with it: the agentic path's L1 safety screen kept reading +# the old anonymous "conv_history" bucket. The desktop app always mints a session +# id, so history was always written under session_hist_ and that read always +# returned "". The half of the crisis score that receives history is the +# ESCALATION half — the one that exists for distress building across several +# turns, where no single message trips the bell on its own. It scored 0 on every +# real conversation for two days, and nothing failed. +# +# The line that broke carried a comment describing this exact bug being fixed +# once already, under issue #9. A comment is not a gate. This is the gate. +# +# ── WHAT IT CHECKS ────────────────────────────────────────────────────────── +# +# DEAD-READ a state_get whose key resolves to something no state_set in the +# tree produces. The direct form of the class. +# +# HAND-ROLLED a state_get/state_set that spells out a literal belonging to a +# key namespace a helper function owns (e.g. "conv_history", owned +# by conv_hist_key()). This is #129's actual shape: the producer +# moved behind the helper and one consumer kept the old spelling +# by hand. DEAD-READ alone does NOT catch #129, because the dead +# handle_chat() still writes that key through the helper — so this +# second check is the one that earns the gate its keep. +# +# ── WHY IT DOES NOT CRY WOLF ──────────────────────────────────────────────── +# +# Keys are usually COMPUTED, not literal, so a naive grep would flood and get +# switched off within a day. scripts/state-key-audit.py resolves computed keys: +# string concatenation (matched on the static prefix), helper functions (resolved +# to their possible return values), keys built into a local variable, and keys +# arriving as a function parameter (resolved through the call sites). Where a key +# genuinely cannot be resolved it is printed under UNRESOLVED and does NOT fail +# the build — visible, never silently ignored. Keep that list short. +# +# On this tree it resolves 278 of 278 sites: UNRESOLVED is 0 and FINDINGS is 0. +# +# Two declaration files, both of which should only ever shrink: +# scripts/state-key-external.txt keys a host outside the El tree writes +# scripts/state-key-baseline.txt findings that predate the gate (real debt) +# +# ── PROVEN TO DISCRIMINATE (2026-08-07) ───────────────────────────────────── +# +# 1. Synthetic: a scratch copy of this tree with agentic_safety_screen reverted +# to the pre-fix state_get("conv_history") — ONE line, nothing else — FAILS +# with `chat.el:2536 ... conv_hist_key() owns this key namespace`. The tree +# as shipped PASSES. One variable, opposite verdicts. +# 2. Independent: run read-only against origin/feat/soul-openai-tools-v2, which +# carries the same defect on its own, the gate reported chat.el:2937 — the +# exact line 43d0449's commit message had named by hand. Against that +# branch's fix (origin/fix/129-on-openai-tools) it passes. +# 3. Producer-moved controls: renaming the sole writer of an EXACT key +# (soul_model) orphans 3 readers across 3 files; renaming the sole writer of +# a PREFIX namespace (agent_workspace_root_*) orphans 3 readers — including +# when the producer moves to a NARROWER namespace, which an earlier, +# sloppier prefix rule let through. +# +# It also found, on its first run, a defect nobody was looking for: soul.el's +# `state_set("soul_identity", ...)` was deleted on 2026-05-13 in b163fa6 (a +# commit about awareness/ISE writes) and five readers in chat.el were left +# behind — the system prompt, the vision handler, the agentic prompt and the +# council handler have been prefixing "" ever since. See state-key-baseline.txt. +# +# ── SAFETY ────────────────────────────────────────────────────────────────── +# Pure static read of .el sources. Starts nothing, opens no port, touches no +# daemon, and never reads or writes ~/.neuron. +# +# ── USAGE ─────────────────────────────────────────────────────────────────── +# scripts/verify-state-keys.sh gate the repo (honours baseline) +# scripts/verify-state-keys.sh --strict ignore the baseline: show the debt +# scripts/verify-state-keys.sh --verbose also dump every write pattern +# scripts/verify-state-keys.sh --root DIR audit a different tree +# exit 0 = clean; 1 = finding(s); 2 = the gate itself could not run. +set -uo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" +STRICT=0 +PASS_THROUGH=() + +while [ $# -gt 0 ]; do + case "$1" in + --strict) STRICT=1; shift ;; + --root) ROOT="${2:?--root needs a directory}"; shift 2 ;; + -h|--help) awk 'NR>1 && /^#/ {print; next} NR>1 {exit}' "${BASH_SOURCE[0]}"; exit 0 ;; + *) PASS_THROUGH+=("$1"); shift ;; + esac +done + +command -v python3 >/dev/null 2>&1 || { + echo "[state-keys] CANNOT RUN: python3 not found" >&2; exit 2; } +[ -d "$ROOT" ] || { echo "[state-keys] CANNOT RUN: no such tree: $ROOT" >&2; exit 2; } + +AUDIT="$SCRIPT_DIR/state-key-audit.py" +[ -f "$AUDIT" ] || { echo "[state-keys] CANNOT RUN: missing $AUDIT" >&2; exit 2; } + +ARGS=("$ROOT" "--external" "$SCRIPT_DIR/state-key-external.txt") +[ "$STRICT" -eq 0 ] && ARGS+=("--baseline" "$SCRIPT_DIR/state-key-baseline.txt") +[ ${#PASS_THROUGH[@]} -gt 0 ] && ARGS+=("${PASS_THROUGH[@]}") + +python3 "$AUDIT" "${ARGS[@]}" +RC=$? +if [ "$RC" -gt 1 ]; then + echo "[state-keys] CANNOT RUN: the audit itself failed (exit $RC)" >&2 + exit 2 +fi +exit "$RC"