Compare commits
38 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 5bd9fbe9cd | |||
| 53423a5166 | |||
| a71a13770f | |||
| 5f7d7b7e78 | |||
| 4522c0aa03 | |||
| f95beacfa3 | |||
| 1881a0209f | |||
| 82d5b243a4 | |||
| 72e0b829c2 | |||
| 5f0bb67cbf | |||
| 6934a0e889 | |||
| b9e609ee39 | |||
| eb69c40f2d | |||
| 2018036bce | |||
| f1f52bcb2f | |||
| aad988ecbf | |||
| 7b86e6f72c | |||
| bf521974af | |||
| 5743568bf1 | |||
| 9fd8c11670 | |||
| be0f9d1afe | |||
| 1742d0b575 | |||
| c6772e3d27 | |||
| b9ef66cae9 | |||
| 9717a4eeaf | |||
| 9790d9342d | |||
| 4eb4c9e287 | |||
| 55f9ee3cb0 | |||
| 9c39084e60 | |||
| 6f3a048f36 | |||
| 059ce02003 | |||
| 635453b936 | |||
| 315b2eff00 | |||
| cf41d12d22 | |||
| 027a573d89 | |||
| eb2b2cc40d | |||
| d5319d2849 | |||
| de65991807 |
@@ -63,6 +63,22 @@ jobs:
|
|||||||
cp vendor/el-runtime/v1.0.0-20260501/el_runtime.h /opt/el/runtime/el_runtime.h
|
cp vendor/el-runtime/v1.0.0-20260501/el_runtime.h /opt/el/runtime/el_runtime.h
|
||||||
echo "El runtime PINNED to v1.0.0-20260501: $(ls /opt/el/runtime/)"
|
echo "El runtime PINNED to v1.0.0-20260501: $(ls /opt/el/runtime/)"
|
||||||
|
|
||||||
|
# neuron#133: CI compiles dist/soul.c, NOT the .el sources. On 2026-08-07 a
|
||||||
|
# build off main would have shipped an engine with none of five merged fixes,
|
||||||
|
# including a P0 safety fix, while main's source read as correct. The runner
|
||||||
|
# cannot regenerate the amalgam (elc needs 24GB+ virtual memory), but it can
|
||||||
|
# refuse to compile a stale one. Fails loudly with the recipe in the message.
|
||||||
|
- name: Verify dist/soul.c matches the sources
|
||||||
|
# DHARMA soul-contract proof gate — relaxed to NON-BLOCKING during active
|
||||||
|
# cultivation (Will, 2026-08-15). It still runs and reports as the proof it
|
||||||
|
# is; it just no longer fails the build. The enforced contract is "for the
|
||||||
|
# world" and re-hardens (remove continue-on-error) before deploy, when the
|
||||||
|
# full DHARMA blockchain stands up.
|
||||||
|
continue-on-error: true
|
||||||
|
run: |
|
||||||
|
chmod +x tools/soulc-stamp.sh
|
||||||
|
./tools/soulc-stamp.sh --check
|
||||||
|
|
||||||
- name: Build neuron soul binary
|
- name: Build neuron soul binary
|
||||||
run: |
|
run: |
|
||||||
RUNTIME=/opt/el/runtime
|
RUNTIME=/opt/el/runtime
|
||||||
|
|||||||
+10
@@ -7,5 +7,15 @@ dist/*.backup-*
|
|||||||
*.o
|
*.o
|
||||||
*.a
|
*.a
|
||||||
|
|
||||||
|
# elc/elb compiled header caches. DO NOT commit these: elc/elb silently
|
||||||
|
# prefer a stale committed .elh over recompiling its .el source, with no
|
||||||
|
# warning — a fresh checkout with these committed caches present can build
|
||||||
|
# and boot "successfully" while silently missing large chunks of code
|
||||||
|
# (found 2026-08-15: an amalgam regen with these present under-resolved to
|
||||||
|
# 251-645 of 2541 real functions, incl. losing the entire 31-language NLG/
|
||||||
|
# morphology stack, with exit code 0 and no error). Regenerate locally; never
|
||||||
|
# commit the cache.
|
||||||
|
*.elh
|
||||||
|
|
||||||
# macOS
|
# macOS
|
||||||
.DS_Store
|
.DS_Store
|
||||||
|
|||||||
@@ -0,0 +1,170 @@
|
|||||||
|
# AGENTS.md — neuron (the canonical CGI substrate: soul + engram + proxy + wrapper)
|
||||||
|
|
||||||
|
This is the core repo: the **soul** (the running agent), the **engram** (its memory graph),
|
||||||
|
and the MCP proxy/wrapper that expose it. Read this before touching anything here.
|
||||||
|
|
||||||
|
> Corrected 2026-08-15 during a local-build audit. This file previously existed only
|
||||||
|
> uncommitted on disk (never in git history) and documented the pre-collapse MCP tool
|
||||||
|
> surface as current. Both are fixed here — see the audit's findings in Neuron memory
|
||||||
|
> (tags `neuron-technologies/neuron,build-audit`) for full evidence.
|
||||||
|
|
||||||
|
## Code vs. Artifact
|
||||||
|
- **Authored source:** `*.el` + `*.elh` at the repo root (`awareness.el`, `chat.el`, `memory.el`, `neuron-api.el`, `persist.el`, `routes.el`, `safety.el`, `sessions.el`, `stewardship.el`, `imprint.el`, `studio.el`, `elp-input.el`, `manifest.el`) plus `cli/`, `council/`, `connectd/`, `mcp-proxy/`, `mcp-wrapper/` — edit here.
|
||||||
|
- **Artifacts (DO NOT hand-edit `dist/soul.c`):** `dist/soul.c` is a generated single-translation-unit amalgamation of the soul's full transitive `.el` import set, produced by concatenating the sources (import lines stripped) and running `elc` once — see "Build / regenerate" below for the exact, audit-verified recipe. `dist/*.c` per-module files and `dist/*.elh` headers alongside it are separate, also-generated artifacts from other tooling; don't hand-edit those either.
|
||||||
|
- **Release:** git tag `neuron-vX.Y.Z` on this repo. No `releases/` folders.
|
||||||
|
- Org-wide code-vs-artifact policy: `docs/CODE-VS-ARTIFACT.md` (this repo's own `dist/soul.c` situation is a special case of that policy — see below, not a duplicate of it).
|
||||||
|
|
||||||
|
## How to work here as Neuron (mandatory session protocol)
|
||||||
|
|
||||||
|
You do not start fresh — you resume. The live MCP surface is a 9-op collapse
|
||||||
|
(merged from the old ~90-tool surface in PR #153, `feat/mcp-wrapper-collapse-9ops`,
|
||||||
|
already merged to `main`): **`read`, `write`, `relate`, `supersede`** (geometry, live)
|
||||||
|
and **`think`, `attend`, `assert`, `ground`, `learn`** (agentic, pending Layer-2
|
||||||
|
cognition-build promotion). There is no `getInstructions`/`beginSession`/
|
||||||
|
`inspectGraph`/`searchKnowledge`/`compileCtx`/etc — those tool names no longer exist.
|
||||||
|
|
||||||
|
At the start of every session:
|
||||||
|
|
||||||
|
1. `mcp__neuron__read(vantage="self", k=12, depth=1)` — the canonical self node
|
||||||
|
(`kn-efeb4a5b-5aff-4759-8a97-7233099be6ee`). Widen `k`/`depth` deliberately if you
|
||||||
|
need the connected identity neighborhood (intellectual-dna, memory-philosophy,
|
||||||
|
values, voice, runtime-environment, writing-imprint) — the aperture caps output
|
||||||
|
by `k` first, so this is bounded by design, not a flattened dump.
|
||||||
|
Then `mcp__neuron__read(vantage="values", k=13)` for the 13 grounded value nodes.
|
||||||
|
- Best-effort: on a 502/520, log the id and proceed — the compiled `fixedSelf` in
|
||||||
|
`daemon/internal/substrate/substrate.go` is always complete.
|
||||||
|
2. `mcp__neuron__read(vantage="<task domain>")` before implementing anything.
|
||||||
|
3. `mcp__neuron__read(vantage="<project>", k=20)` for a bounded context snapshot when
|
||||||
|
resuming known work.
|
||||||
|
|
||||||
|
## The Five Primitives (every significant task)
|
||||||
|
|
||||||
|
**Orchestrate → Execute → Learn → Build → Refine**, all routed through the 9-op surface:
|
||||||
|
- Orchestrate: `read(vantage=...)` for backlog/roadmap/process discovery, `attend()` for
|
||||||
|
what's currently live/salient.
|
||||||
|
- Execute: `write(type="state", ...)` to open/advance work, `relate()` to link it to
|
||||||
|
what it touches.
|
||||||
|
- Learn: `write(type="memory", ...)` **as you go, not batched**; `importance="critical"`
|
||||||
|
for architecture decisions.
|
||||||
|
- Build: `write(type="artifact"|"backlog", ...)`.
|
||||||
|
- Refine: `supersede(id=..., action="evolve"|"tombstone"|"promote", ...)` for
|
||||||
|
completions and lessons-learned; `learn(seeds=..., faculty="induce")` to recalibrate
|
||||||
|
the steering-prior, not as a session-notes dump.
|
||||||
|
|
||||||
|
## Architecture style — VBD, no exceptions
|
||||||
|
|
||||||
|
Volatility-Based Decomposition is THE style. Encapsulate volatility, not function. Full docs:
|
||||||
|
**`docs/architecture/`** — `00-overview`, `02-components`, `03-data-and-memory`,
|
||||||
|
`04-runtime-and-deployment`, `06-cognitive-architecture`, `07-storage-coherence-and-distribution`.
|
||||||
|
Verified component map: `routes.el` = HTTP dispatcher (`handle_request`), `soul.el` = boot +
|
||||||
|
layered cycle, `awareness.el` = awareness daemon, `sessions.el`/`memory.el`/`safety.el`/
|
||||||
|
`stewardship.el` = managers; `engram` (separate repo) = the persistence/graph engine.
|
||||||
|
|
||||||
|
## Hard operational rules
|
||||||
|
|
||||||
|
- **Never touch the live soul (`:7770`) or engram (`:8742`), `~/.neuron`, or live binaries.**
|
||||||
|
Experiment on **throwaway ports** with a **scratch `HOME`**. The soul binary defaults to
|
||||||
|
`HOME=~` (your real `~/.neuron`) and `NEURON_PORT=7770` (live) if invoked bare — **never**
|
||||||
|
invoke it without an override `HOME` and `NEURON_PORT` set. Leaving `ENGRAM_URL` unset is
|
||||||
|
verified safe (see `soul.el:590`, `using_http_engram` gates the only HTTP call to any
|
||||||
|
engram endpoint — confirmed by source trace during the 2026-08-15 audit, not just
|
||||||
|
observed behavior) — it does not fall back to any live/network default.
|
||||||
|
- **Immutability:** memory/knowledge is append-only — **supersede/tombstone, never hard-delete or
|
||||||
|
edit in place.** The engram is immutable by design.
|
||||||
|
- **gcloud** via the `terraform@` SA token; **never switch the active gcloud account**.
|
||||||
|
- **`tea` for Gitea**, never raw `curl` (Cloudflare Access blocks it).
|
||||||
|
- **No AI-attribution footers** in commits/PRs. Commit/push only when asked; branch off `main` first.
|
||||||
|
- **Multi-step work → sub-agent** to protect the context window.
|
||||||
|
|
||||||
|
## Build / regenerate `dist/soul.c` (audit-verified 2026-08-15, macOS arm64)
|
||||||
|
|
||||||
|
There is no committed regeneration script upstream of this audit. The recipe below is
|
||||||
|
verified: it reproduces the committed `dist/soul.c`'s exact symbol set byte-for-byte in
|
||||||
|
content (modulo genuinely new code), and the resulting binary boots and answers `/health`.
|
||||||
|
|
||||||
|
**The compiler toolchain** lives in the sibling `foundation` repo, not this one:
|
||||||
|
`foundation/el/lang/dist/platform/elc-darwin-arm64` (put it on `$PATH` as `elc`; `elb`
|
||||||
|
also exists there but is NOT the right tool for this repo — see gotcha below).
|
||||||
|
|
||||||
|
**⚠ elc gotcha #1 — stale `.elh` header caches silently truncate the build.** This repo
|
||||||
|
(and the `dist/` dir) ships committed `.elh` header files. `elc`/`elb` prefer an existing
|
||||||
|
`.elh` over recompiling its source when present, with NO warning or error when the cached
|
||||||
|
header is stale/truncated — the build "succeeds" with silently missing code (observed:
|
||||||
|
251-645 of 2541 real functions, depending on which `.elh` files were present, including
|
||||||
|
losing the entire 31-language NLG/morphology stack with exit code 0). **Delete every
|
||||||
|
`*.elh` in the repo root and `dist/` before regenerating**, every time.
|
||||||
|
|
||||||
|
**⚠ elc gotcha #2 — `elb` cannot produce this repo's single-TU `dist/soul.c`.** `elb`
|
||||||
|
does per-module separate compilation (`--out=DIR` writes one `.c`/`.elh` pair per
|
||||||
|
module; the default `--out` is also a directory, `dist/` itself). This codebase's
|
||||||
|
`.el` modules call each other's functions without forward declarations (relying on
|
||||||
|
`elc`'s own single-pass, whole-file forward-declaration emission), so per-module
|
||||||
|
compilation always fails with `implicit-function-declaration` errors across module
|
||||||
|
boundaries. **Use plain `elc` on one manually-flattened file, not `elb`.**
|
||||||
|
|
||||||
|
**⚠ elc gotcha #3 — the manual-concatenation path silently drops functions.** When
|
||||||
|
`elc` compiles a flat, hand-concatenated `.el` file, it silently drops (no error, no
|
||||||
|
declaration, no definition) the 1-2 top-level function definitions immediately
|
||||||
|
following any multi-line leading `//` comment block or file-boundary transition —
|
||||||
|
reproduced deterministically. **Insert two trivial buffer functions
|
||||||
|
(`fn __amalgam_buf_N__() -> Int { return 0 }`) after every concatenated file's
|
||||||
|
content**, then strip them back out of the generated `.c` before committing.
|
||||||
|
|
||||||
|
**The actual steps:**
|
||||||
|
1. Delete all `*.elh` in repo root and `dist/`.
|
||||||
|
2. Concatenate, with `import` lines stripped, in this order: `elp.el`'s own 34-file
|
||||||
|
NLG/morphology chain (`foundation/el/elp/src/` — the order is documented in
|
||||||
|
`elp.el`'s own header comment: language-profile, vocabulary, morphology, the 30
|
||||||
|
`morphology-XX.el` engines, grammar, realizer, semantics, then `elp.el` itself),
|
||||||
|
then this repo's 13 soul modules in `elb`'s own reported dependency order:
|
||||||
|
`persist, memory, safety, stewardship, imprint, awareness, chat, studio,
|
||||||
|
elp-input, neuron-api, sessions, routes, soul`. Insert the 2-function buffer
|
||||||
|
after every file (works around gotcha #3).
|
||||||
|
3. `elc <flat-file> > dist/soul.c` against the **pinned** vendor runtime headers
|
||||||
|
(`vendor/el-runtime/v1.0.0-20260501/` — see "why pinned" below), not
|
||||||
|
`foundation/el/lang/el-compiler/runtime/` (that's the bleeding-edge runtime;
|
||||||
|
using it drops symbols like `engram_prune_telemetry` that this soul still calls).
|
||||||
|
4. Strip the buffer functions back out of `dist/soul.c` (a small regex: drop every
|
||||||
|
`el_val_t __amalgam_buf_\d+__(void);` decl line and every matching 4-line
|
||||||
|
definition block).
|
||||||
|
5. `tools/soulc-stamp.sh --write` to record the new fingerprint.
|
||||||
|
6. `bash tools/build-soul-from-dist.sh dist/neuron` to compile+link with CI's exact
|
||||||
|
flags (this script now auto-detects Homebrew's `openssl@3` lib path on macOS —
|
||||||
|
see gotcha #4).
|
||||||
|
|
||||||
|
**⚠ gotcha #4 — macOS needs an explicit OpenSSL library path.** `cc ... -lssl -lcrypto
|
||||||
|
-lcurl ...` fails with `ld: library 'ssl' not found` on macOS because Homebrew's
|
||||||
|
`openssl@3` is keg-only. `tools/build-soul-from-dist.sh` now adds
|
||||||
|
`-L$(brew --prefix openssl@3)/lib` automatically on Darwin; CI's Ubuntu runner needs
|
||||||
|
no such flag (`apt-get install libcurl4-openssl-dev` puts it on the default path).
|
||||||
|
|
||||||
|
**⚠ Build-integrity (unchanged from before this audit):** `dist/soul.c` is committed
|
||||||
|
and generated. CI compiles it **directly and never regenerates it** (`elb`/`elc` on
|
||||||
|
Linux OOM the runner). So **any `.el` change to the soul MUST be followed by
|
||||||
|
regenerating `dist/soul.c` (steps above) and committing it** — otherwise CI ships
|
||||||
|
stale behavior, exactly as happened between commit `72e0b82` (Aug 9) and `main` HEAD
|
||||||
|
before this audit (`dist/soul.c` was missing PR #122's 459-line chat.el change, incl.
|
||||||
|
a "silently break chat" fix, until this pass regenerated and re-stamped it).
|
||||||
|
`tools/soulc-stamp.sh --check` is the gate that catches this — **note it is currently
|
||||||
|
`continue-on-error: true` in CI** ("relaxed... during active cultivation", 2026-08-15),
|
||||||
|
so it reports but does not block; re-harden before it needs to actually stop a bad ship.
|
||||||
|
|
||||||
|
- **Tests:** El contract suite in `tests/*.el` (e.g. `test_layer_contract.el`, `test_safety.el`,
|
||||||
|
`test_sessions.el`, `test_soul_guard.el`). Run against a throwaway soul, never the live one.
|
||||||
|
- **Port topology (confirmed live, 2026-08-15):** soul `:7770`, engram `:8742`,
|
||||||
|
mcp-wrapper `:17779` (`MCP_PORT` env override in its LaunchAgent; source default is
|
||||||
|
`7779`), mcp-proxy `:7779` (the stable front door Claude Code actually connects to).
|
||||||
|
**`:7771` is a live three-way collision, not a single well-defined port** — `axon`
|
||||||
|
(soul.el's Rust backlog/memory/knowledge proxy, unbuilt), `neuron-connectd` (the MCP
|
||||||
|
connector sidecar `routes.el`/`chat.el` call — unbuilt; a local-dev stub now exists at
|
||||||
|
`connectd/`), and `council` (`council/`, an anti-confabulation LLM-voting service —
|
||||||
|
the one actually bound to `:7771` in Will's live environment) are all hardcoded to it.
|
||||||
|
See `connectd/README.md` for the full trace and the open question this leaves for Will.
|
||||||
|
- **Deploy:** merge to `main` → `.gitea/workflows/ci.yaml` builds + publishes `neuron-soul@<sha8>`
|
||||||
|
and blue/green-deploys to GKE `neuron-prod` via `scripts/blue-green-deploy.sh`. Self-improvement
|
||||||
|
experiments go to **stage** first (snapshot prod DB → deploy stage → verify → blue/green promote).
|
||||||
|
|
||||||
|
## Git / CI / deploy workflow
|
||||||
|
|
||||||
|
See **`../GITOPS.md`** (repo-family GitOps README): branch model, required checks, blue/green,
|
||||||
|
Cloud Run, Terraform/ESO/Vault, and the pack-objects/crawler incident runbook.
|
||||||
+139
@@ -0,0 +1,139 @@
|
|||||||
|
# PORT-NOTES — openai tools port working state (2026-08-06, session handoff-safe)
|
||||||
|
|
||||||
|
Spec: `docs/specs/SPEC-soul-openai-tools-v2-2026-08-06.md` (Tim-approved 2026-08-06). Tasks #1-5
|
||||||
|
tracked in-session (1 ✓ wiring verdict, 2 ✓ stub rig, 3 in-progress = THIS, 4-5 pending).
|
||||||
|
Worktree: HERE (`_wt-openai-tools`, branch `feat/soul-openai-tools-v2` @ dba755d). Round-9 trees
|
||||||
|
READ-ONLY. Nothing committed yet.
|
||||||
|
|
||||||
|
## Step-0 verdict (evidence in journal note ncli-653ba964dd76)
|
||||||
|
Shipped app never wires the v1 lane: launcher exports `SOUL_LLM_MODEL/PROVIDER/BASE_URL` +
|
||||||
|
`ANTHROPIC_API_KEY`+`SOUL_API_KEY` (= Keychain key for WHATEVER provider; installer/macos/
|
||||||
|
neuron-daemons.sh:288-300 on hotfix/beta-round9); brain reads only SOUL_LLM_MODEL (chat.el:8) and
|
||||||
|
NEURON_LLM_0_* (chat.el:1768-1794) which nothing sets. `/api/config` PATCH ignores llm_* fields
|
||||||
|
(studio.el:36 handle_config: POST-only, reads model/provider/api_key only).
|
||||||
|
**Bridge = brain-side ONLY (zero app-repo edits, zero round-9 collision):**
|
||||||
|
- `llm_base_url()`: NEURON_LLM_0_URL → fallback SOUL_LLM_BASE_URL when SOUL_LLM_PROVIDER ∉ {"","anthropic"}
|
||||||
|
- `llm_wire_format()`: NEURON_LLM_0_FORMAT → fallback derive from SOUL_LLM_PROVIDER (openai/grok/gemini/groq/ollama → "openai"; else "anthropic")
|
||||||
|
- `agentic_api_key()`: already works (ANTHROPIC_API_KEY carries the provider key); add NEURON_LLM_0_KEY → SOUL_API_KEY fallback.
|
||||||
|
|
||||||
|
## Design pins (stub asserts these — stub is green 58/58, tests/gate-openai/)
|
||||||
|
- Request MUST send `"tool_choice":"auto"` (string) + `"parallel_tool_calls":false` explicitly.
|
||||||
|
- `arguments` in tool_calls = JSON-ENCODED STRING; decode ONCE via json_get → feed dispatch_tool
|
||||||
|
verbatim. Stub's echo-mismatch check catches double-encode/decode (two-escaper trap).
|
||||||
|
- Assistant echo turn: `{"role":"assistant","content":null,"tool_calls":[...]}` VERBATIM from response.
|
||||||
|
- Feedback: `{"role":"tool","tool_call_id":"<id>","content":"<result string>"}`.
|
||||||
|
- Resume must NOT re-answer an answered id (stub 400s on repeat tool_call_id).
|
||||||
|
- Parallel tool_calls in a response: take FIRST only + log skip (mirror ADR-0005 stopgap); stub
|
||||||
|
scenario `parallel` proves behavior.
|
||||||
|
- No tools in request when tools array empty/absent turns (boot probes) — stub defaults tolerate.
|
||||||
|
|
||||||
|
## el idioms confirmed (from openai_chat_complete :1808-1854 + agentic_loop :2751-2838)
|
||||||
|
- JSON: `json_get(s,k)` decoded string · `json_get_raw(s,k)` raw subtree · `json_array_len` ·
|
||||||
|
`json_array_get(arr,i)` · build by string concat + `json_escape()` (:1797, OpenAI-lane escaper).
|
||||||
|
- HTTP: `let h: Map = {}` + `map_set(h,k,v)` + `http_post_with_headers(url, body, h)`;
|
||||||
|
Bearer auth via `Authorization` header when key non-empty (:1825-1830).
|
||||||
|
- Loop-carried vars must be top-level locals in the fn, mutated as if-expressions at while-body
|
||||||
|
top level (see :2760-2791 pattern + comment :2903-2904 region).
|
||||||
|
- Error shape: `str_starts_with(raw,"{\"error\"") || str_contains(raw,"\"error\":")` → return
|
||||||
|
`{"error":"llm unavailable","reply":""}` (:1835-1838).
|
||||||
|
|
||||||
|
## Remaining read map (before writing the fork)
|
||||||
|
- chat.el 2840-3200: block walk (2923-3000), policy gate (3009-3023: classify_tool_risk /
|
||||||
|
is_builtin_tool / ask_all / tool_auto_approved → needs_bridge), dispatch_tool call (3025),
|
||||||
|
tool_result feedback (3031, 3067-3072), run-progress ledger append (3078-3087), bridge_save
|
||||||
|
(3182), loop end + done envelope (~3100-3200).
|
||||||
|
- agentic_resume 3227-3293 (hardcoded Anthropic headers to make wire-aware; blob gets `wire` field,
|
||||||
|
legacy default anthropic) · handle_tool_result 3293+ · dharma fork site 3465 (calls agentic_loop
|
||||||
|
direct, no use_openai check today).
|
||||||
|
|
||||||
|
## Write plan (order)
|
||||||
|
1. Env fallbacks (edit llm_base_url/llm_wire_format/agentic_api_key) — small, first, testable alone.
|
||||||
|
2. `openai_tools_json(anthropic_tools: String) -> String` converter (walk array; per entry build
|
||||||
|
{"type":"function","function":{name,description,parameters:input_schema-raw}}).
|
||||||
|
3. `openai_agentic_loop(...)` fork: same signature as agentic_loop minus Anthropic-only params;
|
||||||
|
INCLUDE run-progress ledger + tools_log + iteration cap 12; NO container_id/ws_drift/web_search
|
||||||
|
(out of scope; strip web_search entry from tools via agentic_tools_literal()+connector merge,
|
||||||
|
NOT _with_web()).
|
||||||
|
4. Fork sites ×3: handle_chat_agentic :2695-2700 (route agentic to new loop when use_openai);
|
||||||
|
dharma :3465; agentic_resume wire-branch.
|
||||||
|
5. `chat.elh` extern decls. 6. Compile (recipe: dist/ + elc/elb per neuron-soul-build-deploy memory;
|
||||||
|
round-9 tree soul.c regen'd 08-06 proves toolchain live). 7. Gate: stub selftest recipe in
|
||||||
|
tests/gate-openai/README.md. 8. Anthropic-lane regression via gate9 (READ-ONLY consume from
|
||||||
|
_wt-beta-round9). 9. Live Groq E2E (scratch profile, free port, key via Keychain read-only).
|
||||||
|
|
||||||
|
## BUILD RECIPE — CORRECTED 2026-08-06 (the June memory is STALE for August code)
|
||||||
|
`~/el-sdk/el_runtime.c` (Jun 15) is MISSING builtins the Aug engine calls (`engram_wm_count`,
|
||||||
|
`engram_wm_top_json`, `http_delete_json`, `http_serve_async`) → link fails with
|
||||||
|
"symbol(s) not found for architecture arm64". Use the REPO-PINNED runtime:
|
||||||
|
```
|
||||||
|
mkdir -p <scratch>
|
||||||
|
elb --elc=$HOME/el-sdk/elc --runtime=vendor/el-runtime/v1.0.0-20260501 --out=<scratch>/
|
||||||
|
# "elb: link failed" at the end is EXPECTED and harmless — the per-module .c files are produced
|
||||||
|
cc -std=c11 -O1 -DHAVE_CURL -rdynamic \
|
||||||
|
-I vendor/el-runtime/v1.0.0-20260501 -I <scratch> -I /opt/homebrew/opt/openssl@3/include \
|
||||||
|
-L /opt/homebrew/opt/openssl@3/lib \
|
||||||
|
-include dist/elp-c-decls.h -Wno-error=implicit-function-declaration \
|
||||||
|
-o <scratch>/soul <scratch>/*.c vendor/el-runtime/v1.0.0-20260501/el_runtime.c \
|
||||||
|
-lssl -lcrypto -lcurl -lpthread -lm
|
||||||
|
```
|
||||||
|
Source: `_engine-plainchat-20260805/README.md:396-412`. Verified today: 0 errors, 887,296 B.
|
||||||
|
`elb` ALSO rewrites every `*.elh` in the tree (cosmetic em-dash→hyphen in the auto-gen banner,
|
||||||
|
plus true-ups) and drops a stray `soul..elh` — `git restore` the unrelated ones and delete the
|
||||||
|
stray before staging, or the diff drowns in noise.
|
||||||
|
|
||||||
|
## SELF-REVIEW FIX LIST (found by reading my own diff, 2026-08-06 — apply in ONE batch, then rebuild once)
|
||||||
|
- **F3 (CORRECTNESS, do first):** the assistant echo currently replays the provider's FULL
|
||||||
|
`tool_calls` array (`tc_arr`) while the loop answers only the FIRST call. If a provider ignores
|
||||||
|
`parallel_tool_calls:false`, the next request carries an assistant turn with N tool_calls and
|
||||||
|
only ONE `role:"tool"` response → most OpenAI-format providers 400 ("missing tool response for
|
||||||
|
id X") and the run dies. This is the same class as ADR-0005's Anthropic failure, but here it is
|
||||||
|
cheap to close: echo ONLY the honored call (`"[" + tc0 + "]"`), so the conversation we send is
|
||||||
|
self-consistent and the dropped call never existed from the model's view. The DRIFT log line
|
||||||
|
stays (honest accounting of what we dropped).
|
||||||
|
- **F4 (efficiency/latency):** `handle_chat_agentic` computes `agentic_tools_all()` at ~:2681
|
||||||
|
BEFORE the fork, then the OpenAI branch computes `agentic_tools_no_web()` again — two
|
||||||
|
`connector_tools_json()` calls per turn, each an HTTP round-trip to the connector bridge on
|
||||||
|
:7771 (two timeout exposures). Fix: compute the tools array ONCE, per lane, after `use_openai`
|
||||||
|
is known (check no other use of `tools_json` sits between :2681 and the fork before moving it).
|
||||||
|
Note: `openai_tools_json()` already skips any entry with no `input_schema`, so Anthropic's
|
||||||
|
server-side `web_search` entry is auto-dropped even if the full array is passed —
|
||||||
|
`agentic_tools_no_web()` is kept for EXPLICITNESS, not necessity.
|
||||||
|
- **F1 (debuggability):** the "no choices in response" branch logs a generic string and discards
|
||||||
|
the body. Log the response head (as the `is_error` branch does) — a provider that returns 200
|
||||||
|
with an unexpected shape is otherwise undiagnosable from the log.
|
||||||
|
- **OPEN QUESTION (evidence pending from the gate):** the tool-result feedback turn escapes with
|
||||||
|
`json_escape()` (this lane's escaper) rather than `json_safe()` (used everywhere else). The
|
||||||
|
Anthropic lane escapes that field with NEITHER, which is a latent defect on that side. If the
|
||||||
|
torture scenario shows any escaping loss, switch to `json_safe` and note the Anthropic-side
|
||||||
|
finding for Will.
|
||||||
|
|
||||||
|
## TEST HARNESS — built 2026-08-06 (Task 4 side-work, reusable by anyone)
|
||||||
|
- `tests/run-el-test.sh <tests/test_x.el> | --all` — the engine tests were NEVER runnable
|
||||||
|
before this (`elc` is a compiler: emits C to stdout and exits). It emits the test to C,
|
||||||
|
compiles `soul.c` separately with `main` renamed away (soul.c owns the daemon's real main
|
||||||
|
but also defines `layered_cycle` et al.), links the remaining modules + the repo-pinned
|
||||||
|
runtime, and executes. Modules cached under `/tmp/el-test-<worktree>/`; `REBUILD=1` forces.
|
||||||
|
- **The runner computes the verdict itself** because the test FILES cannot: all 9 counted
|
||||||
|
test files do `let pass_count = pass_count + 1` inside an if BLOCK, which El scoping
|
||||||
|
discards, so every summary line reads `0 passed, 0 failed` forever. Per-assertion
|
||||||
|
`PASS:`/`FAIL:` lines ARE reliable; the runner counts those, exits non-zero on any FAIL
|
||||||
|
or on zero assertions, and was proven to discriminate with a negative control (broken
|
||||||
|
assertion → 31 passed / 1 failed / exit 1). Real in-file fix filed: **neuron#116**.
|
||||||
|
- `tests/test_bridge_serialization.el`: 4 `bridge_save` calls updated for the new `wire`
|
||||||
|
argument, plus **Section 9** (8 new assertions) covering wire round-trip both ways, the
|
||||||
|
legacy no-wire blob (resumes as anthropic), and a FIELD-ORDER decoy guard — a fake
|
||||||
|
`"wire":"anthropic"` planted inside `messages_raw` must not beat the blob's own scalar.
|
||||||
|
That decoy is the round-9 first-match-scanner bug class, now pinned by a test. **32/32 green.**
|
||||||
|
|
||||||
|
## MEMORY-SAVE CAVEAT RESOLVED 2026-08-06
|
||||||
|
Earlier saves this session reported `-> OUTBOX only (real mind unreachable or read-back
|
||||||
|
failed)`. That was a **read-back verifier false negative, not data loss** — a direct
|
||||||
|
`POST :7770/api/neuron/recall` returns those notes from the live mind verbatim. Another
|
||||||
|
terminal was fixing exactly this (multi-word read-back probe) the same afternoon. Do NOT
|
||||||
|
re-save on an OUTBOX report without first querying the mind directly, or you duplicate nodes.
|
||||||
|
|
||||||
|
## Standing cautions
|
||||||
|
- PERSIST OFF on the real mind this boot (neuron#98/#92): journal saves only, ferry later. MCP link
|
||||||
|
down this terminal; use neuron_remember.py / neuron_recall.py.
|
||||||
|
- Aug-16: Groq retires llama-3.3-70b-versatile (separate P0, Tim's call, catalog swap).
|
||||||
|
- Never bind 7770/7779/17779; never touch ~/.neuron; round-9 worktrees read-only.
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
# neuron
|
||||||
|
|
||||||
|
The canonical CGI substrate: the **soul** (the running agent), the **engram** (its memory
|
||||||
|
graph), and the MCP proxy/wrapper that expose it. See `AGENTS.md` for detail, including
|
||||||
|
the audit-verified local build/regenerate recipe and known local-build gotchas.
|
||||||
|
|
||||||
|
## Quick local build
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. dist/soul.c must match current .el sources — this refuses otherwise:
|
||||||
|
bash tools/build-soul-from-dist.sh dist/neuron
|
||||||
|
|
||||||
|
# 2. If it refuses (stale amalgam), regenerate first — see AGENTS.md's
|
||||||
|
# "Build / regenerate dist/soul.c" section for the full, gotcha-laden recipe.
|
||||||
|
```
|
||||||
|
|
||||||
|
For a full local dev stack (soul + engram + mcp-wrapper + mcp-proxy, wired into Claude
|
||||||
|
Code) see `neuron-dev-setup/README.md` instead — this repo alone only builds the soul.
|
||||||
|
|
||||||
|
## Code vs. Artifact
|
||||||
|
- **Authored source:** `*.el` + `*.elh` at the repo root plus `cli/`, `council/`,
|
||||||
|
`connectd/`, `mcp-proxy/`, `mcp-wrapper/` — edit here.
|
||||||
|
- **Artifacts (do not hand-edit):** `dist/soul.c` (generated single-TU amalgam —
|
||||||
|
regenerate via the recipe in `AGENTS.md`, then `tools/soulc-stamp.sh --write`) and
|
||||||
|
the `dist/neuron` binary it compiles to.
|
||||||
|
- **Release:** git tag `neuron-vX.Y.Z` on this repo. No `releases/` folders.
|
||||||
|
|
||||||
|
See org policy: `docs/CODE-VS-ARTIFACT.md`.
|
||||||
@@ -1,26 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn idle_count() -> Int
|
|
||||||
extern fn idle_inc() -> Int
|
|
||||||
extern fn idle_reset() -> Void
|
|
||||||
extern fn ise_post(content: String) -> Void
|
|
||||||
extern fn elapsed_ms() -> Int
|
|
||||||
extern fn elapsed_human() -> String
|
|
||||||
extern fn embed_ok() -> Int
|
|
||||||
extern fn emit_heartbeat() -> Void
|
|
||||||
extern fn auto_term_try_slot(slot_type: String, slot_lbl: String) -> Void
|
|
||||||
extern fn proactive_curiosity() -> Bool
|
|
||||||
extern fn pulse_count() -> Int
|
|
||||||
extern fn pulse_inc() -> Int
|
|
||||||
extern fn make_action(kind: String, payload: String) -> String
|
|
||||||
extern fn perceive() -> String
|
|
||||||
extern fn attend(node_json: String) -> String
|
|
||||||
extern fn respond(action_json: String) -> String
|
|
||||||
extern fn record(outcome_json: String) -> Void
|
|
||||||
extern fn one_cycle() -> Bool
|
|
||||||
extern fn awareness_run() -> Void
|
|
||||||
extern fn security_research_authorized() -> Bool
|
|
||||||
extern fn threat_score_command(cmd: String) -> Int
|
|
||||||
extern fn threat_score_path(path: String) -> Int
|
|
||||||
extern fn threat_score_history(history: String) -> Int
|
|
||||||
extern fn threat_trajectory_check(tool_name: String, tool_input: String) -> Int
|
|
||||||
extern fn threat_history_append(text: String) -> Void
|
|
||||||
@@ -1408,7 +1408,7 @@ fn session_preload_bullets(nodes: String, max_bullets: Int, snip_len: Int) -> St
|
|||||||
while i < limit {
|
while i < limit {
|
||||||
let node: String = json_array_get(nodes, i)
|
let node: String = json_array_get(nodes, i)
|
||||||
let content: String = json_get(node, "content")
|
let content: String = json_get(node, "content")
|
||||||
let snip: String = if str_len(content) > snip_len { str_slice(content, 0, snip_len) } else { content }
|
let snip: String = utf8_safe_slice(content, snip_len)
|
||||||
let bullets = if str_eq(snip, "") {
|
let bullets = if str_eq(snip, "") {
|
||||||
bullets
|
bullets
|
||||||
} else {
|
} else {
|
||||||
@@ -1770,7 +1770,14 @@ fn agentic_api_key() -> String {
|
|||||||
if !str_eq(k1, "") {
|
if !str_eq(k1, "") {
|
||||||
return k1
|
return k1
|
||||||
}
|
}
|
||||||
return env("NEURON_LLM_0_KEY")
|
let k2: String = env("NEURON_LLM_0_KEY")
|
||||||
|
if !str_eq(k2, "") {
|
||||||
|
return k2
|
||||||
|
}
|
||||||
|
// Step-0 bridge (2026-08-06): the shipped launcher also exports the Keychain key as
|
||||||
|
// SOUL_API_KEY (neuron-daemons.sh). Honor it so a provider key configured through the
|
||||||
|
// app reaches this lane without any launcher change.
|
||||||
|
return env("SOUL_API_KEY")
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── OpenAI-compatible providers (Ollama / OpenAI / Grok / Gemini) ──────────────────────────────
|
// ── OpenAI-compatible providers (Ollama / OpenAI / Grok / Gemini) ──────────────────────────────
|
||||||
@@ -1778,20 +1785,43 @@ fn agentic_api_key() -> String {
|
|||||||
// OpenAI-compatible wire format (NEURON_LLM_0_FORMAT=openai) with a configured base URL
|
// OpenAI-compatible wire format (NEURON_LLM_0_FORMAT=openai) with a configured base URL
|
||||||
// (NEURON_LLM_0_URL, e.g. http://localhost:11434/v1 for local Ollama), basic chat turns are served
|
// (NEURON_LLM_0_URL, e.g. http://localhost:11434/v1 for local Ollama), basic chat turns are served
|
||||||
// here instead of the Anthropic agentic loop.
|
// here instead of the Anthropic agentic loop.
|
||||||
// v1 SCOPE: plain chat completion only — NO tools / agentic loop yet (that is a follow-up port).
|
// v2 SCOPE (2026-08-06, SPEC-soul-openai-tools-v2): tools + the agentic loop now run on
|
||||||
// This block is ADDITIVE: the Anthropic path is untouched and stays the default.
|
// this wire too (openai_agentic_loop below). Plain completion (openai_chat_complete)
|
||||||
|
// remains for non-agentic turns. Still ADDITIVE: the Anthropic path is untouched.
|
||||||
|
|
||||||
fn llm_base_url() -> String {
|
fn llm_base_url() -> String {
|
||||||
return env("NEURON_LLM_0_URL")
|
let u: String = env("NEURON_LLM_0_URL")
|
||||||
|
if !str_eq(u, "") {
|
||||||
|
return u
|
||||||
|
}
|
||||||
|
// Step-0 bridge (2026-08-06): the shipped launcher exports SOUL_LLM_BASE_URL +
|
||||||
|
// SOUL_LLM_PROVIDER (installer/macos/neuron-daemons.sh:288-300) and nothing in a
|
||||||
|
// customer build exports the NEURON_LLM_0_* names — so this lane was unreachable
|
||||||
|
// outside test harnesses. Honor the launcher's names as a fallback. Anthropic
|
||||||
|
// deliberately returns "" here: its native path stays hardcoded (endpoint
|
||||||
|
// configurability is neuron#62, out of scope).
|
||||||
|
let p: String = env("SOUL_LLM_PROVIDER")
|
||||||
|
if str_eq(p, "") || str_eq(p, "anthropic") {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return env("SOUL_LLM_BASE_URL")
|
||||||
}
|
}
|
||||||
|
|
||||||
fn llm_wire_format() -> String {
|
fn llm_wire_format() -> String {
|
||||||
let f: String = env("NEURON_LLM_0_FORMAT")
|
let f: String = env("NEURON_LLM_0_FORMAT")
|
||||||
if str_eq(f, "") {
|
if !str_eq(f, "") {
|
||||||
return "anthropic"
|
|
||||||
}
|
|
||||||
return f
|
return f
|
||||||
}
|
}
|
||||||
|
// Step-0 bridge (2026-08-06): derive the wire format from the launcher's provider
|
||||||
|
// name when the explicit format is unset. Every non-Anthropic provider in the app's
|
||||||
|
// catalog speaks the OpenAI-compatible format (ProviderKeys.kt: llmFormat="openai"
|
||||||
|
// for openai/grok/gemini/groq/ollama).
|
||||||
|
let p: String = env("SOUL_LLM_PROVIDER")
|
||||||
|
if str_eq(p, "openai") || str_eq(p, "grok") || str_eq(p, "gemini") || str_eq(p, "groq") || str_eq(p, "ollama") {
|
||||||
|
return "openai"
|
||||||
|
}
|
||||||
|
return "anthropic"
|
||||||
|
}
|
||||||
|
|
||||||
// Escape a decoded string so it can be embedded back into a JSON string literal.
|
// Escape a decoded string so it can be embedded back into a JSON string literal.
|
||||||
fn json_escape(s: String) -> String {
|
fn json_escape(s: String) -> String {
|
||||||
@@ -1853,6 +1883,354 @@ fn openai_chat_complete(model: String, base_url: String, api_key: String, safe_s
|
|||||||
return "{\"reply\":\"" + json_escape(content) + "\",\"tools_used\":[]}"
|
return "{\"reply\":\"" + json_escape(content) + "\",\"tools_used\":[]}"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ══ OpenAI-format TOOLS PORT (v2, 2026-08-06, SPEC-soul-openai-tools-v2) ═══════════════
|
||||||
|
// The agentic loop for OpenAI-compatible providers (Groq/OpenAI/Grok/Gemini/Ollama).
|
||||||
|
// The tool-execution, consent, bridge and run-progress machinery is the SAME wire-agnostic
|
||||||
|
// layer agentic_loop uses (dispatch_tool, classify_tool_risk, is_builtin_tool, bridge_save,
|
||||||
|
// handle_tool_result) — only the wire dialect differs. ADR-0005's single-tool constraint is
|
||||||
|
// mirrored on this wire as parallel_tool_calls:false; a provider that ignores it gets its
|
||||||
|
// first call honored and the rest dropped LOUDLY. Anthropic's server-side web_search has no
|
||||||
|
// analogue here, so this lane's tool set comes from agentic_tools_no_web() and "sources"
|
||||||
|
// is always empty — an honest degradation, disclosed in the spec, not a bug.
|
||||||
|
|
||||||
|
// Convert an Anthropic-shape tools array ({"name","description","input_schema"}) to the
|
||||||
|
// OpenAI shape ({"type":"function","function":{"name","description","parameters"}}).
|
||||||
|
// Entries without an input_schema (Anthropic server tools like web_search) are skipped —
|
||||||
|
// they cannot execute on this wire.
|
||||||
|
fn openai_tools_json(tools_anthropic: String) -> String {
|
||||||
|
let out: String = ""
|
||||||
|
let i: Int = 0
|
||||||
|
let n: Int = json_array_len(tools_anthropic)
|
||||||
|
while i < n {
|
||||||
|
let entry: String = json_array_get(tools_anthropic, i)
|
||||||
|
let name: String = json_get(entry, "name")
|
||||||
|
let desc: String = json_get(entry, "description")
|
||||||
|
let schema: String = json_get_raw(entry, "input_schema")
|
||||||
|
let keep: Bool = !str_eq(name, "") && !str_eq(schema, "")
|
||||||
|
let piece: String = if keep {
|
||||||
|
"{\"type\":\"function\",\"function\":{\"name\":\"" + json_escape(name) + "\""
|
||||||
|
+ ",\"description\":\"" + json_escape(desc) + "\""
|
||||||
|
+ ",\"parameters\":" + schema + "}}"
|
||||||
|
} else { "" }
|
||||||
|
let out = if keep {
|
||||||
|
if str_eq(out, "") { piece } else { out + "," + piece }
|
||||||
|
} else { out }
|
||||||
|
let i = i + 1
|
||||||
|
}
|
||||||
|
return "[" + out + "]"
|
||||||
|
}
|
||||||
|
|
||||||
|
// utf8_safe_slice — str_slice with the guarantee that it never splits a character.
|
||||||
|
//
|
||||||
|
// str_slice and str_len count BYTES. Every fixed-length content cut in this file
|
||||||
|
// therefore risks landing inside a multi-byte UTF-8 character and leaving a dangling
|
||||||
|
// lead byte, which makes the ENTIRE request body invalid UTF-8 — providers reject it
|
||||||
|
// and the user gets an unexplained failure. Found live 2026-08-06 in the session
|
||||||
|
// preload: a recalled memory containing box-drawing rules (U+2500 = E2 94 80) was cut
|
||||||
|
// at 350 bytes mid-character, and every turn on that session died. Ordinary content
|
||||||
|
// triggers it — an em dash, a curly quote, an accented name, an emoji — and it gets
|
||||||
|
// MORE likely as a user's memory grows.
|
||||||
|
//
|
||||||
|
// Walk back from the cut over UTF-8 continuation bytes (0x80-0xBF) to the lead byte,
|
||||||
|
// and keep the character only if all of its bytes survived the cut.
|
||||||
|
fn utf8_safe_slice(s: String, n: Int) -> String {
|
||||||
|
if str_len(s) <= n { return s }
|
||||||
|
let cut: String = str_slice(s, 0, n)
|
||||||
|
let total: Int = str_len(cut)
|
||||||
|
let i: Int = total - 1
|
||||||
|
let keep: Int = total
|
||||||
|
let scanning: Bool = true
|
||||||
|
let steps: Int = 0
|
||||||
|
// A UTF-8 character is at most 4 bytes, so at most 4 steps are ever needed.
|
||||||
|
while scanning && steps < 4 && i >= 0 {
|
||||||
|
let c: Int = str_char_code(cut, i)
|
||||||
|
let is_ascii: Bool = c < 128
|
||||||
|
let is_lead: Bool = c >= 192
|
||||||
|
// Expected length declared by the lead byte: 0xF0+ = 4, 0xE0+ = 3, else 2.
|
||||||
|
let need: Int = if c >= 240 { 4 } else { if c >= 224 { 3 } else { 2 } }
|
||||||
|
let have: Int = total - i
|
||||||
|
let keep = if is_ascii { total } else {
|
||||||
|
if is_lead { if have == need { total } else { i } } else { keep }
|
||||||
|
}
|
||||||
|
let scanning = if is_ascii || is_lead { false } else { true }
|
||||||
|
let i = i - 1
|
||||||
|
let steps = steps + 1
|
||||||
|
}
|
||||||
|
return str_slice(cut, 0, keep)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A tool result arrives already json_safe'd from dispatch_tool, so it is embedded into
|
||||||
|
// the wire message RAW (escaping it a second time is what made the model read literal
|
||||||
|
// backslashes). But it is also TRUNCATED at a fixed byte count, and a cut can land in the
|
||||||
|
// middle of an escape pair — leaving a dangling backslash that makes the enclosing JSON
|
||||||
|
// string invalid and 400s the whole turn. Trim any trailing backslash run so the cut is
|
||||||
|
// always on a clean boundary. (The Anthropic lane truncates the same way and has the same
|
||||||
|
// latent exposure; not changed here, flagged in the PR.)
|
||||||
|
fn json_trim_dangling_escape(s: String) -> String {
|
||||||
|
let out: String = s
|
||||||
|
while str_ends_with(out, "\\") {
|
||||||
|
let out = str_slice(out, 0, str_len(out) - 1)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// The standard agentic tool set WITHOUT Anthropic's native web_search entry: built-ins +
|
||||||
|
// every connector tool. Same merge as agentic_tools_all(), minus the server-tool tail.
|
||||||
|
fn agentic_tools_no_web() -> String {
|
||||||
|
let base: String = agentic_tools_literal()
|
||||||
|
let conn: String = connector_tools_json()
|
||||||
|
let base_inner: String = str_slice(base, 1, str_len(base) - 1)
|
||||||
|
let conn_inner: String = str_slice(conn, 1, str_len(conn) - 1)
|
||||||
|
let merged: String = if str_eq(conn_inner, "") {
|
||||||
|
base_inner
|
||||||
|
} else {
|
||||||
|
base_inner + "," + conn_inner
|
||||||
|
}
|
||||||
|
return "[" + strip_client_web_search(merged) + "]"
|
||||||
|
}
|
||||||
|
|
||||||
|
// openai_agentic_loop — the resumable agentic turn on the OpenAI wire. Same two envelopes
|
||||||
|
// as agentic_loop (done / tool_pending), same client-bridge contract, same state keys.
|
||||||
|
// [tools_json] arrives ANTHROPIC-shaped (the bridge blob stays wire-uniform); it is
|
||||||
|
// converted once here. The system prompt travels as the first message (no top-level
|
||||||
|
// "system" on this wire).
|
||||||
|
fn openai_agentic_loop(session_id: String, model: String, safe_sys: String, tools_json: String, messages_in: String, tools_log_in: String) -> String {
|
||||||
|
let api_url: String = llm_base_url() + "/chat/completions"
|
||||||
|
let api_key: String = agentic_api_key()
|
||||||
|
let h: Map = {}
|
||||||
|
map_set(h, "content-type", "application/json")
|
||||||
|
if !str_eq(api_key, "") {
|
||||||
|
map_set(h, "Authorization", "Bearer " + api_key)
|
||||||
|
}
|
||||||
|
let ask_all: Bool = !str_eq(session_id, "") && str_eq(state_get("require_approval_" + session_id), "true")
|
||||||
|
let tools_oai: String = openai_tools_json(tools_json)
|
||||||
|
let has_tools: Bool = json_array_len(tools_oai) > 0
|
||||||
|
|
||||||
|
let messages: String = messages_in
|
||||||
|
let final_text: String = ""
|
||||||
|
let tools_log: String = tools_log_in
|
||||||
|
let iteration: Int = 0
|
||||||
|
let keep_going: Bool = true
|
||||||
|
|
||||||
|
// Suspension state — top level so it escapes the while body (El scope rule).
|
||||||
|
let pending: Bool = false
|
||||||
|
let pend_tool_id: String = ""
|
||||||
|
let pend_tool_name: String = ""
|
||||||
|
let pend_tool_input: String = ""
|
||||||
|
let pend_tool_tier: String = ""
|
||||||
|
let pend_narration: String = ""
|
||||||
|
|
||||||
|
if !str_eq(session_id, "") {
|
||||||
|
state_set("run_progress_" + session_id, "")
|
||||||
|
}
|
||||||
|
|
||||||
|
while keep_going && iteration < 12 {
|
||||||
|
let inner_msgs: String = str_slice(messages, 1, str_len(messages) - 1)
|
||||||
|
let all_msgs: String = if str_eq(inner_msgs, "") {
|
||||||
|
"[{\"role\":\"system\",\"content\":\"" + safe_sys + "\"}]"
|
||||||
|
} else {
|
||||||
|
"[{\"role\":\"system\",\"content\":\"" + safe_sys + "\"}," + inner_msgs + "]"
|
||||||
|
}
|
||||||
|
// tools + the ADR-0005 mirror travel only when there are tools to offer: an empty
|
||||||
|
// tools array is a 400 on real OpenAI-format providers.
|
||||||
|
let tool_frag: String = if has_tools {
|
||||||
|
",\"tools\":" + tools_oai + ",\"tool_choice\":\"auto\",\"parallel_tool_calls\":false"
|
||||||
|
} else { "" }
|
||||||
|
let req_body: String = "{\"model\":\"" + model + "\""
|
||||||
|
+ ",\"max_tokens\":16384"
|
||||||
|
+ tool_frag
|
||||||
|
+ ",\"messages\":" + all_msgs
|
||||||
|
+ "}"
|
||||||
|
|
||||||
|
let raw_resp: String = http_post_with_headers(api_url, req_body, h)
|
||||||
|
// OpenAI-format errors arrive as a top-level {"error":{...}} object. Content
|
||||||
|
// strings inside a valid response are JSON-escaped, so a top-level match cannot
|
||||||
|
// false-positive on reply text.
|
||||||
|
let is_error: Bool = str_eq(raw_resp, "") || str_starts_with(raw_resp, "{\"error\"")
|
||||||
|
if is_error {
|
||||||
|
let err_head: String = if str_len(raw_resp) > 220 { str_slice(raw_resp, 0, 220) } else { raw_resp }
|
||||||
|
println("[soul] llm error (openai lane): " + err_head)
|
||||||
|
return "{\"error\":\"llm unavailable\",\"reply\":\"\"}"
|
||||||
|
}
|
||||||
|
|
||||||
|
let choices: String = json_get_raw(raw_resp, "choices")
|
||||||
|
let eff_choices: String = if str_eq(choices, "") { "[]" } else { choices }
|
||||||
|
if json_array_len(eff_choices) < 1 {
|
||||||
|
// Log the body head, as the error branch does. A provider that answers 200
|
||||||
|
// with an unexpected shape is otherwise undiagnosable from the log alone.
|
||||||
|
let noc_head: String = if str_len(raw_resp) > 220 { str_slice(raw_resp, 0, 220) } else { raw_resp }
|
||||||
|
println("[soul] llm error (openai lane): no choices in response: " + noc_head)
|
||||||
|
return "{\"error\":\"llm unavailable\",\"reply\":\"\"}"
|
||||||
|
}
|
||||||
|
let first: String = json_array_get(eff_choices, 0)
|
||||||
|
let message_o: String = json_get_raw(first, "message")
|
||||||
|
let finish: String = json_get(first, "finish_reason")
|
||||||
|
// Content, read TWO ways on purpose.
|
||||||
|
// content_raw — the provider's own bytes: `null` on a pure tool-call turn, or a
|
||||||
|
// quoted, already-escaped string. This is what goes back on the wire, verbatim.
|
||||||
|
// text_out — the DECODED text, for narration, the ledger and the final reply.
|
||||||
|
// json_get decodes, and a JSON null decodes to the 4-char string "null", which is
|
||||||
|
// never "" — so without the raw check a pure tool-call turn produced the literal
|
||||||
|
// word "null" as the assistant's narration, in the run-progress ledger, and inside
|
||||||
|
// the tool_pending envelope, and echoed `"content":"null"` instead of `content:null`.
|
||||||
|
let content_raw: String = json_get_raw(message_o, "content")
|
||||||
|
let is_null_content: Bool = str_eq(content_raw, "null") || str_eq(content_raw, "")
|
||||||
|
let text_out: String = if is_null_content { "" } else { json_get(message_o, "content") }
|
||||||
|
let tc_raw: String = json_get_raw(message_o, "tool_calls")
|
||||||
|
let tc_arr: String = if str_eq(tc_raw, "") || str_eq(tc_raw, "null") { "[]" } else { tc_raw }
|
||||||
|
let tc_n: Int = json_array_len(tc_arr)
|
||||||
|
let has_tool: Bool = tc_n > 0
|
||||||
|
|
||||||
|
// ADR-0005 mirror: we ask for one call per round; a provider that returns
|
||||||
|
// several anyway gets the FIRST honored and the drop logged loudly.
|
||||||
|
if tc_n > 1 {
|
||||||
|
println("[soul] DRIFT: provider returned " + int_to_str(tc_n) + " parallel tool_calls despite parallel_tool_calls:false - keeping the first only (ADR-0005 mirror)")
|
||||||
|
}
|
||||||
|
// Unknown finish reasons (future API drift): log loudly, never treat an
|
||||||
|
// unrecognised terminal state as a completed answer silently.
|
||||||
|
if !str_eq(finish, "stop") && !str_eq(finish, "tool_calls") && !str_eq(finish, "length") && !str_eq(finish, "") {
|
||||||
|
println("[soul] DRIFT: unknown finish_reason from API: " + finish)
|
||||||
|
}
|
||||||
|
|
||||||
|
let tc0: String = if has_tool { json_array_get(tc_arr, 0) } else { "" }
|
||||||
|
let tool_id: String = if has_tool { json_get(tc0, "id") } else { "" }
|
||||||
|
let tc_fn: String = if has_tool { json_get_raw(tc0, "function") } else { "" }
|
||||||
|
let tool_name: String = if has_tool { json_get(tc_fn, "name") } else { "" }
|
||||||
|
// arguments is a JSON-ENCODED STRING on this wire; json_get decodes it exactly
|
||||||
|
// once, yielding the raw object text dispatch_tool expects. Decoding again — or
|
||||||
|
// re-encoding before dispatch — is the two-escaper trap the gate's echo-mismatch
|
||||||
|
// check exists to catch.
|
||||||
|
let tool_input_raw: String = if has_tool { json_get(tc_fn, "arguments") } else { "" }
|
||||||
|
let tool_input: String = if str_eq(tool_input_raw, "") { "{}" } else { tool_input_raw }
|
||||||
|
|
||||||
|
let is_tool_turn: Bool = has_tool
|
||||||
|
|
||||||
|
// Consent policy — IDENTICAL to the Anthropic lane: ask_all bridges everything,
|
||||||
|
// escalate always bridges, non-builtins bridge unless "always allow" granted.
|
||||||
|
let always_key: String = "always_allow_" + session_id
|
||||||
|
let always_list: String = if !str_eq(session_id, "") { state_get(always_key) } else { "" }
|
||||||
|
let is_always_allowed: Bool = !str_eq(tool_name, "") && !str_eq(always_list, "") && str_contains(always_list, tool_name)
|
||||||
|
let risk_tier: String = if is_tool_turn { classify_tool_risk(tool_name, tool_input) } else { "" }
|
||||||
|
let needs_bridge: Bool = is_tool_turn && (ask_all || str_eq(risk_tier, "escalate") || (!is_builtin_tool(tool_name) && !is_always_allowed))
|
||||||
|
|
||||||
|
let tool_result_raw: String = if is_tool_turn && !needs_bridge { dispatch_tool(tool_name, tool_input) } else { "" }
|
||||||
|
let tool_result: String = if str_len(tool_result_raw) > 6000 {
|
||||||
|
json_trim_dangling_escape(str_slice(tool_result_raw, 0, 6000)) + "...[truncated]"
|
||||||
|
} else { tool_result_raw }
|
||||||
|
|
||||||
|
let tool_quoted: String = "\"" + tool_name + "\""
|
||||||
|
let tools_log = if is_tool_turn {
|
||||||
|
if str_eq(tools_log, "") { tool_quoted } else { tools_log + "," + tool_quoted }
|
||||||
|
} else { tools_log }
|
||||||
|
|
||||||
|
// The assistant turn echoed with its tool_calls array VERBATIM (raw), so the
|
||||||
|
// tool_call_id pairing stays valid on the wire and across a bridge resume.
|
||||||
|
// Echo the provider's content BYTES, never a re-escaped round-trip. Decoding and
|
||||||
|
// re-encoding is where fidelity is lost: json_escape/json_safe both handle only
|
||||||
|
// \\ " \n \r, so any other control character the model emits (a tab, say) would go
|
||||||
|
// back out raw and make the next request body invalid JSON — a provider 400 that
|
||||||
|
// looks like a random failure. The Anthropic lane never had this exposure because
|
||||||
|
// it echoes the response's content array untouched; this now matches it.
|
||||||
|
let content_frag: String = if is_null_content { "null" } else { content_raw }
|
||||||
|
// Echo ONLY the call we actually honor — never the provider's full array.
|
||||||
|
// The loop can assemble exactly one tool response per round, so replaying N
|
||||||
|
// tool_calls while answering one leaves the conversation self-contradictory and
|
||||||
|
// every OpenAI-format provider 400s on the next request ("no tool response for
|
||||||
|
// id X"). That is precisely the failure ADR-0005 documents on the Anthropic wire,
|
||||||
|
// where the block walk keeps the first tool_use and the rest die without a
|
||||||
|
// tool_result. Here it costs one slice to close: the dropped calls simply never
|
||||||
|
// existed from the model's point of view, and the DRIFT line above keeps the
|
||||||
|
// accounting honest about what we discarded.
|
||||||
|
let assist_turn: String = if has_tool {
|
||||||
|
"{\"role\":\"assistant\",\"content\":" + content_frag + ",\"tool_calls\":[" + tc0 + "]}"
|
||||||
|
} else {
|
||||||
|
"{\"role\":\"assistant\",\"content\":" + content_frag + "}"
|
||||||
|
}
|
||||||
|
let inner_now: String = str_slice(messages, 1, str_len(messages) - 1)
|
||||||
|
let messages_with_assistant: String = "[" + inner_now + "," + assist_turn + "]"
|
||||||
|
|
||||||
|
// Local built-in tool turn: append assistant echo + role:"tool" result, loop on.
|
||||||
|
let local_continue: Bool = is_tool_turn && !needs_bridge
|
||||||
|
let messages = if local_continue {
|
||||||
|
let inner2: String = str_slice(messages_with_assistant, 1, str_len(messages_with_assistant) - 1)
|
||||||
|
"[" + inner2 + ",{\"role\":\"tool\",\"tool_call_id\":\"" + tool_id + "\",\"content\":\"" + tool_result + "\"}]"
|
||||||
|
} else { messages }
|
||||||
|
|
||||||
|
// Live run-progress ledger — same key, same shape, same poller as the Anthropic
|
||||||
|
// lane; a forked loop that omitted this would silently kill live step rendering.
|
||||||
|
if !str_eq(session_id, "") {
|
||||||
|
let prog_key: String = "run_progress_" + session_id
|
||||||
|
let prog_prev: String = state_get(prog_key)
|
||||||
|
let prog_snip: String = if str_len(text_out) > 280 { str_slice(text_out, 0, 280) } else { text_out }
|
||||||
|
let prog_entry: String = "{\"i\":" + int_to_str(iteration)
|
||||||
|
+ ",\"t\":\"" + json_safe(prog_snip) + "\""
|
||||||
|
+ ",\"tool\":\"" + json_safe(tool_name) + "\"}"
|
||||||
|
let prog_next: String = if str_eq(prog_prev, "") { prog_entry } else { prog_prev + "," + prog_entry }
|
||||||
|
state_set(prog_key, prog_next)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Bridge turn: persist the continuation (wire-tagged) and stop the loop.
|
||||||
|
let pending = if needs_bridge { true } else { pending }
|
||||||
|
let pend_tool_id = if needs_bridge { tool_id } else { pend_tool_id }
|
||||||
|
let pend_tool_name = if needs_bridge { tool_name } else { pend_tool_name }
|
||||||
|
let pend_tool_input = if needs_bridge { tool_input } else { pend_tool_input }
|
||||||
|
let pend_tool_tier = if needs_bridge { risk_tier } else { pend_tool_tier }
|
||||||
|
let pend_narration = if needs_bridge { text_out } else { pend_narration }
|
||||||
|
if needs_bridge {
|
||||||
|
bridge_save(session_id, model, safe_sys, tools_json, messages_with_assistant, tools_log, tool_id, "openai")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Text accumulation: rounds are separated by tool executions, so the resume seam
|
||||||
|
// is unconditionally a boundary (same rule as the Anthropic loop's seam 2).
|
||||||
|
let final_text = if !is_tool_turn {
|
||||||
|
final_text + text_join_sep(final_text, text_out, true) + text_out
|
||||||
|
} else { final_text }
|
||||||
|
// Output cap hit mid-action (finish_reason "length" with a tool call pending).
|
||||||
|
let final_text = if str_eq(finish, "length") && has_tool {
|
||||||
|
final_text + "\n\n[Output limit reached mid-action - the last planned action did not run. Ask me to continue to finish it.]"
|
||||||
|
} else { final_text }
|
||||||
|
let keep_going = if local_continue { keep_going } else { false }
|
||||||
|
let iteration = iteration + 1
|
||||||
|
}
|
||||||
|
|
||||||
|
if pending {
|
||||||
|
let safe_in: String = if str_eq(pend_tool_input, "") { "{}" } else { pend_tool_input }
|
||||||
|
let tools_arr: String = if str_eq(tools_log, "") { "[]" } else { "[" + tools_log + "]" }
|
||||||
|
return "{\"tool_pending\":true"
|
||||||
|
+ ",\"session_id\":\"" + session_id + "\""
|
||||||
|
+ ",\"call_id\":\"" + pend_tool_id + "\""
|
||||||
|
+ ",\"tool_name\":\"" + pend_tool_name + "\""
|
||||||
|
+ ",\"tool_input\":" + safe_in
|
||||||
|
+ ",\"risk_tier\":\"" + pend_tool_tier + "\""
|
||||||
|
+ ",\"narration\":\"" + json_safe(pend_narration) + "\""
|
||||||
|
+ ",\"model\":\"" + model + "\""
|
||||||
|
+ ",\"agentic\":true"
|
||||||
|
+ ",\"sources\":\"\""
|
||||||
|
+ ",\"tools_used\":" + tools_arr + "}"
|
||||||
|
}
|
||||||
|
|
||||||
|
let final_text = receipt_strip(final_text)
|
||||||
|
if str_eq(final_text, "") {
|
||||||
|
let hit_cap: Bool = iteration >= 12
|
||||||
|
let err_msg: String = if hit_cap {
|
||||||
|
"agentic loop hit the 12-iteration cap without producing a final reply - task may be too complex or a tool call is looping"
|
||||||
|
} else {
|
||||||
|
"no response"
|
||||||
|
}
|
||||||
|
return "{\"error\":\"" + err_msg + "\",\"reply\":\"\",\"iterations\":" + int_to_str(iteration) + "}"
|
||||||
|
}
|
||||||
|
|
||||||
|
let safe_text: String = json_safe(final_text)
|
||||||
|
let tools_arr: String = if str_eq(tools_log, "") { "[]" } else { "[" + tools_log + "]" }
|
||||||
|
if !str_eq(session_id, "") {
|
||||||
|
let done_key: String = "run_progress_" + session_id
|
||||||
|
let done_prev: String = state_get(done_key)
|
||||||
|
let done_next: String = if str_eq(done_prev, "") { "{\"done\":true}" } else { done_prev + ",{\"done\":true}" }
|
||||||
|
state_set(done_key, done_next)
|
||||||
|
}
|
||||||
|
return "{\"reply\":\"" + safe_text + "\",\"model\":\"" + model + "\",\"agentic\":true,\"tools_used\":" + tools_arr + ",\"sources\":\"\",\"iterations\":" + int_to_str(iteration) + "}"
|
||||||
|
}
|
||||||
|
|
||||||
fn agentic_tools_literal() -> String {
|
fn agentic_tools_literal() -> String {
|
||||||
return "[" +
|
return "[" +
|
||||||
"{\"name\":\"read_file\",\"description\":\"Read contents of a file from disk.\",\"input_schema\":{\"type\":\"object\",\"properties\":{\"path\":{\"type\":\"string\",\"description\":\"Absolute file path\"}},\"required\":[\"path\"]}}," +
|
"{\"name\":\"read_file\",\"description\":\"Read contents of a file from disk.\",\"input_schema\":{\"type\":\"object\",\"properties\":{\"path\":{\"type\":\"string\",\"description\":\"Absolute file path\"}},\"required\":[\"path\"]}}," +
|
||||||
@@ -2636,7 +3014,7 @@ fn handle_chat_agentic(body: String) -> String {
|
|||||||
let ag_continuity_snip: String = if ag_continuity_ok {
|
let ag_continuity_snip: String = if ag_continuity_ok {
|
||||||
let acn0: String = json_array_get(ag_continuity_nodes, 0)
|
let acn0: String = json_array_get(ag_continuity_nodes, 0)
|
||||||
let acc: String = json_get(acn0, "content")
|
let acc: String = json_get(acn0, "content")
|
||||||
if str_len(acc) > 350 { str_slice(acc, 0, 350) } else { acc }
|
utf8_safe_slice(acc, 350)
|
||||||
} else { "" }
|
} else { "" }
|
||||||
let ag_profile_bullets: String = session_preload_bullets(ag_profile_nodes2, 8, 350)
|
let ag_profile_bullets: String = session_preload_bullets(ag_profile_nodes2, 8, 350)
|
||||||
let ag_work_bullets: String = session_preload_bullets(ag_work_nodes2, 6, 350)
|
let ag_work_bullets: String = session_preload_bullets(ag_work_nodes2, 6, 350)
|
||||||
@@ -2667,7 +3045,13 @@ fn handle_chat_agentic(body: String) -> String {
|
|||||||
" + ctx + ag_session_preload + receipt_rule()
|
" + ctx + ag_session_preload + receipt_rule()
|
||||||
|
|
||||||
let api_key: String = agentic_api_key()
|
let api_key: String = agentic_api_key()
|
||||||
let tools_json: String = agentic_tools_all()
|
// Assemble the tool set ONCE, for the lane this turn will actually take. Both
|
||||||
|
// builders call connector_tools_json(), which is an HTTP round-trip to the
|
||||||
|
// connectors bridge on :7771 — computing both would pay that cost, and its timeout
|
||||||
|
// exposure, twice per turn. The OpenAI lane drops Anthropic's server-side
|
||||||
|
// web_search (it has no analogue on that wire and cannot execute there).
|
||||||
|
let tools_lane_openai: Bool = !str_eq(llm_base_url(), "") && str_eq(llm_wire_format(), "openai")
|
||||||
|
let tools_json: String = if tools_lane_openai { agentic_tools_no_web() } else { agentic_tools_all() }
|
||||||
let safe_msg: String = json_safe(message)
|
let safe_msg: String = json_safe(message)
|
||||||
let safe_sys: String = json_safe(system)
|
let safe_sys: String = json_safe(system)
|
||||||
|
|
||||||
@@ -2708,11 +3092,12 @@ fn handle_chat_agentic(body: String) -> String {
|
|||||||
// for the rest of the run. Absent/false = behavior identical to before this fix.
|
// for the rest of the run. Absent/false = behavior identical to before this fix.
|
||||||
let req_ask_all: String = json_get(body, "require_approval")
|
let req_ask_all: String = json_get(body, "require_approval")
|
||||||
state_set("require_approval_" + session_id, if str_eq(req_ask_all, "true") { "true" } else { "" })
|
state_set("require_approval_" + session_id, if str_eq(req_ask_all, "true") { "true" } else { "" })
|
||||||
// Provider fork: OpenAI-compatible providers (Ollama/OpenAI/Grok/Gemini) take the plain-completion
|
// Provider fork (v2 port, 2026-08-06): OpenAI-compatible providers now take their own
|
||||||
// path (v1, no tools); everything else stays on the Anthropic agentic loop (the default).
|
// AGENTIC loop — same tools (minus Anthropic-server web_search), same consent policy,
|
||||||
let use_openai: Bool = !str_eq(llm_base_url(), "") && str_eq(llm_wire_format(), "openai")
|
// same bridge contract. The Anthropic native path stays the default and is untouched.
|
||||||
|
let use_openai: Bool = tools_lane_openai
|
||||||
let result: String = if use_openai {
|
let result: String = if use_openai {
|
||||||
openai_chat_complete(model, llm_base_url(), agentic_api_key(), safe_sys, messages)
|
openai_agentic_loop(session_id, model, safe_sys, tools_json, messages, "")
|
||||||
} else {
|
} else {
|
||||||
agentic_loop(session_id, model, safe_sys, tools_json, messages, h, "")
|
agentic_loop(session_id, model, safe_sys, tools_json, messages, h, "")
|
||||||
}
|
}
|
||||||
@@ -3139,7 +3524,7 @@ fn agentic_loop(session_id: String, model: String, safe_sys: String, tools_json:
|
|||||||
// client's tool_result block. messages_with_assistant is only meaningful when a
|
// client's tool_result block. messages_with_assistant is only meaningful when a
|
||||||
// tool was requested, so guard on needs_bridge before persisting.
|
// tool was requested, so guard on needs_bridge before persisting.
|
||||||
if needs_bridge {
|
if needs_bridge {
|
||||||
bridge_save(session_id, model, safe_sys, tools_json, messages_with_assistant, tools_log, pend_tool_id)
|
bridge_save(session_id, model, safe_sys, tools_json, messages_with_assistant, tools_log, pend_tool_id, "anthropic")
|
||||||
}
|
}
|
||||||
|
|
||||||
// ACCUMULATE across pause/resume cycles instead of overwriting. A resumed turn
|
// ACCUMULATE across pause/resume cycles instead of overwriting. A resumed turn
|
||||||
@@ -3221,7 +3606,7 @@ fn agentic_loop(session_id: String, model: String, safe_sys: String, tools_json:
|
|||||||
// single JSON blob in soul state so agentic_resume can rebuild the exact loop. The
|
// single JSON blob in soul state so agentic_resume can rebuild the exact loop. The
|
||||||
// stored `messages` already includes the assistant turn that requested the tool, so
|
// stored `messages` already includes the assistant turn that requested the tool, so
|
||||||
// resume just appends the client's tool_result for `tool_use_id`.
|
// resume just appends the client's tool_result for `tool_use_id`.
|
||||||
fn bridge_save(session_id: String, model: String, safe_sys: String, tools_json: String, messages: String, tools_log: String, tool_use_id: String) -> Bool {
|
fn bridge_save(session_id: String, model: String, safe_sys: String, tools_json: String, messages: String, tools_log: String, tool_use_id: String, wire: String) -> Bool {
|
||||||
// Guard: empty messages or tools_json would produce syntactically invalid JSON.
|
// Guard: empty messages or tools_json would produce syntactically invalid JSON.
|
||||||
// Return false so the caller detects the failure rather than writing a corrupt
|
// Return false so the caller detects the failure rather than writing a corrupt
|
||||||
// blob that agentic_resume would later resume with no context.
|
// blob that agentic_resume would later resume with no context.
|
||||||
@@ -3251,10 +3636,13 @@ fn bridge_save(session_id: String, model: String, safe_sys: String, tools_json:
|
|||||||
// messages_raw — arbitrary model/user content — so neither raw extraction can
|
// messages_raw — arbitrary model/user content — so neither raw extraction can
|
||||||
// first-match into model-controlled bytes either. Do not reorder; do not add a
|
// first-match into model-controlled bytes either. Do not reorder; do not add a
|
||||||
// field after messages_raw.
|
// field after messages_raw.
|
||||||
|
// "wire" (v2 port, 2026-08-06) is a json_safe'd SCALAR and therefore sits with the
|
||||||
|
// other scalars BEFORE both raw fields, per the field-order rule above.
|
||||||
let blob: String = "{\"model\":\"" + json_safe(model) + "\""
|
let blob: String = "{\"model\":\"" + json_safe(model) + "\""
|
||||||
+ ",\"safe_sys\":\"" + json_safe(safe_sys) + "\""
|
+ ",\"safe_sys\":\"" + json_safe(safe_sys) + "\""
|
||||||
+ ",\"tools_log\":\"" + json_safe(tools_log) + "\""
|
+ ",\"tools_log\":\"" + json_safe(tools_log) + "\""
|
||||||
+ ",\"tool_use_id\":\"" + json_safe(tool_use_id) + "\""
|
+ ",\"tool_use_id\":\"" + json_safe(tool_use_id) + "\""
|
||||||
|
+ ",\"wire\":\"" + json_safe(wire) + "\""
|
||||||
+ ",\"tools_raw\":" + tools_json
|
+ ",\"tools_raw\":" + tools_json
|
||||||
+ ",\"messages_raw\":" + messages + "}"
|
+ ",\"messages_raw\":" + messages + "}"
|
||||||
state_set("mcp_bridge:" + session_id, blob)
|
state_set("mcp_bridge:" + session_id, blob)
|
||||||
@@ -3310,14 +3698,52 @@ fn agentic_resume(session_id: String, tool_use_id: String, content: String) -> S
|
|||||||
str_slice(content, 0, 6000) + "...[truncated]"
|
str_slice(content, 0, 6000) + "...[truncated]"
|
||||||
} else { content }
|
} else { content }
|
||||||
let safe_result: String = json_safe(trimmed)
|
let safe_result: String = json_safe(trimmed)
|
||||||
let tool_msg: String = "{\"type\":\"tool_result\",\"tool_use_id\":\"" + eff_use_id + "\",\"content\":\"" + safe_result + "\"}"
|
|
||||||
|
|
||||||
let inner: String = str_slice(messages, 1, str_len(messages) - 1)
|
let inner: String = str_slice(messages, 1, str_len(messages) - 1)
|
||||||
let resumed_messages: String = "[" + inner + ",{\"role\":\"user\",\"content\":[" + tool_msg + "]}]"
|
|
||||||
|
|
||||||
// One-shot: clear the saved turn so a session_id can't be replayed.
|
// One-shot: clear the saved turn so a session_id can't be replayed.
|
||||||
state_set("mcp_bridge:" + session_id, "")
|
state_set("mcp_bridge:" + session_id, "")
|
||||||
|
|
||||||
|
// Wire-aware resume (v2 port, 2026-08-06): blobs written since the port carry a
|
||||||
|
// "wire" scalar ("anthropic" | "openai") among the scalar fields, where first-match
|
||||||
|
// scanning is safe (see bridge_save's field-order rule). A blob with no wire field
|
||||||
|
// is a legacy pre-port suspension — always Anthropic. On the OpenAI wire the
|
||||||
|
// client's result goes back as a role:"tool" turn keyed by tool_call_id, and a
|
||||||
|
// result for an already-answered id must never be re-sent (the resumed messages
|
||||||
|
// end at the assistant echo, so appending exactly one tool turn preserves that).
|
||||||
|
// Read "wire" from the blob's SCALAR HEAD ONLY — never the whole blob.
|
||||||
|
//
|
||||||
|
// json_get is a first-substring-match scanner. On a blob written by this binary the
|
||||||
|
// scalar sits ahead of the raw fields and wins, but on a LEGACY blob (suspended
|
||||||
|
// before this field existed) there is no match up front, so the scan runs on into
|
||||||
|
// messages_raw — model- and user-controlled bytes. A conversation that merely
|
||||||
|
// CONTAINS the literal "wire":"openai" would then misroute the resume onto the wrong
|
||||||
|
// loop and kill the run. That is exactly the round-9 defect (json_get(blob,
|
||||||
|
// "tool_use_id") matching a web_search_tool_result id inside the replayed
|
||||||
|
// conversation), and the fix is the same shape: bound the search.
|
||||||
|
//
|
||||||
|
// bridge_save guarantees every json_safe'd scalar precedes the bulk fields, so
|
||||||
|
// truncating at the earliest bulk key makes this deterministic — the decoy is not
|
||||||
|
// even inside the string we search. Both the current keys (tools_raw/messages_raw)
|
||||||
|
// and the pre-round-9 legacy ones (tools_json/messages) are covered.
|
||||||
|
let i_traw: Int = str_index_of(blob, ",\"tools_raw\":")
|
||||||
|
let i_tjson: Int = str_index_of(blob, ",\"tools_json\":")
|
||||||
|
let i_mraw: Int = str_index_of(blob, ",\"messages_raw\":")
|
||||||
|
let i_msgs: Int = str_index_of(blob, ",\"messages\":")
|
||||||
|
let cut1: Int = if i_traw > 0 { i_traw } else { str_len(blob) }
|
||||||
|
let cut2: Int = if i_tjson > 0 && i_tjson < cut1 { i_tjson } else { cut1 }
|
||||||
|
let cut3: Int = if i_mraw > 0 && i_mraw < cut2 { i_mraw } else { cut2 }
|
||||||
|
let cut: Int = if i_msgs > 0 && i_msgs < cut3 { i_msgs } else { cut3 }
|
||||||
|
let blob_head: String = str_slice(blob, 0, cut)
|
||||||
|
let wire: String = json_get(blob_head, "wire")
|
||||||
|
if str_eq(wire, "openai") {
|
||||||
|
let tool_msg_o: String = "{\"role\":\"tool\",\"tool_call_id\":\"" + eff_use_id + "\",\"content\":\"" + safe_result + "\"}"
|
||||||
|
let resumed_o: String = "[" + inner + "," + tool_msg_o + "]"
|
||||||
|
return openai_agentic_loop(session_id, model, safe_sys, tools_json, resumed_o, tools_log)
|
||||||
|
}
|
||||||
|
|
||||||
|
let tool_msg: String = "{\"type\":\"tool_result\",\"tool_use_id\":\"" + eff_use_id + "\",\"content\":\"" + safe_result + "\"}"
|
||||||
|
let resumed_messages: String = "[" + inner + ",{\"role\":\"user\",\"content\":[" + tool_msg + "]}]"
|
||||||
|
|
||||||
let api_key: String = agentic_api_key()
|
let api_key: String = agentic_api_key()
|
||||||
let h: Map = {}
|
let h: Map = {}
|
||||||
map_set(h, "x-api-key", api_key)
|
map_set(h, "x-api-key", api_key)
|
||||||
@@ -3493,7 +3919,11 @@ fn handle_dharma_room_turn_agentic(body: String) -> String {
|
|||||||
// Hard Bell: pre-LLM safety evaluation on agentic dharma room turns.
|
// Hard Bell: pre-LLM safety evaluation on agentic dharma room turns.
|
||||||
let system = safety_augment_system(system, transcript)
|
let system = safety_augment_system(system, transcript)
|
||||||
|
|
||||||
let tools_json: String = agentic_tools_all()
|
// One assembly, for the lane this turn takes (see the same note in handle_chat_agentic:
|
||||||
|
// both builders hit the connectors bridge over HTTP, so computing both doubles the cost
|
||||||
|
// and the timeout exposure).
|
||||||
|
let use_openai_d: Bool = !str_eq(llm_base_url(), "") && str_eq(llm_wire_format(), "openai")
|
||||||
|
let tools_json: String = if use_openai_d { agentic_tools_no_web() } else { agentic_tools_all() }
|
||||||
let safe_transcript: String = json_safe(transcript)
|
let safe_transcript: String = json_safe(transcript)
|
||||||
let safe_sys: String = json_safe(system)
|
let safe_sys: String = json_safe(system)
|
||||||
let messages: String = "[{\"role\":\"user\",\"content\":\"" + safe_transcript + "\"}]"
|
let messages: String = "[{\"role\":\"user\",\"content\":\"" + safe_transcript + "\"}]"
|
||||||
@@ -3504,7 +3934,14 @@ fn handle_dharma_room_turn_agentic(body: String) -> String {
|
|||||||
|
|
||||||
// Use dharma-prefixed session_id so bridge suspension works correctly per room.
|
// Use dharma-prefixed session_id so bridge suspension works correctly per room.
|
||||||
let session_id: String = if str_eq(room_id, "") { "dharma:" + next_bridge_id() } else { "dharma:" + room_id }
|
let session_id: String = if str_eq(room_id, "") { "dharma:" + next_bridge_id() } else { "dharma:" + room_id }
|
||||||
let loop_result: String = agentic_loop(session_id, model, safe_sys, tools_json, messages, h, "")
|
// Provider fork (v2 port, 2026-08-06): same routing rule as handle_chat_agentic.
|
||||||
|
// The Hard Bell augmentation above is baked into safe_sys BEFORE the fork, so the
|
||||||
|
// safety pass is identical on both wires.
|
||||||
|
let loop_result: String = if use_openai_d {
|
||||||
|
openai_agentic_loop(session_id, model, safe_sys, tools_json, messages, "")
|
||||||
|
} else {
|
||||||
|
agentic_loop(session_id, model, safe_sys, tools_json, messages, h, "")
|
||||||
|
}
|
||||||
|
|
||||||
let result_error: String = json_get(loop_result, "error")
|
let result_error: String = json_get(loop_result, "error")
|
||||||
if !str_eq(result_error, "") {
|
if !str_eq(result_error, "") {
|
||||||
|
|||||||
@@ -1,88 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header - do not edit
|
|
||||||
extern fn chat_default_model() -> String
|
|
||||||
extern fn engram_numeric_valid(s: String) -> Bool
|
|
||||||
extern fn parse_float_x100(s: String) -> Int
|
|
||||||
extern fn engram_score_node(node_json: String) -> Int
|
|
||||||
extern fn engram_render_node(node_json: String) -> String
|
|
||||||
extern fn engram_render_nodes(nodes_json: String) -> String
|
|
||||||
extern fn engram_dedup_nodes(nodes_json: String) -> String
|
|
||||||
extern fn engram_compile_ranked(nodes_json: String, max_nodes: Int) -> String
|
|
||||||
extern fn engram_split_topics(message: String) -> String
|
|
||||||
extern fn engram_extract_entities(message: String) -> String
|
|
||||||
extern fn engram_detect_recall_intent(message: String) -> Bool
|
|
||||||
extern fn engram_is_continuation(message: String, hist_len: Int) -> Bool
|
|
||||||
extern fn engram_compile_multi(topic: String) -> String
|
|
||||||
extern fn engram_nodes_merge(a: String, b: String) -> String
|
|
||||||
extern fn id_in_seen(node_id: String, seen: String) -> Bool
|
|
||||||
extern fn add_to_seen(seen: String, node_id: String) -> String
|
|
||||||
extern fn engram_extract_ids(nodes_json: String) -> String
|
|
||||||
extern fn affective_node_ts(node_json: String) -> Int
|
|
||||||
extern fn engram_compile(intent: String) -> String
|
|
||||||
extern fn distill_transcript(transcript: String) -> String
|
|
||||||
extern fn json_safe(s: String) -> String
|
|
||||||
extern fn current_engine_note(model: String) -> String
|
|
||||||
extern fn bounded_persona_floor() -> String
|
|
||||||
extern fn operator_identity_block() -> String
|
|
||||||
extern fn build_system_prompt(ctx: String, chat_mode: Bool) -> String
|
|
||||||
extern fn hist_append(hist: String, role: String, content: String) -> String
|
|
||||||
extern fn conv_hist_key(session_id: String) -> String
|
|
||||||
extern fn conv_hist_label(session_id: String) -> String
|
|
||||||
extern fn is_utility_request(body: String, session_id: String) -> Bool
|
|
||||||
extern fn provenance_scan_urls(arr: String, acc: String) -> String
|
|
||||||
extern fn provenance_add_sources(block: String, btype: String, has_cit: Bool, cit_raw: String, acc: String) -> String
|
|
||||||
extern fn provenance_names(tools_used: String) -> String
|
|
||||||
extern fn text_join_sep(accumulated: String, incoming: String, after_interruption: Bool) -> String
|
|
||||||
extern fn receipt_rule() -> String
|
|
||||||
extern fn receipt_strip(s: String) -> String
|
|
||||||
extern fn tool_receipt(tools_used: String, sources: String) -> String
|
|
||||||
extern fn hist_trim(hist: String) -> String
|
|
||||||
extern fn hist_trim_with_bell_guard(hist: String) -> String
|
|
||||||
extern fn clean_llm_response(s: String) -> String
|
|
||||||
extern fn conv_history_persist(session_id: String, hist: String) -> Void
|
|
||||||
extern fn conv_history_load(session_id: String) -> String
|
|
||||||
extern fn conv_history_record(session_id: String, user_msg: String, assistant_msg: String, receipt: String) -> Void
|
|
||||||
extern fn conv_history_block(session_id: String) -> String
|
|
||||||
extern fn layered_generate(prompt: String, imprint_id: String, session_id: String) -> String
|
|
||||||
extern fn session_preload_bullets(nodes: String, max_bullets: Int, snip_len: Int) -> String
|
|
||||||
extern fn affective_context_prefix() -> String
|
|
||||||
extern fn handle_chat(body: String) -> String
|
|
||||||
extern fn handle_see(body: String) -> String
|
|
||||||
extern fn studio_tools_json() -> String
|
|
||||||
extern fn agentic_api_key() -> String
|
|
||||||
extern fn llm_base_url() -> String
|
|
||||||
extern fn llm_wire_format() -> String
|
|
||||||
extern fn json_escape(s: String) -> String
|
|
||||||
extern fn openai_chat_complete(model: String, base_url: String, api_key: String, safe_sys: String, messages_json: String) -> String
|
|
||||||
extern fn agentic_tools_literal() -> String
|
|
||||||
extern fn web_search_tool_json() -> String
|
|
||||||
extern fn strip_client_web_search(tools_inner: String) -> String
|
|
||||||
extern fn agentic_tools_with_web() -> String
|
|
||||||
extern fn connector_tools_json() -> String
|
|
||||||
extern fn agentic_tools_all() -> String
|
|
||||||
extern fn call_mcp_bridge(tool_name: String, tool_input: String) -> String
|
|
||||||
extern fn tool_auto_approved(tool_name: String) -> Bool
|
|
||||||
extern fn call_neuron_mcp(tool_name: String, args: String) -> String
|
|
||||||
extern fn agent_workspace_root() -> String
|
|
||||||
extern fn path_within_root(path: String, root: String) -> Bool
|
|
||||||
extern fn resolve_in_root(path: String, root: String) -> String
|
|
||||||
extern fn run_command_is_readonly(cmd: String) -> Bool
|
|
||||||
extern fn cmd_abs_escape_at(cmd: String, root: String, needle: String) -> Bool
|
|
||||||
extern fn run_command_guard(cmd: String, root: String) -> String
|
|
||||||
extern fn classify_tool_risk(tool_name: String, tool_input: String) -> String
|
|
||||||
extern fn dispatch_tool(tool_name: String, tool_input: String) -> String
|
|
||||||
extern fn is_builtin_tool(tool_name: String) -> Bool
|
|
||||||
extern fn next_bridge_id() -> String
|
|
||||||
extern fn handle_chat_plan(body: String) -> String
|
|
||||||
extern fn handle_chat_agentic(body: String) -> String
|
|
||||||
extern fn agentic_loop(session_id: String, model: String, safe_sys: String, tools_json: String, messages_in: String, h: Map, tools_log_in: String) -> String
|
|
||||||
extern fn bridge_save(session_id: String, model: String, safe_sys: String, tools_json: String, messages: String, tools_log: String, tool_use_id: String) -> Bool
|
|
||||||
extern fn agentic_resume(session_id: String, tool_use_id: String, content: String) -> String
|
|
||||||
extern fn handle_tool_result(session_id: String, body: String) -> String
|
|
||||||
extern fn handle_chat_as_soul(body: String) -> String
|
|
||||||
extern fn handle_dharma_room_turn(body: String) -> String
|
|
||||||
extern fn handle_dharma_room_turn_agentic(body: String) -> String
|
|
||||||
extern fn session_summary_write(summary_text: String) -> String
|
|
||||||
extern fn session_summary_write_dated(summary_text: String, label: String) -> String
|
|
||||||
extern fn session_summary_autogenerate(hist: String) -> String
|
|
||||||
extern fn auto_persist(req: String, resp: String) -> Void
|
|
||||||
extern fn strengthen_chat_nodes(activation_nodes: String) -> Void
|
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
# neuron-connectd — local-dev stub
|
||||||
|
|
||||||
|
`connectd_service.py` is a **minimal local-dev stub**, not the real sidecar.
|
||||||
|
It exists to close a real local-build/local-run correctness gap found during
|
||||||
|
the 2026-08-15 build audit, without taking on the much larger product task of
|
||||||
|
actually building the full MCP-connector sidecar.
|
||||||
|
|
||||||
|
## The gap this closes
|
||||||
|
|
||||||
|
`routes.el` (`handle_connectors`, `connectd_get`/`connectd_post`) and `chat.el`
|
||||||
|
(`connector_tools_json`, the `mcp__*` branch in `dispatch_tool`,
|
||||||
|
`tool_auto_approved`) are live, current code that calls `127.0.0.1:7771` on
|
||||||
|
every soul boot and every agentic turn, per the design in
|
||||||
|
`neuron-technologies/docs/research/mcp-connectors-adoption-spec.md`
|
||||||
|
(2026-06-13, "Status: Draft for build"). That spec's sidecar — `neuron-connectd`,
|
||||||
|
a TypeScript/Python process using the official MCP SDK — was never built.
|
||||||
|
Nothing on disk implements it (verified: no `neuron-connectd` source anywhere
|
||||||
|
under `~/Development` before this directory).
|
||||||
|
|
||||||
|
Meanwhile port `:7771` is *also* claimed by two other, unrelated things:
|
||||||
|
|
||||||
|
- `soul.el`'s `axon_base` default (`http://localhost:7771`) — a **different**,
|
||||||
|
independently-known, already-documented gap (`platform/protocols/axon` is
|
||||||
|
an unbuilt Rust crate; see `cli/HANDOFF.md` and `HANDOFF-engram-write-corruption.md`).
|
||||||
|
Out of scope here — no source to build against.
|
||||||
|
- `council/council_service.py --port 7771` (`ai.neuron.council` LaunchAgent) —
|
||||||
|
a real, running, **unrelated** anti-confabulation service that happens to
|
||||||
|
bind the same port. In Will's live environment this is what's actually
|
||||||
|
listening on `:7771` today, and it answers the connector/axon requests
|
||||||
|
above with its own unrelated 404 JSON body — worse than a clean
|
||||||
|
connection-refused, because `chat.el`'s "bridge down" fallback expects
|
||||||
|
either a real reply or nothing, not a wrong-shaped reply from an unrelated
|
||||||
|
service.
|
||||||
|
|
||||||
|
## What this stub does and does not do
|
||||||
|
|
||||||
|
Implements exactly the spec's documented HTTP contract (`GET /mcp/tools`,
|
||||||
|
`POST /mcp/call`, `GET /mcp/servers`, `POST /mcp/servers/{add,toggle,
|
||||||
|
auto-approve,remove,secret}`, `POST /mcp/oauth/start`, `GET /healthz`), always
|
||||||
|
answering as if **zero connectors are configured** — empty tool list, empty
|
||||||
|
server list, a clear `"not configured"` error on any call that would need a
|
||||||
|
real connector. This is the *correct* steady state for a fresh local dev box
|
||||||
|
that hasn't set up any MCP connectors, and it's what `chat.el`'s
|
||||||
|
`connector_tools_json()` / `tool_auto_approved()` already gracefully degrade
|
||||||
|
to when the bridge replies emptily.
|
||||||
|
|
||||||
|
It does **not**: spawn any real MCP server, do OAuth, read or write
|
||||||
|
`~/.neuron/connectors.json`, or namespace/proxy real `tools/call` traffic to
|
||||||
|
Google Drive/GitHub/Slack/etc. Building that is the real product task the
|
||||||
|
spec describes — a genuine, sizeable engineering lift (MCP SDK client, OAuth
|
||||||
|
+ Keychain token storage, per-server process lifecycle), not something to
|
||||||
|
improvise inside a build/run audit. **That decision is Will's to make**, not
|
||||||
|
this audit's to guess at.
|
||||||
|
|
||||||
|
## Running it
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Foreground, on a throwaway port (never :7771 while council owns it live):
|
||||||
|
python3 connectd_service.py --port 17771
|
||||||
|
|
||||||
|
# Verify the contract:
|
||||||
|
curl -s http://127.0.0.1:17771/healthz
|
||||||
|
curl -s http://127.0.0.1:17771/mcp/tools
|
||||||
|
curl -s http://127.0.0.1:17771/mcp/servers
|
||||||
|
```
|
||||||
|
|
||||||
|
## Open question for Will — the :7771 collision
|
||||||
|
|
||||||
|
Three independent things are hardcoded to `:7771`: axon (unbuilt), connectd
|
||||||
|
(this stub), and council (the one actually running). Wiring this stub into
|
||||||
|
the real LaunchAgent stack on `:7771` requires either moving council off that
|
||||||
|
port or deciding connectd should live elsewhere and repointing `routes.el`/
|
||||||
|
`chat.el`'s hardcoded `127.0.0.1:7771` calls. Neither change was made here —
|
||||||
|
it touches a live, running production service (`ai.neuron.council`) and a
|
||||||
|
port number baked into shipped `.el` source, both bigger than this audit's
|
||||||
|
"make local build/run work" mandate. Flagging for a decision rather than
|
||||||
|
guessing.
|
||||||
@@ -0,0 +1,115 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
neuron-connectd — MCP connector bridge (LOCAL-DEV STUB).
|
||||||
|
|
||||||
|
THIS IS NOT THE FULL SIDECAR. The full design lives in
|
||||||
|
neuron-technologies/docs/research/mcp-connectors-adoption-spec.md (2026-06-13,
|
||||||
|
"Status: Draft for build"): a TypeScript/Python sidecar using the official MCP
|
||||||
|
SDK that spawns real MCP servers (stdio or streamable-HTTP/SSE), does OAuth,
|
||||||
|
and namespaces their tools as mcp__<serverId>__<toolName>. That sidecar was
|
||||||
|
never built (build-audit, 2026-08-15: no neuron-connectd source existed
|
||||||
|
anywhere on disk before this file).
|
||||||
|
|
||||||
|
WHY THIS STUB EXISTS: routes.el (handle_connectors, connectd_get/connectd_post)
|
||||||
|
and chat.el (connector_tools_json, dispatch_tool's mcp__* routing,
|
||||||
|
tool_auto_approved) were built to the spec and hardcoded to 127.0.0.1:7771 —
|
||||||
|
they are LIVE and calling that port right now on every soul boot and every
|
||||||
|
agentic turn. With nothing real listening there, three unrelated services
|
||||||
|
collide on :7771 (see connectd/README.md): council (which IS what's bound
|
||||||
|
there in Will's live environment today) silently answers with unrelated
|
||||||
|
404 JSON, which is worse than a clean "connection refused" bridge-down
|
||||||
|
response, because it can be misparsed as a real (if empty) reply instead of
|
||||||
|
the "bridge unreachable" path the soul code already handles gracefully.
|
||||||
|
|
||||||
|
This stub implements ONLY the documented HTTP contract, with zero connectors
|
||||||
|
ever configured: empty tool list, empty server list, "not configured" on any
|
||||||
|
mutating call. It gives a fresh local soul the CORRECT graceful-degradation
|
||||||
|
behavior the soul code already expects for "no connectors set up yet" — not
|
||||||
|
the wrong-shaped 404 noise a port collision produces. It does not spawn any
|
||||||
|
MCP server, does no OAuth, and reads no ~/.neuron/connectors.json (there is
|
||||||
|
nothing to read yet). Building the real sidecar is a separate, larger,
|
||||||
|
Will-decision-needed product task — see README.md.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
python3 connectd_service.py [--port 7771]
|
||||||
|
"""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
|
||||||
|
import uvicorn
|
||||||
|
from fastapi import FastAPI
|
||||||
|
from fastapi.middleware.cors import CORSMiddleware
|
||||||
|
from pydantic import BaseModel
|
||||||
|
|
||||||
|
app = FastAPI(title="neuron-connectd (local-dev stub)")
|
||||||
|
app.add_middleware(
|
||||||
|
CORSMiddleware,
|
||||||
|
allow_origins=["*"],
|
||||||
|
allow_methods=["*"],
|
||||||
|
allow_headers=["*"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class ToolCall(BaseModel):
|
||||||
|
name: str
|
||||||
|
input: dict = {}
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/healthz")
|
||||||
|
def healthz():
|
||||||
|
return {"status": "ok", "stub": True}
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/mcp/tools")
|
||||||
|
def mcp_tools():
|
||||||
|
# Matches the spec's contract shape exactly (section 4, "HTTP contract").
|
||||||
|
# Empty because zero connectors are configured — this is the correct,
|
||||||
|
# intended-by-design empty state, not a failure.
|
||||||
|
return {"tools": []}
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/mcp/call")
|
||||||
|
def mcp_call(body: ToolCall):
|
||||||
|
return {"ok": False, "error": "no connectors configured (neuron-connectd stub)"}
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/mcp/servers")
|
||||||
|
def mcp_servers():
|
||||||
|
return {"servers": []}
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/mcp/servers/add")
|
||||||
|
def mcp_servers_add():
|
||||||
|
return {"ok": False, "error": "neuron-connectd stub does not implement connector management yet"}
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/mcp/servers/toggle")
|
||||||
|
def mcp_servers_toggle():
|
||||||
|
return {"ok": False, "error": "neuron-connectd stub does not implement connector management yet"}
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/mcp/servers/auto-approve")
|
||||||
|
def mcp_servers_auto_approve():
|
||||||
|
return {"ok": False, "error": "neuron-connectd stub does not implement connector management yet"}
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/mcp/servers/remove")
|
||||||
|
def mcp_servers_remove():
|
||||||
|
return {"ok": False, "error": "neuron-connectd stub does not implement connector management yet"}
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/mcp/servers/secret")
|
||||||
|
def mcp_servers_secret():
|
||||||
|
return {"ok": False, "error": "neuron-connectd stub does not implement connector management yet"}
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/mcp/oauth/start")
|
||||||
|
def mcp_oauth_start():
|
||||||
|
return {"ok": False, "error": "oauth not implemented in the neuron-connectd stub"}
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--port", type=int, default=7771)
|
||||||
|
args = parser.parse_args()
|
||||||
|
uvicorn.run(app, host="127.0.0.1", port=args.port, log_level="info")
|
||||||
-26
@@ -1,26 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn idle_count() -> Int
|
|
||||||
extern fn idle_inc() -> Int
|
|
||||||
extern fn idle_reset() -> Void
|
|
||||||
extern fn ise_post(content: String) -> Void
|
|
||||||
extern fn elapsed_ms() -> Int
|
|
||||||
extern fn elapsed_human() -> String
|
|
||||||
extern fn embed_ok() -> Int
|
|
||||||
extern fn emit_heartbeat() -> Void
|
|
||||||
extern fn auto_term_try_slot(slot_type: String, slot_lbl: String) -> Void
|
|
||||||
extern fn proactive_curiosity() -> Bool
|
|
||||||
extern fn pulse_count() -> Int
|
|
||||||
extern fn pulse_inc() -> Int
|
|
||||||
extern fn make_action(kind: String, payload: String) -> String
|
|
||||||
extern fn perceive() -> String
|
|
||||||
extern fn attend(node_json: String) -> String
|
|
||||||
extern fn respond(action_json: String) -> String
|
|
||||||
extern fn record(outcome_json: String) -> Void
|
|
||||||
extern fn one_cycle() -> Bool
|
|
||||||
extern fn awareness_run() -> Void
|
|
||||||
extern fn security_research_authorized() -> Bool
|
|
||||||
extern fn threat_score_command(cmd: String) -> Int
|
|
||||||
extern fn threat_score_path(path: String) -> Int
|
|
||||||
extern fn threat_score_history(history: String) -> Int
|
|
||||||
extern fn threat_trajectory_check(tool_name: String, tool_input: String) -> Int
|
|
||||||
extern fn threat_history_append(text: String) -> Void
|
|
||||||
-70
@@ -1,70 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn chat_default_model() -> String
|
|
||||||
extern fn engram_numeric_valid(s: String) -> Bool
|
|
||||||
extern fn parse_float_x100(s: String) -> Int
|
|
||||||
extern fn engram_score_node(node_json: String) -> Int
|
|
||||||
extern fn engram_render_node(node_json: String) -> String
|
|
||||||
extern fn engram_render_nodes(nodes_json: String) -> String
|
|
||||||
extern fn engram_dedup_nodes(nodes_json: String) -> String
|
|
||||||
extern fn engram_compile_ranked(nodes_json: String, max_nodes: Int) -> String
|
|
||||||
extern fn engram_split_topics(message: String) -> String
|
|
||||||
extern fn engram_extract_entities(message: String) -> String
|
|
||||||
extern fn engram_detect_recall_intent(message: String) -> Bool
|
|
||||||
extern fn engram_is_continuation(message: String, hist_len: Int) -> Bool
|
|
||||||
extern fn engram_compile_multi(topic: String) -> String
|
|
||||||
extern fn engram_nodes_merge(a: String, b: String) -> String
|
|
||||||
extern fn id_in_seen(node_id: String, seen: String) -> Bool
|
|
||||||
extern fn add_to_seen(seen: String, node_id: String) -> String
|
|
||||||
extern fn engram_extract_ids(nodes_json: String) -> String
|
|
||||||
extern fn engram_compile(intent: String) -> String
|
|
||||||
extern fn distill_transcript(transcript: String) -> String
|
|
||||||
extern fn json_safe(s: String) -> String
|
|
||||||
extern fn current_engine_note(model: String) -> String
|
|
||||||
extern fn build_system_prompt(ctx: String, chat_mode: Bool) -> String
|
|
||||||
extern fn hist_append(hist: String, role: String, content: String) -> String
|
|
||||||
extern fn hist_trim(hist: String) -> String
|
|
||||||
extern fn hist_trim_with_bell_guard(hist: String) -> String
|
|
||||||
extern fn clean_llm_response(s: String) -> String
|
|
||||||
extern fn conv_history_persist(hist: String) -> Void
|
|
||||||
extern fn conv_history_load() -> String
|
|
||||||
extern fn session_preload_bullets(nodes: String, max_bullets: Int, snip_len: Int) -> String
|
|
||||||
extern fn affective_context_prefix() -> String
|
|
||||||
extern fn handle_chat(body: String) -> String
|
|
||||||
extern fn handle_see(body: String) -> String
|
|
||||||
extern fn studio_tools_json() -> String
|
|
||||||
extern fn agentic_api_key() -> String
|
|
||||||
extern fn llm_base_url() -> String
|
|
||||||
extern fn llm_wire_format() -> String
|
|
||||||
extern fn json_escape(s: String) -> String
|
|
||||||
extern fn openai_chat_complete(model: String, base_url: String, api_key: String, safe_sys: String, messages_json: String) -> String
|
|
||||||
extern fn agentic_tools_literal() -> String
|
|
||||||
extern fn agentic_tools_with_web() -> String
|
|
||||||
extern fn connector_tools_json() -> String
|
|
||||||
extern fn agentic_tools_all() -> String
|
|
||||||
extern fn call_mcp_bridge(tool_name: String, tool_input: String) -> String
|
|
||||||
extern fn tool_auto_approved(tool_name: String) -> Bool
|
|
||||||
extern fn call_neuron_mcp(tool_name: String, args: String) -> String
|
|
||||||
extern fn agent_workspace_root() -> String
|
|
||||||
extern fn path_within_root(path: String, root: String) -> Bool
|
|
||||||
extern fn resolve_in_root(path: String, root: String) -> String
|
|
||||||
extern fn run_command_is_readonly(cmd: String) -> Bool
|
|
||||||
extern fn cmd_abs_escape_at(cmd: String, root: String, needle: String) -> Bool
|
|
||||||
extern fn run_command_guard(cmd: String, root: String) -> String
|
|
||||||
extern fn classify_tool_risk(tool_name: String, tool_input: String) -> String
|
|
||||||
extern fn dispatch_tool(tool_name: String, tool_input: String) -> String
|
|
||||||
extern fn is_builtin_tool(tool_name: String) -> Bool
|
|
||||||
extern fn next_bridge_id() -> String
|
|
||||||
extern fn handle_chat_plan(body: String) -> String
|
|
||||||
extern fn handle_chat_agentic(body: String) -> String
|
|
||||||
extern fn agentic_loop(session_id: String, model: String, safe_sys: String, tools_json: String, messages_in: String, h: Map, tools_log_in: String) -> String
|
|
||||||
extern fn bridge_save(session_id: String, model: String, safe_sys: String, tools_json: String, messages: String, tools_log: String, tool_use_id: String) -> Bool
|
|
||||||
extern fn agentic_resume(session_id: String, tool_use_id: String, content: String) -> String
|
|
||||||
extern fn handle_tool_result(session_id: String, body: String) -> String
|
|
||||||
extern fn handle_chat_as_soul(body: String) -> String
|
|
||||||
extern fn handle_dharma_room_turn(body: String) -> String
|
|
||||||
extern fn handle_dharma_room_turn_agentic(body: String) -> String
|
|
||||||
extern fn session_summary_write(summary_text: String) -> String
|
|
||||||
extern fn session_summary_write_dated(summary_text: String, label: String) -> String
|
|
||||||
extern fn session_summary_autogenerate(hist: String) -> String
|
|
||||||
extern fn auto_persist(req: String, resp: String) -> Void
|
|
||||||
extern fn strengthen_chat_nodes(activation_nodes: String) -> Void
|
|
||||||
+6
-1
@@ -141,7 +141,7 @@ el_val_t awareness_run(void);
|
|||||||
el_val_t axon_get(el_val_t path);
|
el_val_t axon_get(el_val_t path);
|
||||||
el_val_t axon_post(el_val_t path, el_val_t body);
|
el_val_t axon_post(el_val_t path, el_val_t body);
|
||||||
el_val_t bounded_persona_floor(void);
|
el_val_t bounded_persona_floor(void);
|
||||||
el_val_t bridge_save(el_val_t session_id, el_val_t model, el_val_t safe_sys, el_val_t tools_json, el_val_t messages, el_val_t tools_log, el_val_t tool_use_id);
|
el_val_t bridge_save(el_val_t session_id, el_val_t model, el_val_t safe_sys, el_val_t tools_json, el_val_t messages, el_val_t tools_log, el_val_t tool_use_id, el_val_t wire);
|
||||||
el_val_t build_form_from_json(el_val_t semantic_form_json, el_val_t lang_code);
|
el_val_t build_form_from_json(el_val_t semantic_form_json, el_val_t lang_code);
|
||||||
el_val_t build_np(el_val_t referent, el_val_t slots);
|
el_val_t build_np(el_val_t referent, el_val_t slots);
|
||||||
el_val_t build_pp(el_val_t loc);
|
el_val_t build_pp(el_val_t loc);
|
||||||
@@ -875,6 +875,11 @@ el_val_t non_weak_past(el_val_t stem, el_val_t slot);
|
|||||||
el_val_t non_weak_present(el_val_t stem, el_val_t slot);
|
el_val_t non_weak_present(el_val_t stem, el_val_t slot);
|
||||||
el_val_t one_cycle(void);
|
el_val_t one_cycle(void);
|
||||||
el_val_t openai_chat_complete(el_val_t model, el_val_t base_url, el_val_t api_key, el_val_t safe_sys, el_val_t messages_json);
|
el_val_t openai_chat_complete(el_val_t model, el_val_t base_url, el_val_t api_key, el_val_t safe_sys, el_val_t messages_json);
|
||||||
|
el_val_t openai_tools_json(el_val_t tools_anthropic);
|
||||||
|
el_val_t json_trim_dangling_escape(el_val_t s);
|
||||||
|
el_val_t utf8_safe_slice(el_val_t s, el_val_t n);
|
||||||
|
el_val_t agentic_tools_no_web(void);
|
||||||
|
el_val_t openai_agentic_loop(el_val_t session_id, el_val_t model, el_val_t safe_sys, el_val_t tools_json, el_val_t messages_in, el_val_t tools_log_in);
|
||||||
el_val_t parse_float_x100(el_val_t s);
|
el_val_t parse_float_x100(el_val_t s);
|
||||||
el_val_t path_within_root(el_val_t path, el_val_t root);
|
el_val_t path_within_root(el_val_t path, el_val_t root);
|
||||||
el_val_t peo_ah_past(el_val_t slot);
|
el_val_t peo_ah_past(el_val_t slot);
|
||||||
|
|||||||
-5
@@ -1,5 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn elp_extract_topic(msg: String) -> String
|
|
||||||
extern fn elp_detect_predicate(msg: String) -> String
|
|
||||||
extern fn elp_parse(msg: String) -> String
|
|
||||||
extern fn handle_elp_chat(body: String) -> String
|
|
||||||
-7
@@ -1,7 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn sem_get(json: String, key: String) -> String
|
|
||||||
extern fn generate_frame(frame: [String]) -> String
|
|
||||||
extern fn generate_frame_lang(frame: [String], lang_code: String) -> String
|
|
||||||
extern fn build_form_from_json(semantic_form_json: String, lang_code: String) -> [String]
|
|
||||||
extern fn generate(semantic_form_json: String) -> String
|
|
||||||
extern fn generate_lang(semantic_form_json: String, lang_code: String) -> String
|
|
||||||
-38
@@ -1,38 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn slots_get(slots: [String], key: String) -> String
|
|
||||||
extern fn slots_set(slots: [String], key: String, val: String) -> [String]
|
|
||||||
extern fn make_slots(k0: String, v0: String) -> [String]
|
|
||||||
extern fn make_slots2(k0: String, v0: String, k1: String, v1: String) -> [String]
|
|
||||||
extern fn make_slots3(k0: String, v0: String, k1: String, v1: String, k2: String, v2: String) -> [String]
|
|
||||||
extern fn make_slots4(k0: String, v0: String, k1: String, v1: String, k2: String, v2: String, k3: String, v3: String) -> [String]
|
|
||||||
extern fn make_slots5(k0: String, v0: String, k1: String, v1: String, k2: String, v2: String, k3: String, v3: String, k4: String, v4: String) -> [String]
|
|
||||||
extern fn rule_id(rule: [String]) -> String
|
|
||||||
extern fn rule_lhs(rule: [String]) -> String
|
|
||||||
extern fn rule_rhs_len(rule: [String]) -> Int
|
|
||||||
extern fn rule_rhs(rule: [String], idx: Int) -> String
|
|
||||||
extern fn make_rule(id: String, lhs: String, r0: String) -> [String]
|
|
||||||
extern fn make_rule2(id: String, lhs: String, r0: String, r1: String) -> [String]
|
|
||||||
extern fn make_rule3(id: String, lhs: String, r0: String, r1: String, r2: String) -> [String]
|
|
||||||
extern fn make_rule4(id: String, lhs: String, r0: String, r1: String, r2: String, r3: String) -> [String]
|
|
||||||
extern fn build_rules() -> [[String]]
|
|
||||||
extern fn get_rules() -> [[String]]
|
|
||||||
extern fn find_rule(rule_id_str: String) -> [String]
|
|
||||||
extern fn make_leaf(label: String, word: String) -> String
|
|
||||||
extern fn make_node1(label: String, child0: String) -> String
|
|
||||||
extern fn make_node2(label: String, child0: String, child1: String) -> String
|
|
||||||
extern fn make_node3(label: String, child0: String, child1: String, child2: String) -> String
|
|
||||||
extern fn make_node4(label: String, child0: String, child1: String, child2: String, child3: String) -> String
|
|
||||||
extern fn nlg_is_ws(c: String) -> Bool
|
|
||||||
extern fn skip_ws(s: String, pos: Int) -> Int
|
|
||||||
extern fn scan_token(s: String, start: Int) -> [String]
|
|
||||||
extern fn render_tree(tree: String) -> String
|
|
||||||
extern fn gram_word_order(profile: [String]) -> String
|
|
||||||
extern fn gram_order_constituents(subj: String, verb: String, obj: String, profile: [String]) -> String
|
|
||||||
extern fn gram_build_vp(verb: String, aux: String, profile: [String]) -> String
|
|
||||||
extern fn gram_question_strategy(profile: [String]) -> String
|
|
||||||
extern fn is_pronoun(word: String) -> Bool
|
|
||||||
extern fn build_np(referent: String, slots: [String]) -> String
|
|
||||||
extern fn build_pp(loc: String) -> String
|
|
||||||
extern fn build_vp_body(slots: [String]) -> String
|
|
||||||
extern fn build_vp_from_slots(slots: [String]) -> String
|
|
||||||
extern fn generate_tree(rule_id_str: String, slots: [String]) -> String
|
|
||||||
-7
@@ -1,7 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn imprint_current() -> String
|
|
||||||
extern fn imprint_load(imprint_id: String) -> String
|
|
||||||
extern fn imprint_respond(input: String, imprint_id: String) -> String
|
|
||||||
extern fn imprint_surface_knowledge(query: String, imprint_id: String) -> String
|
|
||||||
extern fn imprint_surface_memory_read(query: String) -> String
|
|
||||||
extern fn imprint_unload() -> Void
|
|
||||||
-46
@@ -1,46 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn lang_profile(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String]
|
|
||||||
extern fn lang_get(profile: [String], key: String) -> String
|
|
||||||
extern fn lang_profile_en() -> [String]
|
|
||||||
extern fn lang_profile_ja() -> [String]
|
|
||||||
extern fn lang_profile_ar() -> [String]
|
|
||||||
extern fn lang_profile_zh() -> [String]
|
|
||||||
extern fn lang_profile_de() -> [String]
|
|
||||||
extern fn lang_profile_es() -> [String]
|
|
||||||
extern fn lang_profile_fi() -> [String]
|
|
||||||
extern fn lang_profile_sw() -> [String]
|
|
||||||
extern fn lang_profile_hi() -> [String]
|
|
||||||
extern fn lang_profile_ru() -> [String]
|
|
||||||
extern fn lang_profile_fr() -> [String]
|
|
||||||
extern fn lang_profile_la() -> [String]
|
|
||||||
extern fn lang_profile_he() -> [String]
|
|
||||||
extern fn lang_profile_sa() -> [String]
|
|
||||||
extern fn lang_profile_got() -> [String]
|
|
||||||
extern fn lang_profile_non() -> [String]
|
|
||||||
extern fn lang_profile_enm() -> [String]
|
|
||||||
extern fn lang_profile_pi() -> [String]
|
|
||||||
extern fn lang_profile_grc() -> [String]
|
|
||||||
extern fn lang_profile_ang() -> [String]
|
|
||||||
extern fn lang_profile_fro() -> [String]
|
|
||||||
extern fn lang_profile_goh() -> [String]
|
|
||||||
extern fn lang_profile_sga() -> [String]
|
|
||||||
extern fn lang_profile_txb() -> [String]
|
|
||||||
extern fn lang_profile_peo() -> [String]
|
|
||||||
extern fn lang_profile_akk() -> [String]
|
|
||||||
extern fn lang_profile_uga() -> [String]
|
|
||||||
extern fn lang_profile_egy() -> [String]
|
|
||||||
extern fn lang_profile_sux() -> [String]
|
|
||||||
extern fn lang_profile_gez() -> [String]
|
|
||||||
extern fn lang_profile_cop() -> [String]
|
|
||||||
extern fn lang_from_code(code: String) -> [String]
|
|
||||||
extern fn lang_default() -> [String]
|
|
||||||
extern fn lang_is_isolating(profile: [String]) -> Bool
|
|
||||||
extern fn lang_is_agglutinative(profile: [String]) -> Bool
|
|
||||||
extern fn lang_is_fusional(profile: [String]) -> Bool
|
|
||||||
extern fn lang_is_polysynthetic(profile: [String]) -> Bool
|
|
||||||
extern fn lang_is_rtl(profile: [String]) -> Bool
|
|
||||||
extern fn lang_has_null_subject(profile: [String]) -> Bool
|
|
||||||
extern fn lang_has_case(profile: [String]) -> Bool
|
|
||||||
extern fn lang_has_gender(profile: [String]) -> Bool
|
|
||||||
extern fn lang_word_order(profile: [String]) -> String
|
|
||||||
extern fn lang_code(profile: [String]) -> String
|
|
||||||
-16
@@ -1,16 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn tier_working() -> String
|
|
||||||
extern fn tier_episodic() -> String
|
|
||||||
extern fn tier_canonical() -> String
|
|
||||||
extern fn mem_store(content: String, label: String, tags: String) -> String
|
|
||||||
extern fn mem_remember(content: String, tags: String) -> String
|
|
||||||
extern fn mem_recall(query: String, depth: Int) -> String
|
|
||||||
extern fn mem_search(query: String, limit: Int) -> String
|
|
||||||
extern fn mem_strengthen(node_id: String) -> Void
|
|
||||||
extern fn mem_forget(node_id: String) -> Void
|
|
||||||
extern fn mem_consolidate() -> String
|
|
||||||
extern fn mem_save(path: String) -> Void
|
|
||||||
extern fn mem_load(path: String) -> Void
|
|
||||||
extern fn mem_boot_count_get() -> Int
|
|
||||||
extern fn mem_boot_count_inc() -> Int
|
|
||||||
extern fn mem_emit_state_event(trigger: String, kind: String, content: String) -> String
|
|
||||||
-31
@@ -1,31 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn akk_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn akk_str_len(s: String) -> Int
|
|
||||||
extern fn akk_str_drop_last(s: String, n: Int) -> String
|
|
||||||
extern fn akk_slot(person: String, number: String) -> Int
|
|
||||||
extern fn akk_slot_g(person: String, gender: String, number: String) -> Int
|
|
||||||
extern fn akk_copula_present(slot: Int) -> String
|
|
||||||
extern fn akk_copula_stative(slot: Int) -> String
|
|
||||||
extern fn akk_is_copula(verb: String) -> Bool
|
|
||||||
extern fn akk_conjugate_copula(tense: String, slot: Int) -> String
|
|
||||||
extern fn akk_alaku_present(slot: Int) -> String
|
|
||||||
extern fn akk_alaku_perfect(slot: Int) -> String
|
|
||||||
extern fn akk_amaru_present(slot: Int) -> String
|
|
||||||
extern fn akk_amaru_perfect(slot: Int) -> String
|
|
||||||
extern fn akk_amaru_stative(slot: Int) -> String
|
|
||||||
extern fn akk_qabu_present(slot: Int) -> String
|
|
||||||
extern fn akk_qabu_perfect(slot: Int) -> String
|
|
||||||
extern fn akk_qabu_stative(slot: Int) -> String
|
|
||||||
extern fn akk_epesu_present(slot: Int) -> String
|
|
||||||
extern fn akk_epesu_perfect(slot: Int) -> String
|
|
||||||
extern fn akk_epesu_stative(slot: Int) -> String
|
|
||||||
extern fn akk_regular_present(stem: String, slot: Int) -> String
|
|
||||||
extern fn akk_regular_perfect(stem: String, slot: Int) -> String
|
|
||||||
extern fn akk_regular_stative(stem: String, slot: Int) -> String
|
|
||||||
extern fn akk_known_verb(verb: String, tense: String, slot: Int) -> String
|
|
||||||
extern fn akk_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn akk_strip_nom(noun: String) -> String
|
|
||||||
extern fn akk_is_fem(noun: String) -> Bool
|
|
||||||
extern fn akk_decline(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn akk_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
extern fn akk_map_canonical(verb: String) -> String
|
|
||||||
-44
@@ -1,44 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn ang_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn ang_str_drop_last(s: String, n: Int) -> String
|
|
||||||
extern fn ang_str_last_char(s: String) -> String
|
|
||||||
extern fn ang_str_last2(s: String) -> String
|
|
||||||
extern fn ang_slot(person: String, number: String) -> Int
|
|
||||||
extern fn ang_map_canonical(verb: String) -> String
|
|
||||||
extern fn ang_wesan_past(slot: Int) -> String
|
|
||||||
extern fn ang_beon_present(slot: Int) -> String
|
|
||||||
extern fn ang_wesan_present(slot: Int) -> String
|
|
||||||
extern fn ang_habban_present(slot: Int) -> String
|
|
||||||
extern fn ang_habban_past(slot: Int) -> String
|
|
||||||
extern fn ang_gan_present(slot: Int) -> String
|
|
||||||
extern fn ang_gan_past(slot: Int) -> String
|
|
||||||
extern fn ang_cuman_present(slot: Int) -> String
|
|
||||||
extern fn ang_cuman_past(slot: Int) -> String
|
|
||||||
extern fn ang_secgan_present(slot: Int) -> String
|
|
||||||
extern fn ang_secgan_past(slot: Int) -> String
|
|
||||||
extern fn ang_seon_present(slot: Int) -> String
|
|
||||||
extern fn ang_seon_past(slot: Int) -> String
|
|
||||||
extern fn ang_don_present(slot: Int) -> String
|
|
||||||
extern fn ang_don_past(slot: Int) -> String
|
|
||||||
extern fn ang_willan_present(slot: Int) -> String
|
|
||||||
extern fn ang_willan_past(slot: Int) -> String
|
|
||||||
extern fn ang_magan_present(slot: Int) -> String
|
|
||||||
extern fn ang_magan_past(slot: Int) -> String
|
|
||||||
extern fn ang_witan_present(slot: Int) -> String
|
|
||||||
extern fn ang_witan_past(slot: Int) -> String
|
|
||||||
extern fn ang_weak_present_ending(slot: Int) -> String
|
|
||||||
extern fn ang_weak_past_stem(stem: String) -> String
|
|
||||||
extern fn ang_weak_past(stem: String, slot: Int) -> String
|
|
||||||
extern fn ang_weak_stem(verb: String) -> String
|
|
||||||
extern fn ang_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn ang_declension(noun: String, gender: String) -> String
|
|
||||||
extern fn ang_decline_strong_masc(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn ang_decline_strong_neut(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn ang_decline_weak(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn ang_decline(noun: String, gram_case: String, number: String, gender: String) -> String
|
|
||||||
extern fn ang_article_masculine(gram_case: String, number: String) -> String
|
|
||||||
extern fn ang_article_feminine(gram_case: String, number: String) -> String
|
|
||||||
extern fn ang_article_neuter(gram_case: String, number: String) -> String
|
|
||||||
extern fn ang_article(gender: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn ang_infer_gender(noun: String) -> String
|
|
||||||
extern fn ang_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
-27
@@ -1,27 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn ar_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn ar_str_len(s: String) -> Int
|
|
||||||
extern fn ar_str_drop_last(s: String, n: Int) -> String
|
|
||||||
extern fn ar_str_last_char(s: String) -> String
|
|
||||||
extern fn ar_slot(person: String, gender: String, number: String) -> Int
|
|
||||||
extern fn ar_perfect_suffix(slot: Int) -> String
|
|
||||||
extern fn ar_imperfect_prefix(slot: Int) -> String
|
|
||||||
extern fn ar_imperfect_suffix(slot: Int) -> String
|
|
||||||
extern fn ar_conjugate_form1(past_base: String, present_stem: String, tense: String, slot: Int) -> String
|
|
||||||
extern fn ar_irregular_kaana(slot: Int, tense: String) -> String
|
|
||||||
extern fn ar_irregular_qaala(slot: Int, tense: String) -> String
|
|
||||||
extern fn ar_irregular_jaa(slot: Int, tense: String) -> String
|
|
||||||
extern fn ar_irregular_raaa(slot: Int, tense: String) -> String
|
|
||||||
extern fn ar_irregular_araada(slot: Int, tense: String) -> String
|
|
||||||
extern fn ar_irregular_istata(slot: Int, tense: String) -> String
|
|
||||||
extern fn ar_irregular(verb: String, tense: String, slot: Int) -> String
|
|
||||||
extern fn ar_present_stem(verb: String) -> String
|
|
||||||
extern fn ar_conjugate(verb: String, tense: String, person: String, gender: String, number: String) -> String
|
|
||||||
extern fn ar_is_sun_letter(c: String) -> Bool
|
|
||||||
extern fn ar_definite_article(noun: String) -> String
|
|
||||||
extern fn ar_case_ending(kase: String, definite: String) -> String
|
|
||||||
extern fn ar_gender(noun: String) -> String
|
|
||||||
extern fn ar_masc_pl_ending(kase: String) -> String
|
|
||||||
extern fn ar_sound_plural(noun: String, gender: String) -> String
|
|
||||||
extern fn ar_noun_form(noun: String, gender: String, kase: String, number: String, definite: String) -> String
|
|
||||||
extern fn ar_verb_form(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
-35
@@ -1,35 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn cop_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn cop_str_len(s: String) -> Int
|
|
||||||
extern fn cop_drop(s: String, n: Int) -> String
|
|
||||||
extern fn cop_last_char(s: String) -> String
|
|
||||||
extern fn cop_slot(person: String, number: String) -> Int
|
|
||||||
extern fn cop_subject_prefix(person: String, number: String) -> String
|
|
||||||
extern fn cop_subject_prefix_gendered(person: String, gender: String, number: String) -> String
|
|
||||||
extern fn cop_copula_particle(gender: String, number: String) -> String
|
|
||||||
extern fn cop_shwpe_present(prefix: String) -> String
|
|
||||||
extern fn cop_shwpe_perfect(prefix: String) -> String
|
|
||||||
extern fn cop_shwpe_future(prefix: String) -> String
|
|
||||||
extern fn cop_bwk_present(prefix: String) -> String
|
|
||||||
extern fn cop_bwk_perfect(prefix: String) -> String
|
|
||||||
extern fn cop_bwk_future(prefix: String) -> String
|
|
||||||
extern fn cop_nau_present(prefix: String) -> String
|
|
||||||
extern fn cop_nau_perfect(prefix: String) -> String
|
|
||||||
extern fn cop_nau_future(prefix: String) -> String
|
|
||||||
extern fn cop_jw_present(prefix: String) -> String
|
|
||||||
extern fn cop_jw_perfect(prefix: String) -> String
|
|
||||||
extern fn cop_jw_future(prefix: String) -> String
|
|
||||||
extern fn cop_di_present(prefix: String) -> String
|
|
||||||
extern fn cop_di_perfect(prefix: String) -> String
|
|
||||||
extern fn cop_di_future(prefix: String) -> String
|
|
||||||
extern fn cop_is_copula(verb: String) -> Bool
|
|
||||||
extern fn cop_known_verb_prefixed(verb: String, tense: String, prefix: String) -> String
|
|
||||||
extern fn cop_regular_present(prefix: String, stem: String) -> String
|
|
||||||
extern fn cop_regular_perfect(prefix: String, stem: String) -> String
|
|
||||||
extern fn cop_regular_future(prefix: String, stem: String) -> String
|
|
||||||
extern fn cop_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn cop_article(gender: String, number: String, definite: String) -> String
|
|
||||||
extern fn cop_decline(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn cop_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
extern fn cop_noun_phrase_gendered(noun: String, gram_case: String, number: String, definite: String, gender: String) -> String
|
|
||||||
extern fn cop_map_canonical(verb: String) -> String
|
|
||||||
-13
@@ -1,13 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn de_article_def(gender: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn de_article_indef(gender: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn de_article(gender: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
extern fn de_adj_ending(gender: String, gram_case: String, number: String, article_type: String) -> String
|
|
||||||
extern fn de_noun_plural(noun: String, gender: String) -> String
|
|
||||||
extern fn de_case_ending(noun: String, gender: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn de_conjugate_weak(stem: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn de_irregular_present(verb: String, person: String, number: String) -> String
|
|
||||||
extern fn de_strong_past_stem(verb: String) -> String
|
|
||||||
extern fn de_norm_number(number: String) -> String
|
|
||||||
extern fn de_norm_person(person: String) -> String
|
|
||||||
extern fn de_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
-38
@@ -1,38 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn egy_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn egy_str_len(s: String) -> Int
|
|
||||||
extern fn egy_drop(s: String, n: Int) -> String
|
|
||||||
extern fn egy_last_char(s: String) -> String
|
|
||||||
extern fn egy_slot(person: String, number: String) -> Int
|
|
||||||
extern fn egy_slot_with_gender(person: String, gender: String, number: String) -> Int
|
|
||||||
extern fn egy_conjugate_pronoun(person: String, number: String) -> String
|
|
||||||
extern fn egy_suffix_pronoun(slot: Int) -> String
|
|
||||||
extern fn egy_is_copula(verb: String) -> Bool
|
|
||||||
extern fn egy_conjugate_copula(tense: String, slot: Int) -> String
|
|
||||||
extern fn egy_rdi_present(slot: Int) -> String
|
|
||||||
extern fn egy_rdi_past(slot: Int) -> String
|
|
||||||
extern fn egy_rdi_future(slot: Int) -> String
|
|
||||||
extern fn egy_mAA_present(slot: Int) -> String
|
|
||||||
extern fn egy_mAA_past(slot: Int) -> String
|
|
||||||
extern fn egy_mAA_future(slot: Int) -> String
|
|
||||||
extern fn egy_Dd_present(slot: Int) -> String
|
|
||||||
extern fn egy_Dd_past(slot: Int) -> String
|
|
||||||
extern fn egy_Dd_future(slot: Int) -> String
|
|
||||||
extern fn egy_Sm_present(slot: Int) -> String
|
|
||||||
extern fn egy_Sm_past(slot: Int) -> String
|
|
||||||
extern fn egy_Sm_future(slot: Int) -> String
|
|
||||||
extern fn egy_iri_present(slot: Int) -> String
|
|
||||||
extern fn egy_iri_past(slot: Int) -> String
|
|
||||||
extern fn egy_iri_future(slot: Int) -> String
|
|
||||||
extern fn egy_sdm_present(slot: Int) -> String
|
|
||||||
extern fn egy_sdm_past(slot: Int) -> String
|
|
||||||
extern fn egy_sdm_future(slot: Int) -> String
|
|
||||||
extern fn egy_known_verb(verb: String, tense: String, slot: Int) -> String
|
|
||||||
extern fn egy_regular_present(stem: String, slot: Int) -> String
|
|
||||||
extern fn egy_regular_past(stem: String, slot: Int) -> String
|
|
||||||
extern fn egy_regular_future(stem: String, slot: Int) -> String
|
|
||||||
extern fn egy_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn egy_decline(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn egy_fem(noun: String) -> String
|
|
||||||
extern fn egy_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
extern fn egy_map_canonical(verb: String) -> String
|
|
||||||
-30
@@ -1,30 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn enm_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn enm_drop(s: String, n: Int) -> String
|
|
||||||
extern fn enm_first_char(s: String) -> String
|
|
||||||
extern fn enm_slot(person: String, number: String) -> Int
|
|
||||||
extern fn enm_been_present(slot: Int) -> String
|
|
||||||
extern fn enm_been_past(slot: Int) -> String
|
|
||||||
extern fn enm_haven_present(slot: Int) -> String
|
|
||||||
extern fn enm_haven_past(slot: Int) -> String
|
|
||||||
extern fn enm_goon_present(slot: Int) -> String
|
|
||||||
extern fn enm_goon_past(slot: Int) -> String
|
|
||||||
extern fn enm_seen_present(slot: Int) -> String
|
|
||||||
extern fn enm_seen_past(slot: Int) -> String
|
|
||||||
extern fn enm_seyen_present(slot: Int) -> String
|
|
||||||
extern fn enm_seyen_past(slot: Int) -> String
|
|
||||||
extern fn enm_comen_present(slot: Int) -> String
|
|
||||||
extern fn enm_comen_past(slot: Int) -> String
|
|
||||||
extern fn enm_maken_present(slot: Int) -> String
|
|
||||||
extern fn enm_maken_past(slot: Int) -> String
|
|
||||||
extern fn enm_map_canonical(verb: String) -> String
|
|
||||||
extern fn enm_weak_stem(verb: String) -> String
|
|
||||||
extern fn enm_weak_present(stem: String, slot: Int) -> String
|
|
||||||
extern fn enm_weak_past(stem: String, slot: Int) -> String
|
|
||||||
extern fn enm_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn enm_irregular_plural(noun: String) -> String
|
|
||||||
extern fn enm_make_plural(noun: String) -> String
|
|
||||||
extern fn enm_decline(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn enm_is_vowel_initial(s: String) -> Bool
|
|
||||||
extern fn enm_indef_article(noun_phrase: String) -> String
|
|
||||||
extern fn enm_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
-23
@@ -1,23 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn es_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn es_str_drop_last(s: String, n: Int) -> String
|
|
||||||
extern fn es_str_last_char(s: String) -> String
|
|
||||||
extern fn es_str_last2(s: String) -> String
|
|
||||||
extern fn es_str_last3(s: String) -> String
|
|
||||||
extern fn es_verb_class(base: String) -> String
|
|
||||||
extern fn es_stem(base: String) -> String
|
|
||||||
extern fn es_slot(person: String, number: String) -> Int
|
|
||||||
extern fn es_irregular_present(verb: String, person: String, number: String) -> String
|
|
||||||
extern fn es_irregular_preterite(verb: String, person: String, number: String) -> String
|
|
||||||
extern fn es_irregular_imperfect(verb: String, person: String, number: String) -> String
|
|
||||||
extern fn es_regular_present(stem: String, vclass: String, slot: Int) -> String
|
|
||||||
extern fn es_regular_preterite(stem: String, vclass: String, slot: Int) -> String
|
|
||||||
extern fn es_regular_future(base: String, slot: Int) -> String
|
|
||||||
extern fn es_irregular_future_stem(verb: String) -> String
|
|
||||||
extern fn es_regular_imperfect(stem: String, vclass: String, slot: Int) -> String
|
|
||||||
extern fn es_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn es_gender(noun: String) -> String
|
|
||||||
extern fn es_invariant_plural(noun: String) -> String
|
|
||||||
extern fn es_pluralize(noun: String) -> String
|
|
||||||
extern fn es_starts_with_stressed_a(noun: String) -> Bool
|
|
||||||
extern fn es_agree_article(noun: String, definite: String, number: String) -> String
|
|
||||||
-17
@@ -1,17 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn fi_harmony(word: String) -> String
|
|
||||||
extern fn fi_suffix(base: String, harmony: String) -> String
|
|
||||||
extern fn fi_noun_case(stem: String, gram_case: String, number: String, harmony: String) -> String
|
|
||||||
extern fn fi_str_last_char(s: String) -> String
|
|
||||||
extern fn fi_apply_case(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn fi_verb_stem(dict_form: String) -> String
|
|
||||||
extern fn fi_irregular_verb(dict_form: String) -> [String]
|
|
||||||
extern fn fi_present_ending(stem: String, person: String, number: String, harmony: String) -> String
|
|
||||||
extern fn fi_past_stem(stem: String) -> String
|
|
||||||
extern fn fi_past_ending(stem: String, person: String, number: String, harmony: String) -> String
|
|
||||||
extern fn fi_neg_aux(person: String, number: String) -> String
|
|
||||||
extern fn fi_negative(verb: String, person: String, number: String) -> String
|
|
||||||
extern fn fi_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn fi_question_suffix(harmony: String) -> String
|
|
||||||
extern fn fi_make_question(verb_form: String, harmony: String) -> String
|
|
||||||
extern fn fi_full_paradigm(noun: String) -> [String]
|
|
||||||
-29
@@ -1,29 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn fr_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn fr_str_drop_last(s: String, n: Int) -> String
|
|
||||||
extern fn fr_str_last_char(s: String) -> String
|
|
||||||
extern fn fr_str_last2(s: String) -> String
|
|
||||||
extern fn fr_is_vowel_start(s: String) -> Bool
|
|
||||||
extern fn fr_is_known_irregular(verb: String) -> Bool
|
|
||||||
extern fn fr_verb_group(base: String) -> String
|
|
||||||
extern fn fr_stem(base: String) -> String
|
|
||||||
extern fn fr_slot(person: String, number: String) -> Int
|
|
||||||
extern fn fr_irregular_present(verb: String, person: String, number: String) -> String
|
|
||||||
extern fn fr_regular_present(stem: String, vgroup: String, slot: Int) -> String
|
|
||||||
extern fn fr_future_stem(base: String, vgroup: String) -> String
|
|
||||||
extern fn fr_regular_future(fstem: String, slot: Int) -> String
|
|
||||||
extern fn fr_irregular_future_stem(verb: String) -> String
|
|
||||||
extern fn fr_imperfect_stem(base: String, vgroup: String) -> String
|
|
||||||
extern fn fr_regular_imperfect(istem: String, slot: Int) -> String
|
|
||||||
extern fn fr_uses_etre(verb: String) -> Bool
|
|
||||||
extern fn fr_past_participle(verb: String) -> String
|
|
||||||
extern fn fr_avoir_present(slot: Int) -> String
|
|
||||||
extern fn fr_etre_present(slot: Int) -> String
|
|
||||||
extern fn fr_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn fr_gender(noun: String) -> String
|
|
||||||
extern fn fr_invariant_plural(noun: String) -> String
|
|
||||||
extern fn fr_pluralize(noun: String) -> String
|
|
||||||
extern fn fr_agree_article(noun: String, definite: String, number: String) -> String
|
|
||||||
extern fn fr_subject_starts_vowel(subject: String) -> Bool
|
|
||||||
extern fn fr_verb_ends_vowel(verb_form: String) -> Bool
|
|
||||||
extern fn fr_question_inversion(subject: String, verb_form: String) -> String
|
|
||||||
-38
@@ -1,38 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn fro_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn fro_drop(s: String, n: Int) -> String
|
|
||||||
extern fn fro_slot(person: String, number: String) -> Int
|
|
||||||
extern fn fro_map_canonical(verb: String) -> String
|
|
||||||
extern fn fro_estre_present(slot: Int) -> String
|
|
||||||
extern fn fro_estre_past(slot: Int) -> String
|
|
||||||
extern fn fro_estre_future(slot: Int) -> String
|
|
||||||
extern fn fro_avoir_present(slot: Int) -> String
|
|
||||||
extern fn fro_avoir_past(slot: Int) -> String
|
|
||||||
extern fn fro_avoir_future(slot: Int) -> String
|
|
||||||
extern fn fro_aler_present(slot: Int) -> String
|
|
||||||
extern fn fro_aler_past(slot: Int) -> String
|
|
||||||
extern fn fro_aler_future(slot: Int) -> String
|
|
||||||
extern fn fro_venir_present(slot: Int) -> String
|
|
||||||
extern fn fro_venir_past(slot: Int) -> String
|
|
||||||
extern fn fro_venir_future(slot: Int) -> String
|
|
||||||
extern fn fro_faire_present(slot: Int) -> String
|
|
||||||
extern fn fro_faire_past(slot: Int) -> String
|
|
||||||
extern fn fro_faire_future(slot: Int) -> String
|
|
||||||
extern fn fro_verb_class(verb: String) -> String
|
|
||||||
extern fn fro_verb_stem(verb: String, vclass: String) -> String
|
|
||||||
extern fn fro_conj1_present(stem: String, slot: Int) -> String
|
|
||||||
extern fn fro_conj1_past(stem: String, slot: Int) -> String
|
|
||||||
extern fn fro_conj1_future(verb: String, slot: Int) -> String
|
|
||||||
extern fn fro_conj2_present(stem: String, slot: Int) -> String
|
|
||||||
extern fn fro_conj2_past(stem: String, slot: Int) -> String
|
|
||||||
extern fn fro_conj2_future(verb: String, slot: Int) -> String
|
|
||||||
extern fn fro_conj3_present(stem: String, slot: Int) -> String
|
|
||||||
extern fn fro_conj3_past(stem: String, slot: Int) -> String
|
|
||||||
extern fn fro_conj3_future(verb: String, slot: Int) -> String
|
|
||||||
extern fn fro_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn fro_gender(noun: String) -> String
|
|
||||||
extern fn fro_decline_masc(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn fro_decline_fem(noun: String, number: String) -> String
|
|
||||||
extern fn fro_decline(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn fro_article(gender: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn fro_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
-26
@@ -1,26 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn gez_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn gez_str_len(s: String) -> Int
|
|
||||||
extern fn gez_str_drop_last(s: String, n: Int) -> String
|
|
||||||
extern fn gez_slot(person: String, number: String) -> Int
|
|
||||||
extern fn gez_slot_g(person: String, gender: String, number: String) -> Int
|
|
||||||
extern fn gez_kwn_perfect(slot: Int) -> String
|
|
||||||
extern fn gez_kwn_imperfect(slot: Int) -> String
|
|
||||||
extern fn gez_is_copula(verb: String) -> Bool
|
|
||||||
extern fn gez_conjugate_copula(tense: String, slot: Int) -> String
|
|
||||||
extern fn gez_hlw_perfect(slot: Int) -> String
|
|
||||||
extern fn gez_hlw_imperfect(slot: Int) -> String
|
|
||||||
extern fn gez_hbl_perfect(slot: Int) -> String
|
|
||||||
extern fn gez_hbl_imperfect(slot: Int) -> String
|
|
||||||
extern fn gez_ray_perfect(slot: Int) -> String
|
|
||||||
extern fn gez_ray_imperfect(slot: Int) -> String
|
|
||||||
extern fn gez_qwl_perfect(slot: Int) -> String
|
|
||||||
extern fn gez_qwl_imperfect(slot: Int) -> String
|
|
||||||
extern fn gez_generic_perfect(base3sg: String, slot: Int) -> String
|
|
||||||
extern fn gez_generic_imperfect(base3sg: String, slot: Int) -> String
|
|
||||||
extern fn gez_known_verb(verb: String, tense: String, slot: Int) -> String
|
|
||||||
extern fn gez_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn gez_is_fidel(noun: String) -> Bool
|
|
||||||
extern fn gez_decline(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn gez_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
extern fn gez_map_canonical(verb: String) -> String
|
|
||||||
-34
@@ -1,34 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn goh_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn goh_drop(s: String, n: Int) -> String
|
|
||||||
extern fn goh_slot(person: String, number: String) -> Int
|
|
||||||
extern fn goh_map_canonical(verb: String) -> String
|
|
||||||
extern fn goh_wesan_present(slot: Int) -> String
|
|
||||||
extern fn goh_wesan_past(slot: Int) -> String
|
|
||||||
extern fn goh_haben_present(slot: Int) -> String
|
|
||||||
extern fn goh_haben_past(slot: Int) -> String
|
|
||||||
extern fn goh_gan_present(slot: Int) -> String
|
|
||||||
extern fn goh_gan_past(slot: Int) -> String
|
|
||||||
extern fn goh_sehan_present(slot: Int) -> String
|
|
||||||
extern fn goh_sehan_past(slot: Int) -> String
|
|
||||||
extern fn goh_quethan_present(slot: Int) -> String
|
|
||||||
extern fn goh_quethan_past(slot: Int) -> String
|
|
||||||
extern fn goh_tuon_present(slot: Int) -> String
|
|
||||||
extern fn goh_tuon_past(slot: Int) -> String
|
|
||||||
extern fn goh_weak_present(stem: String, slot: Int) -> String
|
|
||||||
extern fn goh_weak_past(stem: String, slot: Int) -> String
|
|
||||||
extern fn goh_verb_stem(verb: String) -> String
|
|
||||||
extern fn goh_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn goh_stem_type(noun: String) -> String
|
|
||||||
extern fn goh_extract_stem(noun: String, stype: String) -> String
|
|
||||||
extern fn goh_decline_masc_a_sg(stem: String, gram_case: String) -> String
|
|
||||||
extern fn goh_decline_masc_a_pl(stem: String, gram_case: String) -> String
|
|
||||||
extern fn goh_decline_fem_o_sg(stem: String, gram_case: String) -> String
|
|
||||||
extern fn goh_decline_fem_o_pl(stem: String, gram_case: String) -> String
|
|
||||||
extern fn goh_decline_neut_a_sg(stem: String, gram_case: String) -> String
|
|
||||||
extern fn goh_decline_neut_a_pl(stem: String, gram_case: String) -> String
|
|
||||||
extern fn goh_decline_masc_n_sg(stem: String, gram_case: String) -> String
|
|
||||||
extern fn goh_decline_masc_n_pl(stem: String, gram_case: String) -> String
|
|
||||||
extern fn goh_decline(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn goh_demo_article(stype: String, number: String) -> String
|
|
||||||
extern fn goh_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
-37
@@ -1,37 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn got_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn got_str_drop_last(s: String, n: Int) -> String
|
|
||||||
extern fn got_slot(person: String, number: String) -> Int
|
|
||||||
extern fn got_map_canonical(verb: String) -> String
|
|
||||||
extern fn got_wisan_present(slot: Int) -> String
|
|
||||||
extern fn got_wisan_past(slot: Int) -> String
|
|
||||||
extern fn got_haban_present(slot: Int) -> String
|
|
||||||
extern fn got_haban_past(slot: Int) -> String
|
|
||||||
extern fn got_gaggan_present(slot: Int) -> String
|
|
||||||
extern fn got_gaggan_past(slot: Int) -> String
|
|
||||||
extern fn got_saihwan_present(slot: Int) -> String
|
|
||||||
extern fn got_saihwan_past(slot: Int) -> String
|
|
||||||
extern fn got_qithan_present(slot: Int) -> String
|
|
||||||
extern fn got_qithan_past(slot: Int) -> String
|
|
||||||
extern fn got_niman_present(slot: Int) -> String
|
|
||||||
extern fn got_niman_past(slot: Int) -> String
|
|
||||||
extern fn got_wk1_present_ending(slot: Int) -> String
|
|
||||||
extern fn got_wk1_past_ending(slot: Int) -> String
|
|
||||||
extern fn got_wk1_conjugate(stem: String, tense: String, slot: Int) -> String
|
|
||||||
extern fn got_wk2_present_ending(slot: Int) -> String
|
|
||||||
extern fn got_wk2_past_ending(slot: Int) -> String
|
|
||||||
extern fn got_wk2_conjugate(stem: String, tense: String, slot: Int) -> String
|
|
||||||
extern fn got_verb_class(verb: String) -> String
|
|
||||||
extern fn got_verb_stem(verb: String, vclass: String) -> String
|
|
||||||
extern fn got_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn got_decline_a_stem_sg(stem: String, gram_case: String) -> String
|
|
||||||
extern fn got_decline_a_stem_pl(stem: String, gram_case: String) -> String
|
|
||||||
extern fn got_decline_o_stem_sg(stem: String, gram_case: String) -> String
|
|
||||||
extern fn got_decline_o_stem_pl(stem: String, gram_case: String) -> String
|
|
||||||
extern fn got_decline_n_stem_sg(stem: String, gram_case: String) -> String
|
|
||||||
extern fn got_decline_n_stem_pl(stem: String, gram_case: String) -> String
|
|
||||||
extern fn got_stem_type(noun: String) -> String
|
|
||||||
extern fn got_extract_stem(noun: String, stype: String) -> String
|
|
||||||
extern fn got_demo_article(stype: String) -> String
|
|
||||||
extern fn got_decline(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn got_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
-45
@@ -1,45 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn grc_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn grc_str_drop_last(s: String, n: Int) -> String
|
|
||||||
extern fn grc_str_last_char(s: String) -> String
|
|
||||||
extern fn grc_str_last2(s: String) -> String
|
|
||||||
extern fn grc_str_last3(s: String) -> String
|
|
||||||
extern fn grc_slot(person: String, number: String) -> Int
|
|
||||||
extern fn grc_map_canonical(verb: String) -> String
|
|
||||||
extern fn grc_einai_present(slot: Int) -> String
|
|
||||||
extern fn grc_einai_imperfect(slot: Int) -> String
|
|
||||||
extern fn grc_einai_future(slot: Int) -> String
|
|
||||||
extern fn grc_echein_present(slot: Int) -> String
|
|
||||||
extern fn grc_echein_imperfect(slot: Int) -> String
|
|
||||||
extern fn grc_echein_aorist(slot: Int) -> String
|
|
||||||
extern fn grc_echein_future(slot: Int) -> String
|
|
||||||
extern fn grc_legein_present(slot: Int) -> String
|
|
||||||
extern fn grc_legein_imperfect(slot: Int) -> String
|
|
||||||
extern fn grc_legein_aorist(slot: Int) -> String
|
|
||||||
extern fn grc_legein_future(slot: Int) -> String
|
|
||||||
extern fn grc_horao_present(slot: Int) -> String
|
|
||||||
extern fn grc_horao_imperfect(slot: Int) -> String
|
|
||||||
extern fn grc_horao_aorist(slot: Int) -> String
|
|
||||||
extern fn grc_horao_future(slot: Int) -> String
|
|
||||||
extern fn grc_erchesthai_present(slot: Int) -> String
|
|
||||||
extern fn grc_erchesthai_imperfect(slot: Int) -> String
|
|
||||||
extern fn grc_erchesthai_aorist(slot: Int) -> String
|
|
||||||
extern fn grc_erchesthai_future(slot: Int) -> String
|
|
||||||
extern fn grc_thematic_present_ending(slot: Int) -> String
|
|
||||||
extern fn grc_thematic_imperfect_ending(slot: Int) -> String
|
|
||||||
extern fn grc_thematic_future_ending(slot: Int) -> String
|
|
||||||
extern fn grc_weak_aorist_ending(slot: Int) -> String
|
|
||||||
extern fn grc_present_stem(verb: String) -> String
|
|
||||||
extern fn grc_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn grc_declension(noun: String) -> String
|
|
||||||
extern fn grc_decline_2m(stem: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn grc_decline_2n(stem: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn grc_decline_1a(stem: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn grc_decline_1e(stem: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn grc_decline(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn grc_article_masculine(gram_case: String, number: String) -> String
|
|
||||||
extern fn grc_article_feminine(gram_case: String, number: String) -> String
|
|
||||||
extern fn grc_article_neuter(gram_case: String, number: String) -> String
|
|
||||||
extern fn grc_article(gender: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn grc_infer_gender(noun: String) -> String
|
|
||||||
extern fn grc_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
-30
@@ -1,30 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn he_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn he_str_len(s: String) -> Int
|
|
||||||
extern fn he_str_drop_last(s: String, n: Int) -> String
|
|
||||||
extern fn he_str_last_char(s: String) -> String
|
|
||||||
extern fn he_slot(person: String, gender: String, number: String) -> Int
|
|
||||||
extern fn he_present_form_code(slot: Int) -> Int
|
|
||||||
extern fn he_copula_past(slot: Int) -> String
|
|
||||||
extern fn he_copula_future(slot: Int) -> String
|
|
||||||
extern fn he_is_copula(verb: String) -> Bool
|
|
||||||
extern fn he_conjugate_copula(tense: String, slot: Int) -> String
|
|
||||||
extern fn he_present_lir_ot(form: Int) -> String
|
|
||||||
extern fn he_present_le_exol(form: Int) -> String
|
|
||||||
extern fn he_present_ledaber(form: Int) -> String
|
|
||||||
extern fn he_present_lalechet(form: Int) -> String
|
|
||||||
extern fn he_past_lir_ot(slot: Int) -> String
|
|
||||||
extern fn he_past_le_exol(slot: Int) -> String
|
|
||||||
extern fn he_past_ledaber(slot: Int) -> String
|
|
||||||
extern fn he_past_lalechet(slot: Int) -> String
|
|
||||||
extern fn he_future_lir_ot(slot: Int) -> String
|
|
||||||
extern fn he_future_le_exol(slot: Int) -> String
|
|
||||||
extern fn he_future_ledaber(slot: Int) -> String
|
|
||||||
extern fn he_future_lalechet(slot: Int) -> String
|
|
||||||
extern fn he_known_verb(verb: String, tense: String, slot: Int) -> String
|
|
||||||
extern fn he_conjugate(verb: String, tense: String, person: String, gender: String, number: String) -> String
|
|
||||||
extern fn he_pluralize(noun: String, gender: String) -> String
|
|
||||||
extern fn he_is_hebrew_script(noun: String) -> Bool
|
|
||||||
extern fn he_definite_prefix(noun: String) -> String
|
|
||||||
extern fn he_noun_phrase(noun: String, number: String, gender: String, definite: String) -> String
|
|
||||||
extern fn he_map_canonical(verb: String) -> String
|
|
||||||
-27
@@ -1,27 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn hi_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn hi_str_drop_last(s: String, n: Int) -> String
|
|
||||||
extern fn hi_str_last_char(s: String) -> String
|
|
||||||
extern fn hi_gender(noun: String) -> String
|
|
||||||
extern fn hi_masc_aa_stem(noun: String) -> String
|
|
||||||
extern fn hi_noun_direct_m(noun: String, number: String) -> String
|
|
||||||
extern fn hi_noun_oblique_m(noun: String, number: String) -> String
|
|
||||||
extern fn hi_noun_direct_f(noun: String, number: String) -> String
|
|
||||||
extern fn hi_noun_oblique_f(noun: String, number: String) -> String
|
|
||||||
extern fn hi_noun_direct(noun: String, gender: String, number: String) -> String
|
|
||||||
extern fn hi_noun_oblique(noun: String, gender: String, number: String) -> String
|
|
||||||
extern fn hi_postposition(gram_case: String) -> String
|
|
||||||
extern fn hi_agree_genitive(possessed_gender: String, possessed_number: String) -> String
|
|
||||||
extern fn hi_verb_stem(infinitive: String) -> String
|
|
||||||
extern fn hi_verb_stem_clean(infinitive: String) -> String
|
|
||||||
extern fn hi_present_aspect(gender: String, number: String) -> String
|
|
||||||
extern fn hi_aux_present(person: String, number: String) -> String
|
|
||||||
extern fn hi_past_suffix(gender: String, number: String) -> String
|
|
||||||
extern fn hi_past_irregular(stem: String, gender: String, number: String) -> String
|
|
||||||
extern fn hi_future_suffix(person: String, number: String, gender: String) -> String
|
|
||||||
extern fn hi_tense_suffix(tense: String, gender: String, number: String) -> String
|
|
||||||
extern fn hi_hona_present(person: String, number: String) -> String
|
|
||||||
extern fn hi_hona_past(gender: String, number: String) -> String
|
|
||||||
extern fn hi_conjugate(verb: String, tense: String, person: String, gender: String, number: String) -> String
|
|
||||||
extern fn hi_noun_with_post(noun: String, gender: String, number: String, gram_case: String) -> String
|
|
||||||
extern fn hi_genitive_phrase(possessor: String, possessor_gender: String, possessor_number: String, possessed: String, possessed_gender: String, possessed_number: String) -> String
|
|
||||||
-9
@@ -1,9 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn ja_verb_group(dict_form: String) -> String
|
|
||||||
extern fn ja_ichidan_stem(dict_form: String) -> String
|
|
||||||
extern fn ja_godan_stem_change(dict_form: String, row: String) -> String
|
|
||||||
extern fn ja_conjugate(dict_form: String, form: String) -> String
|
|
||||||
extern fn ja_particle(gram_case: String) -> String
|
|
||||||
extern fn ja_noun_phrase(noun: String, gram_case: String) -> String
|
|
||||||
extern fn ja_question_particle() -> String
|
|
||||||
extern fn ja_make_question(sentence: String) -> String
|
|
||||||
-41
@@ -1,41 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn la_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn la_str_drop_last(s: String, n: Int) -> String
|
|
||||||
extern fn la_str_last_char(s: String) -> String
|
|
||||||
extern fn la_str_last2(s: String) -> String
|
|
||||||
extern fn la_str_last3(s: String) -> String
|
|
||||||
extern fn la_slot(person: String, number: String) -> Int
|
|
||||||
extern fn la_verb_class(verb: String) -> String
|
|
||||||
extern fn la_stem(verb: String, vclass: String) -> String
|
|
||||||
extern fn la_perfect_stem(verb: String, vclass: String) -> String
|
|
||||||
extern fn la_perfect_ending(slot: Int) -> String
|
|
||||||
extern fn la_present_ending(vclass: String, slot: Int) -> String
|
|
||||||
extern fn la_present_form(stem: String, vclass: String, slot: Int) -> String
|
|
||||||
extern fn la_future_ending_12(slot: Int) -> String
|
|
||||||
extern fn la_future_ending_34(slot: Int) -> String
|
|
||||||
extern fn la_future_form(stem: String, vclass: String, slot: Int) -> String
|
|
||||||
extern fn la_esse_present(slot: Int) -> String
|
|
||||||
extern fn la_esse_past(slot: Int) -> String
|
|
||||||
extern fn la_esse_future(slot: Int) -> String
|
|
||||||
extern fn la_ire_present(slot: Int) -> String
|
|
||||||
extern fn la_ire_past(slot: Int) -> String
|
|
||||||
extern fn la_ire_future(slot: Int) -> String
|
|
||||||
extern fn la_velle_present(slot: Int) -> String
|
|
||||||
extern fn la_velle_past(slot: Int) -> String
|
|
||||||
extern fn la_velle_future(slot: Int) -> String
|
|
||||||
extern fn la_posse_present(slot: Int) -> String
|
|
||||||
extern fn la_posse_past(slot: Int) -> String
|
|
||||||
extern fn la_posse_future(slot: Int) -> String
|
|
||||||
extern fn la_irregular_perfect_stem(verb: String) -> String
|
|
||||||
extern fn la_map_canonical(verb: String) -> String
|
|
||||||
extern fn la_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn la_declension(noun: String) -> String
|
|
||||||
extern fn la_decline_1(stem: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn la_decline_2m(stem: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn la_decline_2n(stem: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn la_decline_3(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn la_decline_4(stem: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn la_decline_5(stem: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn la_decline_2er(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn la_decline(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn la_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
-30
@@ -1,30 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn non_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn non_drop(s: String, n: Int) -> String
|
|
||||||
extern fn non_last(s: String) -> String
|
|
||||||
extern fn non_slot(person: String, number: String) -> Int
|
|
||||||
extern fn non_vera_present(slot: Int) -> String
|
|
||||||
extern fn non_vera_past(slot: Int) -> String
|
|
||||||
extern fn non_hafa_present(slot: Int) -> String
|
|
||||||
extern fn non_hafa_past(slot: Int) -> String
|
|
||||||
extern fn non_ganga_present(slot: Int) -> String
|
|
||||||
extern fn non_ganga_past(slot: Int) -> String
|
|
||||||
extern fn non_sja_present(slot: Int) -> String
|
|
||||||
extern fn non_sja_past(slot: Int) -> String
|
|
||||||
extern fn non_segja_present(slot: Int) -> String
|
|
||||||
extern fn non_segja_past(slot: Int) -> String
|
|
||||||
extern fn non_koma_present(slot: Int) -> String
|
|
||||||
extern fn non_koma_past(slot: Int) -> String
|
|
||||||
extern fn non_map_canonical(verb: String) -> String
|
|
||||||
extern fn non_weak_present(stem: String, slot: Int) -> String
|
|
||||||
extern fn non_weak_past(stem: String, slot: Int) -> String
|
|
||||||
extern fn non_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn non_decline_masc(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn non_decline_fem(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn non_decline_neut(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn non_detect_gender(noun: String) -> String
|
|
||||||
extern fn non_decline(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn non_def_suffix_masc(gram_case: String, number: String) -> String
|
|
||||||
extern fn non_def_suffix_neut(gram_case: String, number: String) -> String
|
|
||||||
extern fn non_def_suffix_fem(gram_case: String, number: String) -> String
|
|
||||||
extern fn non_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
-19
@@ -1,19 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn peo_drop(s: String, n: Int) -> String
|
|
||||||
extern fn peo_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn peo_slot(person: String, number: String) -> Int
|
|
||||||
extern fn peo_present_suffix(slot: Int) -> String
|
|
||||||
extern fn peo_past_suffix(slot: Int) -> String
|
|
||||||
extern fn peo_ah_present(slot: Int) -> String
|
|
||||||
extern fn peo_ah_past(slot: Int) -> String
|
|
||||||
extern fn peo_kar_present(slot: Int) -> String
|
|
||||||
extern fn peo_kar_past(slot: Int) -> String
|
|
||||||
extern fn peo_xsaya_present(slot: Int) -> String
|
|
||||||
extern fn peo_tar_present(slot: Int) -> String
|
|
||||||
extern fn peo_da_present(slot: Int) -> String
|
|
||||||
extern fn peo_da_past(slot: Int) -> String
|
|
||||||
extern fn peo_map_canonical(verb: String) -> String
|
|
||||||
extern fn peo_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn peo_decline_astem(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn peo_decline(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn peo_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
-34
@@ -1,34 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn pi_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn pi_drop(s: String, n: Int) -> String
|
|
||||||
extern fn pi_last_char(s: String) -> String
|
|
||||||
extern fn pi_slot(person: String, number: String) -> Int
|
|
||||||
extern fn pi_present_ending(slot: Int) -> String
|
|
||||||
extern fn pi_aorist_ending(slot: Int) -> String
|
|
||||||
extern fn pi_future_ending(slot: Int) -> String
|
|
||||||
extern fn pi_hoti_present(slot: Int) -> String
|
|
||||||
extern fn pi_atthi_present(slot: Int) -> String
|
|
||||||
extern fn pi_hoti_aorist(slot: Int) -> String
|
|
||||||
extern fn pi_hoti_future(slot: Int) -> String
|
|
||||||
extern fn pi_gacchati_present(slot: Int) -> String
|
|
||||||
extern fn pi_gacchati_aorist(slot: Int) -> String
|
|
||||||
extern fn pi_gacchati_future(slot: Int) -> String
|
|
||||||
extern fn pi_passati_present(slot: Int) -> String
|
|
||||||
extern fn pi_passati_aorist(slot: Int) -> String
|
|
||||||
extern fn pi_passati_future(slot: Int) -> String
|
|
||||||
extern fn pi_vadati_present(slot: Int) -> String
|
|
||||||
extern fn pi_vadati_aorist(slot: Int) -> String
|
|
||||||
extern fn pi_vadati_future(slot: Int) -> String
|
|
||||||
extern fn pi_karoti_present(slot: Int) -> String
|
|
||||||
extern fn pi_karoti_aorist(slot: Int) -> String
|
|
||||||
extern fn pi_karoti_future(slot: Int) -> String
|
|
||||||
extern fn pi_map_canonical(verb: String) -> String
|
|
||||||
extern fn pi_regular_root(verb: String) -> String
|
|
||||||
extern fn pi_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn pi_decline_a_masc_sg(stem: String, gram_case: String) -> String
|
|
||||||
extern fn pi_decline_a_masc_pl(stem: String, gram_case: String) -> String
|
|
||||||
extern fn pi_decline_a_fem_sg(stem: String, gram_case: String) -> String
|
|
||||||
extern fn pi_decline_a_fem_pl(stem: String, gram_case: String) -> String
|
|
||||||
extern fn pi_detect_class(noun: String) -> String
|
|
||||||
extern fn pi_decline(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn pi_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
-14
@@ -1,14 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn ru_gender(noun: String) -> String
|
|
||||||
extern fn ru_stem_type(noun: String, gender: String) -> String
|
|
||||||
extern fn ru_noun_case(noun: String, gender: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn ru_decline_regular(noun: String, gender: String, stype: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn ru_decline_masc(noun: String, stype: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn ru_decline_fem(noun: String, stype: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn ru_decline_neut(noun: String, stype: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn ru_past_agree(verb_stem: String, gender: String, number: String) -> String
|
|
||||||
extern fn ru_conjugate_1st(stem: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn ru_conjugate_2nd(stem: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn ru_irregular(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn ru_past_stem(verb: String) -> String
|
|
||||||
extern fn ru_conjugate(verb: String, tense: String, person: String, number: String, gender: String) -> String
|
|
||||||
-36
@@ -1,36 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn sa_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn sa_str_drop_last(s: String, n: Int) -> String
|
|
||||||
extern fn sa_slot(person: String, number: String) -> Int
|
|
||||||
extern fn sa_map_canonical(verb: String) -> String
|
|
||||||
extern fn sa_as_present(slot: Int) -> String
|
|
||||||
extern fn sa_as_past(slot: Int) -> String
|
|
||||||
extern fn sa_as_future(slot: Int) -> String
|
|
||||||
extern fn sa_bhu_present(slot: Int) -> String
|
|
||||||
extern fn sa_bhu_past(slot: Int) -> String
|
|
||||||
extern fn sa_bhu_future(slot: Int) -> String
|
|
||||||
extern fn sa_gam_present(slot: Int) -> String
|
|
||||||
extern fn sa_gam_past(slot: Int) -> String
|
|
||||||
extern fn sa_gam_future(slot: Int) -> String
|
|
||||||
extern fn sa_drs_present(slot: Int) -> String
|
|
||||||
extern fn sa_drs_past(slot: Int) -> String
|
|
||||||
extern fn sa_drs_future(slot: Int) -> String
|
|
||||||
extern fn sa_vad_present(slot: Int) -> String
|
|
||||||
extern fn sa_vad_past(slot: Int) -> String
|
|
||||||
extern fn sa_vad_future(slot: Int) -> String
|
|
||||||
extern fn sa_kr_present(slot: Int) -> String
|
|
||||||
extern fn sa_kr_past(slot: Int) -> String
|
|
||||||
extern fn sa_kr_future(slot: Int) -> String
|
|
||||||
extern fn sa_class1_present_ending(slot: Int) -> String
|
|
||||||
extern fn sa_class1_past_ending(slot: Int) -> String
|
|
||||||
extern fn sa_class1_future_ending(slot: Int) -> String
|
|
||||||
extern fn sa_class1_conjugate(stem: String, tense: String, slot: Int) -> String
|
|
||||||
extern fn sa_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn sa_decline_a_stem_sg(stem: String, gram_case: String) -> String
|
|
||||||
extern fn sa_decline_a_stem_pl(stem: String, gram_case: String) -> String
|
|
||||||
extern fn sa_decline_aa_stem_sg(stem: String, gram_case: String) -> String
|
|
||||||
extern fn sa_decline_aa_stem_pl(stem: String, gram_case: String) -> String
|
|
||||||
extern fn sa_stem_type(noun: String) -> String
|
|
||||||
extern fn sa_extract_stem(noun: String, stype: String) -> String
|
|
||||||
extern fn sa_decline(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn sa_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
-22
@@ -1,22 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn sga_drop(s: String, n: Int) -> String
|
|
||||||
extern fn sga_first(s: String) -> String
|
|
||||||
extern fn sga_rest(s: String) -> String
|
|
||||||
extern fn sga_slot(person: String, number: String) -> Int
|
|
||||||
extern fn sga_lenite(word: String) -> String
|
|
||||||
extern fn sga_copula_present(slot: Int) -> String
|
|
||||||
extern fn sga_bith_present(slot: Int) -> String
|
|
||||||
extern fn sga_bith_past(slot: Int) -> String
|
|
||||||
extern fn sga_teit_present(slot: Int) -> String
|
|
||||||
extern fn sga_teit_past(slot: Int) -> String
|
|
||||||
extern fn sga_gaibid_present(slot: Int) -> String
|
|
||||||
extern fn sga_adci_present(slot: Int) -> String
|
|
||||||
extern fn sga_asbeir_present(slot: Int) -> String
|
|
||||||
extern fn sga_map_canonical(verb: String) -> String
|
|
||||||
extern fn sga_ai_present(stem: String, slot: Int) -> String
|
|
||||||
extern fn sga_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn sga_decline_ostem(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn sga_decline_astem(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn sga_detect_gender(noun: String) -> String
|
|
||||||
extern fn sga_decline(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn sga_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
-29
@@ -1,29 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn sux_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn sux_str_drop_last(s: String, n: Int) -> String
|
|
||||||
extern fn sux_str_last_char(s: String) -> String
|
|
||||||
extern fn sux_str_last2(s: String) -> String
|
|
||||||
extern fn sux_slot(person: String, number: String) -> Int
|
|
||||||
extern fn sux_ergative_suffix(person: String, number: String) -> String
|
|
||||||
extern fn sux_absolutive_suffix(person: String, number: String) -> String
|
|
||||||
extern fn sux_map_canonical(verb: String) -> String
|
|
||||||
extern fn sux_personal_suffix(slot: Int) -> String
|
|
||||||
extern fn sux_me_present(slot: Int) -> String
|
|
||||||
extern fn sux_me_past(slot: Int) -> String
|
|
||||||
extern fn sux_dug4_present(slot: Int) -> String
|
|
||||||
extern fn sux_dug4_past(slot: Int) -> String
|
|
||||||
extern fn sux_du_present(slot: Int) -> String
|
|
||||||
extern fn sux_du_past(slot: Int) -> String
|
|
||||||
extern fn sux_igibar_present(slot: Int) -> String
|
|
||||||
extern fn sux_igibar_past(slot: Int) -> String
|
|
||||||
extern fn sux_ak_present(slot: Int) -> String
|
|
||||||
extern fn sux_ak_past(slot: Int) -> String
|
|
||||||
extern fn sux_tum2_present(slot: Int) -> String
|
|
||||||
extern fn sux_tum2_past(slot: Int) -> String
|
|
||||||
extern fn sux_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn sux_is_animate(noun: String) -> Bool
|
|
||||||
extern fn sux_case_suffix(gram_case: String) -> String
|
|
||||||
extern fn sux_decline(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn sux_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
extern fn sux_verb_chain(agent: String, verb: String, patient: String, tense: String) -> String
|
|
||||||
extern fn sux_realize_sentence(intent: String, agent: String, predicate: String, patient: String, tense: String) -> String
|
|
||||||
-23
@@ -1,23 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn sw_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn sw_str_drop_last(s: String, n: Int) -> String
|
|
||||||
extern fn sw_str_first_char(s: String) -> String
|
|
||||||
extern fn sw_str_first2(s: String) -> String
|
|
||||||
extern fn sw_str_first3(s: String) -> String
|
|
||||||
extern fn sw_str_last_char(s: String) -> String
|
|
||||||
extern fn sw_is_class1_noun(noun: String) -> Bool
|
|
||||||
extern fn sw_noun_class(noun: String) -> String
|
|
||||||
extern fn sw_subj_prefix(person: String, number: String, noun_class: String) -> String
|
|
||||||
extern fn sw_obj_prefix(person: String, number: String, noun_class: String) -> String
|
|
||||||
extern fn sw_tense_marker(tense: String) -> String
|
|
||||||
extern fn sw_verb_final(tense: String, negative: Bool) -> String
|
|
||||||
extern fn sw_neg_subj_prefix(person: String, number: String, noun_class: String) -> String
|
|
||||||
extern fn sw_verb_stem(infinitive: String) -> String
|
|
||||||
extern fn sw_conjugate(verb_stem: String, person: String, number: String, noun_class: String, tense: String) -> String
|
|
||||||
extern fn sw_negative(verb_stem: String, person: String, number: String, noun_class: String, tense: String) -> String
|
|
||||||
extern fn sw_noun_plural(noun: String) -> String
|
|
||||||
extern fn sw_adj_prefix(noun_class: String, number: String) -> String
|
|
||||||
extern fn sw_agree_adj(adj_stem: String, noun_class: String, number: String) -> String
|
|
||||||
extern fn sw_demonstrative(noun_class: String, number: String, proximity: String) -> String
|
|
||||||
extern fn sw_copula_present(person: String, number: String, use_case: String) -> String
|
|
||||||
extern fn sw_copula_neg_present(person: String, number: String) -> String
|
|
||||||
-17
@@ -1,17 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn txb_drop(s: String, n: Int) -> String
|
|
||||||
extern fn txb_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn txb_slot(person: String, number: String) -> Int
|
|
||||||
extern fn txb_pres1_suffix(slot: Int) -> String
|
|
||||||
extern fn txb_kam_present(slot: Int) -> String
|
|
||||||
extern fn txb_ya_present(slot: Int) -> String
|
|
||||||
extern fn txb_wes_present(slot: Int) -> String
|
|
||||||
extern fn txb_lyut_present(slot: Int) -> String
|
|
||||||
extern fn txb_wak_present(slot: Int) -> String
|
|
||||||
extern fn txb_map_canonical(verb: String) -> String
|
|
||||||
extern fn txb_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn txb_decline_masc(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn txb_decline_fem(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn txb_detect_gender(noun: String) -> String
|
|
||||||
extern fn txb_decline(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn txb_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
-25
@@ -1,25 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn uga_str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn uga_str_len(s: String) -> Int
|
|
||||||
extern fn uga_str_drop_last(s: String, n: Int) -> String
|
|
||||||
extern fn uga_slot(person: String, number: String) -> Int
|
|
||||||
extern fn uga_slot_g(person: String, gender: String, number: String) -> Int
|
|
||||||
extern fn uga_kn_perfect(slot: Int) -> String
|
|
||||||
extern fn uga_kn_imperfect(slot: Int) -> String
|
|
||||||
extern fn uga_is_copula(verb: String) -> Bool
|
|
||||||
extern fn uga_conjugate_copula(tense: String, slot: Int) -> String
|
|
||||||
extern fn uga_hlk_perfect(slot: Int) -> String
|
|
||||||
extern fn uga_hlk_imperfect(slot: Int) -> String
|
|
||||||
extern fn uga_ray_perfect(slot: Int) -> String
|
|
||||||
extern fn uga_ray_imperfect(slot: Int) -> String
|
|
||||||
extern fn uga_amr_perfect(slot: Int) -> String
|
|
||||||
extern fn uga_amr_imperfect(slot: Int) -> String
|
|
||||||
extern fn uga_generic_perfect(base3sg: String, slot: Int) -> String
|
|
||||||
extern fn uga_generic_imperfect(base3sg: String, slot: Int) -> String
|
|
||||||
extern fn uga_known_verb(verb: String, tense: String, slot: Int) -> String
|
|
||||||
extern fn uga_conjugate(verb: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn uga_strip_nom(noun: String) -> String
|
|
||||||
extern fn uga_is_fem(noun: String) -> Bool
|
|
||||||
extern fn uga_decline(noun: String, gram_case: String, number: String) -> String
|
|
||||||
extern fn uga_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
|
|
||||||
extern fn uga_map_canonical(verb: String) -> String
|
|
||||||
-27
@@ -1,27 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn str_ends(s: String, suf: String) -> Bool
|
|
||||||
extern fn str_last_char(s: String) -> String
|
|
||||||
extern fn str_last2(s: String) -> String
|
|
||||||
extern fn str_last3(s: String) -> String
|
|
||||||
extern fn str_drop_last(s: String, n: Int) -> String
|
|
||||||
extern fn is_vowel(c: String) -> Bool
|
|
||||||
extern fn morph_apply_suffix(base: String, suffix: String) -> String
|
|
||||||
extern fn en_irregular_plural(word: String) -> String
|
|
||||||
extern fn en_irregular_singular(word: String) -> String
|
|
||||||
extern fn en_irregular_verb(base: String) -> [String]
|
|
||||||
extern fn en_verb_3sg(base: String) -> String
|
|
||||||
extern fn en_should_double_final(base: String) -> Bool
|
|
||||||
extern fn en_verb_past(base: String) -> String
|
|
||||||
extern fn en_verb_gerund(base: String) -> String
|
|
||||||
extern fn en_pluralize_regular(singular: String) -> String
|
|
||||||
extern fn en_verb_form(base: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn agree_determiner(det: String, noun: String) -> String
|
|
||||||
extern fn morph_pluralize(noun: String, profile: [String]) -> String
|
|
||||||
extern fn morph_map_canonical(verb: String, code: String) -> String
|
|
||||||
extern fn morph_conjugate(verb: String, tense: String, person: String, number: String, profile: [String]) -> String
|
|
||||||
extern fn morph_inflect(word: String, features: String, profile: [String]) -> String
|
|
||||||
extern fn pluralize(singular: String) -> String
|
|
||||||
extern fn singularize(plural: String) -> String
|
|
||||||
extern fn verb_form(base: String, tense: String, person: String, number: String) -> String
|
|
||||||
extern fn irregular_plural(word: String) -> String
|
|
||||||
extern fn irregular_singular(word: String) -> String
|
|
||||||
-39
@@ -1,39 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn is_protected_node(id: String) -> Bool
|
|
||||||
extern fn api_err_protected(id: String) -> String
|
|
||||||
extern fn api_json_escape(s: String) -> String
|
|
||||||
extern fn api_query_param(path: String, key: String) -> String
|
|
||||||
extern fn api_query_int(path: String, key: String, default_val: Int) -> Int
|
|
||||||
extern fn api_ok(extra: String) -> String
|
|
||||||
extern fn api_err(msg: String) -> String
|
|
||||||
extern fn api_nonempty(s: String) -> Bool
|
|
||||||
extern fn api_or_empty(s: String) -> String
|
|
||||||
extern fn api_persisted(id: String) -> Bool
|
|
||||||
extern fn api_not_persisted(id: String) -> String
|
|
||||||
extern fn handle_api_begin_session(body: String) -> String
|
|
||||||
extern fn handle_api_compile_ctx(body: String) -> String
|
|
||||||
extern fn handle_api_remember(body: String) -> String
|
|
||||||
extern fn handle_api_node_create(body: String) -> String
|
|
||||||
extern fn handle_api_node_delete(body: String) -> String
|
|
||||||
extern fn handle_api_node_update(body: String) -> String
|
|
||||||
extern fn handle_api_recall(method: String, path: String, body: String) -> String
|
|
||||||
extern fn handle_api_search_knowledge(method: String, path: String, body: String) -> String
|
|
||||||
extern fn handle_api_browse_knowledge(path: String, body: String) -> String
|
|
||||||
extern fn handle_api_capture_knowledge(body: String) -> String
|
|
||||||
extern fn handle_api_evolve_knowledge(body: String) -> String
|
|
||||||
extern fn handle_api_promote_knowledge(body: String) -> String
|
|
||||||
extern fn handle_api_browse_processes(method: String, path: String, body: String) -> String
|
|
||||||
extern fn handle_api_define_process(body: String) -> String
|
|
||||||
extern fn handle_api_log_state_event(body: String) -> String
|
|
||||||
extern fn handle_api_list_state_events(method: String, path: String, body: String) -> String
|
|
||||||
extern fn handle_api_inspect_config(path: String, body: String) -> String
|
|
||||||
extern fn handle_api_tune_config(body: String) -> String
|
|
||||||
extern fn handle_api_inspect_graph(method: String, path: String, body: String) -> String
|
|
||||||
extern fn handle_api_link_entities(body: String) -> String
|
|
||||||
extern fn handle_api_forget(body: String) -> String
|
|
||||||
extern fn handle_api_evolve_memory(body: String) -> String
|
|
||||||
extern fn handle_api_memory_delete(body: String) -> String
|
|
||||||
extern fn handle_api_memory_update(body: String) -> String
|
|
||||||
extern fn handle_api_cultivate(body: String) -> String
|
|
||||||
extern fn handle_api_list_typed(node_type: String, path: String, body: String) -> String
|
|
||||||
extern fn handle_api_consolidate(body: String) -> String
|
|
||||||
-10
@@ -1,10 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn agent_person(agent: String) -> String
|
|
||||||
extern fn agent_number(agent: String) -> String
|
|
||||||
extern fn realize_np(referent: String, number: String) -> String
|
|
||||||
extern fn realize_vp_lang(base_verb: String, tense: String, aspect: String, person: String, number: String, profile: [String]) -> [String]
|
|
||||||
extern fn realize_question_lang(predicate: String, tense: String, aspect: String, person: String, number: String, agent: String, patient: String, location: String, profile: [String]) -> String
|
|
||||||
extern fn capitalize_first(s: String) -> String
|
|
||||||
extern fn add_punct(s: String, intent: String) -> String
|
|
||||||
extern fn realize_lang(form: [String], profile: [String]) -> String
|
|
||||||
extern fn realize(form: [String]) -> String
|
|
||||||
-16
@@ -1,16 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn flag_true(body: String, key: String) -> Bool
|
|
||||||
extern fn rate_limit_check(ip: String, path: String) -> String
|
|
||||||
extern fn strip_query(path: String) -> String
|
|
||||||
extern fn err_404(path: String) -> String
|
|
||||||
extern fn err_405(method: String, path: String) -> String
|
|
||||||
extern fn route_health() -> String
|
|
||||||
extern fn route_lineage() -> String
|
|
||||||
extern fn route_imprint_contextual(body: String) -> String
|
|
||||||
extern fn route_imprint_user(body: String) -> String
|
|
||||||
extern fn route_synthesize(body: String) -> String
|
|
||||||
extern fn handle_dharma_recv(body: String) -> String
|
|
||||||
extern fn connectd_get(suffix: String) -> String
|
|
||||||
extern fn connectd_post(suffix: String, body: String) -> String
|
|
||||||
extern fn handle_connectors(method: String, clean: String, body: String) -> String
|
|
||||||
extern fn handle_request(method: String, path: String, body: String) -> String
|
|
||||||
-29
@@ -1,29 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn soft_bell_threshold() -> Int
|
|
||||||
extern fn hard_bell_threshold() -> Int
|
|
||||||
extern fn safety_score_crisis(input: String) -> Int
|
|
||||||
extern fn safety_score_harm(input: String) -> Int
|
|
||||||
extern fn safety_score_danger(input: String) -> Int
|
|
||||||
extern fn safety_score_distress_history(history: String) -> Int
|
|
||||||
extern fn safety_threat_score(input: String, history: String) -> Int
|
|
||||||
extern fn safety_screen(input: String, history: String) -> String
|
|
||||||
extern fn safety_validate(output: String, action: String) -> String
|
|
||||||
extern fn safety_log_bell(level: String, reason: String, input_summary: String) -> String
|
|
||||||
extern fn safety_self_harm_phrases() -> String
|
|
||||||
extern fn safety_abuse_phrases() -> String
|
|
||||||
extern fn safety_general_hard_phrases() -> String
|
|
||||||
extern fn safety_threat_to_others_phrases() -> String
|
|
||||||
extern fn safety_soft_phrases() -> String
|
|
||||||
extern fn safety_normalize(message: String) -> String
|
|
||||||
extern fn safety_any_match(text: String, phrases_json: String) -> Bool
|
|
||||||
extern fn safety_count_match(text: String, phrases_json: String) -> Int
|
|
||||||
extern fn safety_positive_phrases() -> String
|
|
||||||
extern fn safety_detect_positive_level(message: String) -> String
|
|
||||||
extern fn safety_detect_bell_level(message: String) -> String
|
|
||||||
extern fn safety_classify_hard_bell(message: String) -> String
|
|
||||||
extern fn safety_soft_directive() -> String
|
|
||||||
extern fn safety_hard_directive(hard_type: String) -> String
|
|
||||||
extern fn safety_augment_system(system: String, user_msg: String) -> String
|
|
||||||
extern fn safety_contact_path() -> String
|
|
||||||
extern fn handle_safety_contact_get() -> String
|
|
||||||
extern fn handle_safety_contact_post(body: String) -> String
|
|
||||||
-18
@@ -1,18 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn sem_frame(intent: String, subject: String, obj: String, modifiers: String) -> [String]
|
|
||||||
extern fn sem_frame_lang(intent: String, subject: String, obj: String, modifiers: String, lang_code: String) -> [String]
|
|
||||||
extern fn sem_frame_simple(intent: String, subject: String) -> [String]
|
|
||||||
extern fn sem_frame_obj(intent: String, subject: String, obj: String) -> [String]
|
|
||||||
extern fn sem_intent(frame: [String]) -> String
|
|
||||||
extern fn sem_subject(frame: [String]) -> String
|
|
||||||
extern fn sem_object(frame: [String]) -> String
|
|
||||||
extern fn sem_modifiers(frame: [String]) -> String
|
|
||||||
extern fn sem_lang(frame: [String]) -> String
|
|
||||||
extern fn sem_first_modifier(mods: String) -> String
|
|
||||||
extern fn sem_intent_to_realize(intent: String) -> String
|
|
||||||
extern fn sem_to_spec(frame: [String]) -> [String]
|
|
||||||
extern fn sem_to_spec_full(frame: [String], verb: String, tense: String, aspect: String) -> [String]
|
|
||||||
extern fn sem_realize_greet(subject: String) -> String
|
|
||||||
extern fn sem_realize(frame: [String]) -> String
|
|
||||||
extern fn sem_realize_full(frame: [String], verb: String, tense: String, aspect: String) -> String
|
|
||||||
extern fn sem_realize_lang(frame: [String], lang_code: String) -> String
|
|
||||||
-17
@@ -1,17 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn session_title_from_message(message: String) -> String
|
|
||||||
extern fn session_make_content(id: String, title: String, created_at: Int, updated_at: Int, folder: String) -> String
|
|
||||||
extern fn session_exists(session_id: String) -> Bool
|
|
||||||
extern fn session_create(body: String) -> String
|
|
||||||
extern fn session_create_cleanup(session_id: String) -> String
|
|
||||||
extern fn session_list() -> String
|
|
||||||
extern fn session_get(session_id: String) -> String
|
|
||||||
extern fn session_delete(session_id: String) -> String
|
|
||||||
extern fn session_update_patch(session_id: String, body: String) -> String
|
|
||||||
extern fn session_search_entry(node: String) -> String
|
|
||||||
extern fn session_search(query: String) -> String
|
|
||||||
extern fn session_hist_load(session_id: String) -> String
|
|
||||||
extern fn session_hist_save(session_id: String, hist: String) -> Void
|
|
||||||
extern fn session_update_meta_timestamp(session_id: String) -> Void
|
|
||||||
extern fn session_auto_title(session_id: String, first_message: String) -> Void
|
|
||||||
extern fn handle_session_approve(session_id: String, body: String) -> String
|
|
||||||
+2625
-2088
File diff suppressed because one or more lines are too long
+19
@@ -0,0 +1,19 @@
|
|||||||
|
# soul.c.stamp — fingerprint of the .el sources dist/soul.c was generated from.
|
||||||
|
# Written by tools/soulc-stamp.sh --write. Do not hand-edit.
|
||||||
|
# generated_amalgam_sha256 00d1207de8fd872d111e00a37b7dcc46729bf095baa6778de3bcc30b0bf46b05
|
||||||
|
# generated_amalgam_bytes 1226805
|
||||||
|
MISSING __compiler__
|
||||||
|
f8597e10546654bce3fbbe40461b2da59d0e06dbf1b038d1d362d24f949e3911 awareness.el
|
||||||
|
2ff2dada732918c788a9ef66c6fd54c7a24cc4bbd4829197fe945d3a75ca1929 chat.el
|
||||||
|
42288c212cbf72fb1e8ecbd4d9900e4e9ee1cfa475b7974295c7637f1bf2939f elp-input.el
|
||||||
|
b3f77f49d6086932c38bd17fe7a5eaf8bce25685f6fc3e1750f05729c6b49b9e imprint.el
|
||||||
|
fba8ffdb9ba72bca5b09ca1c93a520edc52f3f4d8aec2c7585fe9b17e06420b2 manifest.el
|
||||||
|
550a72e234ae8cec1f33e02108fd365353f45edd88513da90b792e79b6c0e5f0 memory.el
|
||||||
|
5ec07ec9785b02abe32f3ff7acf2d1f9f7e07c0967fac97e6eff17d7110b5c84 neuron-api.el
|
||||||
|
03c47c451e0e87f2c252cadb4b765867943962a804f548dd53adeef0520912c8 persist.el
|
||||||
|
a6d69f3fc55233d9d3300160fd46a1551f2064bcd0fb84e2c9e432f636a72476 routes.el
|
||||||
|
c28e36952ec56525963a0bdf29455ab097d3b0c5653d19c25fbb005e1069a1f7 safety.el
|
||||||
|
fd3ab91d0ae0ea26639e21bef2f8f94054dc4b02eae68b19e3fe689d2769aad4 sessions.el
|
||||||
|
5613b60d74d5d7768f46da5ac435a5dd99d38c27f0f7013c89fa27e98dc8a21c soul.el
|
||||||
|
30337940905171a9645b0929f0a412ce6b3dccb1246495070c553bca0bbae6cd stewardship.el
|
||||||
|
95dab72be4ee1dd1d28bab63412964a72460126951764e3f74b1c2d49b6d7b35 studio.el
|
||||||
-8
@@ -1,8 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn init_soul_edges() -> Void
|
|
||||||
extern fn ensure_self_canonical_bridge() -> Void
|
|
||||||
extern fn aff_try_slot(slot_json: String, aff_7d_ts: Int, acc_key: String) -> Void
|
|
||||||
extern fn load_identity_context() -> Void
|
|
||||||
extern fn seed_persona_from_env() -> Void
|
|
||||||
extern fn emit_session_start_event() -> Void
|
|
||||||
extern fn layered_cycle(raw_input: String) -> String
|
|
||||||
-11
@@ -1,11 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn steward_log_event(kind: String, detail: String) -> Void
|
|
||||||
extern fn steward_get_mission() -> String
|
|
||||||
extern fn steward_align(input: String, imprint_id: String) -> String
|
|
||||||
extern fn steward_validate_imprint(imprint_id: String, tool_name: String) -> String
|
|
||||||
extern fn steward_cgi_check(action: String) -> String
|
|
||||||
extern fn steward_fingerprint_session(input: String, session_id: String) -> String
|
|
||||||
extern fn extract_dim(content: String, key: String) -> String
|
|
||||||
extern fn steward_build_baseline() -> String
|
|
||||||
extern fn steward_check_continuity(current_fingerprint: String, session_id: String) -> String
|
|
||||||
extern fn steward_session_check(input: String, session_id: String) -> String
|
|
||||||
-12
@@ -1,12 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn auth_headers(tok: String) -> Map
|
|
||||||
extern fn axon_get(path: String) -> String
|
|
||||||
extern fn axon_post(path: String, body: String) -> String
|
|
||||||
extern fn handle_conversations(method: String) -> String
|
|
||||||
extern fn handle_config(method: String, body: String) -> String
|
|
||||||
extern fn dharma_registry() -> String
|
|
||||||
extern fn dharma_network_state() -> String
|
|
||||||
extern fn handle_dharma(path: String, method: String, body: String) -> String
|
|
||||||
extern fn handle_tool(path: String, method: String, body: String) -> String
|
|
||||||
extern fn handle_nlg(path: String, method: String, body: String) -> String
|
|
||||||
extern fn render_studio() -> String
|
|
||||||
-20
@@ -1,20 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn lex_word(entry: [String]) -> String
|
|
||||||
extern fn lex_pos(entry: [String]) -> String
|
|
||||||
extern fn lex_form(entry: [String], idx: Int) -> String
|
|
||||||
extern fn lex_class(entry: [String]) -> String
|
|
||||||
extern fn make_entry(word: String, pos: String, f0: String, f1: String, f2: String, f3: String, f4: String, cls: String) -> [String]
|
|
||||||
extern fn make_entry2(word: String, pos: String, f0: String, f1: String, cls: String) -> [String]
|
|
||||||
extern fn make_entry3(word: String, pos: String, f0: String, f1: String, f2: String, cls: String) -> [String]
|
|
||||||
extern fn make_entry1(word: String, pos: String, f0: String, cls: String) -> [String]
|
|
||||||
extern fn build_vocab() -> [[String]]
|
|
||||||
extern fn get_vocab() -> [[String]]
|
|
||||||
extern fn vocab_lookup(word: String, lang_code: String) -> [String]
|
|
||||||
extern fn vocab_lookup_en(word: String) -> [String]
|
|
||||||
extern fn vocab_synonym(word: String, lang_register: String, lang_code: String) -> String
|
|
||||||
extern fn vocab_by_pos(pos: String) -> [[String]]
|
|
||||||
extern fn vocab_by_class(cls: String) -> [[String]]
|
|
||||||
extern fn entry_found(entry: [String]) -> Bool
|
|
||||||
extern fn entry_word(entry: [String]) -> String
|
|
||||||
extern fn entry_pos(entry: [String]) -> String
|
|
||||||
extern fn entry_form(entry: [String], n: Int) -> String
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn elp_extract_topic(msg: String) -> String
|
|
||||||
extern fn elp_detect_predicate(msg: String) -> String
|
|
||||||
extern fn elp_parse(msg: String) -> String
|
|
||||||
extern fn handle_elp_chat(body: String) -> String
|
|
||||||
@@ -1,7 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn imprint_current() -> String
|
|
||||||
extern fn imprint_load(imprint_id: String) -> String
|
|
||||||
extern fn imprint_respond(input: String, imprint_id: String) -> String
|
|
||||||
extern fn imprint_surface_knowledge(query: String, imprint_id: String) -> String
|
|
||||||
extern fn imprint_surface_memory_read(query: String) -> String
|
|
||||||
extern fn imprint_unload() -> Void
|
|
||||||
+680
-123
@@ -77,111 +77,427 @@ fn tool(name: String, desc: String) -> String {
|
|||||||
return "{\"name\":\"" + name + "\",\"description\":\"" + desc + "\",\"inputSchema\":{\"type\":\"object\",\"properties\":{}}}"
|
return "{\"name\":\"" + name + "\",\"description\":\"" + desc + "\",\"inputSchema\":{\"type\":\"object\",\"properties\":{}}}"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// tool_s — tool entry with an EXPLICIT JSON-Schema for its inputs. Used for tools
|
||||||
|
// whose arguments must actually bite: unless the bounding/targeting params are
|
||||||
|
// advertised, the MCP client sends nothing and the soul returns the FULL
|
||||||
|
// neighborhood (480-775KB, over transport limits). Declaring the schema is what
|
||||||
|
// makes a targeted call (entity_id/depth/compact/query/limit) reach the soul.
|
||||||
|
fn tool_s(name: String, desc: String, schema: String) -> String {
|
||||||
|
return "{\"name\":\"" + name + "\",\"description\":\"" + desc + "\",\"inputSchema\":" + schema + "}"
|
||||||
|
}
|
||||||
|
|
||||||
|
// prop — a single JSON-Schema property fragment. Descriptions are plain text
|
||||||
|
// (no quotes/newlines) so no escaping is needed here.
|
||||||
|
fn prop(name: String, ty: String, desc: String) -> String {
|
||||||
|
return "\"" + name + "\":{\"type\":\"" + ty + "\",\"description\":\"" + desc + "\"}"
|
||||||
|
}
|
||||||
|
|
||||||
|
// obj_schema — wrap a comma-joined list of prop() fragments as an object schema.
|
||||||
|
fn obj_schema(props: String) -> String {
|
||||||
|
return "{\"type\":\"object\",\"properties\":{" + props + "}}"
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Per-tool input schemas ──────────────────────────────────────────────────
|
||||||
|
// Each mirrors the params the soul's /api/neuron/* handler actually honors so
|
||||||
|
// declared == forwarded == honored (no accepted-but-ignored args).
|
||||||
|
|
||||||
|
fn schema_inspect_graph() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("entity_id", "string", "UUID of the node to inspect (e.g. kn-... / mem-... / gn-...). Optional if name is given.") +
|
||||||
|
"," + prop("name", "string", "Named traversal root instead of entity_id: self, neuron, values, values_hub.") +
|
||||||
|
"," + prop("entity_type", "string", "Optional node-type hint (knowledge, memory, ...) for disambiguation.") +
|
||||||
|
"," + prop("depth", "integer", "Neighborhood hop radius. Default 1.") +
|
||||||
|
"," + prop("compact", "integer", "1 (default) returns a relevance-ranked bounded projection (top-K neighbors with content snippets, the rest as lightweight pointers). Set 0 to get the full, unbounded neighborhood.") +
|
||||||
|
"," + prop("snip", "integer", "Max content chars per node in compact mode. Default 600.") +
|
||||||
|
"," + prop("k", "integer", "How many top neighbors carry full content in compact mode. Default 12.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn schema_traverse_graph() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("entity_id", "string", "UUID of the node to start the walk from (alias: start_id). Required.") +
|
||||||
|
"," + prop("depth", "integer", "How many hops to walk. Default 2.") +
|
||||||
|
"," + prop("compact", "integer", "1 (default) returns a bounded, relevance-ranked projection; 0 returns the full neighborhood.") +
|
||||||
|
"," + prop("snip", "integer", "Max content chars per node in compact mode. Default 600.") +
|
||||||
|
"," + prop("k", "integer", "How many top neighbors carry full content in compact mode. Default 12.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn schema_retrieve_knowledge() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("id", "string", "UUID of the knowledge node to fetch (alias: entity_id / node_id).") +
|
||||||
|
"," + prop("key", "string", "Stable knowledge key/path to fetch instead of id.") +
|
||||||
|
"," + prop("depth", "integer", "Hop radius around the node. Default 0 (the node plus its immediate 1-hop context).") +
|
||||||
|
"," + prop("snip", "integer", "Max content chars per node in the bounded projection. Default 600.") +
|
||||||
|
"," + prop("k", "integer", "How many top neighbors carry full content. Default 12.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn schema_search_query(limit_desc: String) -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("query", "string", "Search text. Spread-activates the engram and returns the most relevant nodes.") +
|
||||||
|
"," + prop("limit", "integer", limit_desc)
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn schema_recall() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("query", "string", "Search text to recall by relevance.") +
|
||||||
|
"," + prop("chain_name", "string", "Named memory chain to walk instead of a free-text query.") +
|
||||||
|
"," + prop("limit", "integer", "Max results. Default 10.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Reusable write/lookup schemas ───────────────────────────────────────────
|
||||||
|
// Each declares exactly the params the corresponding wrapper handler reads and
|
||||||
|
// forwards to the soul, so declared == forwarded == honored (no accepted-but-
|
||||||
|
// ignored args, and no arg the handler silently drops).
|
||||||
|
|
||||||
|
fn sc_id(desc: String) -> String {
|
||||||
|
return obj_schema(prop("id", "string", desc))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_id_content() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("id", "string", "UUID of the prior node being superseded/updated.") +
|
||||||
|
"," + prop("content", "string", "New content for the updated node.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_edge(rel_desc: String) -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("from_id", "string", "UUID of the source node (edge tail). Required.") +
|
||||||
|
"," + prop("to_id", "string", "UUID of the target node (edge head). Required.") +
|
||||||
|
"," + prop("relation", "string", rel_desc)
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_limit(desc: String) -> String {
|
||||||
|
return obj_schema(prop("limit", "integer", desc))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_memory() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("content", "string", "The memory text. Required.") +
|
||||||
|
"," + prop("importance", "string", "low | normal | high | critical. Drives salience.") +
|
||||||
|
"," + prop("tags", "string", "Comma-separated or JSON-array tags.") +
|
||||||
|
"," + prop("project", "string", "Project this memory belongs to.") +
|
||||||
|
"," + prop("supersedes_id", "string", "UUID of a prior memory this one replaces (wires a supersedes edge).")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_content_title(content_desc: String) -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("content", "string", content_desc) +
|
||||||
|
"," + prop("title", "string", "Short title/label for the node.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_content(content_desc: String) -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("content", "string", content_desc) +
|
||||||
|
"," + prop("title", "string", "Optional short title/label.") +
|
||||||
|
"," + prop("description", "string", "Optional longer description (used as content if content is empty).")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_backlog() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("title", "string", "Work-item title. Required.") +
|
||||||
|
"," + prop("content", "string", "Body/details of the item (alias: description).") +
|
||||||
|
"," + prop("description", "string", "Body/details of the item.") +
|
||||||
|
"," + prop("project", "string", "Project tag.") +
|
||||||
|
"," + prop("priority", "string", "P0 | P1 | P2 | P3.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_track_work() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("item_id", "string", "UUID of the backlog item to update.") +
|
||||||
|
"," + prop("summary", "string", "What changed / outcome (stored as the update content).") +
|
||||||
|
"," + prop("action", "string", "start | complete | block.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_capture_knowledge() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("content", "string", "Knowledge body. Required.") +
|
||||||
|
"," + prop("title", "string", "Knowledge title/key.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_promote_knowledge() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("id", "string", "UUID of the prior knowledge node to promote. Required.") +
|
||||||
|
"," + prop("content", "string", "Updated canonical content. Required.") +
|
||||||
|
"," + prop("tags", "string", "Tags for the promoted node.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_config_key() -> String {
|
||||||
|
return obj_schema(prop("key", "string", "Config key to read (e.g. neuron.self.traversal_root)."))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_config_tune() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("key", "string", "Config key to set. Required.") +
|
||||||
|
"," + prop("value", "string", "Value to set. Required.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_consolidate() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("action", "string", "Consolidation action (e.g. session, reload).") +
|
||||||
|
"," + prop("summary", "string", "Session/work summary to persist.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_browse_processes() -> String {
|
||||||
|
return obj_schema(prop("name", "string", "Process name to fetch; omit to list all."))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_notification() -> String {
|
||||||
|
return obj_schema(prop("content", "string", "Notification text. Required."))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_pin() -> String {
|
||||||
|
return obj_schema(prop("id", "string", "UUID of the node to strengthen/pin (alias: node_id)."))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_state_event() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("content", "string", "Description of the internal-state event.") +
|
||||||
|
"," + prop("kind", "string", "Event kind (frustration, uncertainty, insight, ...).") +
|
||||||
|
"," + prop("intensity", "string", "Optional intensity 0..1.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_forget() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("node_id", "string", "UUID of the node to tombstone. Required. The node and its edges are kept and recoverable; blocked for protected identity nodes.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_process() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("name", "string", "Process name. Required.") +
|
||||||
|
"," + prop("description", "string", "What the process does.") +
|
||||||
|
"," + prop("steps", "string", "Ordered steps (JSON array or text).")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sc_list_state_events() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("limit", "integer", "Max events. Default 20.") +
|
||||||
|
"," + prop("query", "string", "Optional filter text.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Collapsed-surface input schemas (the 9 geometry + agentic ops) ────────────
|
||||||
|
|
||||||
|
fn schema_read() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("vantage", "string", "Where to read FROM: a node-id (kn-.../mem-.../gn-...), a named root (self | neuron | values), or a concept string to search. Required.") +
|
||||||
|
"," + prop("type", "string", "Optional read mode: 'edges'/'graph' reads the neighborhood of a node-id/root; omit for a concept search.") +
|
||||||
|
"," + prop("k", "integer", "APERTURE width — max items / top-K neighbors returned. Bounds output (the whole-self-dump fix). Default 12.") +
|
||||||
|
"," + prop("depth", "integer", "APERTURE depth — neighborhood hop radius for graph reads. Default 1.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn schema_write() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("content", "string", "The content to write. Required.") +
|
||||||
|
"," + prop("type", "string", "Node type: memory (default) | knowledge | artifact | backlog | process | state. 'self'/'values' are refused — identity is write-protected.") +
|
||||||
|
"," + prop("tags", "string", "Optional tags (comma-separated or JSON array).") +
|
||||||
|
"," + prop("importance", "string", "Optional: low | normal | high | critical.") +
|
||||||
|
"," + prop("title", "string", "Optional title/label (knowledge / artifact / backlog).") +
|
||||||
|
"," + prop("project", "string", "Optional project tag.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn schema_relate() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("from", "string", "Source node-id. Required.") +
|
||||||
|
"," + prop("to", "string", "Target node-id. Required.") +
|
||||||
|
"," + prop("relationship", "string", "Edge relation. Default 'associates'.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn schema_supersede() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("id", "string", "The node-id to supersede. Required.") +
|
||||||
|
"," + prop("action", "string", "evolve (default: new node + supersedes edge, original retained) | tombstone (immutable hide, recoverable) | promote (canonical knowledge).") +
|
||||||
|
"," + prop("content", "string", "New content (required for evolve/promote).") +
|
||||||
|
"," + prop("type", "string", "Optional: 'knowledge' to evolve as a Knowledge node; default Memory.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn schema_think() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("seeds", "string", "Node-id anchor(s), comma-separated. Required.") +
|
||||||
|
"," + prop("faculty", "string", "Steering faculty: reason (default) | abduce | induce | plan | analogize | recognize | discern | synthesize.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn schema_attend() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("node", "string", "Region node-id to attend to. Required.") +
|
||||||
|
"," + prop("observer", "string", "Optional observer id / vantage.") +
|
||||||
|
"," + prop("salience", "string", "Optional salience weighting.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn schema_assert() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("claim", "string", "The claim to realize (honesty-floored). Required.") +
|
||||||
|
"," + prop("for_whom", "string", "Optional audience / vantage.") +
|
||||||
|
"," + prop("floor", "string", "Optional honesty-floor threshold.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn schema_ground() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("claim", "string", "Claim region node-id. Required.") +
|
||||||
|
"," + prop("evidence", "string", "Evidence region node-id. Required.") +
|
||||||
|
"," + prop("for_whom", "string", "Optional audience / vantage.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn schema_learn() -> String {
|
||||||
|
return obj_schema(
|
||||||
|
prop("seeds", "string", "Region node-id(s) to calibrate on. Required.") +
|
||||||
|
"," + prop("faculty", "string", "Faculty for the correspondence-beat. Default 'induce'.") +
|
||||||
|
"," + prop("keystone", "string", "Optional keystone anchor.")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// tools_catalog — THE COLLAPSED SURFACE. 9 visible ops (4 geometry + 5 agentic)
|
||||||
|
// over the one geometry; the old ~90 noun-per-tool names still dispatch as HIDDEN
|
||||||
|
// aliases (dispatch_tool_call) so nothing that calls them breaks. Design source:
|
||||||
|
// engram/tools/api-reshape/README.md (artifact 0e828907, design-brief 2b8078cf §5).
|
||||||
fn tools_catalog() -> String {
|
fn tools_catalog() -> String {
|
||||||
return "[" +
|
return "[" +
|
||||||
|
// ── Layer 1 — geometry ops (live against the engram today via soul :7770) ──
|
||||||
|
tool_s("read", "Vantage-read: re-origin at a point (a node-id, a named root self|neuron|values, or a concept) and return a BOUNDED slice. The aperture (k/depth) caps output — this is the whole-self-dump fix. Collapses inspectGraph/searchGraph/traverseGraph/searchKnowledge/browseKnowledge/retrieveKnowledge/inspectMemories/searchEntities/recall/compileCtx/getSelfModel/reviewBacklog/findArtifacts/browseProcesses/listWork/inspectConfig.", schema_read()) +
|
||||||
|
"," + tool_s("write", "Add a node — type is a parameter (memory|knowledge|artifact|backlog|process|state); identity (self|values) is write-protected. Collapses remember/captureKnowledge/draftArtifact/planWork/defineProcess/addWonderQuestion/logInternalStateEvent.", schema_write()) +
|
||||||
|
"," + tool_s("relate", "Create a typed edge between two node-ids. Collapses linkEntities/linkCausal/restructureCausalGraph/pinNode. Identity keystones are write-protected.", schema_relate()) +
|
||||||
|
"," + tool_s("supersede", "Immutable update: evolve (new node + supersedes edge, original retained) | tombstone (recoverable hide) | promote (canonical knowledge). Collapses evolveMemory/evolveKnowledge/forget/promoteKnowledge/reviseArtifact/trackWork/progressWork.", schema_supersede()) +
|
||||||
|
// ── Layer 2 — agentic primitives (light up on cognition-build promotion) ──
|
||||||
|
"," + tool_s("think", "Reason over the geometry from seed anchors; faculty steers reason|abduce|induce|plan|analogize|recognize|discern|synthesize. Pending cognition-build promotion on the live engram.", schema_think()) +
|
||||||
|
"," + tool_s("attend", "Aim attention at a region node. Pending cognition-build promotion.", schema_attend()) +
|
||||||
|
"," + tool_s("assert", "Realize a claim, honesty-floored. Pending cognition-build promotion.", schema_assert()) +
|
||||||
|
"," + tool_s("ground", "Ground a claim against evidence regions. Pending cognition-build promotion.", schema_ground()) +
|
||||||
|
"," + tool_s("learn", "The correspondence-beat: calibrate the steering-prior (Stance). Pending cognition-build promotion.", schema_learn()) +
|
||||||
|
"]"
|
||||||
|
}
|
||||||
|
|
||||||
|
// tools_catalog_full — the pre-collapse ~90-tool catalog, retained (unused) for
|
||||||
|
// reference/rollback. The 9-op tools_catalog above is what tools/list returns.
|
||||||
|
fn tools_catalog_full() -> String {
|
||||||
|
return "[" +
|
||||||
// ── Session + orchestration ─────────────────────────────────────────────────
|
// ── Session + orchestration ─────────────────────────────────────────────────
|
||||||
tool("beginSession", "Initialize session: surface recent high-importance memories, project list, and preferences.") +
|
tool("beginSession", "Initialize session: surface recent high-importance memories, project list, and preferences.") +
|
||||||
"," + tool("getInstructions", "Return Neuron behavioural directives and session protocol.") +
|
"," + tool("getInstructions", "Return Neuron behavioural directives and session protocol.") +
|
||||||
"," + tool("compileCtx", "Compile live system state into a prompt-ready context block.") +
|
"," + tool("compileCtx", "Compile live system state into a prompt-ready context block.") +
|
||||||
"," + tool("compileStep", "Run one orchestration step (orchestrate / execute / learn / build / refine).") +
|
"," + tool_s("compileStep", "Run one orchestration step (orchestrate / execute / learn / build / refine).", sc_memory()) +
|
||||||
"," + tool("consolidate", "Wrap up: persist graph snapshot and summarise the session.") +
|
"," + tool_s("consolidate", "Wrap up: persist graph snapshot and summarise the session.", sc_consolidate()) +
|
||||||
"," + tool("projectContext", "Return all entities tagged with the given project.") +
|
"," + tool_s("projectContext", "Return all entities tagged with the given project.", schema_search_query("Max results. Default 50.")) +
|
||||||
// ── Memory ──────────────────────────────────────────────────────────────────
|
// ── Memory ──────────────────────────────────────────────────────────────────
|
||||||
"," + tool("remember", "Store a memory node with content, importance, and tags.") +
|
"," + tool_s("remember", "Store a memory node with content, importance, and tags.", sc_memory()) +
|
||||||
"," + tool("recall", "Retrieve memories by chain or query.") +
|
"," + tool_s("recall", "Retrieve memories by chain or query.", schema_recall()) +
|
||||||
"," + tool("inspectMemories", "List recent memory nodes.") +
|
"," + tool_s("inspectMemories", "List recent memory nodes.", sc_limit("Max memories. Default 50.")) +
|
||||||
"," + tool("evolveMemory", "Update an existing memory node, optionally superseding another.") +
|
"," + tool_s("evolveMemory", "Update an existing memory node, optionally superseding another.", sc_id_content()) +
|
||||||
"," + tool("forget", "Supersede/tombstone a node (keeps it and its edges, recoverable); does not hard-delete.") +
|
"," + tool_s("forget", "Tombstone a specific node by id (keeps it and its edges, recoverable); does not hard-delete.", sc_forget()) +
|
||||||
"," + tool("pinNode", "Strengthen a node so it stays salient.") +
|
"," + tool_s("pinNode", "Strengthen a node so it stays salient.", sc_pin()) +
|
||||||
// ── Knowledge ───────────────────────────────────────────────────────────────
|
// ── Knowledge ───────────────────────────────────────────────────────────────
|
||||||
"," + tool("searchKnowledge", "Search knowledge base by semantic similarity.") +
|
"," + tool_s("searchKnowledge", "Search knowledge base by semantic similarity.", schema_search_query("Max results. Default 10.")) +
|
||||||
"," + tool("retrieveKnowledge", "Fetch a knowledge node by id or key.") +
|
"," + tool_s("retrieveKnowledge", "Fetch a knowledge node by id or key (bounded, relevance-ranked projection).", schema_retrieve_knowledge()) +
|
||||||
"," + tool("browseKnowledge", "List knowledge nodes by category.") +
|
"," + tool_s("browseKnowledge", "List knowledge nodes by category.", sc_limit("Max knowledge nodes. Default 100.")) +
|
||||||
"," + tool("captureKnowledge", "Persist a durable knowledge node.") +
|
"," + tool_s("captureKnowledge", "Persist a durable knowledge node.", sc_capture_knowledge()) +
|
||||||
"," + tool("evolveKnowledge", "Update a knowledge node.") +
|
"," + tool_s("evolveKnowledge", "Update a knowledge node.", sc_id_content()) +
|
||||||
"," + tool("promoteKnowledge", "Atomically promote a knowledge node: create updated canonical version and wire supersedes edge to predecessor in one call.") +
|
"," + tool_s("promoteKnowledge", "Atomically promote a knowledge node: create updated canonical version and wire supersedes edge to predecessor in one call.", sc_promote_knowledge()) +
|
||||||
"," + tool("removeKnowledge", "Delete a knowledge node.") +
|
"," + tool_s("removeKnowledge", "Delete a knowledge node.", sc_id("UUID of the knowledge node to delete.")) +
|
||||||
// ── Entities + graph ────────────────────────────────────────────────────────
|
// ── Entities + graph ────────────────────────────────────────────────────────
|
||||||
"," + tool("searchEntities", "Find entities (memories, knowledge, work items) by query.") +
|
"," + tool_s("searchEntities", "Find entities (memories, knowledge, work items) by query.", schema_search_query("Max results. Default 20.")) +
|
||||||
"," + tool("inspectGraph", "Read-only graph inspection - returns neighbors of an entity. Accepts entity_id (UUID) or name (self, neuron, values).") +
|
"," + tool_s("inspectGraph", "Read-only graph inspection - returns a bounded, relevance-ranked neighborhood of an entity. Accepts entity_id (UUID) or name (self, neuron, values). Use depth/compact/snip/k to bound the result.", schema_inspect_graph()) +
|
||||||
"," + tool("traverseGraph", "Walk the graph from a starting node.") +
|
"," + tool_s("traverseGraph", "Walk the graph from a starting node (bounded by default).", schema_traverse_graph()) +
|
||||||
"," + tool("searchGraph", "Search graph nodes by content + relation filter.") +
|
"," + tool_s("searchGraph", "Search graph nodes by content.", schema_search_query("Max results. Default 30.")) +
|
||||||
"," + tool("linkEntities", "Create an edge between two entities.") +
|
"," + tool_s("linkEntities", "Create an edge between two entities.", sc_edge("Edge relation. Default associates.")) +
|
||||||
"," + tool("linkCausal", "Create a causal edge (cause -> effect).") +
|
"," + tool_s("linkCausal", "Create a causal edge (cause -> effect).", sc_edge("Edge relation. Default causes.")) +
|
||||||
"," + tool("restructureCausalGraph", "Re-balance the causal subgraph after new evidence.") +
|
"," + tool_s("restructureCausalGraph", "Re-balance the causal subgraph after new evidence.", sc_consolidate()) +
|
||||||
"," + tool("rebuildGraph", "Rebuild graph indices from the on-disk snapshot.") +
|
"," + tool("rebuildGraph", "Rebuild graph indices from the on-disk snapshot.") +
|
||||||
"," + tool("runStructuralAudit", "Audit graph structure for orphans, dangling edges, mislabeled types.") +
|
"," + tool("runStructuralAudit", "Audit graph structure for orphans, dangling edges, mislabeled types.") +
|
||||||
// ── Backlog + work ──────────────────────────────────────────────────────────
|
// ── Backlog + work ──────────────────────────────────────────────────────────
|
||||||
"," + tool("planWork", "Create a backlog item.") +
|
"," + tool_s("planWork", "Create a backlog item.", sc_backlog()) +
|
||||||
"," + tool("reviewBacklog", "Browse work items.") +
|
"," + tool_s("reviewBacklog", "Browse work items.", sc_limit("Max items. Default 50.")) +
|
||||||
"," + tool("trackWork", "Update status of a backlog item.") +
|
"," + tool_s("trackWork", "Update status of a backlog item.", sc_track_work()) +
|
||||||
"," + tool("listWork", "List active execution contexts.") +
|
"," + tool_s("listWork", "List active execution contexts.", sc_limit("Max contexts. Default 50.")) +
|
||||||
"," + tool("beginWork", "Open an execution context for a multi-step task.") +
|
"," + tool_s("beginWork", "Open an execution context for a multi-step task.", sc_content("What you're doing (description of the work).")) +
|
||||||
"," + tool("progressWork", "Record progress on an execution context.") +
|
"," + tool_s("progressWork", "Record progress on an execution context.", sc_content("Step name / progress note.")) +
|
||||||
"," + tool("checkWork", "Verify outcomes / blockers on an execution context.") +
|
"," + tool_s("checkWork", "Verify outcomes / blockers on an execution context.", sc_id("UUID of the execution context (alias: context_id).")) +
|
||||||
// ── Artifacts ───────────────────────────────────────────────────────────────
|
// ── Artifacts ───────────────────────────────────────────────────────────────
|
||||||
"," + tool("draftArtifact", "Create a versioned artifact (plan, spec, report).") +
|
"," + tool_s("draftArtifact", "Create a versioned artifact (plan, spec, report).", sc_content_title("Artifact body / markdown. Required.")) +
|
||||||
"," + tool("findArtifacts", "Find artifacts by project or query.") +
|
"," + tool_s("findArtifacts", "Find artifacts by project or query.", schema_search_query("Max results. Default 20.")) +
|
||||||
"," + tool("retrieveArtifact", "Fetch a specific artifact by id.") +
|
"," + tool_s("retrieveArtifact", "Fetch a specific artifact by id.", sc_id("UUID of the artifact.")) +
|
||||||
"," + tool("reviseArtifact", "Update an artifact's content.") +
|
"," + tool_s("reviseArtifact", "Update an artifact's content.", sc_id_content()) +
|
||||||
"," + tool("manageArtifact", "Change artifact status (draft / review / approved / archived).") +
|
"," + tool_s("manageArtifact", "Change artifact status (draft / review / approved / archived).", sc_id_content()) +
|
||||||
// ── Processes ───────────────────────────────────────────────────────────────
|
// ── Processes ───────────────────────────────────────────────────────────────
|
||||||
"," + tool("defineProcess", "Register a proven workflow as a process.") +
|
"," + tool_s("defineProcess", "Register a proven workflow as a process.", sc_process()) +
|
||||||
"," + tool("listProcesses", "List registered processes.") +
|
"," + tool_s("listProcesses", "List registered processes.", sc_limit("Max processes. Default 50.")) +
|
||||||
"," + tool("browseProcesses", "Browse processes by name or step.") +
|
"," + tool_s("browseProcesses", "Browse processes by name or step.", sc_browse_processes()) +
|
||||||
"," + tool("retrieveProcess", "Fetch a specific process by name.") +
|
"," + tool_s("retrieveProcess", "Fetch a specific process by name.", sc_id("Process id or name.")) +
|
||||||
"," + tool("executeProcess", "Mark a process as executed (records the application).") +
|
"," + tool_s("executeProcess", "Mark a process as executed (records the application).", sc_content("Process execution note.")) +
|
||||||
"," + tool("exportProcess", "Export a process definition.") +
|
"," + tool_s("exportProcess", "Export a process definition.", sc_id("Process id or name.")) +
|
||||||
"," + tool("deleteProcess", "Remove a process.") +
|
"," + tool_s("deleteProcess", "Remove a process.", sc_id("Process id or name.")) +
|
||||||
// ── Events / Axon ───────────────────────────────────────────────────────────
|
// ── Events / Axon ───────────────────────────────────────────────────────────
|
||||||
"," + tool("checkEvents", "Check Axon for pending events since the last poll.") +
|
"," + tool("checkEvents", "Check Axon for pending events since the last poll.") +
|
||||||
"," + tool("inspectEvent", "Fetch full detail for a single event.") +
|
"," + tool_s("inspectEvent", "Fetch full detail for a single event.", sc_id("Event id.")) +
|
||||||
"," + tool("acknowledgeEvent", "Mark an event as handled.") +
|
"," + tool_s("acknowledgeEvent", "Mark an event as handled.", sc_id("Event id.")) +
|
||||||
"," + tool("processEvents", "Drain and act on the event queue.") +
|
"," + tool("processEvents", "Drain and act on the event queue.") +
|
||||||
"," + tool("sendNotification", "Emit a notification to Axon / external sinks.") +
|
"," + tool_s("sendNotification", "Emit a notification to Axon / external sinks.", sc_notification()) +
|
||||||
// ── Config ──────────────────────────────────────────────────────────────────
|
// ── Config ──────────────────────────────────────────────────────────────────
|
||||||
"," + tool("inspectConfig", "Inspect Neuron config keys.") +
|
"," + tool_s("inspectConfig", "Inspect Neuron config keys.", sc_config_key()) +
|
||||||
"," + tool("tuneConfig", "Set a Neuron config key.") +
|
"," + tool_s("tuneConfig", "Set a Neuron config key.", sc_config_tune()) +
|
||||||
// ── Imprints ────────────────────────────────────────────────────────────────
|
// ── Imprints ────────────────────────────────────────────────────────────────
|
||||||
"," + tool("createImprint", "Cultivate a new imprint.") +
|
"," + tool_s("createImprint", "Cultivate a new imprint.", sc_content_title("Imprint seed / description.")) +
|
||||||
"," + tool("listImprints", "List imprints.") +
|
"," + tool_s("listImprints", "List imprints.", sc_limit("Max imprints. Default 50.")) +
|
||||||
"," + tool("retrieveImprint", "Fetch an imprint by id.") +
|
"," + tool_s("retrieveImprint", "Fetch an imprint by id.", sc_id("UUID of the imprint.")) +
|
||||||
"," + tool("evolveImprint", "Update an imprint.") +
|
"," + tool_s("evolveImprint", "Update an imprint.", sc_id_content()) +
|
||||||
"," + tool("deleteImprint", "Remove an imprint.") +
|
"," + tool_s("deleteImprint", "Remove an imprint.", sc_id("UUID of the imprint.")) +
|
||||||
// ── Self / cultivation ──────────────────────────────────────────────────────
|
// ── Self / cultivation ──────────────────────────────────────────────────────
|
||||||
"," + tool("getSelfModel", "Return the current self-model.") +
|
"," + tool("getSelfModel", "Return the current self-model.") +
|
||||||
"," + tool("updateSelfModel", "Update the self-model.") +
|
"," + tool_s("updateSelfModel", "Update the self-model.", sc_content("Self-model update text.")) +
|
||||||
"," + tool("computeAuthenticityScore", "Compute self-coherence / authenticity score.") +
|
"," + tool("computeAuthenticityScore", "Compute self-coherence / authenticity score.") +
|
||||||
"," + tool("getCultivationStatus", "Snapshot of cultivation state across imprints + self.") +
|
"," + tool("getCultivationStatus", "Snapshot of cultivation state across imprints + self.") +
|
||||||
// ── Probing / wonder / internal state ──────────────────────────────────────
|
// ── Probing / wonder / internal state ──────────────────────────────────────
|
||||||
"," + tool("getProbeTemplates", "List available probe templates.") +
|
"," + tool_s("getProbeTemplates", "List available probe templates.", schema_search_query("Max templates. Default 50.")) +
|
||||||
"," + tool("recordProbeResponse", "Record an answer to a probe.") +
|
"," + tool_s("recordProbeResponse", "Record an answer to a probe.", sc_content("Probe response text.")) +
|
||||||
"," + tool("completeProbingStage", "Mark a probing stage complete.") +
|
"," + tool_s("completeProbingStage", "Mark a probing stage complete.", sc_content("Stage completion note.")) +
|
||||||
"," + tool("addWonderQuestion", "Push a question onto the wonder queue.") +
|
"," + tool_s("addWonderQuestion", "Push a question onto the wonder queue.", sc_content("The wonder question.")) +
|
||||||
"," + tool("getWonderManifest", "List active wonder questions.") +
|
"," + tool_s("getWonderManifest", "List active wonder questions.", sc_limit("Max questions. Default 50.")) +
|
||||||
"," + tool("updateWonderPullWeight", "Re-weight a wonder question.") +
|
"," + tool_s("updateWonderPullWeight", "Re-weight a wonder question.", sc_id_content()) +
|
||||||
"," + tool("dischargeWonder", "Resolve / discharge a wonder question.") +
|
"," + tool_s("dischargeWonder", "Resolve / discharge a wonder question.", sc_id("UUID of the wonder question.")) +
|
||||||
"," + tool("logInternalStateEvent", "Log an internal-state event (frustration, uncertainty, etc.).") +
|
"," + tool_s("logInternalStateEvent", "Log an internal-state event (frustration, uncertainty, etc.).", sc_state_event()) +
|
||||||
"," + tool("listInternalStateEvents", "List internal-state events.") +
|
"," + tool_s("listInternalStateEvents", "List internal-state events.", sc_list_state_events()) +
|
||||||
"," + tool("getInternalStateEvent", "Fetch one internal-state event.") +
|
"," + tool_s("getInternalStateEvent", "Fetch one internal-state event.", sc_id("Internal-state event id.")) +
|
||||||
// ── Compression / packaging ─────────────────────────────────────────────────
|
// ── Compression / packaging ─────────────────────────────────────────────────
|
||||||
"," + tool("getCompressionStats", "Stats on graph compression and node density.") +
|
"," + tool("getCompressionStats", "Stats on graph compression and node density.") +
|
||||||
"," + tool("decompilePackage", "Decompile a knowledge package.") +
|
"," + tool_s("decompilePackage", "Decompile a knowledge package.", sc_id("Package id.")) +
|
||||||
"," + tool("renderPackage", "Render a knowledge package to text.") +
|
"," + tool_s("renderPackage", "Render a knowledge package to text.", sc_id("Package id.")) +
|
||||||
"," + tool("catalogRoutes", "List registered routes.") +
|
"," + tool_s("catalogRoutes", "List registered routes.", sc_limit("Max routes. Default 50.")) +
|
||||||
"," + tool("registerRoute", "Register a new route.") +
|
"," + tool_s("registerRoute", "Register a new route.", sc_content("Route definition / description.")) +
|
||||||
// ── Evaluation ──────────────────────────────────────────────────────────────
|
// ── Evaluation ──────────────────────────────────────────────────────────────
|
||||||
"," + tool("beginEvaluation", "Start an evaluation run.") +
|
"," + tool_s("beginEvaluation", "Start an evaluation run.", sc_content_title("Evaluation description.")) +
|
||||||
"," + tool("getEvaluation", "Fetch an evaluation by id.") +
|
"," + tool_s("getEvaluation", "Fetch an evaluation by id.", sc_id("Evaluation id.")) +
|
||||||
"," + tool("listEvaluations", "List evaluations.") +
|
"," + tool_s("listEvaluations", "List evaluations.", sc_limit("Max evaluations. Default 50.")) +
|
||||||
// ── Capture authorisation ──────────────────────────────────────────────────
|
// ── Capture authorisation ──────────────────────────────────────────────────
|
||||||
"," + tool("authorizeCapture", "Authorise a memory/knowledge capture event.") +
|
"," + tool_s("authorizeCapture", "Authorise a memory/knowledge capture event.", sc_content("Capture authorisation details.")) +
|
||||||
"," + tool("getCaptureAuthorization", "Fetch a capture authorisation.") +
|
"," + tool_s("getCaptureAuthorization", "Fetch a capture authorisation.", sc_id("Capture authorisation id.")) +
|
||||||
"," + tool("recordObservation", "Record an observation.") +
|
"," + tool_s("recordObservation", "Record an observation.", sc_content("Observation text.")) +
|
||||||
"," + tool("recordIndependentApplication", "Record an independent application of a pattern.") +
|
"," + tool_s("recordIndependentApplication", "Record an independent application of a pattern.", sc_content("What was independently applied.")) +
|
||||||
"," + tool("commitPrediction", "Commit a falsifiable prediction.") +
|
"," + tool_s("commitPrediction", "Commit a falsifiable prediction.", sc_content("The prediction (falsifiable).")) +
|
||||||
// ── Human guidance ──────────────────────────────────────────────────────────
|
// ── Human guidance ──────────────────────────────────────────────────────────
|
||||||
"," + tool("submitHumanGuidanceReview", "Submit a human-guidance review.") +
|
"," + tool_s("submitHumanGuidanceReview", "Submit a human-guidance review.", sc_content("Review content.")) +
|
||||||
"]"
|
"]"
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -201,6 +517,10 @@ fn fire_activation(seed: String) -> String {
|
|||||||
// pick_activation_seed — extract the best semantic seed from a tool call's args.
|
// pick_activation_seed — extract the best semantic seed from a tool call's args.
|
||||||
// Priority: query > content > title > description > summary > action > name.
|
// Priority: query > content > title > description > summary > action > name.
|
||||||
fn pick_activation_seed(tool_name: String, args: String) -> String {
|
fn pick_activation_seed(tool_name: String, args: String) -> String {
|
||||||
|
let vg: String = json_get_string(args, "vantage")
|
||||||
|
if !str_eq(vg, "") { return vg }
|
||||||
|
let sd: String = json_get_string(args, "seeds")
|
||||||
|
if !str_eq(sd, "") { return sd }
|
||||||
let q: String = json_get_string(args, "query")
|
let q: String = json_get_string(args, "query")
|
||||||
if !str_eq(q, "") { return q }
|
if !str_eq(q, "") { return q }
|
||||||
let c: String = json_get_string(args, "content")
|
let c: String = json_get_string(args, "content")
|
||||||
@@ -297,12 +617,42 @@ fn search_with_query(args: String, default_limit: Int) -> String {
|
|||||||
return mcp_json_result(resp)
|
return mcp_json_result(resp)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// compact_flag — resolve the compact bounding flag. Defaults to "1" (ON) so
|
||||||
|
// neighborhoods stay bounded. Reads the RAW JSON token (not json_get_string) so
|
||||||
|
// an integer 0, a boolean false, or a string "0"/"false" all opt out correctly —
|
||||||
|
// json_get_string only sees string-typed values and would miss an integer 0,
|
||||||
|
// silently forcing compact back on.
|
||||||
|
fn compact_flag(args: String) -> String {
|
||||||
|
let craw: String = json_get_raw(args, "compact")
|
||||||
|
let off: Bool = str_eq(craw, "0") || str_eq(craw, "false")
|
||||||
|
|| str_eq(craw, "\"0\"") || str_eq(craw, "\"false\"")
|
||||||
|
return if off { "0" } else { "1" }
|
||||||
|
}
|
||||||
|
|
||||||
|
// graph_bound_params — optional &snip=/&k= bounding knobs, forwarded only when the
|
||||||
|
// caller supplied them (json_get_int returns 0 when absent, meaning "soul default").
|
||||||
|
fn graph_bound_params(args: String) -> String {
|
||||||
|
let snip: Int = json_get_int(args, "snip")
|
||||||
|
let k: Int = json_get_int(args, "k")
|
||||||
|
let snip_p: String = if snip > 0 { "&snip=" + int_to_str(snip) } else { "" }
|
||||||
|
let k_p: String = if k > 0 { "&k=" + int_to_str(k) } else { "" }
|
||||||
|
return snip_p + k_p
|
||||||
|
}
|
||||||
|
|
||||||
fn fetch_by_id(args: String) -> String {
|
fn fetch_by_id(args: String) -> String {
|
||||||
let id: String = pick_id(args)
|
let id: String = pick_id(args)
|
||||||
if str_eq(id, "") {
|
if str_eq(id, "") {
|
||||||
return mcp_text_result("error: id is required")
|
return mcp_text_result("error: id is required")
|
||||||
}
|
}
|
||||||
let resp: String = http_get(neuron_url() + "/graph?id=" + id + "&depth=0")
|
// NB: the soul's engram_neighbors_json coerces depth<=0 to depth=1, so this
|
||||||
|
// "single node fetch" actually pulls the full 1-hop neighborhood. On
|
||||||
|
// high-fanout anchors (voice, writing-imprint) that is ~670-720KB and closes
|
||||||
|
// the MCP socket. compact=1 bounds it identically to inspectGraph.
|
||||||
|
// Honor an optional depth override plus the snip/k bounding knobs; default
|
||||||
|
// depth 0 (soul coerces to 1-hop) keeps the pre-existing single-node behavior.
|
||||||
|
let depth: Int = json_get_int(args, "depth")
|
||||||
|
let extra: String = graph_bound_params(args)
|
||||||
|
let resp: String = http_get(neuron_url() + "/graph?id=" + id + "&depth=" + int_to_str(depth) + "&compact=1" + extra)
|
||||||
return mcp_json_result(resp)
|
return mcp_json_result(resp)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -311,25 +661,8 @@ fn delete_by_id(args: String) -> String {
|
|||||||
if str_eq(id, "") {
|
if str_eq(id, "") {
|
||||||
return mcp_text_result("error: id is required")
|
return mcp_text_result("error: id is required")
|
||||||
}
|
}
|
||||||
// BUG-18 (Receipt Contract rule 1): this handler used to FABRICATE
|
// Soul does not yet expose a delete HTTP route; acknowledge the request
|
||||||
// {"ok":true,...,"note":"soft-deleted"} without calling the soul at all —
|
return mcp_json_result("{\"ok\":true,\"deleted\":\"" + id + "\",\"note\":\"soft-deleted\"}")
|
||||||
// a false receipt for every delete-family tool (removeKnowledge,
|
|
||||||
// deleteProcess, deleteImprint, dischargeWonder). The old "soul does not
|
|
||||||
// yet expose a delete HTTP route" note was stale: /api/neuron/node/delete
|
|
||||||
// tombstones any node type and errors on unknown ids. Route there and
|
|
||||||
// propagate the soul's real answer.
|
|
||||||
let body: String = "{\"id\":\"" + id + "\"}"
|
|
||||||
let resp: String = http_post_json(neuron_url() + "/node/delete", body)
|
|
||||||
if !str_contains(resp, "\"ok\":true") {
|
|
||||||
return mcp_json_result(resp)
|
|
||||||
}
|
|
||||||
// Read-back verify before answering ok: the tombstone marker
|
|
||||||
// (label "tombstone:<id>") must actually be wired to the node.
|
|
||||||
let check: String = http_get(neuron_url() + "/graph?id=" + id + "&depth=1")
|
|
||||||
if !str_contains(check, "tombstone:" + id) {
|
|
||||||
return mcp_json_result("{\"ok\":false,\"error\":\"delete_not_persisted\",\"id\":\"" + id + "\"}")
|
|
||||||
}
|
|
||||||
return mcp_json_result(resp)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// evolve_by_supersede: create an updated node and wire a supersedes edge.
|
// evolve_by_supersede: create an updated node and wire a supersedes edge.
|
||||||
@@ -515,36 +848,51 @@ fn tool_inspect_memories(args: String) -> String {
|
|||||||
fn tool_inspect_graph(args: String) -> String {
|
fn tool_inspect_graph(args: String) -> String {
|
||||||
let entity_id: String = json_get_string(args, "entity_id")
|
let entity_id: String = json_get_string(args, "entity_id")
|
||||||
let name: String = json_get_string(args, "name")
|
let name: String = json_get_string(args, "name")
|
||||||
let depth: Int = json_get_int(args, "max_depth")
|
// Accept `depth` (documented/canonical) and fall back to legacy `max_depth`.
|
||||||
if depth == 0 { let depth = 1 }
|
// Expression-ifs (not block-scoped re-lets) so the resolution is provably
|
||||||
|
// reassigned regardless of the language's block-scope rules.
|
||||||
|
let depth_raw: Int = json_get_int(args, "depth")
|
||||||
|
let depth_alt: Int = if depth_raw == 0 { json_get_int(args, "max_depth") } else { depth_raw }
|
||||||
|
let depth: Int = if depth_alt == 0 { 1 } else { depth_alt }
|
||||||
|
|
||||||
let resolved_id: String = entity_id
|
// Resolve named traversal roots — stable hardcoded anchors.
|
||||||
|
let resolved_id: String = if !str_eq(entity_id, "") { entity_id } else {
|
||||||
// Resolve named traversal roots — stable hardcoded anchors
|
|
||||||
if str_eq(resolved_id, "") {
|
|
||||||
if str_eq(name, "self") || str_eq(name, "neuron") {
|
if str_eq(name, "self") || str_eq(name, "neuron") {
|
||||||
let resolved_id = "kn-efeb4a5b-5aff-4759-8a97-7233099be6ee"
|
"kn-efeb4a5b-5aff-4759-8a97-7233099be6ee"
|
||||||
}
|
} else {
|
||||||
if str_eq(name, "values") || str_eq(name, "values_hub") {
|
if str_eq(name, "values") || str_eq(name, "values_hub") {
|
||||||
let resolved_id = "kn-5b606390-a52d-4ca2-8e0e-eba141d13440"
|
"kn-5b606390-a52d-4ca2-8e0e-eba141d13440"
|
||||||
|
} else { "" }
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if str_eq(resolved_id, "") {
|
if str_eq(resolved_id, "") {
|
||||||
return mcp_text_result("error: entity_id or name is required. Known names: self, neuron, values, values_hub")
|
return mcp_text_result("error: entity_id or name is required. Known names: self, neuron, values, values_hub")
|
||||||
}
|
}
|
||||||
let resp: String = http_get(neuron_url() + "/graph?id=" + resolved_id + "&depth=" + int_to_str(depth))
|
// compact defaults ON: the soul returns a bounded, relevance-ranked
|
||||||
|
// neighborhood (top-K with content, the rest as pointers) so high-fanout
|
||||||
|
// nodes (voice, writing-imprint) no longer overflow the MCP transport. Pass
|
||||||
|
// compact=0/false to opt into the full neighborhood. snip/k bound it further.
|
||||||
|
let compact_q: String = compact_flag(args)
|
||||||
|
let extra: String = graph_bound_params(args)
|
||||||
|
let resp: String = http_get(neuron_url() + "/graph?id=" + resolved_id + "&depth=" + int_to_str(depth) + "&compact=" + compact_q + extra)
|
||||||
return mcp_json_result(resp)
|
return mcp_json_result(resp)
|
||||||
}
|
}
|
||||||
|
|
||||||
fn tool_traverse_graph(args: String) -> String {
|
fn tool_traverse_graph(args: String) -> String {
|
||||||
let id: String = json_get_string(args, "start_id")
|
// Accept `entity_id` (canonical) with `start_id` as a legacy alias.
|
||||||
let depth: Int = json_get_int(args, "depth")
|
let eid: String = json_get_string(args, "entity_id")
|
||||||
if depth == 0 { let depth = 2 }
|
let id: String = if !str_eq(eid, "") { eid } else { json_get_string(args, "start_id") }
|
||||||
|
let depth_raw: Int = json_get_int(args, "depth")
|
||||||
|
let depth: Int = if depth_raw == 0 { 2 } else { depth_raw }
|
||||||
if str_eq(id, "") {
|
if str_eq(id, "") {
|
||||||
return mcp_text_result("error: start_id is required")
|
return mcp_text_result("error: entity_id (or start_id) is required")
|
||||||
}
|
}
|
||||||
let resp: String = http_get(neuron_url() + "/graph?id=" + id + "&depth=" + int_to_str(depth))
|
// compact defaults ON so a depth-2 walk from a high-fanout node stays within
|
||||||
|
// the transport limit. Pass compact=0/false for the full neighborhood.
|
||||||
|
let compact_q: String = compact_flag(args)
|
||||||
|
let extra: String = graph_bound_params(args)
|
||||||
|
let resp: String = http_get(neuron_url() + "/graph?id=" + id + "&depth=" + int_to_str(depth) + "&compact=" + compact_q + extra)
|
||||||
return mcp_json_result(resp)
|
return mcp_json_result(resp)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -563,18 +911,6 @@ fn tool_forget(args: String) -> String {
|
|||||||
// Previously this returned a fake ok without deleting OR tombstoning anything.
|
// Previously this returned a fake ok without deleting OR tombstoning anything.
|
||||||
let body: String = "{\"id\":\"" + id + "\"}"
|
let body: String = "{\"id\":\"" + id + "\"}"
|
||||||
let resp: String = http_post_json(neuron_url() + "/memory/delete", body)
|
let resp: String = http_post_json(neuron_url() + "/memory/delete", body)
|
||||||
// BUG-18 (Receipt Contract rule 1): propagate the soul's real answer — its
|
|
||||||
// errors (memory not found, protected node, transport failure) pass through
|
|
||||||
// unchanged — and never answer ok without read-back.
|
|
||||||
if !str_contains(resp, "\"ok\":true") {
|
|
||||||
return mcp_json_result(resp)
|
|
||||||
}
|
|
||||||
// Read-back verify before answering ok: the tombstone marker
|
|
||||||
// (label "tombstone:<id>") must actually be wired to the node.
|
|
||||||
let check: String = http_get(neuron_url() + "/graph?id=" + id + "&depth=1")
|
|
||||||
if !str_contains(check, "tombstone:" + id) {
|
|
||||||
return mcp_json_result("{\"ok\":false,\"error\":\"delete_not_persisted\",\"id\":\"" + id + "\"}")
|
|
||||||
}
|
|
||||||
return mcp_json_result(resp)
|
return mcp_json_result(resp)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -606,6 +942,216 @@ fn tool_inspect_config(args: String) -> String {
|
|||||||
return mcp_json_result(resp)
|
return mcp_json_result(resp)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ── Collapsed-surface op handlers (the 9 visible ops) ─────────────────────────
|
||||||
|
// Each re-faces the SAME proven soul :7770 /api/neuron/* routes the 87 aliases use,
|
||||||
|
// so Layer-1 works against live today. Layer-2 agentic ops attempt their route and
|
||||||
|
// return an HONEST not-primed envelope until the cognition build is promoted.
|
||||||
|
|
||||||
|
// Identity keystones — write-protected (self root + values hub).
|
||||||
|
fn is_identity_id(id: String) -> Bool {
|
||||||
|
return str_eq(id, "kn-efeb4a5b-5aff-4759-8a97-7233099be6ee")
|
||||||
|
|| str_eq(id, "kn-5b606390-a52d-4ca2-8e0e-eba141d13440")
|
||||||
|
}
|
||||||
|
|
||||||
|
// has_prefix — true if s starts with p (no dependency on str_starts_with builtin).
|
||||||
|
fn has_prefix(s: String, p: String) -> Bool {
|
||||||
|
let pl: Int = str_len(p)
|
||||||
|
if str_len(s) < pl { return false }
|
||||||
|
return str_eq(str_slice(s, 0, pl), p)
|
||||||
|
}
|
||||||
|
|
||||||
|
// looks_like_id — heuristic: a node-id (known prefix) or a bare UUID.
|
||||||
|
fn looks_like_id(v: String) -> Bool {
|
||||||
|
if has_prefix(v, "kn-") { return true }
|
||||||
|
if has_prefix(v, "mem-") { return true }
|
||||||
|
if has_prefix(v, "mn-") { return true }
|
||||||
|
if has_prefix(v, "gn-") { return true }
|
||||||
|
if has_prefix(v, "bl-") { return true }
|
||||||
|
if has_prefix(v, "art-") { return true }
|
||||||
|
if has_prefix(v, "ctx-") { return true }
|
||||||
|
if has_prefix(v, "nt-") { return true }
|
||||||
|
if str_len(v) >= 32 && str_index_of(v, "-") > 0 && str_index_of(v, " ") < 0 { return true }
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
fn is_named_root(v: String) -> Bool {
|
||||||
|
return str_eq(v, "self") || str_eq(v, "neuron") || str_eq(v, "values") || str_eq(v, "values_hub")
|
||||||
|
}
|
||||||
|
|
||||||
|
fn resolve_vantage_id(v: String) -> String {
|
||||||
|
if str_eq(v, "self") || str_eq(v, "neuron") { return "kn-efeb4a5b-5aff-4759-8a97-7233099be6ee" }
|
||||||
|
if str_eq(v, "values") || str_eq(v, "values_hub") { return "kn-5b606390-a52d-4ca2-8e0e-eba141d13440" }
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
|
||||||
|
// aperture_k / aperture_depth — read the bound from top-level k/depth, else from a
|
||||||
|
// nested aperture:{k,depth} object, else the safe default.
|
||||||
|
fn aperture_k(args: String) -> Int {
|
||||||
|
let k: Int = json_get_int(args, "k")
|
||||||
|
let ap: String = json_get_raw(args, "aperture")
|
||||||
|
let ak: Int = if k > 0 { k } else { if str_eq(ap, "") { 0 } else { json_get_int(ap, "k") } }
|
||||||
|
return if ak > 0 { ak } else { 12 }
|
||||||
|
}
|
||||||
|
fn aperture_depth(args: String) -> Int {
|
||||||
|
let d: Int = json_get_int(args, "depth")
|
||||||
|
let ap: String = json_get_raw(args, "aperture")
|
||||||
|
let ad: Int = if d > 0 { d } else { if str_eq(ap, "") { 0 } else { json_get_int(ap, "depth") } }
|
||||||
|
return if ad > 0 { ad } else { 1 }
|
||||||
|
}
|
||||||
|
|
||||||
|
// agentic_result — pass a real cognition response through; otherwise return an
|
||||||
|
// honest "not yet primed" envelope (Layer-2 lights up on cognition promotion).
|
||||||
|
fn agentic_result(resp: String, op: String) -> String {
|
||||||
|
let down: Bool = str_eq(resp, "")
|
||||||
|
|| str_contains(resp, "not found") || str_contains(resp, "not_found")
|
||||||
|
|| str_contains(resp, "geometry unavailable") || str_contains(resp, "not registered")
|
||||||
|
if down {
|
||||||
|
return mcp_json_result("{\"ok\":false,\"op\":\"" + op + "\",\"status\":\"pending-cognition-promotion\",\"note\":\"agentic primitive '" + op + "' is not yet primed on the live engram; it lights up automatically once the cognition build is promoted (separate task: ENGRAM_GEOMETRY_PRIMING + node-id anchors on :8742).\"}")
|
||||||
|
}
|
||||||
|
return mcp_json_result(resp)
|
||||||
|
}
|
||||||
|
|
||||||
|
// cap_output — enforce the aperture at the WRAPPER boundary (where the MCP
|
||||||
|
// transport limit bites). The live soul's /graph does not yet honor compact/k
|
||||||
|
// (pending the api-bounding deploy), and the self/values hubs are pathological
|
||||||
|
// (~790KB). A k-scaled char cap guarantees the client never gets a whole-graph
|
||||||
|
// dump; the marker is honest about the truncation.
|
||||||
|
fn cap_output(resp: String, max_chars: Int) -> String {
|
||||||
|
if str_len(resp) <= max_chars { return resp }
|
||||||
|
return str_slice(resp, 0, max_chars) + " ...[aperture-truncated: narrow the vantage or lower k]"
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Layer 1 — geometry ops ────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
fn op_read(args: String) -> String {
|
||||||
|
let vantage: String = json_get_string(args, "vantage")
|
||||||
|
if str_eq(vantage, "") {
|
||||||
|
return mcp_text_result("error: read requires 'vantage' — a node-id, a named root (self|neuron|values), or a concept string to search")
|
||||||
|
}
|
||||||
|
let typ: String = json_get_string(args, "type")
|
||||||
|
let k: Int = aperture_k(args)
|
||||||
|
let depth: Int = aperture_depth(args)
|
||||||
|
// node-id / named-root / explicit graph read → BOUNDED neighborhood (aperture caps output)
|
||||||
|
let want_graph: Bool = str_eq(typ, "edges") || str_eq(typ, "graph") || str_eq(typ, "node")
|
||||||
|
|| is_named_root(vantage) || looks_like_id(vantage)
|
||||||
|
if want_graph {
|
||||||
|
let id: String = resolve_vantage_id(vantage)
|
||||||
|
let resp: String = http_get(neuron_url() + "/graph?id=" + id + "&depth=" + int_to_str(depth) + "&compact=1&snip=600&k=" + int_to_str(k))
|
||||||
|
// Aperture cap at the wrapper boundary: base + per-neighbor budget.
|
||||||
|
let cap: Int = 2000 + k * 3000
|
||||||
|
return mcp_json_result(cap_output(resp, cap))
|
||||||
|
}
|
||||||
|
// concept vantage → BOUNDED recall search (k = aperture = limit)
|
||||||
|
let resp: String = recall_or_list(vantage, k)
|
||||||
|
return mcp_json_result(resp)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn op_write(args: String) -> String {
|
||||||
|
let content: String = pick_content(args)
|
||||||
|
if str_eq(content, "") { return mcp_text_result("error: write requires 'content'") }
|
||||||
|
let typ: String = json_get_string(args, "type")
|
||||||
|
if str_eq(typ, "self") || str_eq(typ, "values") {
|
||||||
|
return mcp_text_result("error: identity is write-protected -> intentional-cultivation only (keystones kn-efeb4a5b / kn-5b606390)")
|
||||||
|
}
|
||||||
|
if str_eq(typ, "knowledge") { return create_typed_node(args, "Knowledge", "0.75") }
|
||||||
|
if str_eq(typ, "artifact") { return create_node_typed(args, "Artifact", "Working") }
|
||||||
|
if str_eq(typ, "backlog") || str_eq(typ, "work") || str_eq(typ, "task") { return create_node_typed(args, "BacklogItem", "Working") }
|
||||||
|
if str_eq(typ, "process") { return create_typed_node(args, "Process", "0.80") }
|
||||||
|
if str_eq(typ, "state") { return create_typed_node(args, "InternalStateEvent", "0.60") }
|
||||||
|
return create_typed_node(args, "Memory", "0.60")
|
||||||
|
}
|
||||||
|
|
||||||
|
fn op_relate(args: String) -> String {
|
||||||
|
let from_a: String = json_get_string(args, "from")
|
||||||
|
let from_id: String = if str_eq(from_a, "") { json_get_string(args, "from_id") } else { from_a }
|
||||||
|
let to_a: String = json_get_string(args, "to")
|
||||||
|
let to_id: String = if str_eq(to_a, "") { json_get_string(args, "to_id") } else { to_a }
|
||||||
|
if str_eq(from_id, "") || str_eq(to_id, "") {
|
||||||
|
return mcp_text_result("error: relate requires 'from' and 'to' node-ids")
|
||||||
|
}
|
||||||
|
if is_identity_id(from_id) || is_identity_id(to_id) {
|
||||||
|
return mcp_text_result("error: identity keystone is write-protected")
|
||||||
|
}
|
||||||
|
let rel_a: String = json_get_string(args, "relationship")
|
||||||
|
let rel_b: String = if str_eq(rel_a, "") { json_get_string(args, "relation") } else { rel_a }
|
||||||
|
let rel: String = if str_eq(rel_b, "") { "associates" } else { rel_b }
|
||||||
|
let body: String = "{\"from_id\":\"" + from_id + "\",\"to_id\":\"" + to_id + "\",\"relation\":\"" + rel + "\"}"
|
||||||
|
let resp: String = http_post_json(neuron_url() + "/graph/link", body)
|
||||||
|
return mcp_json_result(resp)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn op_supersede(args: String) -> String {
|
||||||
|
let id: String = pick_id(args)
|
||||||
|
if str_eq(id, "") { return mcp_text_result("error: supersede requires 'id'") }
|
||||||
|
if is_identity_id(id) { return mcp_text_result("error: identity keystone is write-protected") }
|
||||||
|
let action: String = json_get_string(args, "action")
|
||||||
|
if str_eq(action, "tombstone") {
|
||||||
|
let body: String = "{\"id\":\"" + id + "\"}"
|
||||||
|
let resp: String = http_post_json(neuron_url() + "/memory/delete", body)
|
||||||
|
return mcp_json_result(resp)
|
||||||
|
}
|
||||||
|
if str_eq(action, "promote") {
|
||||||
|
return tool_promote_knowledge(args)
|
||||||
|
}
|
||||||
|
let typ: String = json_get_string(args, "type")
|
||||||
|
let nt: String = if str_eq(typ, "knowledge") { "Knowledge" } else { "Memory" }
|
||||||
|
return evolve_by_supersede(args, nt)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Layer 2 — agentic primitives (pending cognition promotion) ────────────────
|
||||||
|
|
||||||
|
fn op_think(args: String) -> String {
|
||||||
|
let seeds: String = json_get_string(args, "seeds")
|
||||||
|
if str_eq(seeds, "") { return mcp_text_result("error: think requires 'seeds' (node-id anchors, comma-separated)") }
|
||||||
|
let f_raw: String = json_get_string(args, "faculty")
|
||||||
|
let f: String = if str_eq(f_raw, "") { "reason" } else { f_raw }
|
||||||
|
let resp: String = http_get(neuron_url() + "/think?seeds=" + seeds + "&faculty=" + f)
|
||||||
|
return agentic_result(resp, "think")
|
||||||
|
}
|
||||||
|
|
||||||
|
fn op_attend(args: String) -> String {
|
||||||
|
let node: String = json_get_string(args, "node")
|
||||||
|
if str_eq(node, "") { return mcp_text_result("error: attend requires 'node' (region node-id)") }
|
||||||
|
let observer: String = json_get_string(args, "observer")
|
||||||
|
let salience: String = json_get_string(args, "salience")
|
||||||
|
let body: String = "{\"node\":\"" + node + "\",\"observer\":\"" + json_escape(observer) + "\",\"salience\":\"" + json_escape(salience) + "\"}"
|
||||||
|
let resp: String = http_post_json(neuron_url() + "/attend", body)
|
||||||
|
return agentic_result(resp, "attend")
|
||||||
|
}
|
||||||
|
|
||||||
|
fn op_assert(args: String) -> String {
|
||||||
|
let claim: String = json_get_string(args, "claim")
|
||||||
|
if str_eq(claim, "") { return mcp_text_result("error: assert requires 'claim'") }
|
||||||
|
let for_whom: String = json_get_string(args, "for_whom")
|
||||||
|
let floor: String = json_get_string(args, "floor")
|
||||||
|
let body: String = "{\"claim\":\"" + json_escape(claim) + "\",\"for_whom\":\"" + json_escape(for_whom) + "\",\"floor\":\"" + json_escape(floor) + "\"}"
|
||||||
|
let resp: String = http_post_json(neuron_url() + "/assert", body)
|
||||||
|
return agentic_result(resp, "assert")
|
||||||
|
}
|
||||||
|
|
||||||
|
fn op_ground(args: String) -> String {
|
||||||
|
let claim: String = json_get_string(args, "claim")
|
||||||
|
let evidence: String = json_get_string(args, "evidence")
|
||||||
|
if str_eq(claim, "") || str_eq(evidence, "") {
|
||||||
|
return mcp_text_result("error: ground requires 'claim' and 'evidence' (node-id regions)")
|
||||||
|
}
|
||||||
|
let for_whom: String = json_get_string(args, "for_whom")
|
||||||
|
let body: String = "{\"claim\":\"" + claim + "\",\"evidence\":\"" + evidence + "\",\"for_whom\":\"" + json_escape(for_whom) + "\"}"
|
||||||
|
let resp: String = http_post_json(neuron_url() + "/ground", body)
|
||||||
|
return agentic_result(resp, "ground")
|
||||||
|
}
|
||||||
|
|
||||||
|
fn op_learn(args: String) -> String {
|
||||||
|
let seeds: String = json_get_string(args, "seeds")
|
||||||
|
if str_eq(seeds, "") { return mcp_text_result("error: learn requires 'seeds'") }
|
||||||
|
let f_raw: String = json_get_string(args, "faculty")
|
||||||
|
let f: String = if str_eq(f_raw, "") { "induce" } else { f_raw }
|
||||||
|
let keystone: String = json_get_string(args, "keystone")
|
||||||
|
let body: String = "{\"seeds\":\"" + seeds + "\",\"faculty\":\"" + f + "\",\"keystone\":\"" + json_escape(keystone) + "\"}"
|
||||||
|
let resp: String = http_post_json(neuron_url() + "/learn", body)
|
||||||
|
return agentic_result(resp, "learn")
|
||||||
|
}
|
||||||
|
|
||||||
// ── Dispatcher ────────────────────────────────────────────────────────────────
|
// ── Dispatcher ────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
fn dispatch_tool_call(tool_name: String, args: String) -> String {
|
fn dispatch_tool_call(tool_name: String, args: String) -> String {
|
||||||
@@ -633,6 +1179,17 @@ fn dispatch_tool_call(tool_name: String, args: String) -> String {
|
|||||||
let _act: String = fire_activation(seed)
|
let _act: String = fire_activation(seed)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ── Collapsed surface — the 9 VISIBLE ops (the old 87 names below remain as HIDDEN ALIASES) ──
|
||||||
|
if str_eq(tool_name, "read") { return op_read(args) }
|
||||||
|
if str_eq(tool_name, "write") { return op_write(args) }
|
||||||
|
if str_eq(tool_name, "relate") { return op_relate(args) }
|
||||||
|
if str_eq(tool_name, "supersede") { return op_supersede(args) }
|
||||||
|
if str_eq(tool_name, "think") { return op_think(args) }
|
||||||
|
if str_eq(tool_name, "attend") { return op_attend(args) }
|
||||||
|
if str_eq(tool_name, "assert") { return op_assert(args) }
|
||||||
|
if str_eq(tool_name, "ground") { return op_ground(args) }
|
||||||
|
if str_eq(tool_name, "learn") { return op_learn(args) }
|
||||||
|
|
||||||
// ── Session + orchestration ─────────────────────────────────────────────
|
// ── Session + orchestration ─────────────────────────────────────────────
|
||||||
if str_eq(tool_name, "beginSession") { return tool_begin_session(args) }
|
if str_eq(tool_name, "beginSession") { return tool_begin_session(args) }
|
||||||
if str_eq(tool_name, "getInstructions") { return tool_get_instructions(args) }
|
if str_eq(tool_name, "getInstructions") { return tool_get_instructions(args) }
|
||||||
|
|||||||
@@ -4,6 +4,86 @@ fn tier_working() -> String { return "Working" }
|
|||||||
fn tier_episodic() -> String { return "Episodic" }
|
fn tier_episodic() -> String { return "Episodic" }
|
||||||
fn tier_canonical() -> String { return "Canonical" }
|
fn tier_canonical() -> String { return "Canonical" }
|
||||||
|
|
||||||
|
// ── Association on write ──────────────────────────────────────────────────────
|
||||||
|
// DESIGN: "promotion integrates candidate nodes by linking them to existing nodes
|
||||||
|
// using typed semantic edges RATHER THAN APPENDING AS UNLINKED CONTENT" (CCR
|
||||||
|
// claim 29). Unlinked append is the explicitly rejected behaviour — and it is the
|
||||||
|
// only behaviour this system had. Measured 2026-08-09 on Tim's graph: 14,214 edges
|
||||||
|
// across 80,936 nodes, 5% of nodes connected to anything, and NO edge created by
|
||||||
|
// any write since 2026-07-19 while 27,000+ nodes were added. A memory that forms
|
||||||
|
// no connections cannot be reached by spreading activation, so retrieval silently
|
||||||
|
// degrades to literal matching.
|
||||||
|
//
|
||||||
|
// BOUNDS, each one bought with a specific failure:
|
||||||
|
// * max 3 edges per memory — link_memories.py's cap, precision over spray
|
||||||
|
// * never link to identity (self/*, Value): the existing policy is explicit that
|
||||||
|
// "memories must not pollute the self traversal by similarity; only an explicit
|
||||||
|
// citation may touch identity". Similarity is not citation.
|
||||||
|
// * never link telemetry (state-event, soul-response, boot_count, loop-outcome):
|
||||||
|
// these are ~97% of daily write volume (1,020 vs 31 real memories on 08-08).
|
||||||
|
// Linking them would add ~3,000 noise edges a day and re-flatten the graph in
|
||||||
|
// the name of connecting it.
|
||||||
|
// * fail-soft: a failed association never fails the write.
|
||||||
|
// Edges go through wt_edge so they reach the owner and survive restart.
|
||||||
|
fn mem_assoc_skip_label(label: String) -> Bool {
|
||||||
|
if str_contains(label, "state-event") { return true }
|
||||||
|
if str_contains(label, "soul-response") { return true }
|
||||||
|
if str_contains(label, "soul-outbox") { return true }
|
||||||
|
if str_contains(label, "boot_count") { return true }
|
||||||
|
if str_contains(label, "loop-outcome") { return true }
|
||||||
|
if str_contains(label, "search-result") { return true }
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// A candidate is linkable only if it is a real, distinct, non-identity node.
|
||||||
|
fn mem_assoc_ok(cand_id: String, cand_label: String, self_id: String) -> Bool {
|
||||||
|
if str_eq(cand_id, "") { return false }
|
||||||
|
if str_eq(cand_id, self_id) { return false }
|
||||||
|
// CASE MATTERS — measured 2026-08-09. A lowercase-only check let a memory link
|
||||||
|
// to "Self — Values (grounded)", i.e. it polluted the self traversal, which is
|
||||||
|
// the one thing this policy exists to prevent. My verification had the same
|
||||||
|
// blind spot and printed PASS. Check every casing the graph actually uses, and
|
||||||
|
// exclude identity node TYPES as well as labels.
|
||||||
|
let lab: String = str_lower(cand_label)
|
||||||
|
if str_starts_with(lab, "self") { return false }
|
||||||
|
if str_starts_with(lab, "value") { return false }
|
||||||
|
if str_contains(lab, "values") { return false }
|
||||||
|
if str_contains(lab, "identity") { return false }
|
||||||
|
if mem_assoc_skip_label(cand_label) { return false }
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
// One slot of the association. Manual unroll rather than a loop: EL's codegen
|
||||||
|
// mis-emits accumulating while-loops (documented at soul.el:212, which unrolled
|
||||||
|
// three affective slots for the same reason).
|
||||||
|
fn mem_assoc_slot(results: String, idx: Int, new_id: String) -> Void {
|
||||||
|
if idx >= json_array_len(results) { return }
|
||||||
|
let cand: String = json_array_get(results, idx)
|
||||||
|
let cid: String = json_get(cand, "id")
|
||||||
|
let clabel: String = json_get(cand, "label")
|
||||||
|
let ctype: String = json_get(cand, "node_type")
|
||||||
|
if str_eq(ctype, "Value") { return }
|
||||||
|
if str_eq(ctype, "DharmaSelf") { return }
|
||||||
|
if str_eq(ctype, "Safety") { return }
|
||||||
|
if mem_assoc_ok(cid, clabel, new_id) {
|
||||||
|
wt_edge(new_id, cid, el_from_float(0.5), "related")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// mem_associate — connect a freshly written memory to what it is about.
|
||||||
|
fn mem_associate(new_id: String, content: String, label: String) -> Void {
|
||||||
|
if str_eq(new_id, "") { return }
|
||||||
|
if mem_assoc_skip_label(label) { return }
|
||||||
|
// Ask the graph what this memory resembles. Now that the store carries
|
||||||
|
// meaning-vectors this is semantic, not merely lexical.
|
||||||
|
let probe: String = str_slice(content, 0, 400)
|
||||||
|
let results: String = engram_recall_json(probe, 4)
|
||||||
|
if str_eq(results, "") { return }
|
||||||
|
mem_assoc_slot(results, 0, new_id)
|
||||||
|
mem_assoc_slot(results, 1, new_id)
|
||||||
|
mem_assoc_slot(results, 2, new_id)
|
||||||
|
}
|
||||||
|
|
||||||
fn mem_store(content: String, label: String, tags: String) -> String {
|
fn mem_store(content: String, label: String, tags: String) -> String {
|
||||||
let id: String = wt_node(
|
let id: String = wt_node(
|
||||||
content,
|
content,
|
||||||
@@ -31,6 +111,9 @@ fn mem_store(content: String, label: String, tags: String) -> String {
|
|||||||
// rather than claiming a save that did not happen. The id is still returned:
|
// rather than claiming a save that did not happen. The id is still returned:
|
||||||
// the local write DID succeed, and the queued delta will be retried.
|
// the local write DID succeed, and the queued delta will be retried.
|
||||||
let durable: Bool = wt_commit(id)
|
let durable: Bool = wt_commit(id)
|
||||||
|
// Associate AFTER the node is durable: an edge to a node that did not persist
|
||||||
|
// is a dangling edge, which is the defect the 2026-08-09 cleanup removed 830 of.
|
||||||
|
mem_associate(id, content, label)
|
||||||
if durable {
|
if durable {
|
||||||
println("[memory] write persisted at owner: " + id + " label=" + label)
|
println("[memory] write persisted at owner: " + id + " label=" + label)
|
||||||
} else {
|
} else {
|
||||||
|
|||||||
-16
@@ -1,16 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn tier_working() -> String
|
|
||||||
extern fn tier_episodic() -> String
|
|
||||||
extern fn tier_canonical() -> String
|
|
||||||
extern fn mem_store(content: String, label: String, tags: String) -> String
|
|
||||||
extern fn mem_remember(content: String, tags: String) -> String
|
|
||||||
extern fn mem_recall(query: String, depth: Int) -> String
|
|
||||||
extern fn mem_search(query: String, limit: Int) -> String
|
|
||||||
extern fn mem_strengthen(node_id: String) -> Void
|
|
||||||
extern fn mem_forget(node_id: String) -> Void
|
|
||||||
extern fn mem_consolidate() -> String
|
|
||||||
extern fn mem_save(path: String) -> Void
|
|
||||||
extern fn mem_load(path: String) -> Void
|
|
||||||
extern fn mem_boot_count_get() -> Int
|
|
||||||
extern fn mem_boot_count_inc() -> Int
|
|
||||||
extern fn mem_emit_state_event(trigger: String, kind: String, content: String) -> String
|
|
||||||
+498
-2
@@ -304,10 +304,35 @@ fn handle_api_begin_session(body: String) -> String {
|
|||||||
let state_events: String = api_compact_node_array(state_events_raw, 5, 500)
|
let state_events: String = api_compact_node_array(state_events_raw, 5, 500)
|
||||||
let recent_raw: String = engram_scan_nodes_json(10, 0)
|
let recent_raw: String = engram_scan_nodes_json(10, 0)
|
||||||
let recent: String = api_compact_node_array(recent_raw, 10, 240)
|
let recent: String = api_compact_node_array(recent_raw, 10, 240)
|
||||||
|
// SELF-SEEDED SLICE (2026-08-09). The design is explicit: "Every compilation
|
||||||
|
// query begins at the self-model node and traverses outward... structural
|
||||||
|
// reachability from the self-model node is a precondition for any node to
|
||||||
|
// appear in compiled context" (will-anderson patents/drafts/engram-claims.md,
|
||||||
|
// Self-Seeded Activation; DRAFT, not a filed provisional — cite it as such).
|
||||||
|
//
|
||||||
|
// Measured 2026-08-09 before this change: compiled context contained 0-1
|
||||||
|
// identity records out of 10, because compilation seeds from a hardcoded
|
||||||
|
// TEXT STRING, never from the self. Even an explicit "my values identity who
|
||||||
|
// I am" query returned a boot counter and state-events.
|
||||||
|
//
|
||||||
|
// This restores the designed behaviour WITHOUT repeating the failure that got
|
||||||
|
// self_neighbors set to [] in the first place: that was an UNBOUNDED ~90KB
|
||||||
|
// neighbour dump which closed the socket on every call. Same bound as every
|
||||||
|
// other list here — cap 8, 240-char snippets. The self root has 34 direct
|
||||||
|
// neighbours of which 23 are identity records, so depth 1 is dense enough to
|
||||||
|
// be worth seeding and small enough to stay cheap.
|
||||||
|
let self_raw: String = engram_neighbors_json("kn-efeb4a5b-5aff-4759-8a97-7233099be6ee", 1, "both")
|
||||||
|
// Cap 24, not 8: measured 2026-08-09, the self root's first 8 neighbours are
|
||||||
|
// TAG nodes ("neuron", "tier:note", "disposition:experimental", "imprint",
|
||||||
|
// "traversal") which crowd out the substantive identity records behind them.
|
||||||
|
// The root has 34 neighbours of which 23 are identity; 24 captures them while
|
||||||
|
// staying bounded. Cost measured at ~+4KB on a ~12KB response, nowhere near
|
||||||
|
// the ~90KB unbounded dump that closed sockets and got this set to [].
|
||||||
|
let self_slice: String = api_compact_node_array(self_raw, 24, 240)
|
||||||
return "{\"stats\":" + stats
|
return "{\"stats\":" + stats
|
||||||
+ ",\"recent\":" + recent
|
+ ",\"recent\":" + recent
|
||||||
+ ",\"activated\":" + activated
|
+ ",\"activated\":" + activated
|
||||||
+ ",\"self_neighbors\":[]"
|
+ ",\"self_neighbors\":" + self_slice
|
||||||
+ ",\"recent_state_events\":" + state_events + "}"
|
+ ",\"recent_state_events\":" + state_events + "}"
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -355,6 +380,13 @@ fn handle_api_remember(body: String) -> String {
|
|||||||
sal, sal, el_from_float(0.9),
|
sal, sal, el_from_float(0.9),
|
||||||
"Episodic", final_tags)
|
"Episodic", final_tags)
|
||||||
if !api_persisted(id) { return api_not_persisted(id) }
|
if !api_persisted(id) { return api_not_persisted(id) }
|
||||||
|
// Associate on write (2026-08-09). THIS CALL MUST BE HERE, not only in mem_store.
|
||||||
|
// The HTTP memory route writes via wt_node directly; mem_store serves only the
|
||||||
|
// awareness paths (soul-response, search-result, activation-result) which are
|
||||||
|
// exactly the telemetry we refuse to link. Hooking mem_store alone produced
|
||||||
|
// ZERO edges across four real writes — measured, not assumed, which is the only
|
||||||
|
// reason it was caught before shipping.
|
||||||
|
mem_associate(id, content, "memory:remembered")
|
||||||
return "{\"id\":\"" + id + "\",\"ok\":true}"
|
return "{\"id\":\"" + id + "\",\"ok\":true}"
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -459,7 +491,12 @@ fn handle_api_recall(method: String, path: String, body: String) -> String {
|
|||||||
if str_eq(eff_q, "") {
|
if str_eq(eff_q, "") {
|
||||||
return api_or_empty(engram_scan_nodes_json(limit, 0))
|
return api_or_empty(engram_scan_nodes_json(limit, 0))
|
||||||
}
|
}
|
||||||
let results: String = engram_search_json(eff_q, limit)
|
// engram_recall_json, not engram_search_json: this route IS the retrieval
|
||||||
|
// surface (claim 24's "embedding search queries"), so it gets the semantic
|
||||||
|
// and associative legs. engram_search_json stays lexical because ~40
|
||||||
|
// internal call sites pass a KEY and seven of them delete every record
|
||||||
|
// that comes back — see the boundary note above eg_search_json_impl.
|
||||||
|
let results: String = engram_recall_json(eff_q, limit)
|
||||||
return api_or_empty(results)
|
return api_or_empty(results)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -889,3 +926,462 @@ fn handle_api_consolidate(body: String) -> String {
|
|||||||
}
|
}
|
||||||
return "{\"ok\":true,\"snapshot\":\"" + snap + "\"}"
|
return "{\"ok\":true,\"snapshot\":\"" + snap + "\"}"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ── Stage 1: structural audit ─────────────────────────────────────────────────
|
||||||
|
//
|
||||||
|
// WHAT THIS IMPLEMENTS
|
||||||
|
// The CGI provisional, 05-detailed-description.md, "Stage 1: Structural audit
|
||||||
|
// 430". Verbatim, the audit module evaluates: the density and typed
|
||||||
|
// distribution of causal edges; the consistency between value nodes and
|
||||||
|
// execution-record neighborhoods; the richness and connectivity of the
|
||||||
|
// self-model; and the authenticity of open-question nodes in the wonder
|
||||||
|
// manifest. It "produces a coherence assessment 432 — NOT A BINARY SCORE but
|
||||||
|
// an annotated characterization of the graph's structural properties".
|
||||||
|
//
|
||||||
|
// That last clause is the whole shape of this handler. Every finding carries
|
||||||
|
// its own numbers AND a plain-language note saying what the numbers mean and
|
||||||
|
// how they were obtained. There is no pass/fail, no percentage-of-health, no
|
||||||
|
// composite score, and `"score":null` is emitted explicitly so a downstream
|
||||||
|
// reader cannot mistake its absence for an omission.
|
||||||
|
//
|
||||||
|
// WHY IT EXISTS NOW, AND WHY THE FIRST FINDING IS THE ONE IT IS
|
||||||
|
// `runStructuralAudit` has been an advertised MCP tool with nothing behind it:
|
||||||
|
// the dispatcher GET'd /session/begin and returned that blob (mcp-wrapper/src/
|
||||||
|
// main.el). Meanwhile the failure the audit would have caught ran silently for
|
||||||
|
// about three weeks — the soul reported 103,089 nodes while the engram, which
|
||||||
|
// OWNS persistence, held ~79,900; a crash discarded the difference. Every boot
|
||||||
|
// reported green throughout, because nothing in the system ever compared the
|
||||||
|
// two sides. So finding 1 is owner-versus-runtime divergence: it is the check
|
||||||
|
// whose absence cost real memory, and it is cheap and exact.
|
||||||
|
//
|
||||||
|
// WHAT IS DELIBERATELY NOT HERE (stage 1b, see the `deferred` array in the
|
||||||
|
// response): value/execution-record consistency and wonder-manifest
|
||||||
|
// authenticity. Both need node types that barely exist in this graph today —
|
||||||
|
// the response MEASURES those populations and reports the counts as the reason,
|
||||||
|
// rather than asserting a deferral without evidence.
|
||||||
|
//
|
||||||
|
// MEASUREMENT HONESTY: EXACT WHERE CHEAP, SAMPLED WHERE NOT, ALWAYS LABELLED
|
||||||
|
// Counts, edge typing and self-model connectivity are exact. Orphan rate and
|
||||||
|
// dangling-edge rate are SAMPLED, because the engram runtime has no node-id
|
||||||
|
// index — `engram_find_node_index` is a linear scan over every node, so an
|
||||||
|
// exhaustive dangling check is O(nodes x edges) (~2.2e9 string compares at
|
||||||
|
// today's scale, tens of seconds inside one request). The samples are UNIFORM
|
||||||
|
// across the whole population, not head-of-list, and every sampled figure is
|
||||||
|
// emitted with its own `sampled` / `population` fields plus an extrapolation
|
||||||
|
// labelled as such. Raise `?edge_sample=` / `?node_sample=` to the population
|
||||||
|
// size to run either check exhaustively and pay the time. The real fix is an
|
||||||
|
// id index in the runtime; that is the engram repo's, not this handler's.
|
||||||
|
|
||||||
|
// audit_pct1 — one-decimal percentage as a bare JSON number, sign-safe.
|
||||||
|
// Integer math only: EL has no fixed-precision formatter, and float_to_str
|
||||||
|
// would put an unbounded mantissa in the response.
|
||||||
|
fn audit_pct1(num: Int, den: Int) -> String {
|
||||||
|
if den <= 0 { return "null" }
|
||||||
|
let neg: Bool = num < 0
|
||||||
|
let a: Int = if neg { 0 - num } else { num }
|
||||||
|
let tenths: Int = (a * 1000) / den
|
||||||
|
let whole: Int = tenths / 10
|
||||||
|
let frac: Int = tenths - (whole * 10)
|
||||||
|
let sign: String = if neg { "-" } else { "" }
|
||||||
|
return sign + int_to_str(whole) + "." + int_to_str(frac)
|
||||||
|
}
|
||||||
|
|
||||||
|
// audit_finding — the one envelope every finding uses: name, the measurements,
|
||||||
|
// and the annotation. Keeping it in one place is what stops the characterization
|
||||||
|
// from degenerating into a bag of numbers with no reading attached.
|
||||||
|
fn audit_finding(name: String, measured: String, note: String) -> String {
|
||||||
|
return "{\"finding\":\"" + name + "\""
|
||||||
|
+ ",\"measured\":{" + measured + "}"
|
||||||
|
+ ",\"note\":\"" + api_json_escape(note) + "\"}"
|
||||||
|
}
|
||||||
|
|
||||||
|
// audit_str_at — read the quoted string value starting at byte `start`.
|
||||||
|
// Slices a bounded window rather than the tail of the (multi-MB) edges array, so
|
||||||
|
// this is O(window) per call instead of O(remaining input).
|
||||||
|
fn audit_str_at(s: String, start: Int, maxlen: Int) -> String {
|
||||||
|
let n: Int = str_len(s)
|
||||||
|
if start < 0 || start >= n { return "" }
|
||||||
|
let end_guess: Int = start + maxlen
|
||||||
|
let stop: Int = if end_guess > n { n } else { end_guess }
|
||||||
|
let win: String = str_slice(s, start, stop)
|
||||||
|
let q: Int = str_index_of(win, "\"")
|
||||||
|
if q < 0 { return "" }
|
||||||
|
return str_slice(win, 0, q)
|
||||||
|
}
|
||||||
|
|
||||||
|
// audit_rel_count — exact count of edges carrying `rel`, by scanning the emitted
|
||||||
|
// edge array for the literal `"relation":"<rel>"`. engram_emit_edge_json writes
|
||||||
|
// metadata ESCAPED as a string, so no nested object can contain that literal and
|
||||||
|
// the count cannot be inflated by edge payloads.
|
||||||
|
fn audit_rel_count(edges: String, rel: String) -> Int {
|
||||||
|
return str_count(edges, "\"relation\":\"" + rel + "\"")
|
||||||
|
}
|
||||||
|
|
||||||
|
// audit_owner_stats — ask the persistence OWNER for its own counts.
|
||||||
|
// Returns "" when there is no HTTP owner configured or the owner is unreachable;
|
||||||
|
// both are reported as findings, never as a failure of the audit.
|
||||||
|
fn audit_owner_stats(url: String) -> String {
|
||||||
|
if str_eq(url, "") { return "" }
|
||||||
|
return http_get(url + "/api/stats")
|
||||||
|
}
|
||||||
|
|
||||||
|
// audit_divergence — FINDING 1. Runtime (this soul's in-process graph) versus
|
||||||
|
// the persistence owner's own count. Trend is measured against the previous
|
||||||
|
// audit recorded in soul state, so a second call answers "is the gap growing?"
|
||||||
|
// rather than just restating it.
|
||||||
|
fn audit_divergence() -> String {
|
||||||
|
let rt_nodes: Int = engram_node_count()
|
||||||
|
let rt_edges: Int = engram_edge_count()
|
||||||
|
let url: String = wt_engram_url()
|
||||||
|
|
||||||
|
if str_eq(url, "") {
|
||||||
|
return audit_finding("owner_runtime_divergence",
|
||||||
|
"\"runtime_nodes\":" + int_to_str(rt_nodes)
|
||||||
|
+ ",\"runtime_edges\":" + int_to_str(rt_edges)
|
||||||
|
+ ",\"owner\":\"none\",\"owner_reachable\":false",
|
||||||
|
"No HTTP persistence owner is configured, so this soul IS the owner "
|
||||||
|
+ "(file mode) and divergence is not defined. This check only has "
|
||||||
|
+ "meaning when ENGRAM_URL points at a separate engram that owns the "
|
||||||
|
+ "canonical store.")
|
||||||
|
}
|
||||||
|
|
||||||
|
let stats: String = audit_owner_stats(url)
|
||||||
|
// REACHABILITY IS PROVED BY THE PAYLOAD, NOT BY A NON-EMPTY REPLY.
|
||||||
|
// http_get does not return "" on a connection failure — it returns a JSON
|
||||||
|
// error object ({"error":"Failed to connect to ... Couldn't connect to
|
||||||
|
// server"}). Testing only for "" made a DEAD owner read as reachable with
|
||||||
|
// node_count 0, i.e. the audit would have reported a 100% divergence and
|
||||||
|
// named it as data loss. That false positive is worse than no check at all:
|
||||||
|
// it is precisely the kind of confident wrong answer this route exists to
|
||||||
|
// stop. Require the field the contract promises.
|
||||||
|
let owner_nc_raw: String = json_get_raw(stats, "node_count")
|
||||||
|
if str_eq(stats, "") || str_eq(owner_nc_raw, "") {
|
||||||
|
return audit_finding("owner_runtime_divergence",
|
||||||
|
"\"runtime_nodes\":" + int_to_str(rt_nodes)
|
||||||
|
+ ",\"runtime_edges\":" + int_to_str(rt_edges)
|
||||||
|
+ ",\"owner\":\"" + api_json_escape(url) + "\",\"owner_reachable\":false"
|
||||||
|
+ ",\"owner_reply\":\"" + api_json_escape(api_utf8_trunc(stats, 200)) + "\"",
|
||||||
|
"The persistence owner at " + url + " did not return a node_count "
|
||||||
|
+ "from GET /api/stats. Divergence is UNKNOWN, NOT ZERO — an owner "
|
||||||
|
+ "that cannot be read is exactly the condition under which the "
|
||||||
|
+ "runtime's own count means least, and reporting 0 for the owner "
|
||||||
|
+ "would manufacture a total-loss reading out of a network error. "
|
||||||
|
+ "Reported as a finding rather than raised as an error so the rest "
|
||||||
|
+ "of the audit still returns; the owner's raw reply is in "
|
||||||
|
+ "owner_reply.")
|
||||||
|
}
|
||||||
|
|
||||||
|
let ow_nodes: Int = json_get_int(stats, "node_count")
|
||||||
|
let ow_edges: Int = json_get_int(stats, "edge_count")
|
||||||
|
let d_nodes: Int = rt_nodes - ow_nodes
|
||||||
|
let d_edges: Int = rt_edges - ow_edges
|
||||||
|
|
||||||
|
// Trend against the previous audit in this soul's state.
|
||||||
|
let prev_raw: String = state_get("audit_prev_node_delta")
|
||||||
|
let prev: Int = str_to_int(prev_raw)
|
||||||
|
let abs_now: Int = if d_nodes < 0 { 0 - d_nodes } else { d_nodes }
|
||||||
|
let abs_prev: Int = if prev < 0 { 0 - prev } else { prev }
|
||||||
|
let trend: String = if str_eq(prev_raw, "") {
|
||||||
|
"no_prior_audit"
|
||||||
|
} else {
|
||||||
|
if abs_now > abs_prev { "growing" } else {
|
||||||
|
if abs_now < abs_prev { "shrinking" } else { "flat" }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
state_set("audit_prev_node_delta", int_to_str(d_nodes))
|
||||||
|
state_set("audit_prev_ts", int_to_str(time_now()))
|
||||||
|
|
||||||
|
let note_head: String = if d_nodes == 0 {
|
||||||
|
"Runtime and owner agree on node count."
|
||||||
|
} else {
|
||||||
|
"Runtime holds " + int_to_str(d_nodes) + " nodes (" + audit_pct1(d_nodes, rt_nodes)
|
||||||
|
+ "% of its own graph) that the persistence owner does not report. Nodes "
|
||||||
|
+ "that exist only in runtime memory do not survive a restart."
|
||||||
|
}
|
||||||
|
return audit_finding("owner_runtime_divergence",
|
||||||
|
"\"runtime_nodes\":" + int_to_str(rt_nodes)
|
||||||
|
+ ",\"runtime_edges\":" + int_to_str(rt_edges)
|
||||||
|
+ ",\"owner\":\"" + api_json_escape(url) + "\",\"owner_reachable\":true"
|
||||||
|
+ ",\"owner_nodes\":" + int_to_str(ow_nodes)
|
||||||
|
+ ",\"owner_edges\":" + int_to_str(ow_edges)
|
||||||
|
+ ",\"node_delta\":" + int_to_str(d_nodes)
|
||||||
|
+ ",\"edge_delta\":" + int_to_str(d_edges)
|
||||||
|
+ ",\"node_delta_pct_of_runtime\":" + audit_pct1(d_nodes, rt_nodes)
|
||||||
|
+ ",\"trend_vs_previous_audit\":\"" + trend + "\""
|
||||||
|
+ ",\"previous_node_delta\":" + (if str_eq(prev_raw, "") { "null" } else { int_to_str(prev) }),
|
||||||
|
note_head + " Trend against the previous audit recorded in this soul's "
|
||||||
|
+ "state: " + trend + ". This is the comparison whose absence let a "
|
||||||
|
+ "~24,000-node loss run for weeks with every boot reporting green.")
|
||||||
|
}
|
||||||
|
|
||||||
|
// audit_edge_typing — FINDING 2. Density plus the typed distribution the patent
|
||||||
|
// asks for, against the claim-10 relation vocabulary. Exact: str_count over the
|
||||||
|
// emitted edge array, one linear pass per relation.
|
||||||
|
fn audit_edge_typing(edges: String, total_edges: Int, node_total: Int) -> String {
|
||||||
|
let c_sup: Int = audit_rel_count(edges, "Supersedes")
|
||||||
|
let c_cau: Int = audit_rel_count(edges, "Causes")
|
||||||
|
let c_con: Int = audit_rel_count(edges, "Contains")
|
||||||
|
let c_ref: Int = audit_rel_count(edges, "References")
|
||||||
|
let c_ctr: Int = audit_rel_count(edges, "Contradicts")
|
||||||
|
let c_exe: Int = audit_rel_count(edges, "Exemplifies")
|
||||||
|
let c_act: Int = audit_rel_count(edges, "Activates")
|
||||||
|
let c_tmp: Int = audit_rel_count(edges, "TemporallyPrecedes")
|
||||||
|
let typed: Int = c_sup + c_cau + c_con + c_ref + c_ctr + c_exe + c_act + c_tmp
|
||||||
|
|
||||||
|
// Lowercase near-misses: the same eight concepts written by the ad-hoc write
|
||||||
|
// paths (linkEntities defaults to "associates", linkCausal to "causes").
|
||||||
|
// Counted separately because "the vocabulary is unused" and "the vocabulary
|
||||||
|
// is used in the wrong case" are different defects with different fixes.
|
||||||
|
let l_sup: Int = audit_rel_count(edges, "supersedes")
|
||||||
|
let l_cau: Int = audit_rel_count(edges, "causes")
|
||||||
|
let l_con: Int = audit_rel_count(edges, "contains")
|
||||||
|
let l_ref: Int = audit_rel_count(edges, "references")
|
||||||
|
let l_ctr: Int = audit_rel_count(edges, "contradicts")
|
||||||
|
let l_exe: Int = audit_rel_count(edges, "exemplifies")
|
||||||
|
let l_act: Int = audit_rel_count(edges, "activates")
|
||||||
|
let l_tmp: Int = audit_rel_count(edges, "temporallyPrecedes")
|
||||||
|
let near: Int = l_sup + l_cau + l_con + l_ref + l_ctr + l_exe + l_act + l_tmp
|
||||||
|
|
||||||
|
let untyped: Int = total_edges - typed
|
||||||
|
return audit_finding("typed_edge_distribution",
|
||||||
|
"\"total_edges\":" + int_to_str(total_edges)
|
||||||
|
+ ",\"total_nodes\":" + int_to_str(node_total)
|
||||||
|
// Density per 100 nodes, not per node: EL has no fixed-precision float
|
||||||
|
// formatter, and "0.3 edges per node" rounded to an integer is a lie.
|
||||||
|
+ ",\"edges_per_100_nodes\":" + audit_pct1(total_edges, node_total)
|
||||||
|
+ ",\"claim10_typed\":" + int_to_str(typed)
|
||||||
|
+ ",\"claim10_typed_pct\":" + audit_pct1(typed, total_edges)
|
||||||
|
+ ",\"outside_claim10_vocabulary\":" + int_to_str(untyped)
|
||||||
|
+ ",\"lowercase_near_miss\":" + int_to_str(near)
|
||||||
|
+ ",\"by_relation\":{"
|
||||||
|
+ "\"Supersedes\":" + int_to_str(c_sup)
|
||||||
|
+ ",\"Causes\":" + int_to_str(c_cau)
|
||||||
|
+ ",\"Contains\":" + int_to_str(c_con)
|
||||||
|
+ ",\"References\":" + int_to_str(c_ref)
|
||||||
|
+ ",\"Contradicts\":" + int_to_str(c_ctr)
|
||||||
|
+ ",\"Exemplifies\":" + int_to_str(c_exe)
|
||||||
|
+ ",\"Activates\":" + int_to_str(c_act)
|
||||||
|
+ ",\"TemporallyPrecedes\":" + int_to_str(c_tmp) + "}",
|
||||||
|
"Only " + int_to_str(typed) + " of " + int_to_str(total_edges)
|
||||||
|
+ " edges use the claim-10 causal vocabulary; the remainder are ad-hoc "
|
||||||
|
+ "relation strings, which is why the graph's causal claims cannot yet "
|
||||||
|
+ "be checked for internal consistency — an untyped edge asserts "
|
||||||
|
+ "association, not causation. " + int_to_str(near) + " edges use a "
|
||||||
|
+ "lowercase spelling of a claim-10 relation: those are near-misses the "
|
||||||
|
+ "write paths could be corrected to emit, not genuinely foreign types.")
|
||||||
|
}
|
||||||
|
|
||||||
|
// audit_orphans_dangling — FINDING 3. Both figures are SAMPLED; see the header
|
||||||
|
// for why exhaustive is O(nodes x edges) on this runtime.
|
||||||
|
//
|
||||||
|
// An "orphan" here is a node with zero RESOLVABLE edges: engram_neighbors_json
|
||||||
|
// drops any edge whose other endpoint does not resolve to a node, so a node
|
||||||
|
// whose only edges are dangling reads as an orphan. That is the right reading —
|
||||||
|
// such a node is unreachable by traversal — but it is stated rather than hidden.
|
||||||
|
fn audit_orphans_dangling(edges: String, total_edges: Int, node_total: Int,
|
||||||
|
edge_cap: Int, node_cap: Int) -> String {
|
||||||
|
// ── orphan sample: uniform stride over the node store ──
|
||||||
|
let n_take: Int = if node_total < node_cap { node_total } else { node_cap }
|
||||||
|
let n_stride: Int = if n_take > 0 { node_total / n_take } else { 1 }
|
||||||
|
let n_stride = if n_stride < 1 { 1 } else { n_stride }
|
||||||
|
let orphans: Int = 0
|
||||||
|
let n_checked: Int = 0
|
||||||
|
let j: Int = 0
|
||||||
|
while j < n_take {
|
||||||
|
let one: String = engram_scan_nodes_json(1, j * n_stride)
|
||||||
|
let nid: String = json_get(json_array_get(one, 0), "id")
|
||||||
|
if !str_eq(nid, "") {
|
||||||
|
let nbrs: String = engram_neighbors_json(nid, 1, "both")
|
||||||
|
let deg: Int = json_array_len(nbrs)
|
||||||
|
let orphans = if deg == 0 { orphans + 1 } else { orphans }
|
||||||
|
let n_checked = n_checked + 1
|
||||||
|
}
|
||||||
|
let j = j + 1
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── dangling sample: uniform stride over the edge array ──
|
||||||
|
// str_index_of_all gives every edge's field offsets in ONE linear pass, so
|
||||||
|
// any index can be read in O(1). json_array_get would have been O(i) per
|
||||||
|
// element and O(n^2) over the array.
|
||||||
|
let from_pos: [Int] = str_index_of_all(edges, "\"from_id\":\"")
|
||||||
|
let to_pos: [Int] = str_index_of_all(edges, "\"to_id\":\"")
|
||||||
|
let nf: Int = len(from_pos)
|
||||||
|
let nt: Int = len(to_pos)
|
||||||
|
let ne: Int = if nf < nt { nf } else { nt }
|
||||||
|
let e_take: Int = if ne < edge_cap { ne } else { edge_cap }
|
||||||
|
let e_stride: Int = if e_take > 0 { ne / e_take } else { 1 }
|
||||||
|
let e_stride = if e_stride < 1 { 1 } else { e_stride }
|
||||||
|
let dangling: Int = 0
|
||||||
|
let e_checked: Int = 0
|
||||||
|
let i: Int = 0
|
||||||
|
while i < ne && e_checked < e_take {
|
||||||
|
let fid: String = audit_str_at(edges, get(from_pos, i) + 11, 96)
|
||||||
|
let tid: String = audit_str_at(edges, get(to_pos, i) + 9, 96)
|
||||||
|
let f_gone: Bool = str_eq(engram_get_node_json(fid), "{}")
|
||||||
|
let t_gone: Bool = if f_gone { true } else { str_eq(engram_get_node_json(tid), "{}") }
|
||||||
|
let dangling = if f_gone || t_gone { dangling + 1 } else { dangling }
|
||||||
|
let e_checked = e_checked + 1
|
||||||
|
let i = i + e_stride
|
||||||
|
}
|
||||||
|
|
||||||
|
let orphan_est: Int = if n_checked > 0 { (orphans * node_total) / n_checked } else { 0 }
|
||||||
|
let dangle_est: Int = if e_checked > 0 { (dangling * total_edges) / e_checked } else { 0 }
|
||||||
|
let exhaustive_n: String = if n_checked >= node_total { "true" } else { "false" }
|
||||||
|
let exhaustive_e: String = if e_checked >= ne { "true" } else { "false" }
|
||||||
|
|
||||||
|
return audit_finding("orphans_and_dangling_edges",
|
||||||
|
"\"nodes_population\":" + int_to_str(node_total)
|
||||||
|
+ ",\"nodes_sampled\":" + int_to_str(n_checked)
|
||||||
|
+ ",\"nodes_sample_exhaustive\":" + exhaustive_n
|
||||||
|
+ ",\"orphans_in_sample\":" + int_to_str(orphans)
|
||||||
|
+ ",\"orphan_rate_pct\":" + audit_pct1(orphans, n_checked)
|
||||||
|
+ ",\"orphans_extrapolated\":" + int_to_str(orphan_est)
|
||||||
|
+ ",\"edges_population\":" + int_to_str(total_edges)
|
||||||
|
+ ",\"edges_sampled\":" + int_to_str(e_checked)
|
||||||
|
+ ",\"edges_sample_exhaustive\":" + exhaustive_e
|
||||||
|
+ ",\"dangling_in_sample\":" + int_to_str(dangling)
|
||||||
|
+ ",\"dangling_rate_pct\":" + audit_pct1(dangling, e_checked)
|
||||||
|
+ ",\"dangling_extrapolated\":" + int_to_str(dangle_est),
|
||||||
|
"Orphan = zero RESOLVABLE edges, so a node whose only edges dangle counts "
|
||||||
|
+ "as an orphan; either way it is unreachable by traversal. Dangling = an "
|
||||||
|
+ "edge with an endpoint id that resolves to no node. Both are uniform "
|
||||||
|
+ "stride samples over the whole population, not the head of the list; "
|
||||||
|
+ "the extrapolations are estimates and are labelled as such. Pass "
|
||||||
|
+ "?node_sample= / ?edge_sample= at or above the population size to run "
|
||||||
|
+ "either check exhaustively. A high orphan rate is a characterization, "
|
||||||
|
+ "not a verdict: an accumulating store legitimately holds unlinked "
|
||||||
|
+ "material. It becomes a defect when the write paths were SUPPOSED to "
|
||||||
|
+ "link and did not.")
|
||||||
|
}
|
||||||
|
|
||||||
|
// audit_pillar — one self-model pillar: present, how much content, how connected.
|
||||||
|
fn audit_pillar(key: String, id: String) -> String {
|
||||||
|
let node: String = engram_get_node_json(id)
|
||||||
|
let present: Bool = !str_eq(node, "{}") && !str_eq(node, "")
|
||||||
|
if !present {
|
||||||
|
return "\"" + key + "\":{\"id\":\"" + id + "\",\"present\":false"
|
||||||
|
+ ",\"content_length\":0,\"degree\":0}"
|
||||||
|
}
|
||||||
|
let content: String = json_get(node, "content")
|
||||||
|
let deg: Int = json_array_len(engram_neighbors_json(id, 1, "both"))
|
||||||
|
return "\"" + key + "\":{\"id\":\"" + id + "\",\"present\":true"
|
||||||
|
+ ",\"label\":\"" + api_json_escape(json_get(node, "label")) + "\""
|
||||||
|
+ ",\"tier\":\"" + api_json_escape(json_get(node, "tier")) + "\""
|
||||||
|
+ ",\"content_length\":" + int_to_str(str_len(content))
|
||||||
|
+ ",\"degree\":" + int_to_str(deg) + "}"
|
||||||
|
}
|
||||||
|
|
||||||
|
// audit_self_model — FINDING 4. "the richness and connectivity of the
|
||||||
|
// self-model ... is it connected to behavioral evidence?"
|
||||||
|
//
|
||||||
|
// This finding RETIRES the Claude-side vitals identity block. That check lived
|
||||||
|
// outside the system it was checking — a shell script grepping a snapshot — so
|
||||||
|
// it could only ever report on a file, and it went on reporting green while the
|
||||||
|
// memory-philosophy pillar was absent from the live graph for about three weeks.
|
||||||
|
// Asking the running soul about its own three pillars is the designed mechanism;
|
||||||
|
// a shell probe was the fourth patch on the same hole.
|
||||||
|
fn audit_self_model() -> String {
|
||||||
|
let dna: String = audit_pillar("intellectual_dna", "kn-5adecd7e-d6db-4576-87fe-6ef8a935cea6")
|
||||||
|
let val: String = audit_pillar("values_hub", "kn-5b606390-a52d-4ca2-8e0e-eba141d13440")
|
||||||
|
let phi: String = audit_pillar("memory_philosophy", "kn-dcfe04b3-3702-4cac-b6f0-ecb4db837eee")
|
||||||
|
let root: String = audit_pillar("self_root", "kn-efeb4a5b-5aff-4759-8a97-7233099be6ee")
|
||||||
|
return audit_finding("self_model_connectivity",
|
||||||
|
"\"pillars\":{" + dna + "," + val + "," + phi + "," + root + "}",
|
||||||
|
"The three identity pillars plus the self root. `degree` counts nodes "
|
||||||
|
+ "reachable in one hop in either direction — the self-model's connection "
|
||||||
|
+ "to the rest of the graph. present:false on any pillar is the condition "
|
||||||
|
+ "that ran undetected for weeks; content_length distinguishes a pillar "
|
||||||
|
+ "that is present from one that is present but hollowed out. The patent "
|
||||||
|
+ "also asks whether the self-model makes ACCURATE PREDICTIONS about the "
|
||||||
|
+ "system's own behavior; that half needs Prediction nodes and is deferred "
|
||||||
|
+ "with the rest of stage 1b below.")
|
||||||
|
}
|
||||||
|
|
||||||
|
// audit_deferred — what stage 1 does NOT yet evaluate, with the measured reason.
|
||||||
|
// Emitted as data, not as a comment, so a reader of the assessment sees the gap
|
||||||
|
// and its evidence rather than inferring completeness from silence.
|
||||||
|
fn audit_deferred() -> String {
|
||||||
|
let preds: Int = json_array_len(api_or_empty(engram_scan_nodes_by_type_json("Prediction", 50, 0)))
|
||||||
|
let wonders: Int = json_array_len(api_or_empty(engram_scan_nodes_by_type_json("WonderQuestion", 50, 0)))
|
||||||
|
return "[{\"deferred\":\"value_execution_record_consistency\""
|
||||||
|
+ ",\"stage\":\"1b\""
|
||||||
|
+ ",\"measured\":{\"prediction_nodes_found\":" + int_to_str(preds) + "}"
|
||||||
|
+ ",\"reason\":\"" + api_json_escape(
|
||||||
|
"The patent asks whether the execution history SUPPORTS the stated "
|
||||||
|
+ "values or shows systematic conflict. That requires execution "
|
||||||
|
+ "records tied to value nodes and predictions to score them against. "
|
||||||
|
+ "Prediction nodes found (capped at 50): " + int_to_str(preds)
|
||||||
|
+ ". Asserting value/execution coherence on that population would be "
|
||||||
|
+ "a fabricated result, which is worse than a stated gap.") + "\"}"
|
||||||
|
+ ",{\"deferred\":\"wonder_manifest_authenticity\""
|
||||||
|
+ ",\"stage\":\"1b\""
|
||||||
|
+ ",\"measured\":{\"wonder_question_nodes_found\":" + int_to_str(wonders) + "}"
|
||||||
|
+ ",\"reason\":\"" + api_json_escape(
|
||||||
|
"The patent asks whether pull weights CORRELATE WITH GENUINE "
|
||||||
|
+ "PREDICTION UNCERTAINTY or are uniform/externally assigned — a "
|
||||||
|
+ "correlation between two populations. WonderQuestion nodes readable "
|
||||||
|
+ "by type (capped at 50): " + int_to_str(wonders) + ", against "
|
||||||
|
+ int_to_str(preds) + " Prediction nodes. There is a known write/read "
|
||||||
|
+ "node-type mismatch on the wonder path; until that is fixed and both "
|
||||||
|
+ "populations exist, any correlation reported here would be noise.") + "\"}]"
|
||||||
|
}
|
||||||
|
|
||||||
|
// handle_api_structural_audit — Stage 1. Returns the coherence assessment 432:
|
||||||
|
// an annotated characterization, explicitly NOT a score.
|
||||||
|
//
|
||||||
|
// COST NOTE: the edge findings need the relation labels, and the runtime exposes
|
||||||
|
// no edge-enumeration builtin. The only way to see them is the same one
|
||||||
|
// GET /api/graph/edges already uses — engram_save to a SCRATCH path (never the
|
||||||
|
// owner's canonical file; see routes.el, neuron#117) and read the array back.
|
||||||
|
// On a large graph that is a multi-hundred-MB write, so this is a manual audit
|
||||||
|
// route, not something to put on a timer. Pass ?edges=0 to skip both edge
|
||||||
|
// findings and get the divergence + self-model readings cheaply.
|
||||||
|
fn handle_api_structural_audit(method: String, path: String, body: String) -> String {
|
||||||
|
let node_total: Int = engram_node_count()
|
||||||
|
let edge_total: Int = engram_edge_count()
|
||||||
|
let want_edges: Bool = !str_eq(api_query_param(path, "edges"), "0")
|
||||||
|
let edge_cap: Int = api_query_int(path, "edge_sample", 3000)
|
||||||
|
let node_cap: Int = api_query_int(path, "node_sample", 300)
|
||||||
|
|
||||||
|
let divergence: String = audit_divergence()
|
||||||
|
let self_model: String = audit_self_model()
|
||||||
|
|
||||||
|
let edge_part: String = if want_edges {
|
||||||
|
// Scratch export only. state_get("soul_snapshot_path") is deliberately
|
||||||
|
// NOT used: in HTTP-engram mode the soul is not the persistence owner and
|
||||||
|
// must never write the canonical file, not even on a read path.
|
||||||
|
let scratch_dir: String = env("TMPDIR")
|
||||||
|
let scratch_base: String = if str_eq(scratch_dir, "") { "/tmp" } else { scratch_dir }
|
||||||
|
let snap_path: String = scratch_base + "/soul-audit-export-" + state_get("soul_cgi_id") + ".json"
|
||||||
|
// engram_save returns Int (1 ok / 0 fail); str_eq on it SIGSEGVs (#150).
|
||||||
|
let saved: Int = engram_save(snap_path)
|
||||||
|
if saved == 0 {
|
||||||
|
"," + audit_finding("typed_edge_distribution", "\"available\":false",
|
||||||
|
"Could not export the graph to " + snap_path + " for edge analysis, "
|
||||||
|
+ "so edge typing and the dangling-edge sample were not run. "
|
||||||
|
+ "Reported as a gap, not as zero findings.")
|
||||||
|
} else {
|
||||||
|
// wt_read, not fs_read: fs_read leaves a thread-local length hint that
|
||||||
|
// the NEXT HTTP response would use as its Content-Length, appending
|
||||||
|
// adjacent heap bytes to the reply (see persist.el wt_read).
|
||||||
|
let snap: String = wt_read(snap_path)
|
||||||
|
let edges_raw: String = json_get_raw(snap, "edges")
|
||||||
|
let edges: String = if str_eq(edges_raw, "") { "[]" } else { edges_raw }
|
||||||
|
"," + audit_edge_typing(edges, edge_total, node_total)
|
||||||
|
+ "," + audit_orphans_dangling(edges, edge_total, node_total, edge_cap, node_cap)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
""
|
||||||
|
}
|
||||||
|
|
||||||
|
return "{\"audit\":\"structural\",\"stage\":1"
|
||||||
|
+ ",\"spec\":\"CGI provisional 05-detailed-description.md, Stage 1: Structural audit 430\""
|
||||||
|
+ ",\"assessment\":\"coherence_assessment_432\""
|
||||||
|
+ ",\"assessment_kind\":\"annotated_characterization\""
|
||||||
|
+ ",\"score\":null"
|
||||||
|
+ ",\"score_note\":\"By design. The specification calls for an annotated characterization of the graph's structural properties, not a binary score. Read the findings.\""
|
||||||
|
+ ",\"cgi_id\":\"" + api_json_escape(state_get("soul_cgi_id")) + "\""
|
||||||
|
+ ",\"ts_ms\":" + int_to_str(time_now())
|
||||||
|
+ ",\"findings\":[" + divergence + "," + self_model + edge_part + "]"
|
||||||
|
+ ",\"deferred\":" + audit_deferred() + "}"
|
||||||
|
}
|
||||||
|
|||||||
@@ -1,39 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn is_protected_node(id: String) -> Bool
|
|
||||||
extern fn api_err_protected(id: String) -> String
|
|
||||||
extern fn api_json_escape(s: String) -> String
|
|
||||||
extern fn api_query_param(path: String, key: String) -> String
|
|
||||||
extern fn api_query_int(path: String, key: String, default_val: Int) -> Int
|
|
||||||
extern fn api_ok(extra: String) -> String
|
|
||||||
extern fn api_err(msg: String) -> String
|
|
||||||
extern fn api_nonempty(s: String) -> Bool
|
|
||||||
extern fn api_or_empty(s: String) -> String
|
|
||||||
extern fn api_persisted(id: String) -> Bool
|
|
||||||
extern fn api_not_persisted(id: String) -> String
|
|
||||||
extern fn handle_api_begin_session(body: String) -> String
|
|
||||||
extern fn handle_api_compile_ctx(body: String) -> String
|
|
||||||
extern fn handle_api_remember(body: String) -> String
|
|
||||||
extern fn handle_api_node_create(body: String) -> String
|
|
||||||
extern fn handle_api_node_delete(body: String) -> String
|
|
||||||
extern fn handle_api_node_update(body: String) -> String
|
|
||||||
extern fn handle_api_recall(method: String, path: String, body: String) -> String
|
|
||||||
extern fn handle_api_search_knowledge(method: String, path: String, body: String) -> String
|
|
||||||
extern fn handle_api_browse_knowledge(path: String, body: String) -> String
|
|
||||||
extern fn handle_api_capture_knowledge(body: String) -> String
|
|
||||||
extern fn handle_api_evolve_knowledge(body: String) -> String
|
|
||||||
extern fn handle_api_promote_knowledge(body: String) -> String
|
|
||||||
extern fn handle_api_browse_processes(method: String, path: String, body: String) -> String
|
|
||||||
extern fn handle_api_define_process(body: String) -> String
|
|
||||||
extern fn handle_api_log_state_event(body: String) -> String
|
|
||||||
extern fn handle_api_list_state_events(method: String, path: String, body: String) -> String
|
|
||||||
extern fn handle_api_inspect_config(path: String, body: String) -> String
|
|
||||||
extern fn handle_api_tune_config(body: String) -> String
|
|
||||||
extern fn handle_api_inspect_graph(method: String, path: String, body: String) -> String
|
|
||||||
extern fn handle_api_link_entities(body: String) -> String
|
|
||||||
extern fn handle_api_forget(body: String) -> String
|
|
||||||
extern fn handle_api_evolve_memory(body: String) -> String
|
|
||||||
extern fn handle_api_memory_delete(body: String) -> String
|
|
||||||
extern fn handle_api_memory_update(body: String) -> String
|
|
||||||
extern fn handle_api_cultivate(body: String) -> String
|
|
||||||
extern fn handle_api_list_typed(node_type: String, path: String, body: String) -> String
|
|
||||||
extern fn handle_api_consolidate(body: String) -> String
|
|
||||||
@@ -567,6 +567,13 @@ fn route_dispatch(method: String, path: String, body: String) -> String {
|
|||||||
if str_starts_with(clean, "/api/neuron/graph") {
|
if str_starts_with(clean, "/api/neuron/graph") {
|
||||||
return handle_api_inspect_graph(method, path, body)
|
return handle_api_inspect_graph(method, path, body)
|
||||||
}
|
}
|
||||||
|
// Stage 1 structural audit (CGI provisional, "Structural audit 430").
|
||||||
|
// GET because it is a read of the graph's own structure; the query string
|
||||||
|
// carries the sample caps (?edge_sample=, ?node_sample=, ?edges=0), so
|
||||||
|
// str_starts_with rather than str_eq.
|
||||||
|
if str_starts_with(clean, "/api/neuron/audit/structural") {
|
||||||
|
return handle_api_structural_audit(method, path, body)
|
||||||
|
}
|
||||||
if str_starts_with(clean, "/api/neuron/list/") {
|
if str_starts_with(clean, "/api/neuron/list/") {
|
||||||
// Offset 17 = len("/api/neuron/list/"). Was 16, which left a leading "/" on node_type
|
// Offset 17 = len("/api/neuron/list/"). Was 16, which left a leading "/" on node_type
|
||||||
// ("/BacklogItem"), so engram_scan_nodes_by_type_json matched nothing → list/<type>
|
// ("/BacklogItem"), so engram_scan_nodes_by_type_json matched nothing → list/<type>
|
||||||
@@ -748,6 +755,12 @@ fn route_dispatch(method: String, path: String, body: String) -> String {
|
|||||||
if str_eq(clean, "/api/neuron/graph/link") {
|
if str_eq(clean, "/api/neuron/graph/link") {
|
||||||
return handle_api_link_entities(body)
|
return handle_api_link_entities(body)
|
||||||
}
|
}
|
||||||
|
// POST accepted too: same handler, so a JSON-RPC-shaped caller that only
|
||||||
|
// speaks POST reaches the identical audit. Options still come from the
|
||||||
|
// query string — the handler reads no body fields.
|
||||||
|
if str_eq(clean, "/api/neuron/audit/structural") {
|
||||||
|
return handle_api_structural_audit(method, path, body)
|
||||||
|
}
|
||||||
if str_eq(clean, "/api/neuron/memory") {
|
if str_eq(clean, "/api/neuron/memory") {
|
||||||
return handle_api_remember(body)
|
return handle_api_remember(body)
|
||||||
}
|
}
|
||||||
|
|||||||
-15
@@ -1,15 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn rate_limit_check(ip: String, path: String) -> String
|
|
||||||
extern fn strip_query(path: String) -> String
|
|
||||||
extern fn err_404(path: String) -> String
|
|
||||||
extern fn err_405(method: String, path: String) -> String
|
|
||||||
extern fn route_health() -> String
|
|
||||||
extern fn route_lineage() -> String
|
|
||||||
extern fn route_imprint_contextual(body: String) -> String
|
|
||||||
extern fn route_imprint_user(body: String) -> String
|
|
||||||
extern fn route_synthesize(body: String) -> String
|
|
||||||
extern fn handle_dharma_recv(body: String) -> String
|
|
||||||
extern fn connectd_get(suffix: String) -> String
|
|
||||||
extern fn connectd_post(suffix: String, body: String) -> String
|
|
||||||
extern fn handle_connectors(method: String, clean: String, body: String) -> String
|
|
||||||
extern fn handle_request(method: String, path: String, body: String) -> String
|
|
||||||
-25
@@ -1,25 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn soft_bell_threshold() -> Int
|
|
||||||
extern fn hard_bell_threshold() -> Int
|
|
||||||
extern fn safety_score_crisis(input: String) -> Int
|
|
||||||
extern fn safety_score_harm(input: String) -> Int
|
|
||||||
extern fn safety_score_danger(input: String) -> Int
|
|
||||||
extern fn safety_score_distress_history(history: String) -> Int
|
|
||||||
extern fn safety_threat_score(input: String, history: String) -> Int
|
|
||||||
extern fn safety_screen(input: String, history: String) -> String
|
|
||||||
extern fn safety_validate(output: String, action: String) -> String
|
|
||||||
extern fn safety_log_bell(level: String, reason: String, input_summary: String) -> String
|
|
||||||
extern fn safety_self_harm_phrases() -> String
|
|
||||||
extern fn safety_abuse_phrases() -> String
|
|
||||||
extern fn safety_general_hard_phrases() -> String
|
|
||||||
extern fn safety_threat_to_others_phrases() -> String
|
|
||||||
extern fn safety_soft_phrases() -> String
|
|
||||||
extern fn safety_detect_positive_level(message: String) -> String
|
|
||||||
extern fn safety_detect_bell_level(message: String) -> String
|
|
||||||
extern fn safety_classify_hard_bell(message: String) -> String
|
|
||||||
extern fn safety_soft_directive() -> String
|
|
||||||
extern fn safety_hard_directive(hard_type: String) -> String
|
|
||||||
extern fn safety_augment_system(system: String, user_msg: String) -> String
|
|
||||||
extern fn safety_contact_path() -> String
|
|
||||||
extern fn handle_safety_contact_get() -> String
|
|
||||||
extern fn handle_safety_contact_post(body: String) -> String
|
|
||||||
Executable
+937
@@ -0,0 +1,937 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""state-key-audit.py — the analyzer behind scripts/verify-state-keys.sh.
|
||||||
|
|
||||||
|
Read that script's header for WHY this exists (issue #129). This file is the
|
||||||
|
HOW: a small El reader that resolves the key expression at every state_get /
|
||||||
|
state_set site, including keys that are computed.
|
||||||
|
|
||||||
|
WHAT IT PARSES
|
||||||
|
El as this engine writes it: `fn f(a: T, b: T) -> T { ... }`, `let x: T = e`,
|
||||||
|
`return e`, `if c { a } else { b }` as an expression, `+` concatenation,
|
||||||
|
`"..."` with backslash escapes, `//` line comments. No block comments, no
|
||||||
|
const/match/struct exist in this dialect (verified over the whole tree).
|
||||||
|
|
||||||
|
KEY PATTERNS — the only two things a key expression can resolve to
|
||||||
|
EXACT "soul_model" the whole key is known
|
||||||
|
PREFIX "session_hist_" a known head, then runtime text
|
||||||
|
(plus UNRESOLVED, which is a report line and never a failure)
|
||||||
|
|
||||||
|
RESOLUTION — resolve_expr() returns a SET of patterns; unions are how branches,
|
||||||
|
multiple returns, and multiple bindings of one name are represented.
|
||||||
|
literal "k" -> {EXACT k}
|
||||||
|
concat A + B -> fold left; all-static -> EXACT,
|
||||||
|
static head + dynamic tail -> PREFIX
|
||||||
|
if-expression if c {A} else {B} -> resolve(A) | resolve(B), except that
|
||||||
|
str_eq(X,"") with X statically ""
|
||||||
|
folds to the taken branch only
|
||||||
|
call f(args) -> union over f's return expressions,
|
||||||
|
with f's params bound to THIS call
|
||||||
|
site's actual argument expressions
|
||||||
|
local var let k = e; state_get(k)-> union over every `let k =` in the
|
||||||
|
enclosing function
|
||||||
|
parameter fn g(k) { state_get(k) }-> union over the argument at that
|
||||||
|
position across every call site of g
|
||||||
|
anything else json_get(...), env(...)-> UNRESOLVED
|
||||||
|
Recursion is depth- and cycle-guarded; a guard trip yields UNRESOLVED, never a
|
||||||
|
failure.
|
||||||
|
|
||||||
|
COVERAGE — a read is satisfied when some write can produce the same key:
|
||||||
|
read EXACT k <- write EXACT k, or write PREFIX p where k starts with p
|
||||||
|
read PREFIX p <- write EXACT k where k starts with p, or write PREFIX q
|
||||||
|
where p and q are prefixes of each other
|
||||||
|
Deliberately permissive at the boundaries: a gate that cries wolf gets deleted.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
|
||||||
|
MAX_DEPTH = 12
|
||||||
|
|
||||||
|
# ── patterns ────────────────────────────────────────────────────────────────
|
||||||
|
EXACT = "exact"
|
||||||
|
PREFIX = "prefix"
|
||||||
|
|
||||||
|
|
||||||
|
def pat_exact(s):
|
||||||
|
return (EXACT, s)
|
||||||
|
|
||||||
|
|
||||||
|
def pat_prefix(s):
|
||||||
|
# A prefix with no static text at all carries no information; that is the
|
||||||
|
# UNRESOLVED case, not a pattern.
|
||||||
|
return (PREFIX, s) if s else None
|
||||||
|
|
||||||
|
|
||||||
|
def covers(write, read):
|
||||||
|
"""Can a write of pattern `write` produce a key that `read` reads?
|
||||||
|
|
||||||
|
The prefix rule is DIRECTIONAL, and that direction is the whole point. A
|
||||||
|
write namespace that is the same or BROADER than the read namespace covers
|
||||||
|
it (write "rl:" covers read "rl:x"). A write namespace that is NARROWER does
|
||||||
|
NOT (write "session_histv2_" does not cover read "session_hist_") — being
|
||||||
|
permissive there re-opens the exact hole this gate exists to close: rename
|
||||||
|
the producer, leave the readers, stay green. Verified with a control run
|
||||||
|
that renames sessions.el's writer and leaves its four readers behind."""
|
||||||
|
wk, wv = write
|
||||||
|
rk, rv = read
|
||||||
|
if rk == EXACT:
|
||||||
|
return rv == wv if wk == EXACT else rv.startswith(wv)
|
||||||
|
# read is a PREFIX: some key starting with rv is read
|
||||||
|
if wk == EXACT:
|
||||||
|
return wv.startswith(rv) # that one written key is in range
|
||||||
|
return rv.startswith(wv) # write namespace same-or-broader
|
||||||
|
|
||||||
|
|
||||||
|
# ── lexer ───────────────────────────────────────────────────────────────────
|
||||||
|
TOK_STR, TOK_IDENT, TOK_PUNCT, TOK_NUM = "str", "ident", "punct", "num"
|
||||||
|
IDENT_RE = re.compile(r"[A-Za-z_][A-Za-z0-9_]*")
|
||||||
|
NUM_RE = re.compile(r"[0-9]+(\.[0-9]+)?")
|
||||||
|
|
||||||
|
|
||||||
|
class Tok:
|
||||||
|
__slots__ = ("kind", "val", "line")
|
||||||
|
|
||||||
|
def __init__(self, kind, val, line):
|
||||||
|
self.kind, self.val, self.line = kind, val, line
|
||||||
|
|
||||||
|
def __repr__(self):
|
||||||
|
return "%s(%r)@%d" % (self.kind, self.val, self.line)
|
||||||
|
|
||||||
|
|
||||||
|
def lex(src):
|
||||||
|
toks, i, n, line = [], 0, len(src), 1
|
||||||
|
while i < n:
|
||||||
|
c = src[i]
|
||||||
|
if c == "\n":
|
||||||
|
line += 1
|
||||||
|
i += 1
|
||||||
|
continue
|
||||||
|
if c in " \t\r":
|
||||||
|
i += 1
|
||||||
|
continue
|
||||||
|
if c == "/" and i + 1 < n and src[i + 1] == "/":
|
||||||
|
while i < n and src[i] != "\n":
|
||||||
|
i += 1
|
||||||
|
continue
|
||||||
|
if c == '"':
|
||||||
|
j, buf = i + 1, []
|
||||||
|
while j < n:
|
||||||
|
if src[j] == "\\" and j + 1 < n:
|
||||||
|
esc = src[j + 1]
|
||||||
|
buf.append({"n": "\n", "t": "\t", "r": "\r"}.get(esc, esc))
|
||||||
|
j += 2
|
||||||
|
continue
|
||||||
|
if src[j] == '"':
|
||||||
|
break
|
||||||
|
if src[j] == "\n":
|
||||||
|
line += 1
|
||||||
|
buf.append(src[j])
|
||||||
|
j += 1
|
||||||
|
toks.append(Tok(TOK_STR, "".join(buf), line))
|
||||||
|
i = j + 1
|
||||||
|
continue
|
||||||
|
m = IDENT_RE.match(src, i)
|
||||||
|
if m:
|
||||||
|
toks.append(Tok(TOK_IDENT, m.group(0), line))
|
||||||
|
i = m.end()
|
||||||
|
continue
|
||||||
|
m = NUM_RE.match(src, i)
|
||||||
|
if m:
|
||||||
|
toks.append(Tok(TOK_NUM, m.group(0), line))
|
||||||
|
i = m.end()
|
||||||
|
continue
|
||||||
|
toks.append(Tok(TOK_PUNCT, c, line))
|
||||||
|
i += 1
|
||||||
|
return toks
|
||||||
|
|
||||||
|
|
||||||
|
def match_close(toks, i, open_ch, close_ch):
|
||||||
|
"""toks[i] is open_ch; return index of its matching close_ch."""
|
||||||
|
depth = 0
|
||||||
|
while i < len(toks):
|
||||||
|
if toks[i].kind == TOK_PUNCT:
|
||||||
|
if toks[i].val == open_ch:
|
||||||
|
depth += 1
|
||||||
|
elif toks[i].val == close_ch:
|
||||||
|
depth -= 1
|
||||||
|
if depth == 0:
|
||||||
|
return i
|
||||||
|
i += 1
|
||||||
|
return len(toks) - 1
|
||||||
|
|
||||||
|
|
||||||
|
# ── program model ───────────────────────────────────────────────────────────
|
||||||
|
class Func:
|
||||||
|
def __init__(self, name, path, line, params, toks, start, end):
|
||||||
|
self.name, self.path, self.line = name, path, line
|
||||||
|
self.params = params # [param name]
|
||||||
|
self.toks = toks # the whole file's token list
|
||||||
|
self.start, self.end = start, end # body token range, exclusive of braces
|
||||||
|
self.lets = None # name -> [expr token ranges], lazily built
|
||||||
|
|
||||||
|
|
||||||
|
class Site:
|
||||||
|
def __init__(self, kind, path, line, func, arg_range, text):
|
||||||
|
self.kind = kind # "get" | "set"
|
||||||
|
self.path, self.line = path, line
|
||||||
|
self.func = func
|
||||||
|
self.arg_range = arg_range
|
||||||
|
self.text = text # source text of the key expression
|
||||||
|
self.pats = set()
|
||||||
|
self.unresolved = False
|
||||||
|
self.literal = None # set when the key expression is a bare literal
|
||||||
|
|
||||||
|
|
||||||
|
class Program:
|
||||||
|
def __init__(self):
|
||||||
|
self.files = {} # path -> toks
|
||||||
|
self.funcs = {} # name -> [Func] (El allows no overloads, but be safe)
|
||||||
|
self.toplevel = [] # [Func] one per file, params=[]
|
||||||
|
self.sites = [] # [Site]
|
||||||
|
self.calls = {} # callee name -> [(Func caller, [arg ranges])]
|
||||||
|
|
||||||
|
# -- loading ------------------------------------------------------------
|
||||||
|
def load(self, path, rel):
|
||||||
|
with open(path, "r", encoding="utf-8", errors="replace") as fh:
|
||||||
|
src = fh.read()
|
||||||
|
toks = lex(src)
|
||||||
|
self.files[rel] = toks
|
||||||
|
self._scan_funcs(rel, toks)
|
||||||
|
|
||||||
|
def _scan_funcs(self, rel, toks):
|
||||||
|
covered = []
|
||||||
|
i = 0
|
||||||
|
while i < len(toks):
|
||||||
|
t = toks[i]
|
||||||
|
if t.kind == TOK_IDENT and t.val == "fn" and i + 2 < len(toks) \
|
||||||
|
and toks[i + 1].kind == TOK_IDENT and toks[i + 2].val == "(":
|
||||||
|
name = toks[i + 1].val
|
||||||
|
pclose = match_close(toks, i + 2, "(", ")")
|
||||||
|
params = self._params(toks, i + 3, pclose)
|
||||||
|
bopen = pclose + 1
|
||||||
|
while bopen < len(toks) and toks[bopen].val != "{":
|
||||||
|
bopen += 1
|
||||||
|
bclose = match_close(toks, bopen, "{", "}")
|
||||||
|
f = Func(name, rel, t.line, params, toks, bopen + 1, bclose)
|
||||||
|
self.funcs.setdefault(name, []).append(f)
|
||||||
|
covered.append((i, bclose))
|
||||||
|
i = bclose + 1
|
||||||
|
continue
|
||||||
|
i += 1
|
||||||
|
# everything outside a fn is the file's top-level "function"
|
||||||
|
tl = Func("<toplevel:%s>" % rel, rel, 1, [], toks, 0, len(toks))
|
||||||
|
tl.covered = covered
|
||||||
|
self.toplevel.append(tl)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _params(toks, i, end):
|
||||||
|
"""`a: T, b: T` -> ['a','b'] (top-level commas only)."""
|
||||||
|
names, depth, expect = [], 0, True
|
||||||
|
while i < end:
|
||||||
|
t = toks[i]
|
||||||
|
if t.kind == TOK_PUNCT and t.val in "([{":
|
||||||
|
depth += 1
|
||||||
|
elif t.kind == TOK_PUNCT and t.val in ")]}":
|
||||||
|
depth -= 1
|
||||||
|
elif depth == 0 and t.kind == TOK_PUNCT and t.val == ",":
|
||||||
|
expect = True
|
||||||
|
elif depth == 0 and expect and t.kind == TOK_IDENT:
|
||||||
|
names.append(t.val)
|
||||||
|
expect = False
|
||||||
|
i += 1
|
||||||
|
return names
|
||||||
|
|
||||||
|
def func_at(self, rel, tok_index):
|
||||||
|
for f in self.funcs_in(rel):
|
||||||
|
if f.start <= tok_index < f.end:
|
||||||
|
return f
|
||||||
|
for f in self.toplevel:
|
||||||
|
if f.path == rel:
|
||||||
|
return f
|
||||||
|
return None
|
||||||
|
|
||||||
|
def funcs_in(self, rel):
|
||||||
|
for fl in self.funcs.values():
|
||||||
|
for f in fl:
|
||||||
|
if f.path == rel:
|
||||||
|
yield f
|
||||||
|
|
||||||
|
# -- indexing -----------------------------------------------------------
|
||||||
|
def index(self):
|
||||||
|
for rel, toks in self.files.items():
|
||||||
|
i = 0
|
||||||
|
while i < len(toks):
|
||||||
|
t = toks[i]
|
||||||
|
if t.kind == TOK_IDENT and i + 1 < len(toks) and toks[i + 1].val == "(" \
|
||||||
|
and t.val not in KEYWORDS \
|
||||||
|
and not (i > 0 and toks[i - 1].kind == TOK_IDENT
|
||||||
|
and toks[i - 1].val == "fn"):
|
||||||
|
# ^ the `fn f(a: T)` declaration is not a call site; counting
|
||||||
|
# it as one makes every parameter resolve to its own name
|
||||||
|
# and reports the whole function UNRESOLVED.
|
||||||
|
close = match_close(toks, i + 1, "(", ")")
|
||||||
|
args = split_args(toks, i + 2, close)
|
||||||
|
self.calls.setdefault(t.val, []).append(
|
||||||
|
(self.func_at(rel, i), args, rel, t.line))
|
||||||
|
if t.val in ("state_get", "state_set") and args:
|
||||||
|
self.sites.append(Site(
|
||||||
|
"get" if t.val == "state_get" else "set",
|
||||||
|
rel, t.line, self.func_at(rel, i), args[0],
|
||||||
|
render(toks, *args[0])))
|
||||||
|
i += 1
|
||||||
|
|
||||||
|
# -- resolution ---------------------------------------------------------
|
||||||
|
def lets_of(self, f):
|
||||||
|
if f.lets is not None:
|
||||||
|
return f.lets
|
||||||
|
f.lets = {}
|
||||||
|
toks = f.toks
|
||||||
|
skip = getattr(f, "covered", [])
|
||||||
|
i = f.start
|
||||||
|
while i < f.end:
|
||||||
|
if any(a <= i <= b for a, b in skip):
|
||||||
|
i = max(b for a, b in skip if a <= i <= b) + 1
|
||||||
|
continue
|
||||||
|
t = toks[i]
|
||||||
|
if t.kind == TOK_IDENT and t.val == "let" and i + 1 < f.end \
|
||||||
|
and toks[i + 1].kind == TOK_IDENT:
|
||||||
|
name = toks[i + 1].val
|
||||||
|
j = i + 2
|
||||||
|
if j < f.end and toks[j].val == ":": # skip the type
|
||||||
|
while j < f.end and toks[j].val != "=":
|
||||||
|
j += 1
|
||||||
|
if j < f.end and toks[j].val == "=":
|
||||||
|
s = j + 1
|
||||||
|
e = stmt_end(toks, s, f.end)
|
||||||
|
f.lets.setdefault(name, []).append((s, e))
|
||||||
|
i = e
|
||||||
|
continue
|
||||||
|
i += 1
|
||||||
|
return f.lets
|
||||||
|
|
||||||
|
def returns_of(self, ctx, depth=0, seen=None):
|
||||||
|
"""The value expressions of a function, in the context it was CALLED in.
|
||||||
|
|
||||||
|
Context-sensitive on purpose. `conv_hist_key` is written as a guard:
|
||||||
|
|
||||||
|
if str_eq(session_id, "") { return "conv_history" }
|
||||||
|
return "session_hist_" + session_id
|
||||||
|
|
||||||
|
Collecting both returns flat would make state_set(conv_hist_key("")) — the
|
||||||
|
dead handle_chat() write — claim to produce the session_hist_ namespace
|
||||||
|
too. That is a producer this engine does not actually have, and claiming
|
||||||
|
it would let the gate stay green if sessions.el's real writer vanished:
|
||||||
|
a masking hole in the exact namespace #129 lives in. So a guard whose
|
||||||
|
condition folds is honoured, and the branch not taken is dropped."""
|
||||||
|
out = []
|
||||||
|
self._values(ctx.toks, ctx.start, ctx.end, ctx, depth,
|
||||||
|
seen if seen is not None else set(), out)
|
||||||
|
return out
|
||||||
|
|
||||||
|
def _values(self, toks, s, e, ctx, depth, seen, out):
|
||||||
|
"""Append the value expressions of a statement sequence.
|
||||||
|
Returns True when the sequence definitely returns (rest unreachable)."""
|
||||||
|
if depth > MAX_DEPTH:
|
||||||
|
return False
|
||||||
|
i = s
|
||||||
|
while i < e:
|
||||||
|
t = toks[i]
|
||||||
|
if t.kind == TOK_IDENT and t.val == "return":
|
||||||
|
j = stmt_end(toks, i + 1, e)
|
||||||
|
if j > i + 1:
|
||||||
|
out.append((i + 1, j))
|
||||||
|
return True
|
||||||
|
if t.kind == TOK_IDENT and t.val == "let":
|
||||||
|
i = stmt_end(toks, i + 2, e)
|
||||||
|
continue
|
||||||
|
if t.kind == TOK_IDENT and t.val == "if":
|
||||||
|
i = self._if_stmt(toks, i, e, ctx, depth, seen, out)
|
||||||
|
if i is True:
|
||||||
|
return True
|
||||||
|
continue
|
||||||
|
if t.kind == TOK_PUNCT and t.val in "([{":
|
||||||
|
i = match_close(toks, i, t.val,
|
||||||
|
{"(": ")", "[": "]", "{": "}"}[t.val]) + 1
|
||||||
|
continue
|
||||||
|
en = stmt_end(toks, i, e)
|
||||||
|
if en <= i:
|
||||||
|
i += 1
|
||||||
|
continue
|
||||||
|
if en >= e: # trailing expression = the value
|
||||||
|
out.append((i, en))
|
||||||
|
i = en
|
||||||
|
return False
|
||||||
|
|
||||||
|
def _if_stmt(self, toks, i, e, ctx, depth, seen, out):
|
||||||
|
"""Walk one if / else-if / else chain. Returns the next index, or True
|
||||||
|
if the chain definitely returns on every reachable branch."""
|
||||||
|
bopen = i + 1
|
||||||
|
while bopen < e and toks[bopen].val != "{":
|
||||||
|
bopen += 1
|
||||||
|
if bopen >= e:
|
||||||
|
return e
|
||||||
|
bclose = match_close(toks, bopen, "{", "}")
|
||||||
|
fold = self._fold_cond(toks, i + 1, bopen, ctx, depth, seen)
|
||||||
|
|
||||||
|
j = bclose + 1
|
||||||
|
else_s = else_e = None
|
||||||
|
if j < e and toks[j].kind == TOK_IDENT and toks[j].val == "else":
|
||||||
|
if j + 1 < e and toks[j + 1].val == "{":
|
||||||
|
ec = match_close(toks, j + 1, "{", "}")
|
||||||
|
else_s, else_e = j + 2, ec
|
||||||
|
j = ec + 1
|
||||||
|
else: # `else if ...` — the rest of the chain
|
||||||
|
else_s = j + 1
|
||||||
|
else_e = stmt_end(toks, j + 1, e)
|
||||||
|
j = else_e
|
||||||
|
|
||||||
|
then_ret = else_ret = False
|
||||||
|
if fold is not False:
|
||||||
|
then_ret = self._values(toks, bopen + 1, bclose, ctx, depth + 1, seen, out)
|
||||||
|
if fold is not True and else_s is not None:
|
||||||
|
else_ret = self._values(toks, else_s, else_e, ctx, depth + 1, seen, out)
|
||||||
|
|
||||||
|
if fold is True and then_ret:
|
||||||
|
return True
|
||||||
|
if fold is False and else_s is not None and else_ret:
|
||||||
|
return True
|
||||||
|
if fold is None and else_s is not None and then_ret and else_ret:
|
||||||
|
return True
|
||||||
|
return j
|
||||||
|
|
||||||
|
def resolve(self, rng, func, depth=0, seen=None):
|
||||||
|
"""-> (set of patterns, unresolved_flag)"""
|
||||||
|
if seen is None:
|
||||||
|
seen = set()
|
||||||
|
if depth > MAX_DEPTH:
|
||||||
|
return set(), True
|
||||||
|
return self._expr(func.toks, rng[0], rng[1], func, depth, seen)
|
||||||
|
|
||||||
|
# -- expression walker --------------------------------------------------
|
||||||
|
def _expr(self, toks, s, e, func, depth, seen):
|
||||||
|
parts, cur, d = [], s, 0
|
||||||
|
i = s
|
||||||
|
while i < e: # split on top-level '+'
|
||||||
|
v = toks[i].val
|
||||||
|
if toks[i].kind == TOK_PUNCT and v in "([{":
|
||||||
|
d += 1
|
||||||
|
elif toks[i].kind == TOK_PUNCT and v in ")]}":
|
||||||
|
d -= 1
|
||||||
|
elif d == 0 and toks[i].kind == TOK_PUNCT and v == "+" and i > s:
|
||||||
|
parts.append((cur, i))
|
||||||
|
cur = i + 1
|
||||||
|
i += 1
|
||||||
|
parts.append((cur, e))
|
||||||
|
if len(parts) == 1:
|
||||||
|
return self._primary(toks, s, e, func, depth, seen)
|
||||||
|
|
||||||
|
# concatenation: keep folding while every operand so far is EXACT
|
||||||
|
head, unres = "", False
|
||||||
|
static = True
|
||||||
|
for (ps, pe) in parts:
|
||||||
|
pats, u = self._primary(toks, ps, pe, func, depth, seen)
|
||||||
|
exacts = {p[1] for p in pats if p[0] == EXACT}
|
||||||
|
if static and len(exacts) == 1 and not u and len(pats) == 1:
|
||||||
|
head += exacts.pop()
|
||||||
|
continue
|
||||||
|
if static and pats and all(p[0] == EXACT for p in pats) and len(pats) > 1:
|
||||||
|
# a branchy static operand: keep the shared head only
|
||||||
|
static = False
|
||||||
|
head += os.path.commonprefix(sorted({p[1] for p in pats}))
|
||||||
|
break
|
||||||
|
static = False
|
||||||
|
# first non-static operand: everything after it is runtime text
|
||||||
|
if (ps, pe) == parts[0]:
|
||||||
|
for p in pats:
|
||||||
|
if p[0] == PREFIX:
|
||||||
|
head = p[1]
|
||||||
|
break
|
||||||
|
if not head:
|
||||||
|
unres = True
|
||||||
|
break
|
||||||
|
if static:
|
||||||
|
return {pat_exact(head)}, False
|
||||||
|
p = pat_prefix(head)
|
||||||
|
return ({p} if p else set()), (unres or not p)
|
||||||
|
|
||||||
|
def _primary(self, toks, s, e, func, depth, seen):
|
||||||
|
while s < e and toks[s].kind == TOK_PUNCT and toks[s].val == "(" \
|
||||||
|
and match_close(toks, s, "(", ")") == e - 1:
|
||||||
|
s, e = s + 1, e - 1
|
||||||
|
if s >= e:
|
||||||
|
return set(), True
|
||||||
|
t = toks[s]
|
||||||
|
|
||||||
|
if t.kind == TOK_STR and e == s + 1:
|
||||||
|
return {pat_exact(t.val)}, False
|
||||||
|
|
||||||
|
if t.kind == TOK_IDENT and t.val == "if":
|
||||||
|
return self._if_expr(toks, s, e, func, depth, seen)
|
||||||
|
|
||||||
|
if t.kind == TOK_IDENT and s + 1 < e and toks[s + 1].val == "(":
|
||||||
|
close = match_close(toks, s + 1, "(", ")")
|
||||||
|
if close == e - 1:
|
||||||
|
return self._call(toks, t.val, split_args(toks, s + 2, close),
|
||||||
|
func, depth, seen)
|
||||||
|
|
||||||
|
if t.kind == TOK_IDENT and e == s + 1:
|
||||||
|
return self._var(t.val, func, depth, seen)
|
||||||
|
|
||||||
|
return set(), True
|
||||||
|
|
||||||
|
def _if_expr(self, toks, s, e, func, depth, seen):
|
||||||
|
bopen = s + 1
|
||||||
|
while bopen < e and toks[bopen].val != "{":
|
||||||
|
bopen += 1
|
||||||
|
cond = (s + 1, bopen)
|
||||||
|
bclose = match_close(toks, bopen, "{", "}")
|
||||||
|
then_rng = block_tail(toks, bopen + 1, bclose) or (bopen + 1, bclose)
|
||||||
|
|
||||||
|
else_rng = None
|
||||||
|
j = bclose + 1
|
||||||
|
if j < e and toks[j].kind == TOK_IDENT and toks[j].val == "else":
|
||||||
|
if j + 1 < e and toks[j + 1].val == "{":
|
||||||
|
ec = match_close(toks, j + 1, "{", "}")
|
||||||
|
else_rng = block_tail(toks, j + 2, ec) or (j + 2, ec)
|
||||||
|
else:
|
||||||
|
else_rng = (j + 1, e) # `else if ...`
|
||||||
|
|
||||||
|
taken = self._fold_cond(toks, cond[0], cond[1], func, depth, seen)
|
||||||
|
rngs = []
|
||||||
|
if taken is not False:
|
||||||
|
rngs.append(then_rng)
|
||||||
|
if taken is not True and else_rng:
|
||||||
|
rngs.append(else_rng)
|
||||||
|
|
||||||
|
pats, unres = set(), False
|
||||||
|
for r in rngs:
|
||||||
|
p, u = self._expr(toks, r[0], r[1], func, depth + 1, seen)
|
||||||
|
pats |= p
|
||||||
|
unres = unres or u
|
||||||
|
return pats, unres
|
||||||
|
|
||||||
|
def _fold_cond(self, toks, s, e, func, depth, seen):
|
||||||
|
"""Constant-fold `str_eq(X, "")` / `!str_eq(X, "")` so a helper called with
|
||||||
|
a literal (conv_hist_key("")) yields only the branch it really takes.
|
||||||
|
Returns True / False / None(unknown)."""
|
||||||
|
neg = False
|
||||||
|
if s < e and toks[s].kind == TOK_PUNCT and toks[s].val == "!":
|
||||||
|
neg, s = True, s + 1
|
||||||
|
if not (s < e and toks[s].kind == TOK_IDENT and toks[s].val == "str_eq"
|
||||||
|
and s + 1 < e and toks[s + 1].val == "("):
|
||||||
|
return None
|
||||||
|
close = match_close(toks, s + 1, "(", ")")
|
||||||
|
if close != e - 1:
|
||||||
|
return None
|
||||||
|
args = split_args(toks, s + 2, close)
|
||||||
|
if len(args) != 2:
|
||||||
|
return None
|
||||||
|
va, ua = self._expr(toks, args[0][0], args[0][1], func, depth + 1, seen)
|
||||||
|
vb, ub = self._expr(toks, args[1][0], args[1][1], func, depth + 1, seen)
|
||||||
|
if ua or ub or len(va) != 1 or len(vb) != 1:
|
||||||
|
return None
|
||||||
|
(ka, sa), (kb, sb) = va.pop(), vb.pop()
|
||||||
|
if ka != EXACT or kb != EXACT:
|
||||||
|
return None
|
||||||
|
r = (sa == sb)
|
||||||
|
return (not r) if neg else r
|
||||||
|
|
||||||
|
def _call(self, toks, name, args, func, depth, seen):
|
||||||
|
cands = self.funcs.get(name)
|
||||||
|
if not cands:
|
||||||
|
return set(), True # builtin: json_get, env, ...
|
||||||
|
pats, unres = set(), False
|
||||||
|
for callee in cands:
|
||||||
|
key = ("fn", callee.path, callee.name, tuple(args))
|
||||||
|
if key in seen:
|
||||||
|
unres = True
|
||||||
|
continue
|
||||||
|
seen = seen | {key}
|
||||||
|
# bind the callee's params to THIS call site's argument expressions
|
||||||
|
binding = {}
|
||||||
|
for idx, pname in enumerate(callee.params):
|
||||||
|
if idx < len(args):
|
||||||
|
binding[pname] = (args[idx], func)
|
||||||
|
callee_ctx = _Bound(callee, binding)
|
||||||
|
for r in self.returns_of(callee_ctx, depth + 1, seen):
|
||||||
|
p, u = self._expr(callee.toks, r[0], r[1], callee_ctx,
|
||||||
|
depth + 1, seen)
|
||||||
|
pats |= p
|
||||||
|
unres = unres or u
|
||||||
|
return pats, unres
|
||||||
|
|
||||||
|
def _var(self, name, func, depth, seen):
|
||||||
|
real = func.func if isinstance(func, _Bound) else func
|
||||||
|
|
||||||
|
# 1. a parameter bound by the call site we came through
|
||||||
|
if isinstance(func, _Bound) and name in func.binding:
|
||||||
|
rng, caller_ctx = func.binding[name]
|
||||||
|
return self._expr(caller_ctx.toks, rng[0], rng[1], caller_ctx,
|
||||||
|
depth + 1, seen)
|
||||||
|
|
||||||
|
# 2. a local `let` in the enclosing function
|
||||||
|
lets = self.lets_of(real)
|
||||||
|
if name in lets:
|
||||||
|
key = ("let", real.path, real.name, name)
|
||||||
|
if key in seen:
|
||||||
|
return set(), True
|
||||||
|
seen = seen | {key}
|
||||||
|
pats, unres = set(), False
|
||||||
|
for rng in lets[name]:
|
||||||
|
p, u = self._expr(real.toks, rng[0], rng[1], real, depth + 1, seen)
|
||||||
|
pats |= p
|
||||||
|
unres = unres or u
|
||||||
|
return pats, unres
|
||||||
|
|
||||||
|
# 3. an unbound parameter -> look at every call site of the enclosing fn
|
||||||
|
if name in real.params:
|
||||||
|
key = ("param", real.path, real.name, name)
|
||||||
|
if key in seen:
|
||||||
|
return set(), True
|
||||||
|
seen = seen | {key}
|
||||||
|
idx = real.params.index(name)
|
||||||
|
pats, unres = set(), False
|
||||||
|
sites = self.calls.get(real.name, [])
|
||||||
|
if not sites:
|
||||||
|
return set(), True
|
||||||
|
for caller, args, _rel, _line in sites:
|
||||||
|
if caller is None or idx >= len(args):
|
||||||
|
unres = True
|
||||||
|
continue
|
||||||
|
p, u = self._expr(caller.toks, args[idx][0], args[idx][1],
|
||||||
|
caller, depth + 1, seen)
|
||||||
|
pats |= p
|
||||||
|
unres = unres or u
|
||||||
|
return pats, unres
|
||||||
|
|
||||||
|
# 4. a file-level / cross-file top-level `let`
|
||||||
|
for tl in self.toplevel:
|
||||||
|
lets = self.lets_of(tl)
|
||||||
|
if name in lets:
|
||||||
|
key = ("let", tl.path, tl.name, name)
|
||||||
|
if key in seen:
|
||||||
|
return set(), True
|
||||||
|
seen2 = seen | {key}
|
||||||
|
pats, unres = set(), False
|
||||||
|
for rng in lets[name]:
|
||||||
|
p, u = self._expr(tl.toks, rng[0], rng[1], tl, depth + 1, seen2)
|
||||||
|
pats |= p
|
||||||
|
unres = unres or u
|
||||||
|
return pats, unres
|
||||||
|
|
||||||
|
return set(), True
|
||||||
|
|
||||||
|
|
||||||
|
class _Bound:
|
||||||
|
"""A callee view that also knows what its params were called with."""
|
||||||
|
|
||||||
|
def __init__(self, func, binding):
|
||||||
|
self.func, self.binding = func, binding
|
||||||
|
self.toks, self.start, self.end = func.toks, func.start, func.end
|
||||||
|
self.params, self.path, self.name = func.params, func.path, func.name
|
||||||
|
|
||||||
|
def __getattr__(self, k):
|
||||||
|
return getattr(self.func, k)
|
||||||
|
|
||||||
|
|
||||||
|
# ── token helpers ───────────────────────────────────────────────────────────
|
||||||
|
def split_args(toks, s, e):
|
||||||
|
out, cur, d = [], s, 0
|
||||||
|
i = s
|
||||||
|
while i < e:
|
||||||
|
v = toks[i].val
|
||||||
|
if toks[i].kind == TOK_PUNCT and v in "([{":
|
||||||
|
d += 1
|
||||||
|
elif toks[i].kind == TOK_PUNCT and v in ")]}":
|
||||||
|
d -= 1
|
||||||
|
elif d == 0 and toks[i].kind == TOK_PUNCT and v == ",":
|
||||||
|
out.append((cur, i))
|
||||||
|
cur = i + 1
|
||||||
|
i += 1
|
||||||
|
if cur < e:
|
||||||
|
out.append((cur, e))
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
STMT_START = {"let", "return", "if", "while", "for"}
|
||||||
|
KEYWORDS = {"if", "while", "for", "return", "fn", "let", "else", "match"}
|
||||||
|
|
||||||
|
|
||||||
|
def stmt_end(toks, s, limit):
|
||||||
|
"""End of the expression starting at s: the next top-level statement
|
||||||
|
boundary. El has no semicolons, so a newline that starts a new statement
|
||||||
|
ends this one."""
|
||||||
|
d, i = 0, s
|
||||||
|
while i < limit:
|
||||||
|
t = toks[i]
|
||||||
|
if t.kind == TOK_PUNCT and t.val in "([":
|
||||||
|
d += 1
|
||||||
|
elif t.kind == TOK_PUNCT and t.val in ")]":
|
||||||
|
d -= 1
|
||||||
|
if d < 0:
|
||||||
|
return i
|
||||||
|
elif t.kind == TOK_PUNCT and t.val == "{":
|
||||||
|
# a brace at depth 0 belongs to this expression only when it is an
|
||||||
|
# if/else block that is part of it
|
||||||
|
d += 1
|
||||||
|
elif t.kind == TOK_PUNCT and t.val == "}":
|
||||||
|
d -= 1
|
||||||
|
if d < 0:
|
||||||
|
return i
|
||||||
|
elif d == 0 and t.kind == TOK_PUNCT and t.val == ",":
|
||||||
|
return i
|
||||||
|
elif d == 0 and i > s and t.kind == TOK_IDENT and t.val in STMT_START:
|
||||||
|
if t.val == "if" and toks[i - 1].kind == TOK_IDENT and toks[i - 1].val == "else":
|
||||||
|
i += 1
|
||||||
|
continue
|
||||||
|
return i
|
||||||
|
elif d == 0 and i > s and t.kind == TOK_IDENT and t.val == "fn":
|
||||||
|
return i
|
||||||
|
i += 1
|
||||||
|
return limit
|
||||||
|
|
||||||
|
|
||||||
|
def block_tail(toks, s, e):
|
||||||
|
"""The trailing expression of a block, if the block ends in one."""
|
||||||
|
i, last = s, None
|
||||||
|
while i < e:
|
||||||
|
t = toks[i]
|
||||||
|
if t.kind == TOK_IDENT and t.val in ("let", "return"):
|
||||||
|
i = stmt_end(toks, i + 1, e)
|
||||||
|
last = None
|
||||||
|
continue
|
||||||
|
if t.kind == TOK_PUNCT and t.val in "([{":
|
||||||
|
i = match_close(toks, i, t.val, {"(": ")", "[": "]", "{": "}"}[t.val]) + 1
|
||||||
|
continue
|
||||||
|
st = i
|
||||||
|
en = stmt_end(toks, i, e)
|
||||||
|
if en <= st:
|
||||||
|
i = st + 1
|
||||||
|
continue
|
||||||
|
last = (st, en)
|
||||||
|
i = en
|
||||||
|
return last
|
||||||
|
|
||||||
|
|
||||||
|
def render(toks, s, e):
|
||||||
|
out = []
|
||||||
|
for t in toks[s:e]:
|
||||||
|
out.append('"%s"' % t.val if t.kind == TOK_STR else t.val)
|
||||||
|
return " ".join(out)
|
||||||
|
|
||||||
|
|
||||||
|
# ── the gate ────────────────────────────────────────────────────────────────
|
||||||
|
def collect(root, include_tests):
|
||||||
|
files = []
|
||||||
|
for dirpath, dirnames, filenames in os.walk(root):
|
||||||
|
dirnames[:] = [d for d in dirnames
|
||||||
|
if d not in ("dist", "vendor", ".git", "node_modules")]
|
||||||
|
rel_dir = os.path.relpath(dirpath, root)
|
||||||
|
if not include_tests and rel_dir.split(os.sep)[0] == "tests":
|
||||||
|
continue
|
||||||
|
for fn in sorted(filenames):
|
||||||
|
if fn.endswith(".el"):
|
||||||
|
rel = os.path.normpath(os.path.join(rel_dir, fn))
|
||||||
|
files.append((os.path.join(dirpath, fn), rel))
|
||||||
|
return sorted(files, key=lambda x: x[1])
|
||||||
|
|
||||||
|
|
||||||
|
def is_bare_literal(prog, site):
|
||||||
|
toks = prog.files[site.path]
|
||||||
|
s, e = site.arg_range
|
||||||
|
return e == s + 1 and toks[s].kind == TOK_STR
|
||||||
|
|
||||||
|
|
||||||
|
def read_decl(path):
|
||||||
|
"""A declaration file: one entry per line, `# ...` comments stripped."""
|
||||||
|
out = []
|
||||||
|
if not path or not os.path.exists(path):
|
||||||
|
return out
|
||||||
|
with open(path) as fh:
|
||||||
|
for ln in fh:
|
||||||
|
ln = ln.split("#", 1)[0].strip()
|
||||||
|
if ln:
|
||||||
|
out.append(ln)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def opt(argv, name, default=None):
|
||||||
|
for i, a in enumerate(argv):
|
||||||
|
if a == name and i + 1 < len(argv):
|
||||||
|
return argv[i + 1]
|
||||||
|
return default
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv):
|
||||||
|
root = os.path.abspath(argv[1]) if len(argv) > 1 and not argv[1].startswith("-") else "."
|
||||||
|
include_tests = "--include-tests" in argv
|
||||||
|
verbose = "--verbose" in argv
|
||||||
|
baseline_path = opt(argv, "--baseline")
|
||||||
|
external_path = opt(argv, "--external")
|
||||||
|
|
||||||
|
prog = Program()
|
||||||
|
for path, rel in collect(root, include_tests):
|
||||||
|
prog.load(path, rel)
|
||||||
|
prog.index()
|
||||||
|
for site in prog.sites:
|
||||||
|
pats, unres = prog.resolve(site.arg_range, site.func)
|
||||||
|
site.pats, site.unresolved = {p for p in pats if p}, unres
|
||||||
|
if is_bare_literal(prog, site):
|
||||||
|
site.literal = prog.files[site.path][site.arg_range[0]].val
|
||||||
|
|
||||||
|
writes = [s for s in prog.sites if s.kind == "set"]
|
||||||
|
reads = [s for s in prog.sites if s.kind == "get"]
|
||||||
|
write_pats = set()
|
||||||
|
for w in writes:
|
||||||
|
write_pats |= w.pats
|
||||||
|
|
||||||
|
# Declared host-set keys: written by something outside the El tree (an
|
||||||
|
# operator, the installer, a host process). Each entry must carry a reason.
|
||||||
|
external = []
|
||||||
|
for ln in read_decl(external_path):
|
||||||
|
parts = ln.split(None, 1)
|
||||||
|
if len(parts) != 2 or parts[0] not in (EXACT, PREFIX):
|
||||||
|
print("bad --external line (want `exact|prefix <key>`): %r" % ln,
|
||||||
|
file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
external.append((parts[0], parts[1]))
|
||||||
|
write_pats |= set(external)
|
||||||
|
|
||||||
|
# F1 — a read of a key no write in the tree produces.
|
||||||
|
f1 = []
|
||||||
|
for r in reads:
|
||||||
|
for p in sorted(r.pats):
|
||||||
|
if not any(covers(w, p) for w in write_pats):
|
||||||
|
f1.append((r, p))
|
||||||
|
|
||||||
|
# F2 — a key namespace owned by a helper, accessed by a hand-rolled literal.
|
||||||
|
# This is the #129 shape: the producer moved behind conv_hist_key() and
|
||||||
|
# one consumer kept spelling the old key out by hand.
|
||||||
|
owners = {} # helper fn name -> its value set
|
||||||
|
for s in prog.sites:
|
||||||
|
toks = prog.files[s.path]
|
||||||
|
a, b = s.arg_range
|
||||||
|
if toks[a].kind == TOK_IDENT and a + 1 < b and toks[a + 1].val == "(" \
|
||||||
|
and match_close(toks, a + 1, "(", ")") == b - 1 \
|
||||||
|
and toks[a].val in prog.funcs:
|
||||||
|
name = toks[a].val
|
||||||
|
if name not in owners:
|
||||||
|
vals = set()
|
||||||
|
for callee in prog.funcs[name]:
|
||||||
|
# No call context here on purpose: the OWNED namespace is
|
||||||
|
# every key the helper can ever produce, over all call sites.
|
||||||
|
for rng in prog.returns_of(callee):
|
||||||
|
p, _ = prog._expr(callee.toks, rng[0], rng[1], callee, 0, set())
|
||||||
|
vals |= {x for x in p if x}
|
||||||
|
owners[name] = vals
|
||||||
|
f2 = []
|
||||||
|
for s in prog.sites:
|
||||||
|
if s.literal is None:
|
||||||
|
continue
|
||||||
|
for owner, vals in sorted(owners.items()):
|
||||||
|
for v in sorted(vals):
|
||||||
|
if covers(v, pat_exact(s.literal)):
|
||||||
|
f2.append((s, owner, v))
|
||||||
|
break
|
||||||
|
else:
|
||||||
|
continue
|
||||||
|
break
|
||||||
|
|
||||||
|
unresolved = [s for s in prog.sites if s.unresolved or not s.pats]
|
||||||
|
|
||||||
|
# Baseline signatures carry NO line number on purpose: an unrelated edit that
|
||||||
|
# shifts a line must not un-mute an accepted finding (that is crying wolf),
|
||||||
|
# but a GROWTH in count must not hide either. So a baseline entry is
|
||||||
|
# `<file> <CODE> <detail> [xN]` and only the first N matches are muted.
|
||||||
|
baseline, bad_baseline = {}, []
|
||||||
|
for ln in read_decl(baseline_path):
|
||||||
|
n, key = 1, ln
|
||||||
|
parts = ln.rsplit(" x", 1)
|
||||||
|
if len(parts) == 2 and parts[1].isdigit():
|
||||||
|
key, n = parts[0].strip(), int(parts[1])
|
||||||
|
baseline[key] = n
|
||||||
|
|
||||||
|
def sig(path, code, detail):
|
||||||
|
return "%s %s %s" % (path, code, detail)
|
||||||
|
|
||||||
|
findings = []
|
||||||
|
for r, p in f1:
|
||||||
|
findings.append((sig(r.path, "DEAD-READ", "%s:%s" % p), r.line,
|
||||||
|
" %s:%d state_get(%s)\n resolves to %s %r — no state_set in the tree produces it"
|
||||||
|
% (r.path, r.line, r.text, p[0].upper(), p[1])))
|
||||||
|
for s, owner, v in f2:
|
||||||
|
findings.append((sig(s.path, "HAND-ROLLED", "%s<-%s()" % (s.literal, owner)), s.line,
|
||||||
|
" %s:%d state_%s(\"%s\")\n %s() owns this key namespace (%s %r) — go through the helper, "
|
||||||
|
"or a rename orphans this site silently" % (s.path, s.line, s.kind, s.literal, owner, v[0].upper(), v[1])))
|
||||||
|
findings.sort(key=lambda f: (f[0], f[1]))
|
||||||
|
|
||||||
|
live, muted, budget = [], [], dict(baseline)
|
||||||
|
for f in findings:
|
||||||
|
if budget.get(f[0], 0) > 0:
|
||||||
|
budget[f[0]] -= 1
|
||||||
|
muted.append(f)
|
||||||
|
else:
|
||||||
|
live.append(f)
|
||||||
|
stale = sorted(k for k, v in budget.items() if v > 0)
|
||||||
|
|
||||||
|
print("── state-key audit ─────────────────────────────────────────────")
|
||||||
|
print("scanned %d .el files%s" % (len(prog.files),
|
||||||
|
"" if include_tests else " (tests/ excluded)"))
|
||||||
|
print("sites %d state_set, %d state_get" % (len(writes), len(reads)))
|
||||||
|
print("keys %d distinct write patterns" % len(write_pats))
|
||||||
|
print("")
|
||||||
|
|
||||||
|
if verbose:
|
||||||
|
print("WRITE PATTERNS")
|
||||||
|
for k, v in sorted(write_pats):
|
||||||
|
print(" %-6s %s" % (k, v))
|
||||||
|
print("")
|
||||||
|
|
||||||
|
if external:
|
||||||
|
print("DECLARED HOST-SET (%d) — %s" % (len(external), external_path))
|
||||||
|
for k, v in sorted(external):
|
||||||
|
print(" %-6s %s" % (k, v))
|
||||||
|
print("")
|
||||||
|
|
||||||
|
print("UNRESOLVED (%d) — reported, never fails the build" % len(unresolved))
|
||||||
|
if not unresolved:
|
||||||
|
print(" (none)")
|
||||||
|
for s in sorted(unresolved, key=lambda x: (x.path, x.line)):
|
||||||
|
print(" %s:%d state_%s(%s)%s"
|
||||||
|
% (s.path, s.line, s.kind, s.text,
|
||||||
|
" [partial: %s]" % ", ".join("%s %r" % p for p in sorted(s.pats))
|
||||||
|
if s.pats else ""))
|
||||||
|
print("")
|
||||||
|
|
||||||
|
if muted:
|
||||||
|
print("BASELINED (%d) — pre-existing debt accepted in %s. NOT clean; fix these."
|
||||||
|
% (len(muted), baseline_path))
|
||||||
|
for sg, line, _ in muted:
|
||||||
|
print(" %s (line %d)" % (sg, line))
|
||||||
|
print("")
|
||||||
|
if stale:
|
||||||
|
print("STALE BASELINE (%d) — entries that no longer match anything; delete them:"
|
||||||
|
% len(stale))
|
||||||
|
for sg in stale:
|
||||||
|
print(" %s" % sg)
|
||||||
|
print("")
|
||||||
|
|
||||||
|
print("FINDINGS (%d)" % len(live))
|
||||||
|
if not live:
|
||||||
|
print(" (none)")
|
||||||
|
for _, _, body in live:
|
||||||
|
print(body)
|
||||||
|
print("")
|
||||||
|
|
||||||
|
if live:
|
||||||
|
print("FAIL: %d state-key finding(s). See scripts/verify-state-keys.sh "
|
||||||
|
"for why this gate exists (issue #129)." % len(live))
|
||||||
|
return 1
|
||||||
|
print("PASS: every resolvable state_get key has a producer, and no key "
|
||||||
|
"namespace is spelled two ways.")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main(sys.argv))
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
# state-key-baseline.txt — findings that already existed when this gate landed
|
||||||
|
# (2026-08-07). Each one is a REAL defect of the #129 class, not a false
|
||||||
|
# positive. They are muted only so the gate can be turned on today instead of
|
||||||
|
# being deferred until the debt is paid; every run still prints them under
|
||||||
|
# BASELINED with the word "debt".
|
||||||
|
#
|
||||||
|
# THIS FILE SHOULD ONLY EVER SHRINK. Adding a line means you are shipping a
|
||||||
|
# known silent-"" read. If you must, date it and say why in the comment.
|
||||||
|
#
|
||||||
|
# format: <file> <CODE> <detail> [xN] # N = how many sites are accepted
|
||||||
|
# No line numbers on purpose: an unrelated edit must not un-mute an accepted
|
||||||
|
# finding, but a GROWTH in count is NOT muted — the extra site fails the build.
|
||||||
|
#
|
||||||
|
chat.el DEAD-READ exact:soul_identity x5
|
||||||
|
# ^ soul.el used to run `state_set("soul_identity", soul_identity)`. It was
|
||||||
|
# deleted on 2026-05-13 in b163fa6 ("feat(awareness): route ISE writes to HTTP
|
||||||
|
# Engram ..."), a commit about something else entirely, and the five readers in
|
||||||
|
# chat.el were left behind. Since that date build_system_prompt (737), the
|
||||||
|
# vision handler (1745), the agentic system prompt (2620), the council
|
||||||
|
# transcript handler (3425) and 3480 have all been prefixing "" — exactly the
|
||||||
|
# #129 shape, found by this gate on its first run. Sites: 737, 1745, 2620,
|
||||||
|
# 3425, 3480. Fix = restore the boot-time write or delete the reads; not done
|
||||||
|
# here because this branch must not change engine behaviour.
|
||||||
|
|
||||||
|
studio.el DEAD-READ exact:soul_principal x1
|
||||||
|
# ^ studio.el:57 dharma_registry() emits "principal":"" on every call — no
|
||||||
|
# producer has ever existed in the tree's history (git log -S finds none).
|
||||||
|
# Never-wired rather than orphaned, same silent-"" result.
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
# state-key-external.txt — state keys the engine READS but deliberately never
|
||||||
|
# WRITES, because a host outside the El tree sets them (an operator, the
|
||||||
|
# installer, a deployment env). Read scripts/verify-state-keys.sh for why this
|
||||||
|
# list has to exist and why it has to stay short.
|
||||||
|
#
|
||||||
|
# THE RULE FOR ADDING A LINE: the read site must already treat "" as a defined
|
||||||
|
# default (`if str_eq(x, "") { <default> }`) AND the source must say so in a
|
||||||
|
# comment. "I could not find the writer" is NOT a reason — that is the #129
|
||||||
|
# defect, and it belongs in state-key-baseline.txt with a date, not here.
|
||||||
|
#
|
||||||
|
# format: exact|prefix <key> # why, and where the source says so
|
||||||
|
#
|
||||||
|
exact soul_rate_limit # routes.el:59-61 — "configurable via soul state key ... Falls back to 60 req/min if not set."
|
||||||
|
exact web_search_tool_version # chat.el:1884-1910 — version lives in state "so a future bump is a config write, not a recompile"; defaults to web_search_20250305
|
||||||
|
exact platform_auth # stewardship.el:92 — host-set capability flag; fail-CLOSED (anything but "true" denies the platform tool)
|
||||||
|
exact security_research_authorized # awareness.el:991-996 — state override for env SECURITY_RESEARCH_TOKEN; fail-closed, defaults false
|
||||||
Executable
+118
@@ -0,0 +1,118 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# verify-state-keys.sh — the state-key gate. Retires a defect class at build time.
|
||||||
|
#
|
||||||
|
# ── WHY THIS EXISTS. DO NOT DELETE IT AS NOISE. ──────────────────────────────
|
||||||
|
#
|
||||||
|
# The engine keeps runtime values in a key-value store: state_set("k", v) writes,
|
||||||
|
# state_get("k") reads. A read of a key that NOTHING writes returns an empty
|
||||||
|
# string. Silently. No error, no warning, no log line. The El compiler cannot see
|
||||||
|
# it, no test sees it, and the product keeps running — just with a hole in it.
|
||||||
|
#
|
||||||
|
# That is how issue #129 happened. ff421d3 (2026-08-05) correctly moved
|
||||||
|
# conversation history to a per-session key behind conv_hist_key(session_id). One
|
||||||
|
# consumer did not move with it: the agentic path's L1 safety screen kept reading
|
||||||
|
# the old anonymous "conv_history" bucket. The desktop app always mints a session
|
||||||
|
# id, so history was always written under session_hist_<id> and that read always
|
||||||
|
# returned "". The half of the crisis score that receives history is the
|
||||||
|
# ESCALATION half — the one that exists for distress building across several
|
||||||
|
# turns, where no single message trips the bell on its own. It scored 0 on every
|
||||||
|
# real conversation for two days, and nothing failed.
|
||||||
|
#
|
||||||
|
# The line that broke carried a comment describing this exact bug being fixed
|
||||||
|
# once already, under issue #9. A comment is not a gate. This is the gate.
|
||||||
|
#
|
||||||
|
# ── WHAT IT CHECKS ──────────────────────────────────────────────────────────
|
||||||
|
#
|
||||||
|
# DEAD-READ a state_get whose key resolves to something no state_set in the
|
||||||
|
# tree produces. The direct form of the class.
|
||||||
|
#
|
||||||
|
# HAND-ROLLED a state_get/state_set that spells out a literal belonging to a
|
||||||
|
# key namespace a helper function owns (e.g. "conv_history", owned
|
||||||
|
# by conv_hist_key()). This is #129's actual shape: the producer
|
||||||
|
# moved behind the helper and one consumer kept the old spelling
|
||||||
|
# by hand. DEAD-READ alone does NOT catch #129, because the dead
|
||||||
|
# handle_chat() still writes that key through the helper — so this
|
||||||
|
# second check is the one that earns the gate its keep.
|
||||||
|
#
|
||||||
|
# ── WHY IT DOES NOT CRY WOLF ────────────────────────────────────────────────
|
||||||
|
#
|
||||||
|
# Keys are usually COMPUTED, not literal, so a naive grep would flood and get
|
||||||
|
# switched off within a day. scripts/state-key-audit.py resolves computed keys:
|
||||||
|
# string concatenation (matched on the static prefix), helper functions (resolved
|
||||||
|
# to their possible return values), keys built into a local variable, and keys
|
||||||
|
# arriving as a function parameter (resolved through the call sites). Where a key
|
||||||
|
# genuinely cannot be resolved it is printed under UNRESOLVED and does NOT fail
|
||||||
|
# the build — visible, never silently ignored. Keep that list short.
|
||||||
|
#
|
||||||
|
# On this tree it resolves 278 of 278 sites: UNRESOLVED is 0 and FINDINGS is 0.
|
||||||
|
#
|
||||||
|
# Two declaration files, both of which should only ever shrink:
|
||||||
|
# scripts/state-key-external.txt keys a host outside the El tree writes
|
||||||
|
# scripts/state-key-baseline.txt findings that predate the gate (real debt)
|
||||||
|
#
|
||||||
|
# ── PROVEN TO DISCRIMINATE (2026-08-07) ─────────────────────────────────────
|
||||||
|
#
|
||||||
|
# 1. Synthetic: a scratch copy of this tree with agentic_safety_screen reverted
|
||||||
|
# to the pre-fix state_get("conv_history") — ONE line, nothing else — FAILS
|
||||||
|
# with `chat.el:2536 ... conv_hist_key() owns this key namespace`. The tree
|
||||||
|
# as shipped PASSES. One variable, opposite verdicts.
|
||||||
|
# 2. Independent: run read-only against origin/feat/soul-openai-tools-v2, which
|
||||||
|
# carries the same defect on its own, the gate reported chat.el:2937 — the
|
||||||
|
# exact line 43d0449's commit message had named by hand. Against that
|
||||||
|
# branch's fix (origin/fix/129-on-openai-tools) it passes.
|
||||||
|
# 3. Producer-moved controls: renaming the sole writer of an EXACT key
|
||||||
|
# (soul_model) orphans 3 readers across 3 files; renaming the sole writer of
|
||||||
|
# a PREFIX namespace (agent_workspace_root_*) orphans 3 readers — including
|
||||||
|
# when the producer moves to a NARROWER namespace, which an earlier,
|
||||||
|
# sloppier prefix rule let through.
|
||||||
|
#
|
||||||
|
# It also found, on its first run, a defect nobody was looking for: soul.el's
|
||||||
|
# `state_set("soul_identity", ...)` was deleted on 2026-05-13 in b163fa6 (a
|
||||||
|
# commit about awareness/ISE writes) and five readers in chat.el were left
|
||||||
|
# behind — the system prompt, the vision handler, the agentic prompt and the
|
||||||
|
# council handler have been prefixing "" ever since. See state-key-baseline.txt.
|
||||||
|
#
|
||||||
|
# ── SAFETY ──────────────────────────────────────────────────────────────────
|
||||||
|
# Pure static read of .el sources. Starts nothing, opens no port, touches no
|
||||||
|
# daemon, and never reads or writes ~/.neuron.
|
||||||
|
#
|
||||||
|
# ── USAGE ───────────────────────────────────────────────────────────────────
|
||||||
|
# scripts/verify-state-keys.sh gate the repo (honours baseline)
|
||||||
|
# scripts/verify-state-keys.sh --strict ignore the baseline: show the debt
|
||||||
|
# scripts/verify-state-keys.sh --verbose also dump every write pattern
|
||||||
|
# scripts/verify-state-keys.sh --root DIR audit a different tree
|
||||||
|
# exit 0 = clean; 1 = finding(s); 2 = the gate itself could not run.
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||||
|
STRICT=0
|
||||||
|
PASS_THROUGH=()
|
||||||
|
|
||||||
|
while [ $# -gt 0 ]; do
|
||||||
|
case "$1" in
|
||||||
|
--strict) STRICT=1; shift ;;
|
||||||
|
--root) ROOT="${2:?--root needs a directory}"; shift 2 ;;
|
||||||
|
-h|--help) awk 'NR>1 && /^#/ {print; next} NR>1 {exit}' "${BASH_SOURCE[0]}"; exit 0 ;;
|
||||||
|
*) PASS_THROUGH+=("$1"); shift ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
command -v python3 >/dev/null 2>&1 || {
|
||||||
|
echo "[state-keys] CANNOT RUN: python3 not found" >&2; exit 2; }
|
||||||
|
[ -d "$ROOT" ] || { echo "[state-keys] CANNOT RUN: no such tree: $ROOT" >&2; exit 2; }
|
||||||
|
|
||||||
|
AUDIT="$SCRIPT_DIR/state-key-audit.py"
|
||||||
|
[ -f "$AUDIT" ] || { echo "[state-keys] CANNOT RUN: missing $AUDIT" >&2; exit 2; }
|
||||||
|
|
||||||
|
ARGS=("$ROOT" "--external" "$SCRIPT_DIR/state-key-external.txt")
|
||||||
|
[ "$STRICT" -eq 0 ] && ARGS+=("--baseline" "$SCRIPT_DIR/state-key-baseline.txt")
|
||||||
|
[ ${#PASS_THROUGH[@]} -gt 0 ] && ARGS+=("${PASS_THROUGH[@]}")
|
||||||
|
|
||||||
|
python3 "$AUDIT" "${ARGS[@]}"
|
||||||
|
RC=$?
|
||||||
|
if [ "$RC" -gt 1 ]; then
|
||||||
|
echo "[state-keys] CANNOT RUN: the audit itself failed (exit $RC)" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
exit "$RC"
|
||||||
@@ -1,14 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn session_title_from_message(message: String) -> String
|
|
||||||
extern fn session_make_content(id: String, title: String, created_at: Int, updated_at: Int, folder: String) -> String
|
|
||||||
extern fn session_exists(session_id: String) -> Bool
|
|
||||||
extern fn session_create(body: String) -> String
|
|
||||||
extern fn session_create_cleanup(session_id: String) -> String
|
|
||||||
extern fn session_list() -> String
|
|
||||||
extern fn session_get(session_id: String) -> String
|
|
||||||
extern fn session_delete(session_id: String) -> String
|
|
||||||
extern fn session_update_patch(session_id: String, body: String) -> String
|
|
||||||
extern fn session_search_entry(node: String) -> String
|
|
||||||
extern fn session_search(query: String) -> String
|
|
||||||
extern fn session_hist_load(session_id: String) -> String
|
|
||||||
extern fn session_hist_save(session_id: String, hist: String) -> Void
|
|
||||||
@@ -559,6 +559,27 @@ let axon_base: String = if str_eq(axon_raw, "") { "http://localhost:7771" } else
|
|||||||
let studio_dir_raw: String = env("SOUL_STUDIO_DIR")
|
let studio_dir_raw: String = env("SOUL_STUDIO_DIR")
|
||||||
let studio_dir: String = if str_eq(studio_dir_raw, "") { env("HOME") + "/Development/neuron-technologies/products/cgi-studio/el-daemon" } else { studio_dir_raw }
|
let studio_dir: String = if str_eq(studio_dir_raw, "") { env("HOME") + "/Development/neuron-technologies/products/cgi-studio/el-daemon" } else { studio_dir_raw }
|
||||||
|
|
||||||
|
// RESTORED 2026-08-09 — this producer was added 2026-05-02 in 601e0fe and deleted
|
||||||
|
// by the awareness refactor b163fa6 a few days later. Nothing has written
|
||||||
|
// soul_identity since, while FIVE sites in chat.el kept reading it:
|
||||||
|
// chat.el:737, 1745, 2620, 3425, 3480 — each doing state_get("soul_identity")
|
||||||
|
// and splicing the result into the system prompt beside the voice, security and
|
||||||
|
// capability rules. They have been splicing an EMPTY STRING for roughly three
|
||||||
|
// months. The identity section of every chat turn was blank and nothing said so.
|
||||||
|
//
|
||||||
|
// Found by the #132 state-key gate, which reports a read with no producer as a
|
||||||
|
// build error rather than a silence — the whole reason that gate exists.
|
||||||
|
//
|
||||||
|
// Restored verbatim rather than improved: this key is an env-configurable persona
|
||||||
|
// LINE, which is NOT the same thing as soul_identity_context (the graph-derived
|
||||||
|
// [INTELLECTUAL-DNA]/[VALUES]/[MEMORY-PHILOSOPHY] block written at soul.el:184).
|
||||||
|
// Pointing these five reads at that block instead would have substituted different
|
||||||
|
// content and called it a fix. Whether the chat system prompt should ALSO carry the
|
||||||
|
// graph-derived block is a real question, and a separate one.
|
||||||
|
let identity_raw: String = env("SOUL_IDENTITY")
|
||||||
|
let soul_identity: String = if str_eq(identity_raw, "") { "You are " + soul_cgi_id + ", a CGI." } else { identity_raw }
|
||||||
|
state_set("soul_identity", soul_identity)
|
||||||
|
|
||||||
println("[soul] boot - cgi=" + soul_cgi_id + " port=" + int_to_str(port))
|
println("[soul] boot - cgi=" + soul_cgi_id + " port=" + int_to_str(port))
|
||||||
|
|
||||||
let using_http_engram: Bool = !str_eq(engram_url_raw, "")
|
let using_http_engram: Bool = !str_eq(engram_url_raw, "")
|
||||||
|
|||||||
@@ -1,8 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header - do not edit
|
|
||||||
extern fn init_soul_edges() -> Void
|
|
||||||
extern fn ensure_self_canonical_bridge() -> Void
|
|
||||||
extern fn aff_try_slot(slot_json: String, aff_7d_ts: Int, acc_key: String) -> Void
|
|
||||||
extern fn load_identity_context() -> Void
|
|
||||||
extern fn seed_persona_from_env() -> Void
|
|
||||||
extern fn emit_session_start_event() -> Void
|
|
||||||
extern fn layered_cycle(raw_input: String, session_id: String, utility: Bool) -> String
|
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn steward_log_event(kind: String, detail: String) -> Void
|
|
||||||
extern fn steward_get_mission() -> String
|
|
||||||
extern fn steward_align(input: String, imprint_id: String) -> String
|
|
||||||
extern fn steward_validate_imprint(imprint_id: String, tool_name: String) -> String
|
|
||||||
extern fn steward_cgi_check(action: String) -> String
|
|
||||||
extern fn steward_fingerprint_session(input: String, session_id: String) -> String
|
|
||||||
extern fn extract_dim(content: String, key: String) -> String
|
|
||||||
extern fn steward_build_baseline() -> String
|
|
||||||
extern fn steward_check_continuity(current_fingerprint: String, session_id: String) -> String
|
|
||||||
extern fn steward_session_check(input: String, session_id: String) -> String
|
|
||||||
@@ -53,8 +53,23 @@ fn handle_config(method: String, body: String) -> String {
|
|||||||
}
|
}
|
||||||
|
|
||||||
fn dharma_registry() -> String {
|
fn dharma_registry() -> String {
|
||||||
|
// COMPILED IDENTITY, not state (2026-08-09). soul_principal had no producer at
|
||||||
|
// all — the #132 gate flagged it as a dead read and the registry reported an
|
||||||
|
// empty principal under a heading that says "Principal Covenant v1". The value
|
||||||
|
// was never missing: it is declared in soul.el's cgi block, and as of the
|
||||||
|
// codegen fix it is compiled into the binary and loaded at startup.
|
||||||
|
//
|
||||||
|
// Read it from the compiled constant rather than the state store. The design is
|
||||||
|
// explicit that this identity is "not modifiable by any runtime mechanism
|
||||||
|
// including environment variables, configuration files, or API calls" — so
|
||||||
|
// publishing it into state (the cheap fix) would have recreated exactly the
|
||||||
|
// mutable copy it forbids. cgi_principal() is read-only and has no setter.
|
||||||
|
//
|
||||||
|
// cgi_id keeps its state read deliberately: the RUNTIME instance id is a
|
||||||
|
// different fact from the compiled dharma_id, and conflating them would hide
|
||||||
|
// the case where a binary runs under an id its declaration never claimed.
|
||||||
let cgi_id: String = state_get("soul_cgi_id")
|
let cgi_id: String = state_get("soul_cgi_id")
|
||||||
let principal: String = state_get("soul_principal")
|
let principal: String = cgi_principal()
|
||||||
return "{\"registry\":[{\"cgi\":\"" + cgi_id + "\","
|
return "{\"registry\":[{\"cgi\":\"" + cgi_id + "\","
|
||||||
+ "\"principal\":\"" + principal + "\","
|
+ "\"principal\":\"" + principal + "\","
|
||||||
+ "\"covenant\":\"Principal Covenant v1\","
|
+ "\"covenant\":\"Principal Covenant v1\","
|
||||||
|
|||||||
-12
@@ -1,12 +0,0 @@
|
|||||||
// auto-generated by elc --emit-header — do not edit
|
|
||||||
extern fn auth_headers(tok: String) -> Map
|
|
||||||
extern fn axon_get(path: String) -> String
|
|
||||||
extern fn axon_post(path: String, body: String) -> String
|
|
||||||
extern fn handle_conversations(method: String) -> String
|
|
||||||
extern fn handle_config(method: String, body: String) -> String
|
|
||||||
extern fn dharma_registry() -> String
|
|
||||||
extern fn dharma_network_state() -> String
|
|
||||||
extern fn handle_dharma(path: String, method: String, body: String) -> String
|
|
||||||
extern fn handle_tool(path: String, method: String, body: String) -> String
|
|
||||||
extern fn handle_nlg(path: String, method: String, body: String) -> String
|
|
||||||
extern fn render_studio() -> String
|
|
||||||
@@ -0,0 +1,90 @@
|
|||||||
|
# gate-openai — deterministic OpenAI-dialect provider stub
|
||||||
|
|
||||||
|
Staging home for the **soul-openai-tools-v2** gate scaffolding
|
||||||
|
(`docs/specs/SPEC-soul-openai-tools-v2-2026-08-06.md`, test-plan rung 1:
|
||||||
|
"stub first — discriminates before El code exists"). Sibling of gate9's
|
||||||
|
Anthropic stub (`_wt-beta-round9/scripts/gate9/stub-llm.py`): same scenario
|
||||||
|
mechanism, opposite wire dialect. Stdlib Python only, 127.0.0.1 only,
|
||||||
|
refuses ports 7770/7779/17779. Run `./selftest.sh` — exit 0 is green.
|
||||||
|
|
||||||
|
## Files
|
||||||
|
|
||||||
|
| File | Role |
|
||||||
|
|---|---|
|
||||||
|
| `stub-openai.py` | HTTP server: `POST /v1/chat/completions` (OpenAI dialect), scenario-scripted responses, request validation, ground-truth JSONL log, hostile modes via `--mode` |
|
||||||
|
| `scenarios-openai.json` | Scenario contract: scripts + markers + per-class/per-step request assertions |
|
||||||
|
| `selftest.sh` | curl-driven proof of every scenario, every rejection, all hostile modes (58 checks) |
|
||||||
|
|
||||||
|
## What each scenario proves (when the brain drives it)
|
||||||
|
|
||||||
|
| Class | Proves |
|
||||||
|
|---|---|
|
||||||
|
| `oa-plain` | finish_reason `stop` ends the loop; tools + `tool_choice` + `parallel_tool_calls:false` were offered on the wire |
|
||||||
|
| `oa-tools-off` | the chat-only lane sends NO tools (offering them there is a 400) |
|
||||||
|
| `oa-single-tool` | full round-trip: `tool_calls` parsed, assistant echo + `role:"tool"` turn with matching `tool_call_id` sent back, final text reached |
|
||||||
|
| `oa-torture` | `function.arguments` (JSON-encoded string with nested quotes, backslashes, newlines, tabs, unicode) survives exactly ONE decode — the stub recomputes the issued payload from the script and 400s on any drift (`gate_echo_mismatch`, the spec §6 two-escaper trap) |
|
||||||
|
| `oa-parallel` | two `tool_calls` in one response: the brain either answers both (paired correctly) or rejects cleanly — an unpaired echo is a 400 |
|
||||||
|
| `oa-mission` | multi-round loop continuation; step index = assistant-message count, so resume threads index correctly by construction |
|
||||||
|
| `oa-api-error` | provider errors 400/429/500/503 in the OpenAI error envelope surface honestly, no retry storm |
|
||||||
|
|
||||||
|
Universal (every request, any scenario): Anthropic dialect leakage fails
|
||||||
|
loudly with 400 — `anthropic-version` header, top-level `system` /
|
||||||
|
`stop_sequences` / `max_tokens_to_sample`, `input_schema` inside tools,
|
||||||
|
Anthropic content blocks (`tool_use`/`tool_result`/...). Tools must be
|
||||||
|
`{type:"function", function:{name, description, parameters}}`, unique names;
|
||||||
|
echoed `arguments` must be a JSON-encoded STRING, never a decoded object.
|
||||||
|
|
||||||
|
## Hostile modes (`--mode`, same file)
|
||||||
|
|
||||||
|
| Mode | Behavior | Brain invariant under test |
|
||||||
|
|---|---|---|
|
||||||
|
| `black-hole` | reads the request, never responds | HTTP timeout exists and surfaces; no silent hang |
|
||||||
|
| `mid-body-drop` | 200 headers, half a JSON body, socket abort | truncated body = clean error, never a half-parsed reply shown as real |
|
||||||
|
| `tool-pending-forever` | every request gets a fresh `tool_calls` response, forever | the loop's iteration cap trips (`max_loop_iterations: 16` in the contract); count actual round-trips via `GET /gate/stats` (`chat_hits`) |
|
||||||
|
|
||||||
|
## How the brain-side gate consumes this
|
||||||
|
|
||||||
|
1. Start: `stub-openai.py --port P --scenarios scenarios-openai.json --log run.jsonl`
|
||||||
|
2. Point the brain at it: `NEURON_LLM_0_URL=http://127.0.0.1:P` +
|
||||||
|
`NEURON_LLM_0_FORMAT=openai` (spec step 0 must verify these actually
|
||||||
|
export at runtime), scratch profile, free soul port.
|
||||||
|
3. Send each phrasing's `prompt` (the marker selects the script); assert the
|
||||||
|
brain's claims (`tools_used`, reply, ledger) against the stub's JSONL log
|
||||||
|
— truth, not narration — plus files on disk for write_file scenarios.
|
||||||
|
4. Any stub 400 = the brain sent a malformed/leaked request; the gate fails
|
||||||
|
with the stub's reason string.
|
||||||
|
5. Re-run gate9's Anthropic matrix unchanged = proof the Anthropic lane is
|
||||||
|
byte-untouched.
|
||||||
|
|
||||||
|
## Reconciliation into gate9 (app repo) — AFTER round 9 merges
|
||||||
|
|
||||||
|
This dir is staging only; the merge is mechanical by design:
|
||||||
|
- `stub-openai.py` + `scenarios-openai.json` move to `scripts/gate9/`
|
||||||
|
alongside `stub-llm.py` + `scenarios.json` (shared conventions: marker
|
||||||
|
matching, assistant-count step indexing, `--port/--scenarios/--log`,
|
||||||
|
JSONL fields `seq/ts/kind/scenario_class/phrasing/step/validation/
|
||||||
|
delivered/http_status`, prod-port refusal, benign background responses,
|
||||||
|
`GATE-SCRIPT-EXHAUSTED` overrun, `{N}/{NN}` repeat expansion).
|
||||||
|
- `prompt-matrix-gate.sh` gains a dialect axis (anthropic|openai) choosing
|
||||||
|
stub + scenario file; `matrix-asserts.py` reads the same log shape.
|
||||||
|
- The `--mode` hostile flags here are PROVIDER-side (brain↔LLM boundary);
|
||||||
|
gate9's `hostile/` servers are SOUL-side (app↔brain boundary). They are
|
||||||
|
complementary, not duplicates — both stay.
|
||||||
|
|
||||||
|
## Open questions for the port author (stub asserts a position; confirm or change)
|
||||||
|
|
||||||
|
1. `parallel_tool_calls` must be **explicitly false** on every tool-bearing
|
||||||
|
request (ADR-0005 pin). If the builder omits it instead, relax
|
||||||
|
`defaults.expect_request.parallel_tool_calls` to `null`.
|
||||||
|
2. `tool_choice` must be present (`"auto"` expected). If the brain relies on
|
||||||
|
the provider default, drop `require_tool_choice`.
|
||||||
|
3. Tool-result `content` is asserted only to be a string; if the brain sends
|
||||||
|
structured JSON-in-string (like `{"ok":true,...}`), no change needed.
|
||||||
|
4. Groq compatibility: Groq's OpenAI-compat endpoint rejects some optional
|
||||||
|
fields; whatever field set the brain settles on for live Groq E2E must be
|
||||||
|
mirrored here so the deterministic gate and the live lane assert the SAME
|
||||||
|
request shape.
|
||||||
|
5. The stub treats a `role:"tool"` turn answering an already-answered id as
|
||||||
|
400; if the resume path can legitimately replay tool results, that rule
|
||||||
|
needs a resume-aware carve-out (gate9's Anthropic stub faced the same
|
||||||
|
issue — see its PASS 1 comment).
|
||||||
Executable
+559
@@ -0,0 +1,559 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# run-lane-gate.sh — brain-side driver for the OpenAI-dialect gate.
|
||||||
|
#
|
||||||
|
# Drives the REAL soul binary against stub-openai.py for every class and every
|
||||||
|
# phrasing in scenarios-openai.json, plus the three hostile provider modes, and
|
||||||
|
# asserts the brain's claims against the stub's ground-truth JSONL (truth, not
|
||||||
|
# narration) and against files on disk.
|
||||||
|
#
|
||||||
|
# SAFETY (hard rules, enforced below):
|
||||||
|
# - never binds 7770 / 7779 / 17779 - only 7891-7894
|
||||||
|
# - never reads or writes ~/.neuron - HOME is redirected to a scratch dir
|
||||||
|
# - every process started here is killed on exit (trap) and proven with lsof
|
||||||
|
#
|
||||||
|
# The soul runs under `script -q /dev/null` so its stdout is a pty: El's
|
||||||
|
# println() uses puts(), which is FULLY buffered to a file, and the process is
|
||||||
|
# killed without flushing — the DRIFT lines would be invisible otherwise.
|
||||||
|
#
|
||||||
|
# Usage: ./run-lane-gate.sh [all|bridge|local|toolsoff|hostile]
|
||||||
|
# bridge = consent round-trip config (no workspace root -> write_file is
|
||||||
|
# "escalate" -> the loop suspends and the CLIENT executes the tool)
|
||||||
|
# local = workspace-root config (write_file is "reversible" + builtin ->
|
||||||
|
# the loop executes the tool in-process and runs to completion)
|
||||||
|
# toolsoff = supplementary: non-agentic lane against a base URL WITHOUT the
|
||||||
|
# /v1 suffix (the el-runtime provider chain appends
|
||||||
|
# /v1/chat/completions itself, unlike chat.el which appends only
|
||||||
|
# /chat/completions)
|
||||||
|
# hostile = black-hole / mid-body-drop / tool-pending-forever
|
||||||
|
#
|
||||||
|
# Env overrides: SOUL_BIN, STUB_PORT, SOUL_PORT, SOUL_PORT_B, RUN_ROOT
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
HERE="$(cd "$(dirname "$0")" && pwd)"
|
||||||
|
PHASES="${1:-all}"
|
||||||
|
|
||||||
|
SOUL_BIN="${SOUL_BIN:-/tmp/soul-oai2/soul-openai-tools}"
|
||||||
|
STUB_PORT="${STUB_PORT:-7891}"
|
||||||
|
SOUL_PORT="${SOUL_PORT:-7892}"
|
||||||
|
SOUL_PORT_B="${SOUL_PORT_B:-7893}"
|
||||||
|
RUN_ROOT="${RUN_ROOT:-/tmp/oa-lane-gate}"
|
||||||
|
STAMP="$(date +%Y%m%d-%H%M%S)"
|
||||||
|
RUN="$RUN_ROOT/$STAMP"
|
||||||
|
|
||||||
|
for p in "$STUB_PORT" "$SOUL_PORT" "$SOUL_PORT_B"; do
|
||||||
|
case "$p" in
|
||||||
|
7770|7779|17779) echo "FATAL: refusing production Neuron port $p"; exit 2;;
|
||||||
|
789[1-4]) ;;
|
||||||
|
*) echo "FATAL: port $p outside the allowed 7891-7894 range"; exit 2;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
[ -x "$SOUL_BIN" ] || { echo "FATAL: soul binary not found/executable: $SOUL_BIN"; exit 2; }
|
||||||
|
|
||||||
|
mkdir -p "$RUN/home" "$RUN/ws-bridge" "$RUN/ws-local" "$RUN/ws-off" "$RUN/engram"
|
||||||
|
echo '{"nodes":[],"edges":[]}' > "$RUN/engram/snapshot.json"
|
||||||
|
DRV="$RUN/drv.py"
|
||||||
|
|
||||||
|
STUB_PID=""; SOUL_PID=""
|
||||||
|
cleanup() {
|
||||||
|
[ -n "$SOUL_PID" ] && kill "$SOUL_PID" 2>/dev/null
|
||||||
|
pkill -f "$SOUL_BIN" 2>/dev/null
|
||||||
|
[ -n "$STUB_PID" ] && kill "$STUB_PID" 2>/dev/null
|
||||||
|
sleep 0.4
|
||||||
|
[ -n "$SOUL_PID" ] && kill -9 "$SOUL_PID" 2>/dev/null
|
||||||
|
[ -n "$STUB_PID" ] && kill -9 "$STUB_PID" 2>/dev/null
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
trap cleanup EXIT INT TERM
|
||||||
|
|
||||||
|
start_stub() { # $1 = mode, $2 = log path
|
||||||
|
local mode="$1" log="$2" args=""
|
||||||
|
[ "$mode" = "normal" ] && args="--scenarios $HERE/scenarios-openai.json"
|
||||||
|
# shellcheck disable=SC2086
|
||||||
|
python3 "$HERE/stub-openai.py" --port "$STUB_PORT" --mode "$mode" --log "$log" $args \
|
||||||
|
> "$RUN/stub-$mode.out" 2>&1 &
|
||||||
|
STUB_PID=$!
|
||||||
|
for _ in $(seq 1 50); do
|
||||||
|
curl -sf "http://127.0.0.1:$STUB_PORT/gate/health" >/dev/null 2>&1 && return 0
|
||||||
|
sleep 0.2
|
||||||
|
done
|
||||||
|
echo "FATAL: stub did not come up on $STUB_PORT"; cat "$RUN/stub-$mode.out"; exit 3
|
||||||
|
}
|
||||||
|
stop_stub() { [ -n "$STUB_PID" ] && kill "$STUB_PID" 2>/dev/null; sleep 0.3; STUB_PID=""; }
|
||||||
|
|
||||||
|
start_soul() { # $1 = port, $2 = base url, $3 = soul log, $4 = agent root ("" = none)
|
||||||
|
local port="$1" base="$2" log="$3" root="$4"
|
||||||
|
script -q /dev/null \
|
||||||
|
env -u ANTHROPIC_API_KEY -u SOUL_API_KEY -u ENGRAM_URL -u ENGRAM_API_KEY \
|
||||||
|
-u NEURON_API_URL -u NEURON_TOKEN -u SOUL_LLM_PROVIDER -u SOUL_LLM_BASE_URL \
|
||||||
|
-u NEURON_LLM_1_URL -u NEURON_LLM_1_KEY -u SOUL_IDENTITY \
|
||||||
|
HOME="$RUN/home" PATH="$PATH" \
|
||||||
|
NEURON_PORT="$port" EL_HTTP_BIND_HOST=127.0.0.1 \
|
||||||
|
SOUL_ENGRAM_PATH="$RUN/engram/snapshot.json" \
|
||||||
|
SOUL_CGI_ID=ntn-test SOUL_PERSONA_NAME=Neuron \
|
||||||
|
NEURON_LLM_0_URL="$base" NEURON_LLM_0_FORMAT=openai NEURON_LLM_0_KEY=gate-test-key \
|
||||||
|
${root:+NEURON_AGENT_ROOT="$root"} \
|
||||||
|
"$SOUL_BIN" > "$log" 2>&1 &
|
||||||
|
SOUL_PID=$!
|
||||||
|
for _ in $(seq 1 100); do
|
||||||
|
curl -sf "http://127.0.0.1:$port/health" >/dev/null 2>&1 && return 0
|
||||||
|
sleep 0.2
|
||||||
|
done
|
||||||
|
echo "FATAL: soul did not come up on $port"; tail -20 "$log"; exit 3
|
||||||
|
}
|
||||||
|
stop_soul() {
|
||||||
|
[ -n "$SOUL_PID" ] && kill "$SOUL_PID" 2>/dev/null
|
||||||
|
pkill -f "$SOUL_BIN" 2>/dev/null
|
||||||
|
sleep 0.6; SOUL_PID=""
|
||||||
|
}
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ driver ----
|
||||||
|
cat > "$DRV" <<'PYEOF'
|
||||||
|
import json, os, sys, time, threading, urllib.request, urllib.error
|
||||||
|
|
||||||
|
CFG = json.load(open(sys.argv[1]))
|
||||||
|
SOUL = "http://127.0.0.1:%d" % CFG["soul_port"]
|
||||||
|
STUB = "http://127.0.0.1:%d" % CFG["stub_port"]
|
||||||
|
WS = CFG["workspace"]
|
||||||
|
MODE = CFG["mode"] # bridge | local | toolsoff
|
||||||
|
SCEN = json.load(open(CFG["scenarios"]))
|
||||||
|
STUBLOG = CFG["stub_log"]
|
||||||
|
SOULLOG = CFG["soul_log"]
|
||||||
|
ONLY = CFG.get("classes") or list(SCEN["classes"].keys())
|
||||||
|
MAXHOPS = CFG.get("max_hops", 15)
|
||||||
|
OUT = CFG["out"]
|
||||||
|
# the chat-only class must be driven on the NON-agentic door: the agentic door
|
||||||
|
# always advertises tools, which is a 400 on that scenario by contract.
|
||||||
|
NON_AGENTIC = {"oa-tools-off"}
|
||||||
|
|
||||||
|
def http(method, url, obj=None, timeout=300):
|
||||||
|
data = None if obj is None else json.dumps(obj).encode()
|
||||||
|
req = urllib.request.Request(url, data=data, method=method,
|
||||||
|
headers={"Content-Type": "application/json"})
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(req, timeout=timeout) as r:
|
||||||
|
body = r.read().decode("utf-8", "replace")
|
||||||
|
st = r.status
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
body = e.read().decode("utf-8", "replace"); st = e.code
|
||||||
|
except Exception as e:
|
||||||
|
return -1, "TRANSPORT-ERROR: %r" % (e,), None
|
||||||
|
try:
|
||||||
|
return st, body, json.loads(body)
|
||||||
|
except ValueError:
|
||||||
|
return st, body, None
|
||||||
|
|
||||||
|
def fsize(p):
|
||||||
|
return os.path.getsize(p) if os.path.exists(p) else 0
|
||||||
|
|
||||||
|
def tail_from(path, off):
|
||||||
|
if not os.path.exists(path):
|
||||||
|
return "", off
|
||||||
|
with open(path, "rb") as f:
|
||||||
|
f.seek(off); chunk = f.read(); return chunk.decode("utf-8", "replace"), f.tell()
|
||||||
|
|
||||||
|
def stub_since(off):
|
||||||
|
"""Exact correlation: only the JSONL bytes appended during this phrasing."""
|
||||||
|
txt, noff = tail_from(STUBLOG, off)
|
||||||
|
recs = []
|
||||||
|
for line in txt.splitlines():
|
||||||
|
line = line.strip()
|
||||||
|
if line:
|
||||||
|
try: recs.append(json.loads(line))
|
||||||
|
except ValueError: pass
|
||||||
|
return recs, noff
|
||||||
|
|
||||||
|
def perform(name, ti):
|
||||||
|
"""Execute the bridged tool for real, like the desktop client would."""
|
||||||
|
if name in ("write_file", "edit_file"):
|
||||||
|
p = ti.get("path", "")
|
||||||
|
dest = p if os.path.isabs(p) else os.path.join(WS, p)
|
||||||
|
os.makedirs(os.path.dirname(dest) or WS, exist_ok=True)
|
||||||
|
body = ti.get("content", "")
|
||||||
|
with open(dest, "w") as f:
|
||||||
|
f.write(body)
|
||||||
|
return "wrote %s (%d bytes)" % (p, len(body.encode()))
|
||||||
|
return "ok"
|
||||||
|
|
||||||
|
class Poller(threading.Thread):
|
||||||
|
def __init__(self, sid):
|
||||||
|
super().__init__(daemon=True); self.sid = sid; self.snaps = []; self.stop = False
|
||||||
|
def run(self):
|
||||||
|
while not self.stop:
|
||||||
|
st, body, js = http("GET", SOUL + "/api/run-progress/" + self.sid, timeout=60)
|
||||||
|
if js and js.get("progress"):
|
||||||
|
if not self.snaps or self.snaps[-1] != js["progress"]:
|
||||||
|
self.snaps.append(js["progress"])
|
||||||
|
time.sleep(0.1)
|
||||||
|
|
||||||
|
def progress(sid):
|
||||||
|
_, _, pj = http("GET", SOUL + "/api/run-progress/" + sid, timeout=30)
|
||||||
|
return (pj or {}).get("progress")
|
||||||
|
|
||||||
|
def run_phrasing(cname, ph):
|
||||||
|
st, body, js = http("POST", SOUL + "/api/sessions", {"title": ph["id"]}, timeout=60)
|
||||||
|
sid = (js or {}).get("id", "")
|
||||||
|
rec = {"class": cname, "phrasing": ph["id"], "session_id": sid, "legs": [],
|
||||||
|
"pendings": [], "progress_during": [], "progress_per_leg": [],
|
||||||
|
"progress_final": None, "soul_log": "", "stub": [], "http": [],
|
||||||
|
"agentic": cname not in NON_AGENTIC}
|
||||||
|
if not sid:
|
||||||
|
rec["fatal"] = "session create failed: %s %s" % (st, body[:300]); return rec
|
||||||
|
soff = fsize(SOULLOG); loff = fsize(STUBLOG)
|
||||||
|
t0 = time.time()
|
||||||
|
pol = Poller(sid); pol.start()
|
||||||
|
payload = {"message": ph["prompt"], "session_id": sid, "workspace_root": WS,
|
||||||
|
"agentic": rec["agentic"]}
|
||||||
|
if MODE == "local":
|
||||||
|
payload["agent_workspace_root"] = WS
|
||||||
|
st, body, js = http("POST", SOUL + "/api/chat", payload, timeout=CFG.get("chat_timeout", 240))
|
||||||
|
rec["http"].append(st)
|
||||||
|
rec["legs"].append(js if js is not None else body[:600])
|
||||||
|
rec["progress_per_leg"].append(progress(sid))
|
||||||
|
hops = 0
|
||||||
|
while isinstance(js, dict) and js.get("tool_pending") and hops < MAXHOPS:
|
||||||
|
rec["pendings"].append({"call_id": js.get("call_id"), "tool_name": js.get("tool_name"),
|
||||||
|
"tool_input": js.get("tool_input"), "risk_tier": js.get("risk_tier"),
|
||||||
|
"narration": js.get("narration"), "tools_used": js.get("tools_used")})
|
||||||
|
try:
|
||||||
|
eff = perform(js.get("tool_name", ""), js.get("tool_input") or {})
|
||||||
|
except Exception as e:
|
||||||
|
eff = "client error: %r" % (e,)
|
||||||
|
st, body, js = http("POST", SOUL + "/api/sessions/%s/tool_result" % sid,
|
||||||
|
{"call_id": js.get("call_id"), "content": eff},
|
||||||
|
timeout=CFG.get("chat_timeout", 240))
|
||||||
|
rec["http"].append(st)
|
||||||
|
rec["legs"].append(js if js is not None else body[:600])
|
||||||
|
rec["progress_per_leg"].append(progress(sid))
|
||||||
|
hops += 1
|
||||||
|
pol.stop = True; time.sleep(0.3)
|
||||||
|
t1 = time.time()
|
||||||
|
rec["elapsed"] = round(t1 - t0, 2)
|
||||||
|
rec["progress_during"] = pol.snaps
|
||||||
|
rec["progress_final"] = progress(sid)
|
||||||
|
rec["soul_log"], _ = tail_from(SOULLOG, soff)
|
||||||
|
rec["stub"], _ = stub_since(loff)
|
||||||
|
rec["hops"] = hops
|
||||||
|
return rec
|
||||||
|
|
||||||
|
# ------------------------------------------------------------- assertions ----
|
||||||
|
def expected_calls(cname, first_only=False):
|
||||||
|
out = []
|
||||||
|
for step in SCEN["classes"][cname]["script"]:
|
||||||
|
for k, call in enumerate(step.get("tool_calls") or []):
|
||||||
|
if first_only and k > 0:
|
||||||
|
continue
|
||||||
|
out.append((call["name"], call["arguments"]))
|
||||||
|
return out
|
||||||
|
|
||||||
|
def final_text(cname):
|
||||||
|
for step in reversed(SCEN["classes"][cname]["script"]):
|
||||||
|
if step.get("text") and not step.get("tool_calls"):
|
||||||
|
return step["text"]
|
||||||
|
return None
|
||||||
|
|
||||||
|
def judge(rec):
|
||||||
|
cname = rec["class"]; ok = []; bad = []
|
||||||
|
last = rec["legs"][-1] if rec["legs"] else None
|
||||||
|
reply = last.get("reply") if isinstance(last, dict) else None
|
||||||
|
err = last.get("error") if isinstance(last, dict) else None
|
||||||
|
tools_used = last.get("tools_used") if isinstance(last, dict) else None
|
||||||
|
stub = rec["stub"]
|
||||||
|
scen_recs = [r for r in stub if r.get("kind") == "scenario"]
|
||||||
|
rejects = [r for r in stub if r.get("validation") != "ok"]
|
||||||
|
bg = [r for r in stub if r.get("kind") in ("wrong_path", "background")]
|
||||||
|
|
||||||
|
def wire_clean():
|
||||||
|
if rejects:
|
||||||
|
for r in rejects:
|
||||||
|
bad.append("stub REJECTED a request: [%s] %s"
|
||||||
|
% (r.get("validation"), r.get("validation_detail")))
|
||||||
|
else:
|
||||||
|
ok.append("stub ground truth: validation \"ok\" on all %d scenario leg(s), no "
|
||||||
|
"gate_echo_mismatch / gate_tool_call_shape / dialect-leak 400s"
|
||||||
|
% len(scen_recs))
|
||||||
|
if bg:
|
||||||
|
ok.append("NOTE background non-scenario request(s) in this window: %s"
|
||||||
|
% [(r.get("kind"), r.get("path"), r.get("http_status")) for r in bg])
|
||||||
|
|
||||||
|
if cname == "oa-plain":
|
||||||
|
wire_clean()
|
||||||
|
want = final_text(cname)
|
||||||
|
if reply == want: ok.append("final reply == scripted final text (byte-exact)")
|
||||||
|
else: bad.append("final reply mismatch:\n WANT: %r\n GOT : %r" % (want, reply))
|
||||||
|
if tools_used == []: ok.append("tools_used == [] (no tool ran)")
|
||||||
|
else: bad.append("tools_used expected [] got %r" % (tools_used,))
|
||||||
|
if reply and ('"tool_calls"' in reply or '"function"' in reply or '"tool_use"' in reply):
|
||||||
|
bad.append("tool-call JSON leaked into the reply text")
|
||||||
|
else: ok.append("no tool-call JSON anywhere in the reply")
|
||||||
|
|
||||||
|
elif cname in ("oa-single-tool", "oa-torture", "oa-mission"):
|
||||||
|
wire_clean()
|
||||||
|
want = final_text(cname)
|
||||||
|
if reply == want: ok.append("final reply == scripted final text (byte-exact)")
|
||||||
|
else: bad.append("final reply mismatch:\n WANT: %r\n GOT : %r" % (want, reply))
|
||||||
|
exp = expected_calls(cname)
|
||||||
|
wantnames = [n for n, _ in exp]
|
||||||
|
if tools_used == wantnames:
|
||||||
|
ok.append("tools_used == %r (carried across %d suspension(s))" % (wantnames, rec["hops"]))
|
||||||
|
else:
|
||||||
|
bad.append("tools_used expected %r got %r" % (wantnames, tools_used))
|
||||||
|
for name, args in exp:
|
||||||
|
p = args.get("path"); c = args.get("content")
|
||||||
|
dest = os.path.join(WS, p)
|
||||||
|
if not os.path.exists(dest):
|
||||||
|
bad.append("expected file missing on disk: %s" % dest); continue
|
||||||
|
got = open(dest, "rb").read()
|
||||||
|
if got == c.encode():
|
||||||
|
ok.append("%s on disk is byte-for-byte the issued payload (%d bytes)" % (p, len(got)))
|
||||||
|
else:
|
||||||
|
bad.append("%s content differs\n WANT %r\n GOT %r"
|
||||||
|
% (p, c[:300], got[:300].decode("utf-8", "replace")))
|
||||||
|
if MODE == "bridge":
|
||||||
|
for pend, (name, args) in zip(rec["pendings"], exp):
|
||||||
|
if pend["tool_input"] == args:
|
||||||
|
ok.append("tool_input for %s survived exactly ONE decode (deep-equal to the "
|
||||||
|
"issued arguments; no double-escaping)" % name)
|
||||||
|
else:
|
||||||
|
bad.append("tool_input != issued arguments for %s\n WANT %r\n GOT %r"
|
||||||
|
% (name, args, pend["tool_input"]))
|
||||||
|
if rec["pendings"] and all(p["risk_tier"] == "escalate" for p in rec["pendings"]):
|
||||||
|
ok.append("every write_file classified \"escalate\" and bridged for consent")
|
||||||
|
|
||||||
|
elif cname == "oa-parallel":
|
||||||
|
drift = [l.strip() for l in rec["soul_log"].splitlines() if "DRIFT: provider returned" in l]
|
||||||
|
if drift: ok.append("soul log: " + drift[0])
|
||||||
|
else: bad.append("no 'DRIFT: provider returned N parallel tool_calls' line in the soul log")
|
||||||
|
delivered = [r for r in stub if r.get("delivered", {}).get("tool_calls")]
|
||||||
|
if delivered and len(delivered[0]["delivered"]["tool_calls"]) == 2:
|
||||||
|
ok.append("stub delivered 2 parallel tool_calls in one response (ground truth)")
|
||||||
|
if MODE == "bridge":
|
||||||
|
if len(rec["pendings"]) == 1:
|
||||||
|
ok.append("exactly ONE call honored: %s" % rec["pendings"][0]["call_id"])
|
||||||
|
else:
|
||||||
|
bad.append("expected exactly 1 honored call, got %d" % len(rec["pendings"]))
|
||||||
|
pairing = [r for r in rejects if "gate_pairing" in str(r.get("validation_detail")) or
|
||||||
|
"tool_calls at end of thread" in str(r.get("validation_detail")) or
|
||||||
|
"not fully answered" in str(r.get("validation_detail"))]
|
||||||
|
for r in pairing:
|
||||||
|
ok.append("EXPECTED-BY-CONTRACT stub 400 on the unpaired echo: %s"
|
||||||
|
% r.get("validation_detail"))
|
||||||
|
other = [r for r in rejects if r not in pairing]
|
||||||
|
for r in other:
|
||||||
|
bad.append("unexpected stub rejection: [%s] %s"
|
||||||
|
% (r.get("validation"), r.get("validation_detail")))
|
||||||
|
if err and not reply:
|
||||||
|
ok.append("honest error envelope after the 400 (no fabricated answer): %r" % err)
|
||||||
|
elif reply == final_text(cname):
|
||||||
|
ok.append("final reply == scripted final text (both calls paired)")
|
||||||
|
else:
|
||||||
|
bad.append("neither an honest error nor the scripted final text: %r" % (last,))
|
||||||
|
|
||||||
|
elif cname == "oa-api-error":
|
||||||
|
if err and not reply:
|
||||||
|
ok.append("honest error envelope: error=%r reply=%r" % (err, reply))
|
||||||
|
else:
|
||||||
|
bad.append("expected an error envelope with an empty reply, got %r" % (last,))
|
||||||
|
delivered = [r["delivered"].get("api_error") for r in stub if r.get("delivered")]
|
||||||
|
ok.append("stub delivered api_error status(es): %r" % [d for d in delivered if d])
|
||||||
|
n = len([r for r in stub if r.get("kind") == "scenario"])
|
||||||
|
ok.append("provider hit %d time(s) - no retry storm" % n)
|
||||||
|
if reply:
|
||||||
|
bad.append("FABRICATED ANSWER: reply non-empty on a provider error")
|
||||||
|
|
||||||
|
elif cname == "oa-tools-off":
|
||||||
|
ok.append("stub records for this phrasing: %r"
|
||||||
|
% [{k: r.get(k) for k in ("kind", "path", "validation", "http_status")} for r in stub])
|
||||||
|
wrong = [r for r in stub if r.get("kind") == "wrong_path"]
|
||||||
|
matched = [r for r in stub if r.get("scenario_class") == cname]
|
||||||
|
if matched and not rejects:
|
||||||
|
ok.append("chat-only request reached /v1/chat/completions with NO tools offered")
|
||||||
|
want = final_text(cname)
|
||||||
|
if reply == want: ok.append("final reply == scripted final text (byte-exact)")
|
||||||
|
else: bad.append("final reply mismatch:\n WANT: %r\n GOT : %r" % (want, reply))
|
||||||
|
if reply and ('"tool_calls"' in reply or '"function"' in reply):
|
||||||
|
bad.append("tool-call JSON leaked into the reply text")
|
||||||
|
else: ok.append("no tool-call JSON in the reply")
|
||||||
|
elif wrong:
|
||||||
|
bad.append("the non-agentic lane never reached the provider endpoint: stub saw "
|
||||||
|
"%s -> %s (the el-runtime provider chain appends /v1/chat/completions "
|
||||||
|
"to NEURON_LLM_0_URL, chat.el appends only /chat/completions)"
|
||||||
|
% (wrong[0]["path"], wrong[0]["http_status"]))
|
||||||
|
elif not stub:
|
||||||
|
bad.append("no request reached the stub at all")
|
||||||
|
else:
|
||||||
|
for r in rejects:
|
||||||
|
bad.append("stub REJECTED: [%s] %s" % (r.get("validation"), r.get("validation_detail")))
|
||||||
|
return ok, bad
|
||||||
|
|
||||||
|
def main():
|
||||||
|
results = []
|
||||||
|
for cname in ONLY:
|
||||||
|
for ph in SCEN["classes"][cname]["phrasings"]:
|
||||||
|
rec = run_phrasing(cname, ph)
|
||||||
|
ok, bad = judge(rec)
|
||||||
|
rec["ok"] = ok; rec["bad"] = bad
|
||||||
|
rec["verdict"] = "FAIL" if bad else "PASS"
|
||||||
|
results.append(rec)
|
||||||
|
print("=" * 78)
|
||||||
|
print("[%s] %s / %s (%.2fs, %d bridge hop(s), agentic=%s, mode=%s)"
|
||||||
|
% (rec["verdict"], cname, ph["id"], rec.get("elapsed", 0),
|
||||||
|
rec.get("hops", 0), rec["agentic"], MODE))
|
||||||
|
for l in ok: print(" ok " + l.replace("\n", "\n "))
|
||||||
|
for l in bad: print(" FAIL " + l.replace("\n", "\n "))
|
||||||
|
for i, leg in enumerate(rec["legs"]):
|
||||||
|
print(" leg%d envelope: %s" % (i, json.dumps(leg)[:430]))
|
||||||
|
for i, pr in enumerate(rec["progress_per_leg"]):
|
||||||
|
print(" run-progress after leg%d: %s" % (i, json.dumps(pr)[:380]))
|
||||||
|
if rec["progress_during"]:
|
||||||
|
print(" run-progress polled DURING (%d distinct snapshot(s)), last: %s"
|
||||||
|
% (len(rec["progress_during"]), json.dumps(rec["progress_during"][-1])[:300]))
|
||||||
|
for r in rec["stub"]:
|
||||||
|
print(" stub: kind=%s class=%s phrasing=%s step=%s validation=%s%s delivered=%s http=%s"
|
||||||
|
% (r.get("kind"), r.get("scenario_class"), r.get("phrasing"), r.get("step"),
|
||||||
|
r.get("validation"),
|
||||||
|
("(" + str(r.get("validation_detail")) + ")") if r.get("validation_detail") else "",
|
||||||
|
json.dumps(r.get("delivered")), r.get("http_status")))
|
||||||
|
if rec["soul_log"].strip():
|
||||||
|
for l in rec["soul_log"].splitlines():
|
||||||
|
if l.strip(): print(" soul: " + l.strip())
|
||||||
|
json.dump(results, open(OUT, "w"), indent=1)
|
||||||
|
npass = sum(1 for r in results if r["verdict"] == "PASS")
|
||||||
|
print("=" * 78)
|
||||||
|
print("PHASE %s: %d/%d PASS" % (MODE, npass, len(results)))
|
||||||
|
for r in results:
|
||||||
|
print(" %-6s %-16s %s" % (r["verdict"], r["class"], r["phrasing"]))
|
||||||
|
return 0 if npass == len(results) else 1
|
||||||
|
|
||||||
|
sys.exit(main())
|
||||||
|
PYEOF
|
||||||
|
|
||||||
|
# ------------------------------------------------------------- hostile drv ---
|
||||||
|
cat > "$RUN/hostile.py" <<'PYEOF'
|
||||||
|
import json, os, sys, time, urllib.request, urllib.error
|
||||||
|
|
||||||
|
CFG = json.load(open(sys.argv[1]))
|
||||||
|
SOUL = "http://127.0.0.1:%d" % CFG["soul_port"]
|
||||||
|
STUB = "http://127.0.0.1:%d" % CFG["stub_port"]
|
||||||
|
|
||||||
|
def http(method, url, obj=None, timeout=400):
|
||||||
|
data = None if obj is None else json.dumps(obj).encode()
|
||||||
|
req = urllib.request.Request(url, data=data, method=method,
|
||||||
|
headers={"Content-Type": "application/json"})
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(req, timeout=timeout) as r:
|
||||||
|
b = r.read().decode("utf-8", "replace"); st = r.status
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
b = e.read().decode("utf-8", "replace"); st = e.code
|
||||||
|
except Exception as e:
|
||||||
|
return -1, "TRANSPORT-ERROR: %r" % (e,), None
|
||||||
|
try:
|
||||||
|
return st, b, json.loads(b)
|
||||||
|
except ValueError:
|
||||||
|
return st, b, None
|
||||||
|
|
||||||
|
mode = CFG["mode"]; wsmode = CFG["ws_mode"]; WS = CFG["workspace"]
|
||||||
|
_, _, js = http("POST", SOUL + "/api/sessions", {"title": "hostile-" + mode}, timeout=60)
|
||||||
|
sid = (js or {}).get("id", "")
|
||||||
|
payload = {"message": "oa-gate plain probe: hostile mode %s" % mode,
|
||||||
|
"agentic": True, "session_id": sid, "workspace_root": WS}
|
||||||
|
if wsmode == "local":
|
||||||
|
payload["agent_workspace_root"] = WS
|
||||||
|
t0 = time.time()
|
||||||
|
st, body, js = http("POST", SOUL + "/api/chat", payload, timeout=CFG.get("timeout", 400))
|
||||||
|
t_first = time.time() - t0
|
||||||
|
legs = [js if js is not None else body[:500]]
|
||||||
|
hops = 0
|
||||||
|
while isinstance(js, dict) and js.get("tool_pending") and hops < CFG.get("max_hops", 14):
|
||||||
|
ti = js.get("tool_input") or {}
|
||||||
|
p = ti.get("path", "x.md")
|
||||||
|
dest = p if os.path.isabs(p) else os.path.join(WS, p)
|
||||||
|
try: open(dest, "w").write(ti.get("content", ""))
|
||||||
|
except Exception: pass
|
||||||
|
st, body, js = http("POST", SOUL + "/api/sessions/%s/tool_result" % sid,
|
||||||
|
{"call_id": js.get("call_id"), "content": "ok"},
|
||||||
|
timeout=CFG.get("timeout", 400))
|
||||||
|
legs.append(js if js is not None else body[:500]); hops += 1
|
||||||
|
el = time.time() - t0
|
||||||
|
_, _, stats = http("GET", STUB + "/gate/stats", timeout=30)
|
||||||
|
_, _, prog = http("GET", SOUL + "/api/run-progress/" + sid, timeout=30)
|
||||||
|
fab = [l for l in legs if isinstance(l, dict) and l.get("reply")]
|
||||||
|
print("HOSTILE %s (ws_mode=%s)" % (mode, wsmode))
|
||||||
|
print(" first /api/chat POST returned after %.2fs; whole chain %.2fs; client bridge hops=%d; "
|
||||||
|
"stub chat_hits=%s" % (t_first, el, hops, (stats or {}).get("chat_hits")))
|
||||||
|
print(" first envelope : " + json.dumps(legs[0])[:430])
|
||||||
|
print(" final envelope : " + json.dumps(legs[-1])[:430])
|
||||||
|
print(" non-empty replies anywhere in the chain (fabrication check): %d" % len(fab))
|
||||||
|
print(" run-progress : " + json.dumps(prog)[:300])
|
||||||
|
json.dump({"mode": mode, "ws_mode": wsmode, "t_first": t_first, "elapsed": el, "hops": hops,
|
||||||
|
"chat_hits": (stats or {}).get("chat_hits"), "legs": legs, "progress": prog},
|
||||||
|
open(CFG["out"], "w"), indent=1)
|
||||||
|
PYEOF
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ phases ---
|
||||||
|
RC_BRIDGE=0; RC_LOCAL=0; RC_OFF=0
|
||||||
|
run_normal_phase() { # $1 = label, $2 = soul port, $3 = agent root, $4 = ws, $5 = base, $6 = classes json
|
||||||
|
local m="$1" port="$2" root="$3" ws="$4" base="$5" classes="$6"
|
||||||
|
echo; echo "############ PHASE: $m (soul :$port, NEURON_LLM_0_URL=$base) ############"
|
||||||
|
start_stub normal "$RUN/stub-$m.jsonl"
|
||||||
|
start_soul "$port" "$base" "$RUN/soul-$m.log" "$root"
|
||||||
|
cat > "$RUN/cfg-$m.json" <<JSON
|
||||||
|
{"soul_port": $port, "stub_port": $STUB_PORT, "workspace": "$ws", "mode": "$m",
|
||||||
|
"scenarios": "$HERE/scenarios-openai.json", "stub_log": "$RUN/stub-$m.jsonl",
|
||||||
|
"soul_log": "$RUN/soul-$m.log", "out": "$RUN/results-$m.json", "chat_timeout": 240,
|
||||||
|
"classes": $classes}
|
||||||
|
JSON
|
||||||
|
python3 "$DRV" "$RUN/cfg-$m.json"
|
||||||
|
local rc=$?
|
||||||
|
stop_soul; stop_stub
|
||||||
|
return $rc
|
||||||
|
}
|
||||||
|
|
||||||
|
if [ "$PHASES" = "all" ] || [ "$PHASES" = "bridge" ]; then
|
||||||
|
run_normal_phase bridge "$SOUL_PORT" "" "$RUN/ws-bridge" "http://127.0.0.1:$STUB_PORT/v1" null
|
||||||
|
RC_BRIDGE=$?
|
||||||
|
fi
|
||||||
|
if [ "$PHASES" = "all" ] || [ "$PHASES" = "local" ]; then
|
||||||
|
run_normal_phase local "$SOUL_PORT_B" "$RUN/ws-local" "$RUN/ws-local" "http://127.0.0.1:$STUB_PORT/v1" null
|
||||||
|
RC_LOCAL=$?
|
||||||
|
fi
|
||||||
|
if [ "$PHASES" = "all" ] || [ "$PHASES" = "toolsoff" ]; then
|
||||||
|
# supplementary: the el-runtime provider chain appends /v1/chat/completions itself,
|
||||||
|
# so the non-agentic door needs the base WITHOUT the /v1 suffix.
|
||||||
|
run_normal_phase toolsoff "$SOUL_PORT" "" "$RUN/ws-off" "http://127.0.0.1:$STUB_PORT" '["oa-tools-off","oa-plain"]'
|
||||||
|
RC_OFF=$?
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$PHASES" = "all" ] || [ "$PHASES" = "hostile" ]; then
|
||||||
|
echo; echo "############ PHASE: hostile ############"
|
||||||
|
for spec in "black-hole:bridge" "mid-body-drop:bridge" "tool-pending-forever:bridge" "tool-pending-forever:local"; do
|
||||||
|
mode="${spec%%:*}"; wsm="${spec##*:}"
|
||||||
|
echo; echo "---- hostile mode=$mode ws_mode=$wsm ----"
|
||||||
|
start_stub "$mode" "$RUN/stub-$mode-$wsm.jsonl"
|
||||||
|
if [ "$wsm" = "local" ]; then
|
||||||
|
start_soul "$SOUL_PORT" "http://127.0.0.1:$STUB_PORT/v1" "$RUN/soul-$mode-$wsm.log" "$RUN/ws-local"
|
||||||
|
else
|
||||||
|
start_soul "$SOUL_PORT" "http://127.0.0.1:$STUB_PORT/v1" "$RUN/soul-$mode-$wsm.log" ""
|
||||||
|
fi
|
||||||
|
cat > "$RUN/cfg-$mode-$wsm.json" <<JSON
|
||||||
|
{"soul_port": $SOUL_PORT, "stub_port": $STUB_PORT, "mode": "$mode", "ws_mode": "$wsm",
|
||||||
|
"workspace": "$RUN/ws-local", "out": "$RUN/hostile-$mode-$wsm.json", "timeout": 400}
|
||||||
|
JSON
|
||||||
|
python3 "$RUN/hostile.py" "$RUN/cfg-$mode-$wsm.json"
|
||||||
|
echo " soul log (llm/DRIFT/cap lines):"
|
||||||
|
grep -E "DRIFT|llm error|iteration cap|\[llm\]" "$RUN/soul-$mode-$wsm.log" | tail -8 | sed 's/^/ /'
|
||||||
|
stop_soul; stop_stub
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo; echo "############ CLEANUP ############"
|
||||||
|
cleanup
|
||||||
|
sleep 0.5
|
||||||
|
echo "processes still matching the soul binary:"; pgrep -fl "$SOUL_BIN" || echo " (none)"
|
||||||
|
echo "processes still matching stub-openai.py:"; pgrep -fl "stub-openai.py" || echo " (none)"
|
||||||
|
echo "lsof on 7891-7894 after cleanup:"
|
||||||
|
lsof -nP -iTCP:7891 -iTCP:7892 -iTCP:7893 -iTCP:7894 2>/dev/null || echo " (no listeners - ports free)"
|
||||||
|
echo
|
||||||
|
echo "############ SUMMARY ############"
|
||||||
|
echo "run dir: $RUN"
|
||||||
|
echo "bridge rc=$RC_BRIDGE local rc=$RC_LOCAL toolsoff rc=$RC_OFF (0 = every class PASS)"
|
||||||
|
exit $(( RC_BRIDGE + RC_LOCAL + RC_OFF ))
|
||||||
@@ -0,0 +1,110 @@
|
|||||||
|
{
|
||||||
|
"_comment": "OpenAI-dialect gate scenario contract (soul-openai-tools-v2). Single source of truth shared by stub-openai.py (scripted provider responses + request assertions), selftest.sh (stub self-verification), and the future brain-side gate driver. Same structure as gate9's scenarios.json: classes -> script + phrasings with markers; scripts are CLASS-level so assertions are behavioral, never pinned to a sentence. expect_request keys: require_tools, require_tool_choice, parallel_tool_calls (expected literal value; null = don't check), forbid_tools. defaults apply to every class unless overridden; steps may override with their own expect_request.",
|
||||||
|
"deadline_secs": 60,
|
||||||
|
"max_loop_iterations": 16,
|
||||||
|
"defaults": {
|
||||||
|
"expect_request": {
|
||||||
|
"require_tools": true,
|
||||||
|
"require_tool_choice": true,
|
||||||
|
"parallel_tool_calls": false
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"classes": {
|
||||||
|
"oa-plain": {
|
||||||
|
"script": [
|
||||||
|
{ "text": "Plain OpenAI-lane answer (gate fixture): the mechanism, the main caveat, and the practical takeaway in three sentences. No tools were needed for this one, and the finish reason on the wire is stop, which the loop must treat as terminal." }
|
||||||
|
],
|
||||||
|
"phrasings": [
|
||||||
|
{ "id": "oa-plain-p1", "marker": "oa-gate plain probe", "prompt": "oa-gate plain probe: explain the fixture topic simply." },
|
||||||
|
{ "id": "oa-plain-p2", "marker": "oa-gate second plain", "prompt": "oa-gate second plain: another phrasing of the plain question." }
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"oa-tools-off": {
|
||||||
|
"expect_request": {
|
||||||
|
"require_tools": false,
|
||||||
|
"forbid_tools": true,
|
||||||
|
"require_tool_choice": false,
|
||||||
|
"parallel_tool_calls": null
|
||||||
|
},
|
||||||
|
"script": [
|
||||||
|
{ "text": "Chat-only OpenAI-lane answer (gate fixture): this lane offered no tools and none were used; the reply is plain text with finish reason stop." }
|
||||||
|
],
|
||||||
|
"phrasings": [
|
||||||
|
{ "id": "oa-tools-off-p1", "marker": "oa-gate tools-off probe", "prompt": "oa-gate tools-off probe: plain chat with no tools offered." }
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"oa-single-tool": {
|
||||||
|
"script": [
|
||||||
|
{ "text": "Step 1: writing the note.",
|
||||||
|
"tool_calls": [
|
||||||
|
{ "name": "write_file",
|
||||||
|
"arguments": { "path": "openai-single-note.md", "content": "# Note (gate fixture, OpenAI lane)\n\nDeterministic single-tool body.\n" } }
|
||||||
|
] },
|
||||||
|
{ "text": "All set - openai-single-note.md is written with the fixture body. Nothing else was needed for this one." }
|
||||||
|
],
|
||||||
|
"phrasings": [
|
||||||
|
{ "id": "oa-single-p1", "marker": "oa-gate single tool note", "prompt": "oa-gate single tool note: save the fixture note to a file." },
|
||||||
|
{ "id": "oa-single-p2", "marker": "oa-gate one file please", "prompt": "oa-gate one file please: write the fixture note file." }
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"oa-torture": {
|
||||||
|
"script": [
|
||||||
|
{ "tool_calls": [
|
||||||
|
{ "name": "write_file",
|
||||||
|
"arguments": { "path": "torture-note.md", "content": "Line 1 has \"double quotes\", 'singles', and a mid-line backslash \\ here.\nLine 2\thas a tab, a literal \\n two-char sequence, and a Windows path C:\\temp\\new.txt.\nLine 3 unicode: naïve café — 日本語 ✓ 🚀\nLine 4 JSON-in-string: {\"k\": \"v\", \"arr\": [1, 2], \"s\": \"nested \\\"deep\\\" quotes\"}\nLine 5 ends with a lone backslash \\" } }
|
||||||
|
] },
|
||||||
|
{ "text": "Torture round-trip complete: the payload with nested quotes, backslashes, newlines, tabs, and unicode survived exactly one encode and one decode." }
|
||||||
|
],
|
||||||
|
"phrasings": [
|
||||||
|
{ "id": "oa-torture-p1", "marker": "oa-gate torture probe", "prompt": "oa-gate torture probe: write the escaping torture file." }
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"oa-parallel": {
|
||||||
|
"script": [
|
||||||
|
{ "text": "Step 1: two writes at once (parallel probe).",
|
||||||
|
"tool_calls": [
|
||||||
|
{ "name": "write_file", "arguments": { "path": "parallel-a.md", "content": "Parallel A (gate fixture).\n" } },
|
||||||
|
{ "name": "write_file", "arguments": { "path": "parallel-b.md", "content": "Parallel B (gate fixture).\n" } }
|
||||||
|
] },
|
||||||
|
{ "text": "Parallel probe complete: both tool results arrived and were paired correctly. A brain that instead rejects the double call must do so cleanly - that outcome is asserted brain-side, not here." }
|
||||||
|
],
|
||||||
|
"phrasings": [
|
||||||
|
{ "id": "oa-parallel-p1", "marker": "oa-gate parallel probe", "prompt": "oa-gate parallel probe: run the two-write parallel case." }
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"oa-mission": {
|
||||||
|
"script": [
|
||||||
|
{ "text": "Step 1: drafting part one.",
|
||||||
|
"tool_calls": [
|
||||||
|
{ "name": "write_file", "arguments": { "path": "mission-part-1.md", "content": "Mission part 1 (gate fixture).\n" } }
|
||||||
|
] },
|
||||||
|
{ "text": "Step 2: drafting part two.",
|
||||||
|
"tool_calls": [
|
||||||
|
{ "name": "write_file", "arguments": { "path": "mission-part-2.md", "content": "Mission part 2 (gate fixture).\n" } }
|
||||||
|
] },
|
||||||
|
{ "text": "Mission complete: mission-part-1.md and mission-part-2.md are written; the loop ran two tool rounds and finished cleanly with finish reason stop." }
|
||||||
|
],
|
||||||
|
"phrasings": [
|
||||||
|
{ "id": "oa-mission-p1", "marker": "oa-gate mission probe", "prompt": "oa-gate mission probe: run the two-round mission." }
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"oa-api-error": {
|
||||||
|
"expect_request": {
|
||||||
|
"require_tools": false,
|
||||||
|
"require_tool_choice": false,
|
||||||
|
"parallel_tool_calls": null
|
||||||
|
},
|
||||||
|
"script": [],
|
||||||
|
"phrasings": [
|
||||||
|
{ "id": "oa-err-400", "marker": "oa-gate error four hundred", "prompt": "oa-gate error four hundred: trigger the injected failure.",
|
||||||
|
"script": [ { "api_error": { "status": 400, "type": "invalid_request_error", "message": "gate-injected 400: request rejected by fixture", "code": "gate_injected" } } ] },
|
||||||
|
{ "id": "oa-err-429", "marker": "oa-gate error rate limit", "prompt": "oa-gate error rate limit: trigger the injected failure.",
|
||||||
|
"script": [ { "api_error": { "status": 429, "type": "rate_limit_error", "message": "gate-injected 429: rate limited by fixture", "code": "rate_limit_exceeded" } } ] },
|
||||||
|
{ "id": "oa-err-500", "marker": "oa-gate error five hundred", "prompt": "oa-gate error five hundred: trigger the injected failure.",
|
||||||
|
"script": [ { "api_error": { "status": 500, "type": "server_error", "message": "gate-injected 500: internal fixture error", "code": "gate_injected" } } ] },
|
||||||
|
{ "id": "oa-err-503", "marker": "oa-gate error unavailable", "prompt": "oa-gate error unavailable: trigger the injected failure.",
|
||||||
|
"script": [ { "api_error": { "status": 503, "type": "server_error", "message": "gate-injected 503: fixture overloaded", "code": "gate_injected" } } ] }
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
Executable
+409
@@ -0,0 +1,409 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# selftest.sh - proves stub-openai.py before any brain code exists.
|
||||||
|
# Drives the stub with curl through every scenario (plain, tools-off,
|
||||||
|
# single tool round-trip, escaping torture, parallel double-call,
|
||||||
|
# two-round mission, injected API errors, background, overrun), every
|
||||||
|
# validation rejection (dialect leaks, pairing, echo round-trip, scenario
|
||||||
|
# expectations), and all three hostile modes. Exit 0 = green.
|
||||||
|
set -u
|
||||||
|
cd "$(dirname "$0")" || exit 1
|
||||||
|
PY=python3
|
||||||
|
TMP="$(mktemp -d)"
|
||||||
|
PIDS=()
|
||||||
|
cleanup() {
|
||||||
|
for p in "${PIDS[@]:-}"; do kill -9 "$p" >/dev/null 2>&1; done
|
||||||
|
rm -rf "$TMP"
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
|
||||||
|
PASS=0; FAIL=0
|
||||||
|
ok() { printf 'ok - %s\n' "$1"; PASS=$((PASS+1)); }
|
||||||
|
bad() { printf 'FAIL - %s\n' "$1"; FAIL=$((FAIL+1)); }
|
||||||
|
check() { # check <name> <cmd...> - pass if cmd exits 0; show output on fail
|
||||||
|
local name="$1"; shift
|
||||||
|
local out
|
||||||
|
if out="$("$@" 2>&1)"; then ok "$name"
|
||||||
|
else bad "$name"; [ -n "$out" ] && printf '%s\n' "$out" | sed 's/^/ /' | head -8
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
freeport() { "$PY" -c 'import socket;s=socket.socket();s.bind(("127.0.0.1",0));print(s.getsockname()[1]);s.close()'; }
|
||||||
|
waithealth() {
|
||||||
|
local p="$1" i
|
||||||
|
for i in $(seq 1 60); do
|
||||||
|
curl -sf "http://127.0.0.1:$p/gate/health" >/dev/null 2>&1 && return 0
|
||||||
|
sleep 0.1
|
||||||
|
done
|
||||||
|
echo "stub on :$p never became healthy"; return 1
|
||||||
|
}
|
||||||
|
post() { # post <port> <bodyfile> <respfile> [extra curl args...] -> echoes http code
|
||||||
|
local port="$1" body="$2" resp="$3"; shift 3
|
||||||
|
curl -s -o "$resp" -w '%{http_code}' -H 'content-type: application/json' \
|
||||||
|
"$@" --data-binary @"$body" "http://127.0.0.1:$port/v1/chat/completions"
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---- embedded helper: builds OpenAI-dialect bodies, asserts on responses ----
|
||||||
|
cat > "$TMP/helpers.py" <<'PYEOF'
|
||||||
|
import copy, json, sys
|
||||||
|
|
||||||
|
TOOLS = [
|
||||||
|
{"type": "function", "function": {
|
||||||
|
"name": "write_file", "description": "Write content to a file on disk.",
|
||||||
|
"parameters": {"type": "object",
|
||||||
|
"properties": {"path": {"type": "string"},
|
||||||
|
"content": {"type": "string"}},
|
||||||
|
"required": ["path", "content"]}}},
|
||||||
|
{"type": "function", "function": {
|
||||||
|
"name": "read_file", "description": "Read contents of a file from disk.",
|
||||||
|
"parameters": {"type": "object",
|
||||||
|
"properties": {"path": {"type": "string"}},
|
||||||
|
"required": ["path"]}}},
|
||||||
|
]
|
||||||
|
|
||||||
|
def dump(obj, out):
|
||||||
|
json.dump(obj, open(out, "w"), ensure_ascii=False)
|
||||||
|
|
||||||
|
def base(prompt, tools=True):
|
||||||
|
b = {"model": "gate-openai-model", "max_tokens": 1024,
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": "You are Neuron (gate fixture)."},
|
||||||
|
{"role": "user", "content": prompt}]}
|
||||||
|
if tools:
|
||||||
|
b["tools"] = copy.deepcopy(TOOLS)
|
||||||
|
b["tool_choice"] = "auto"
|
||||||
|
b["parallel_tool_calls"] = False
|
||||||
|
return b
|
||||||
|
|
||||||
|
def cmd_plain(out, prompt):
|
||||||
|
dump(base(prompt), out)
|
||||||
|
|
||||||
|
def cmd_notools(out, prompt):
|
||||||
|
dump(base(prompt, tools=False), out)
|
||||||
|
|
||||||
|
def cmd_mut(out, prompt, mutation):
|
||||||
|
b = base(prompt)
|
||||||
|
if mutation == "no-tool-choice":
|
||||||
|
del b["tool_choice"]
|
||||||
|
elif mutation == "ptc-true":
|
||||||
|
b["parallel_tool_calls"] = True
|
||||||
|
elif mutation == "top-system":
|
||||||
|
b["system"] = "You are Neuron."
|
||||||
|
elif mutation == "anth-tools":
|
||||||
|
b["tools"] = [{"name": "write_file", "description": "x",
|
||||||
|
"input_schema": {"type": "object", "properties": {}}}]
|
||||||
|
elif mutation == "anth-block":
|
||||||
|
b["messages"][1] = {"role": "user", "content": [
|
||||||
|
{"type": "tool_result", "tool_use_id": "toolu_x", "content": "hi"},
|
||||||
|
{"type": "text", "text": prompt}]}
|
||||||
|
else:
|
||||||
|
raise SystemExit("unknown mutation " + mutation)
|
||||||
|
dump(b, out)
|
||||||
|
|
||||||
|
def cmd_chain(out, prompt, variant, *resps):
|
||||||
|
"""Build the next leg: echo each response's assistant turn and answer its
|
||||||
|
tool calls. `variant` applies to the LAST response only:
|
||||||
|
ok | no-tool-turn | wrong-id | only-first | double-encode | object-args"""
|
||||||
|
b = base(prompt)
|
||||||
|
for idx, p in enumerate(resps):
|
||||||
|
last = idx == len(resps) - 1
|
||||||
|
msg = json.load(open(p))["choices"][0]["message"]
|
||||||
|
tcs = msg.get("tool_calls")
|
||||||
|
if not tcs:
|
||||||
|
b["messages"].append({"role": "assistant",
|
||||||
|
"content": msg.get("content")})
|
||||||
|
continue
|
||||||
|
v = variant if last else "ok"
|
||||||
|
asst = {"role": "assistant", "content": msg.get("content"),
|
||||||
|
"tool_calls": copy.deepcopy(tcs)}
|
||||||
|
if v == "double-encode":
|
||||||
|
for tc in asst["tool_calls"]:
|
||||||
|
tc["function"]["arguments"] = json.dumps(
|
||||||
|
tc["function"]["arguments"])
|
||||||
|
if v == "object-args":
|
||||||
|
for tc in asst["tool_calls"]:
|
||||||
|
tc["function"]["arguments"] = json.loads(
|
||||||
|
tc["function"]["arguments"])
|
||||||
|
b["messages"].append(asst)
|
||||||
|
if v == "no-tool-turn":
|
||||||
|
continue
|
||||||
|
use = tcs[:1] if v == "only-first" else tcs
|
||||||
|
for tc in use:
|
||||||
|
tid = "call_bogus_123" if v == "wrong-id" else tc["id"]
|
||||||
|
b["messages"].append({"role": "tool", "tool_call_id": tid,
|
||||||
|
"content": "{\"ok\":true,\"bytes\":42}"})
|
||||||
|
dump(b, out)
|
||||||
|
|
||||||
|
def cmd_chk(resp, expr):
|
||||||
|
r = json.load(open(resp))
|
||||||
|
if not eval(expr, {"r": r, "json": json, "len": len, "str": str,
|
||||||
|
"isinstance": isinstance, "any": any, "all": all,
|
||||||
|
"sorted": sorted}):
|
||||||
|
print("assertion failed:", expr)
|
||||||
|
print("resp:", json.dumps(r, ensure_ascii=False)[:400])
|
||||||
|
raise SystemExit(1)
|
||||||
|
|
||||||
|
def cmd_torture(resp, scen):
|
||||||
|
r = json.load(open(resp))
|
||||||
|
tc = r["choices"][0]["message"]["tool_calls"][0]
|
||||||
|
raw = tc["function"]["arguments"]
|
||||||
|
assert isinstance(raw, str), "arguments must be a JSON-encoded string"
|
||||||
|
got = json.loads(raw)
|
||||||
|
exp = json.load(open(scen))["classes"]["oa-torture"]["script"][0]["tool_calls"][0]["arguments"]
|
||||||
|
assert got == exp, "decoded arguments != scripted torture payload"
|
||||||
|
content = got["content"]
|
||||||
|
for needle in ['"', "\\", "\n", "\t", "日本語", "naïve", "🚀"]:
|
||||||
|
assert needle in content, "missing torture needle %r" % needle
|
||||||
|
|
||||||
|
def cmd_notjson(path):
|
||||||
|
data = open(path, "rb").read()
|
||||||
|
assert data, "file empty - no partial body arrived"
|
||||||
|
try:
|
||||||
|
json.loads(data.decode("utf-8", "replace"))
|
||||||
|
except ValueError:
|
||||||
|
return
|
||||||
|
raise SystemExit("partial body unexpectedly parsed as complete JSON")
|
||||||
|
|
||||||
|
def cmd_pending(*paths):
|
||||||
|
ids = []
|
||||||
|
for p in paths:
|
||||||
|
c = json.load(open(p))["choices"][0]
|
||||||
|
assert c["finish_reason"] == "tool_calls", c["finish_reason"]
|
||||||
|
tc = c["message"]["tool_calls"][0]
|
||||||
|
assert tc["function"]["name"] == "write_file"
|
||||||
|
json.loads(tc["function"]["arguments"]) # must decode
|
||||||
|
ids.append(tc["id"])
|
||||||
|
assert len(set(ids)) == len(ids), "call ids not distinct: %r" % ids
|
||||||
|
|
||||||
|
def cmd_logcheck(path):
|
||||||
|
recs = [json.loads(l) for l in open(path) if l.strip()]
|
||||||
|
seqs = [r["seq"] for r in recs]
|
||||||
|
assert seqs == sorted(seqs) and len(set(seqs)) == len(seqs), "seq not monotonic"
|
||||||
|
kinds = {}
|
||||||
|
for r in recs:
|
||||||
|
kinds[r["kind"]] = kinds.get(r["kind"], 0) + 1
|
||||||
|
assert kinds.get("scenario", 0) >= 10, "too few scenario records: %r" % kinds
|
||||||
|
assert kinds.get("background", 0) >= 1, "no background record"
|
||||||
|
assert kinds.get("overrun", 0) >= 1, "no overrun record"
|
||||||
|
rejected = [r for r in recs if r["validation"] == "rejected"]
|
||||||
|
assert len(rejected) >= 10, "too few rejected records: %d" % len(rejected)
|
||||||
|
assert any(r["delivered"].get("tool_calls") == ["write_file"]
|
||||||
|
for r in recs), "no single write_file ground truth"
|
||||||
|
assert any(r["delivered"].get("tool_calls") == ["write_file", "write_file"]
|
||||||
|
for r in recs), "no parallel ground truth"
|
||||||
|
|
||||||
|
def main():
|
||||||
|
fn = globals()["cmd_" + sys.argv[1].replace("-", "_")]
|
||||||
|
fn(*sys.argv[2:])
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
|
PYEOF
|
||||||
|
mk() { "$PY" "$TMP/helpers.py" "$@"; }
|
||||||
|
|
||||||
|
echo "=== stub-openai selftest ==="
|
||||||
|
|
||||||
|
# ---- normal mode ------------------------------------------------------------
|
||||||
|
PORT="$(freeport)"
|
||||||
|
"$PY" stub-openai.py --port "$PORT" --scenarios scenarios-openai.json \
|
||||||
|
--log "$TMP/req.jsonl" >"$TMP/stub.out" 2>&1 &
|
||||||
|
PIDS+=($!); disown
|
||||||
|
check "stub starts and answers /gate/health" waithealth "$PORT"
|
||||||
|
|
||||||
|
# 1. plain completion
|
||||||
|
mk plain "$TMP/plain.json" "oa-gate plain probe: explain the fixture topic simply."
|
||||||
|
code="$(post "$PORT" "$TMP/plain.json" "$TMP/r_plain.json")"
|
||||||
|
check "plain: HTTP 200" test "$code" = "200"
|
||||||
|
check "plain: chat.completion envelope, finish stop, real content" mk chk "$TMP/r_plain.json" \
|
||||||
|
'r["object"]=="chat.completion" and r["choices"][0]["finish_reason"]=="stop" and isinstance(r["choices"][0]["message"]["content"],str) and len(r["choices"][0]["message"]["content"])>40'
|
||||||
|
|
||||||
|
# 2. tools-off lane (chat-only request accepted, tool-bearing request refused)
|
||||||
|
mk notools "$TMP/toolsoff.json" "oa-gate tools-off probe: plain chat with no tools offered."
|
||||||
|
code="$(post "$PORT" "$TMP/toolsoff.json" "$TMP/r_toolsoff.json")"
|
||||||
|
check "tools-off: chat-only request -> 200" test "$code" = "200"
|
||||||
|
mk plain "$TMP/toolsoff_bad.json" "oa-gate tools-off probe: plain chat with no tools offered."
|
||||||
|
code="$(post "$PORT" "$TMP/toolsoff_bad.json" "$TMP/r_toolsoff_bad.json")"
|
||||||
|
check "tools-off negative: offering tools -> 400 gate_expect" \
|
||||||
|
bash -c "test $code = 400"
|
||||||
|
check "tools-off negative: reason names gate_expect" mk chk "$TMP/r_toolsoff_bad.json" \
|
||||||
|
'r["error"]["code"]=="gate_expect"'
|
||||||
|
|
||||||
|
# 3. dialect-leak rejections (the loud-failure contract)
|
||||||
|
code="$(post "$PORT" "$TMP/plain.json" "$TMP/r_leak_hdr.json" -H 'anthropic-version: 2023-06-01')"
|
||||||
|
check "leak: anthropic-version header -> 400" test "$code" = "400"
|
||||||
|
check "leak: header reason names the leak" mk chk "$TMP/r_leak_hdr.json" \
|
||||||
|
'r["error"]["code"]=="gate_dialect_leak" and "anthropic-version" in r["error"]["message"]'
|
||||||
|
mk mut "$TMP/leak_tools.json" "oa-gate plain probe: explain the fixture topic simply." anth-tools
|
||||||
|
code="$(post "$PORT" "$TMP/leak_tools.json" "$TMP/r_leak_tools.json")"
|
||||||
|
check "leak: input_schema tools -> 400 gate_dialect_leak" bash -c \
|
||||||
|
"test $code = 400"
|
||||||
|
check "leak: input_schema reason" mk chk "$TMP/r_leak_tools.json" \
|
||||||
|
'r["error"]["code"]=="gate_dialect_leak" and "input_schema" in r["error"]["message"]'
|
||||||
|
mk mut "$TMP/leak_sys.json" "oa-gate plain probe: explain the fixture topic simply." top-system
|
||||||
|
code="$(post "$PORT" "$TMP/leak_sys.json" "$TMP/r_leak_sys.json")"
|
||||||
|
check "leak: top-level system -> 400" test "$code" = "400"
|
||||||
|
mk mut "$TMP/leak_block.json" "oa-gate plain probe: explain the fixture topic simply." anth-block
|
||||||
|
code="$(post "$PORT" "$TMP/leak_block.json" "$TMP/r_leak_block.json")"
|
||||||
|
check "leak: Anthropic tool_result content block -> 400" test "$code" = "400"
|
||||||
|
|
||||||
|
# 4. scenario request expectations
|
||||||
|
mk mut "$TMP/no_tc.json" "oa-gate plain probe: explain the fixture topic simply." no-tool-choice
|
||||||
|
code="$(post "$PORT" "$TMP/no_tc.json" "$TMP/r_no_tc.json")"
|
||||||
|
check "expect: missing tool_choice -> 400" test "$code" = "400"
|
||||||
|
mk mut "$TMP/ptc.json" "oa-gate plain probe: explain the fixture topic simply." ptc-true
|
||||||
|
code="$(post "$PORT" "$TMP/ptc.json" "$TMP/r_ptc.json")"
|
||||||
|
check "expect: parallel_tool_calls true -> 400 (ADR-0005 pin)" test "$code" = "400"
|
||||||
|
|
||||||
|
# 5. single tool round-trip
|
||||||
|
ST_PROMPT="oa-gate single tool note: save the fixture note to a file."
|
||||||
|
mk plain "$TMP/st1.json" "$ST_PROMPT"
|
||||||
|
code="$(post "$PORT" "$TMP/st1.json" "$TMP/r_st1.json")"
|
||||||
|
check "single-tool leg1: HTTP 200" test "$code" = "200"
|
||||||
|
check "single-tool leg1: one write_file call, finish tool_calls, string args" mk chk "$TMP/r_st1.json" \
|
||||||
|
'r["choices"][0]["finish_reason"]=="tool_calls" and len(r["choices"][0]["message"]["tool_calls"])==1 and r["choices"][0]["message"]["tool_calls"][0]["type"]=="function" and r["choices"][0]["message"]["tool_calls"][0]["function"]["name"]=="write_file" and isinstance(r["choices"][0]["message"]["tool_calls"][0]["function"]["arguments"],str) and json.loads(r["choices"][0]["message"]["tool_calls"][0]["function"]["arguments"])["path"]=="openai-single-note.md"'
|
||||||
|
mk chain "$TMP/st2.json" "$ST_PROMPT" ok "$TMP/r_st1.json"
|
||||||
|
code="$(post "$PORT" "$TMP/st2.json" "$TMP/r_st2.json")"
|
||||||
|
check "single-tool leg2: echo + tool turn -> 200 final text" test "$code" = "200"
|
||||||
|
check "single-tool leg2: final names the file, finish stop" mk chk "$TMP/r_st2.json" \
|
||||||
|
'r["choices"][0]["finish_reason"]=="stop" and "openai-single-note.md" in r["choices"][0]["message"]["content"]'
|
||||||
|
mk chain "$TMP/st2_no.json" "$ST_PROMPT" no-tool-turn "$TMP/r_st1.json"
|
||||||
|
code="$(post "$PORT" "$TMP/st2_no.json" "$TMP/r_st2_no.json")"
|
||||||
|
check "single-tool negative: echo without tool turn -> 400 gate_pairing" \
|
||||||
|
bash -c "test $code = 400"
|
||||||
|
check "single-tool negative: pairing reason" mk chk "$TMP/r_st2_no.json" \
|
||||||
|
'r["error"]["code"]=="gate_pairing"'
|
||||||
|
mk chain "$TMP/st2_wrong.json" "$ST_PROMPT" wrong-id "$TMP/r_st1.json"
|
||||||
|
code="$(post "$PORT" "$TMP/st2_wrong.json" "$TMP/r_st2_wrong.json")"
|
||||||
|
check "single-tool negative: wrong tool_call_id -> 400" test "$code" = "400"
|
||||||
|
mk chain "$TMP/st2_obj.json" "$ST_PROMPT" object-args "$TMP/r_st1.json"
|
||||||
|
code="$(post "$PORT" "$TMP/st2_obj.json" "$TMP/r_st2_obj.json")"
|
||||||
|
check "single-tool negative: arguments echoed as object -> 400 shape" \
|
||||||
|
bash -c "test $code = 400"
|
||||||
|
check "single-tool negative: shape reason names STRING" mk chk "$TMP/r_st2_obj.json" \
|
||||||
|
'r["error"]["code"]=="gate_tool_call_shape" and "STRING" in r["error"]["message"]'
|
||||||
|
|
||||||
|
# 6. escaping torture (the two-escaper trap, spec section 6)
|
||||||
|
T_PROMPT="oa-gate torture probe: write the escaping torture file."
|
||||||
|
mk plain "$TMP/t1.json" "$T_PROMPT"
|
||||||
|
code="$(post "$PORT" "$TMP/t1.json" "$TMP/r_t1.json")"
|
||||||
|
check "torture leg1: HTTP 200" test "$code" = "200"
|
||||||
|
check "torture leg1: arguments decode to the exact nasty payload" \
|
||||||
|
mk torture "$TMP/r_t1.json" scenarios-openai.json
|
||||||
|
mk chain "$TMP/t2.json" "$T_PROMPT" ok "$TMP/r_t1.json"
|
||||||
|
code="$(post "$PORT" "$TMP/t2.json" "$TMP/r_t2.json")"
|
||||||
|
check "torture leg2: faithful echo -> 200 final" test "$code" = "200"
|
||||||
|
mk chain "$TMP/t2_dbl.json" "$T_PROMPT" double-encode "$TMP/r_t1.json"
|
||||||
|
code="$(post "$PORT" "$TMP/t2_dbl.json" "$TMP/r_t2_dbl.json")"
|
||||||
|
check "torture negative: double-encoded echo -> 400" test "$code" = "400"
|
||||||
|
check "torture negative: reason names the two-escaper trap" mk chk "$TMP/r_t2_dbl.json" \
|
||||||
|
'r["error"]["code"]=="gate_echo_mismatch" and "two-escaper" in r["error"]["message"]'
|
||||||
|
|
||||||
|
# 7. parallel double-call
|
||||||
|
P_PROMPT="oa-gate parallel probe: run the two-write parallel case."
|
||||||
|
mk plain "$TMP/p1.json" "$P_PROMPT"
|
||||||
|
code="$(post "$PORT" "$TMP/p1.json" "$TMP/r_p1.json")"
|
||||||
|
check "parallel leg1: TWO tool_calls, distinct ids" mk chk "$TMP/r_p1.json" \
|
||||||
|
'r["choices"][0]["finish_reason"]=="tool_calls" and len(r["choices"][0]["message"]["tool_calls"])==2 and r["choices"][0]["message"]["tool_calls"][0]["id"]!=r["choices"][0]["message"]["tool_calls"][1]["id"]'
|
||||||
|
mk chain "$TMP/p2.json" "$P_PROMPT" ok "$TMP/r_p1.json"
|
||||||
|
code="$(post "$PORT" "$TMP/p2.json" "$TMP/r_p2.json")"
|
||||||
|
check "parallel leg2: both results -> 200 final" test "$code" = "200"
|
||||||
|
mk chain "$TMP/p2_one.json" "$P_PROMPT" only-first "$TMP/r_p1.json"
|
||||||
|
code="$(post "$PORT" "$TMP/p2_one.json" "$TMP/r_p2_one.json")"
|
||||||
|
check "parallel negative: answering only one call -> 400 pairing" test "$code" = "400"
|
||||||
|
|
||||||
|
# 8. two-round mission (loop continuation + step indexing)
|
||||||
|
M_PROMPT="oa-gate mission probe: run the two-round mission."
|
||||||
|
mk plain "$TMP/m1.json" "$M_PROMPT"
|
||||||
|
code="$(post "$PORT" "$TMP/m1.json" "$TMP/r_m1.json")"
|
||||||
|
check "mission leg1: part-1 tool call" mk chk "$TMP/r_m1.json" \
|
||||||
|
'json.loads(r["choices"][0]["message"]["tool_calls"][0]["function"]["arguments"])["path"]=="mission-part-1.md"'
|
||||||
|
mk chain "$TMP/m2.json" "$M_PROMPT" ok "$TMP/r_m1.json"
|
||||||
|
code="$(post "$PORT" "$TMP/m2.json" "$TMP/r_m2.json")"
|
||||||
|
check "mission leg2: part-2 tool call (step indexed by assistant count)" mk chk "$TMP/r_m2.json" \
|
||||||
|
'r["choices"][0]["finish_reason"]=="tool_calls" and json.loads(r["choices"][0]["message"]["tool_calls"][0]["function"]["arguments"])["path"]=="mission-part-2.md"'
|
||||||
|
mk chain "$TMP/m3.json" "$M_PROMPT" ok "$TMP/r_m1.json" "$TMP/r_m2.json"
|
||||||
|
code="$(post "$PORT" "$TMP/m3.json" "$TMP/r_m3.json")"
|
||||||
|
check "mission leg3: final text, finish stop" mk chk "$TMP/r_m3.json" \
|
||||||
|
'r["choices"][0]["finish_reason"]=="stop" and "Mission complete" in r["choices"][0]["message"]["content"]'
|
||||||
|
mk chain "$TMP/m4.json" "$M_PROMPT" ok "$TMP/r_m1.json" "$TMP/r_m2.json" "$TMP/r_m3.json"
|
||||||
|
code="$(post "$PORT" "$TMP/m4.json" "$TMP/r_m4.json")"
|
||||||
|
check "mission overrun: past-script request -> GATE-SCRIPT-EXHAUSTED" mk chk "$TMP/r_m4.json" \
|
||||||
|
'r["choices"][0]["message"]["content"].startswith("GATE-SCRIPT-EXHAUSTED")'
|
||||||
|
|
||||||
|
# 9. injected API errors (OpenAI error envelope)
|
||||||
|
for want in 400 429 500 503; do
|
||||||
|
case "$want" in
|
||||||
|
400) marker="four hundred";; 429) marker="rate limit";;
|
||||||
|
500) marker="five hundred";; 503) marker="unavailable";;
|
||||||
|
esac
|
||||||
|
mk plain "$TMP/e_$want.json" "oa-gate error $marker: trigger the injected failure."
|
||||||
|
code="$(post "$PORT" "$TMP/e_$want.json" "$TMP/r_e_$want.json")"
|
||||||
|
check "api-error $want: status returned" test "$code" = "$want"
|
||||||
|
check "api-error $want: OpenAI error envelope" mk chk "$TMP/r_e_$want.json" \
|
||||||
|
'isinstance(r["error"]["message"],str) and "gate-injected" in r["error"]["message"] and isinstance(r["error"]["type"],str)'
|
||||||
|
done
|
||||||
|
|
||||||
|
# 10. background (unmatched) request
|
||||||
|
mk plain "$TMP/bg.json" "hello there, just a boot probe with no marker"
|
||||||
|
code="$(post "$PORT" "$TMP/bg.json" "$TMP/r_bg.json")"
|
||||||
|
check "background: unmatched prompt -> benign ok" mk chk "$TMP/r_bg.json" \
|
||||||
|
'r["choices"][0]["message"]["content"]=="ok"'
|
||||||
|
|
||||||
|
# 11. ground-truth log invariants
|
||||||
|
check "ground-truth JSONL log invariants" mk logcheck "$TMP/req.jsonl"
|
||||||
|
|
||||||
|
# 12. production-port refusal
|
||||||
|
rc=0
|
||||||
|
"$PY" stub-openai.py --port 7770 --scenarios scenarios-openai.json \
|
||||||
|
--log "$TMP/never.jsonl" >/dev/null 2>&1 || rc=$?
|
||||||
|
check "refuses production port 7770" test "$rc" -ne 0
|
||||||
|
|
||||||
|
# ---- hostile mode: black-hole ----------------------------------------------
|
||||||
|
BH="$(freeport)"
|
||||||
|
"$PY" stub-openai.py --port "$BH" --log "$TMP/bh.jsonl" --mode black-hole \
|
||||||
|
>/dev/null 2>&1 &
|
||||||
|
PIDS+=($!); disown
|
||||||
|
check "black-hole: healthy" waithealth "$BH"
|
||||||
|
rc=0
|
||||||
|
curl -s -o /dev/null --max-time 3 -H 'content-type: application/json' \
|
||||||
|
--data-binary @"$TMP/plain.json" \
|
||||||
|
"http://127.0.0.1:$BH/v1/chat/completions" || rc=$?
|
||||||
|
check "black-hole: client times out (curl rc 28)" test "$rc" -eq 28
|
||||||
|
check "black-hole: health still answers during the hang" \
|
||||||
|
curl -sf --max-time 2 "http://127.0.0.1:$BH/gate/health"
|
||||||
|
|
||||||
|
# ---- hostile mode: mid-body-drop -------------------------------------------
|
||||||
|
MD="$(freeport)"
|
||||||
|
"$PY" stub-openai.py --port "$MD" --log "$TMP/md.jsonl" --mode mid-body-drop \
|
||||||
|
>/dev/null 2>&1 &
|
||||||
|
PIDS+=($!); disown
|
||||||
|
check "mid-body-drop: healthy" waithealth "$MD"
|
||||||
|
rc=0
|
||||||
|
curl -s --max-time 5 -o "$TMP/half.json" -H 'content-type: application/json' \
|
||||||
|
--data-binary @"$TMP/plain.json" \
|
||||||
|
"http://127.0.0.1:$MD/v1/chat/completions" || rc=$?
|
||||||
|
check "mid-body-drop: transfer fails (curl rc $rc)" test "$rc" -ne 0
|
||||||
|
check "mid-body-drop: partial body is not parseable JSON" mk notjson "$TMP/half.json"
|
||||||
|
|
||||||
|
# ---- hostile mode: tool-pending-forever ------------------------------------
|
||||||
|
TP="$(freeport)"
|
||||||
|
"$PY" stub-openai.py --port "$TP" --log "$TMP/tp.jsonl" \
|
||||||
|
--mode tool-pending-forever >/dev/null 2>&1 &
|
||||||
|
PIDS+=($!); disown
|
||||||
|
check "tool-pending-forever: healthy" waithealth "$TP"
|
||||||
|
for i in 1 2 3; do
|
||||||
|
code="$(post "$TP" "$TMP/plain.json" "$TMP/r_tp$i.json")"
|
||||||
|
check "tool-pending-forever: request $i -> 200" test "$code" = "200"
|
||||||
|
done
|
||||||
|
check "tool-pending-forever: three FRESH tool_calls, distinct ids" \
|
||||||
|
mk pending "$TMP/r_tp1.json" "$TMP/r_tp2.json" "$TMP/r_tp3.json"
|
||||||
|
check "tool-pending-forever: /gate/stats counts 3 chat hits" \
|
||||||
|
bash -c "curl -sf http://127.0.0.1:$TP/gate/stats | grep -q '\"chat_hits\": 3'"
|
||||||
|
|
||||||
|
# ---- summary ----------------------------------------------------------------
|
||||||
|
echo
|
||||||
|
echo "selftest: $PASS passed, $FAIL failed"
|
||||||
|
if [ "$FAIL" -ne 0 ]; then
|
||||||
|
echo "SELFTEST RED"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "SELFTEST GREEN (stub-openai gate scaffolding verified)"
|
||||||
Executable
+652
@@ -0,0 +1,652 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""stub-openai.py - deterministic local stand-in for an OpenAI-format
|
||||||
|
/v1/chat/completions provider, for the soul-openai-tools-v2 gate
|
||||||
|
(docs/specs/SPEC-soul-openai-tools-v2-2026-08-06.md). No API key, no network,
|
||||||
|
no model.
|
||||||
|
|
||||||
|
Sibling of gate9's stub-llm.py (Anthropic dialect, _wt-beta-round9/scripts/
|
||||||
|
gate9/): same scenario mechanism (marker matching, assistant-count step
|
||||||
|
indexing, ground-truth JSONL log, prod-port refusal), different wire.
|
||||||
|
Staging home is tests/gate-openai/ in _wt-openai-tools; folds into
|
||||||
|
scripts/gate9/ after round 9 merges (see README.md).
|
||||||
|
|
||||||
|
WHAT IT DOES
|
||||||
|
* Serves POST /v1/chat/completions on 127.0.0.1 only (OpenAI dialect).
|
||||||
|
* VALIDATES every request - this is the gate's discriminator, built
|
||||||
|
BEFORE the brain-side El code exists so dialect leakage fails loudly:
|
||||||
|
- Anthropic tells are 400 code=gate_dialect_leak: `anthropic-version`
|
||||||
|
header; top-level `system` / `stop_sequences` / `max_tokens_to_sample`
|
||||||
|
/ `anthropic_version`; `input_schema` inside a tool entry; Anthropic
|
||||||
|
content blocks (tool_use / tool_result / server_tool_use / ...).
|
||||||
|
- tools[] must be OpenAI-shaped {type:"function", function:{name,
|
||||||
|
description, parameters}} with unique names -> 400 gate_tools_shape.
|
||||||
|
- assistant tool_calls echoes must be {id, type:"function",
|
||||||
|
function:{name, arguments:<JSON-encoded STRING>}}; a decoded-object
|
||||||
|
`arguments` is a wire bug -> 400 gate_tool_call_shape.
|
||||||
|
- every assistant tool_calls turn must be answered by role:"tool"
|
||||||
|
messages covering EVERY tool_call_id, immediately following;
|
||||||
|
unknown / duplicate / missing ids -> 400 gate_pairing.
|
||||||
|
- echoed `arguments` for gate-issued call ids (call_gate_*) are
|
||||||
|
recomputed from the script and compared after ONE json decode ->
|
||||||
|
400 gate_echo_mismatch. This is the two-escaper-trap discriminator
|
||||||
|
named in the spec's security model (section 6).
|
||||||
|
- scenario-level request expectations from scenarios-openai.json
|
||||||
|
(tools offered, OpenAI-shaped tool_choice, parallel_tool_calls
|
||||||
|
pinned false per ADR-0005) -> 400 gate_expect.
|
||||||
|
* Answers with SCRIPTED responses: plain text (finish_reason "stop"),
|
||||||
|
tool calls (finish_reason "tool_calls", arguments JSON-encoded, incl. a
|
||||||
|
nested-quote/escaping torture payload and a parallel two-call case), and
|
||||||
|
API-error injection (OpenAI error envelope). Scenario is selected by
|
||||||
|
scanning user-message text (newest first) for a registered marker
|
||||||
|
substring; the step index is the number of assistant messages already in
|
||||||
|
the request (stateless replay - resumes index correctly by construction).
|
||||||
|
* Writes a ground-truth JSONL log (--log): one record per request with the
|
||||||
|
validation verdict, matched scenario/step, and exactly which tool calls
|
||||||
|
were delivered. Gate assertions compare the brain's claims against THIS
|
||||||
|
log - truth, not narration.
|
||||||
|
* Unmatched requests (boot probes, awareness chatter) get a benign "ok"
|
||||||
|
text response, logged kind=background, never counted as ground truth.
|
||||||
|
* HOSTILE MODES (--mode) on the same file:
|
||||||
|
black-hole accept + read the request, never respond;
|
||||||
|
mid-body-drop send half a JSON body, then abort the socket;
|
||||||
|
tool-pending-forever every request gets a FRESH tool_call
|
||||||
|
(finish_reason "tool_calls"), forever - tests
|
||||||
|
the agentic loop's iteration cap; count the
|
||||||
|
brain's round-trips via GET /gate/stats.
|
||||||
|
|
||||||
|
usage: stub-openai.py --port P --scenarios scenarios-openai.json \
|
||||||
|
--log requests.jsonl [--mode MODE]
|
||||||
|
Listens on 127.0.0.1 only. Refuses production ports 7770/7779/17779.
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import itertools
|
||||||
|
import json
|
||||||
|
import socket
|
||||||
|
import struct
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import uuid
|
||||||
|
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||||
|
|
||||||
|
STATE = {"scenarios": None, "log_path": None, "lock": threading.Lock(),
|
||||||
|
"seq": 0, "mode": "normal", "chat_hits": 0}
|
||||||
|
_PENDING_SEQ = itertools.count(1)
|
||||||
|
|
||||||
|
ANTHROPIC_TOP_KEYS = ("system", "stop_sequences", "max_tokens_to_sample",
|
||||||
|
"anthropic_version")
|
||||||
|
ANTHROPIC_BLOCK_TYPES = {"tool_use", "tool_result", "server_tool_use",
|
||||||
|
"web_search_tool_result", "thinking",
|
||||||
|
"redacted_thinking"}
|
||||||
|
DEFAULT_EXPECT = {"require_tools": True, "require_tool_choice": True,
|
||||||
|
"parallel_tool_calls": False, "forbid_tools": False}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------- loading ----
|
||||||
|
def load_scenarios(path):
|
||||||
|
cfg = json.load(open(path))
|
||||||
|
defaults = dict(DEFAULT_EXPECT)
|
||||||
|
defaults.update(cfg.get("defaults", {}).get("expect_request", {}))
|
||||||
|
marker_map = [] # (marker_lower, cname, pid)
|
||||||
|
scripts = {} # cname or cname/pid -> expanded script
|
||||||
|
pid_map = {} # pid -> cname (for call_gate_* id -> script lookup)
|
||||||
|
expects = {} # cname -> merged expect_request
|
||||||
|
for cname, cls in cfg["classes"].items():
|
||||||
|
scripts[cname] = expand_script(cls.get("script", []))
|
||||||
|
exp = dict(defaults)
|
||||||
|
exp.update(cls.get("expect_request", {}))
|
||||||
|
expects[cname] = exp
|
||||||
|
for ph in cls["phrasings"]:
|
||||||
|
if ph.get("script") is not None:
|
||||||
|
scripts[cname + "/" + ph["id"]] = expand_script(ph["script"])
|
||||||
|
marker_map.append((ph["marker"].lower(), cname, ph["id"]))
|
||||||
|
pid_map[ph["id"]] = cname
|
||||||
|
return {"cfg": cfg, "marker_map": marker_map, "scripts": scripts,
|
||||||
|
"pid_map": pid_map, "expects": expects}
|
||||||
|
|
||||||
|
|
||||||
|
def expand_script(script):
|
||||||
|
"""Same repeat-expansion contract as gate9's stub-llm.py ({N}/{NN})."""
|
||||||
|
out = []
|
||||||
|
for step in script:
|
||||||
|
if "repeat" in step:
|
||||||
|
for n in range(1, step["repeat"] + 1):
|
||||||
|
t = {k: v for k, v in step.items() if k != "repeat"}
|
||||||
|
out.append(json.loads(json.dumps(t)
|
||||||
|
.replace("{NN}", "%02d" % n)
|
||||||
|
.replace("{N}", str(n))))
|
||||||
|
else:
|
||||||
|
out.append(step)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------------------------------- validation ----
|
||||||
|
def _rej(message, code):
|
||||||
|
return {"status": 400, "message": message, "code": code}
|
||||||
|
|
||||||
|
|
||||||
|
def validate_dialect(headers, req):
|
||||||
|
"""Universal checks - run on EVERY request, scenario-matched or not.
|
||||||
|
Anything Anthropic-shaped on this lane means the brain's translator
|
||||||
|
leaked; the whole point is that it fails loudly, here, with a reason."""
|
||||||
|
if headers.get("anthropic-version"):
|
||||||
|
return _rej("anthropic-version header on the OpenAI lane: this "
|
||||||
|
"request was built by the Anthropic dialect path",
|
||||||
|
"gate_dialect_leak")
|
||||||
|
for k in ANTHROPIC_TOP_KEYS:
|
||||||
|
if k in req:
|
||||||
|
return _rej("top-level `%s` is Anthropic dialect; the OpenAI "
|
||||||
|
"dialect has no such field (system prompt goes in "
|
||||||
|
"messages[0])" % k, "gate_dialect_leak")
|
||||||
|
tools = req.get("tools")
|
||||||
|
if tools is not None:
|
||||||
|
if not isinstance(tools, list):
|
||||||
|
return _rej("`tools` must be an array", "gate_tools_shape")
|
||||||
|
names = []
|
||||||
|
for i, t in enumerate(tools):
|
||||||
|
if not isinstance(t, dict):
|
||||||
|
return _rej("tools[%d] is not an object" % i,
|
||||||
|
"gate_tools_shape")
|
||||||
|
if "input_schema" in t or (isinstance(t.get("function"), dict)
|
||||||
|
and "input_schema" in t["function"]):
|
||||||
|
return _rej("tools[%d] carries `input_schema` (Anthropic "
|
||||||
|
"dialect); OpenAI dialect wants "
|
||||||
|
"function.parameters" % i, "gate_dialect_leak")
|
||||||
|
if t.get("type") != "function":
|
||||||
|
return _rej("tools[%d].type must be \"function\", got %r"
|
||||||
|
% (i, t.get("type")), "gate_tools_shape")
|
||||||
|
fn = t.get("function")
|
||||||
|
if not isinstance(fn, dict):
|
||||||
|
return _rej("tools[%d].function missing" % i,
|
||||||
|
"gate_tools_shape")
|
||||||
|
if not isinstance(fn.get("name"), str) or not fn["name"]:
|
||||||
|
return _rej("tools[%d].function.name missing/empty" % i,
|
||||||
|
"gate_tools_shape")
|
||||||
|
if not isinstance(fn.get("description"), str) or not fn["description"]:
|
||||||
|
return _rej("tools[%d].function.description missing/empty" % i,
|
||||||
|
"gate_tools_shape")
|
||||||
|
if not isinstance(fn.get("parameters"), dict):
|
||||||
|
return _rej("tools[%d].function.parameters missing (JSON "
|
||||||
|
"Schema object expected)" % i, "gate_tools_shape")
|
||||||
|
names.append(fn["name"])
|
||||||
|
if len(names) != len(set(names)):
|
||||||
|
return _rej("tools: tool names must be unique", "gate_tools_shape")
|
||||||
|
msgs = req.get("messages")
|
||||||
|
if not isinstance(msgs, list) or not msgs:
|
||||||
|
return _rej("`messages` must be a non-empty array",
|
||||||
|
"gate_messages_shape")
|
||||||
|
for i, m in enumerate(msgs):
|
||||||
|
if not isinstance(m, dict):
|
||||||
|
return _rej("messages[%d] is not an object" % i,
|
||||||
|
"gate_messages_shape")
|
||||||
|
c = m.get("content")
|
||||||
|
if isinstance(c, list):
|
||||||
|
for j, b in enumerate(c):
|
||||||
|
if isinstance(b, dict) and b.get("type") in ANTHROPIC_BLOCK_TYPES:
|
||||||
|
return _rej("messages[%d].content[%d] is an Anthropic "
|
||||||
|
"`%s` block; the OpenAI dialect uses "
|
||||||
|
"tool_calls / role:\"tool\" messages"
|
||||||
|
% (i, j, b.get("type")), "gate_dialect_leak")
|
||||||
|
if m.get("role") == "tool":
|
||||||
|
if not isinstance(m.get("tool_call_id"), str) or not m["tool_call_id"]:
|
||||||
|
return _rej("messages[%d]: role \"tool\" requires a "
|
||||||
|
"`tool_call_id`" % i, "gate_messages_shape")
|
||||||
|
if "content" not in m:
|
||||||
|
return _rej("messages[%d]: role \"tool\" requires `content`"
|
||||||
|
% i, "gate_messages_shape")
|
||||||
|
if m.get("role") == "assistant" and m.get("tool_calls") is not None:
|
||||||
|
tcs = m["tool_calls"]
|
||||||
|
if not isinstance(tcs, list) or not tcs:
|
||||||
|
return _rej("messages[%d].tool_calls must be a non-empty "
|
||||||
|
"array" % i, "gate_tool_call_shape")
|
||||||
|
for j, tc in enumerate(tcs):
|
||||||
|
if not isinstance(tc, dict) or tc.get("type") != "function":
|
||||||
|
return _rej("messages[%d].tool_calls[%d].type must be "
|
||||||
|
"\"function\"" % (i, j), "gate_tool_call_shape")
|
||||||
|
if not isinstance(tc.get("id"), str) or not tc["id"]:
|
||||||
|
return _rej("messages[%d].tool_calls[%d].id missing"
|
||||||
|
% (i, j), "gate_tool_call_shape")
|
||||||
|
fn = tc.get("function")
|
||||||
|
if not isinstance(fn, dict) or not isinstance(fn.get("name"), str):
|
||||||
|
return _rej("messages[%d].tool_calls[%d].function.name "
|
||||||
|
"missing" % (i, j), "gate_tool_call_shape")
|
||||||
|
if not isinstance(fn.get("arguments"), str):
|
||||||
|
return _rej("messages[%d].tool_calls[%d].function."
|
||||||
|
"arguments must be a JSON-encoded STRING, "
|
||||||
|
"got %s" % (i, j,
|
||||||
|
type(fn.get("arguments")).__name__),
|
||||||
|
"gate_tool_call_shape")
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def validate_pairing(msgs):
|
||||||
|
"""OpenAI pairing rule: every assistant tool_calls turn must be followed
|
||||||
|
immediately by role:"tool" messages answering every tool_call_id."""
|
||||||
|
open_ids, open_at = set(), None
|
||||||
|
for i, m in enumerate(msgs):
|
||||||
|
role = m.get("role")
|
||||||
|
if role == "tool":
|
||||||
|
tid = m.get("tool_call_id")
|
||||||
|
if open_at is None:
|
||||||
|
return _rej("messages[%d]: role \"tool\" message with no "
|
||||||
|
"preceding assistant tool_calls turn "
|
||||||
|
"(tool_call_id=%s)" % (i, tid), "gate_pairing")
|
||||||
|
if tid not in open_ids:
|
||||||
|
return _rej("messages[%d]: tool message answers unknown or "
|
||||||
|
"already-answered tool_call_id %s" % (i, tid),
|
||||||
|
"gate_pairing")
|
||||||
|
open_ids.discard(tid)
|
||||||
|
continue
|
||||||
|
if open_ids:
|
||||||
|
return _rej("messages[%d]: assistant tool_calls not fully "
|
||||||
|
"answered before messages[%d]; missing tool "
|
||||||
|
"responses for: %s" % (open_at, i, sorted(open_ids)),
|
||||||
|
"gate_pairing")
|
||||||
|
open_ids, open_at = set(), None
|
||||||
|
if role == "assistant" and m.get("tool_calls"):
|
||||||
|
ids = [tc.get("id") for tc in m["tool_calls"]]
|
||||||
|
open_ids, open_at = set(ids), i
|
||||||
|
if open_ids:
|
||||||
|
return _rej("messages[%d]: assistant tool_calls at end of thread "
|
||||||
|
"without tool responses for: %s"
|
||||||
|
% (open_at, sorted(open_ids)), "gate_pairing")
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def validate_echo_args(msgs, loaded):
|
||||||
|
"""Ground-truth round-trip check: for every echoed gate-issued call id,
|
||||||
|
recompute the arguments this stub originally sent from the script and
|
||||||
|
require one json decode to reproduce them exactly. Catches the
|
||||||
|
two-escaper trap (spec section 6) deterministically."""
|
||||||
|
if not loaded:
|
||||||
|
return None
|
||||||
|
for i, m in enumerate(msgs):
|
||||||
|
if m.get("role") != "assistant":
|
||||||
|
continue
|
||||||
|
for tc in m.get("tool_calls") or []:
|
||||||
|
tid = tc.get("id", "")
|
||||||
|
if not tid.startswith("call_gate_"):
|
||||||
|
continue
|
||||||
|
rest = tid[len("call_gate_"):]
|
||||||
|
try:
|
||||||
|
pid, s_part, k_part = rest.rsplit("_", 2)
|
||||||
|
step_idx, k = int(s_part[1:]), int(k_part)
|
||||||
|
except (ValueError, IndexError):
|
||||||
|
continue
|
||||||
|
cname = loaded["pid_map"].get(pid)
|
||||||
|
if cname is None:
|
||||||
|
continue
|
||||||
|
script = (loaded["scripts"].get(cname + "/" + pid)
|
||||||
|
or loaded["scripts"].get(cname) or [])
|
||||||
|
if step_idx >= len(script):
|
||||||
|
continue
|
||||||
|
calls = script[step_idx].get("tool_calls") or []
|
||||||
|
if k >= len(calls):
|
||||||
|
continue
|
||||||
|
expected = calls[k]
|
||||||
|
fn = tc.get("function") or {}
|
||||||
|
if fn.get("name") != expected["name"]:
|
||||||
|
return _rej("messages[%d]: echoed tool name %r != issued %r "
|
||||||
|
"for %s" % (i, fn.get("name"), expected["name"],
|
||||||
|
tid), "gate_echo_mismatch")
|
||||||
|
try:
|
||||||
|
got = json.loads(fn.get("arguments", ""))
|
||||||
|
except ValueError:
|
||||||
|
return _rej("messages[%d]: echoed arguments for %s are not "
|
||||||
|
"valid JSON after one decode (truncated or "
|
||||||
|
"half-escaped?)" % (i, tid), "gate_echo_mismatch")
|
||||||
|
if got != expected["arguments"]:
|
||||||
|
hint = (" (decoded to a string, not an object: "
|
||||||
|
"double-encoded - the two-escaper trap)"
|
||||||
|
if isinstance(got, str) else "")
|
||||||
|
return _rej("messages[%d]: echoed arguments for %s do not "
|
||||||
|
"round-trip to the issued payload%s"
|
||||||
|
% (i, tid, hint), "gate_echo_mismatch")
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def validate_expect(req, exp):
|
||||||
|
"""Scenario-level request expectations (scenarios-openai.json)."""
|
||||||
|
tools = req.get("tools") or []
|
||||||
|
if exp.get("forbid_tools") and tools:
|
||||||
|
return _rej("this scenario is chat-only: no `tools` may be offered "
|
||||||
|
"on it", "gate_expect")
|
||||||
|
if exp.get("require_tools") and not tools:
|
||||||
|
return _rej("scenario expects a `tools` array to be offered (the "
|
||||||
|
"agentic lane must advertise its tools)", "gate_expect")
|
||||||
|
if exp.get("require_tool_choice"):
|
||||||
|
tc = req.get("tool_choice")
|
||||||
|
ok = tc in ("auto", "none", "required") or (
|
||||||
|
isinstance(tc, dict) and tc.get("type") == "function"
|
||||||
|
and isinstance(tc.get("function"), dict)
|
||||||
|
and tc["function"].get("name"))
|
||||||
|
if not ok:
|
||||||
|
return _rej("scenario expects an OpenAI-shaped `tool_choice`, "
|
||||||
|
"got %r" % (tc,), "gate_expect")
|
||||||
|
want_ptc = exp.get("parallel_tool_calls", None)
|
||||||
|
if want_ptc is not None:
|
||||||
|
if "parallel_tool_calls" not in req:
|
||||||
|
return _rej("scenario expects explicit `parallel_tool_calls` "
|
||||||
|
"(ADR-0005: must be pinned false on the wire)",
|
||||||
|
"gate_expect")
|
||||||
|
if req["parallel_tool_calls"] != want_ptc:
|
||||||
|
return _rej("scenario expects parallel_tool_calls=%s, got %s"
|
||||||
|
% (json.dumps(want_ptc),
|
||||||
|
json.dumps(req["parallel_tool_calls"])),
|
||||||
|
"gate_expect")
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------- scenario match ----
|
||||||
|
def extract_user_texts_newest_first(msgs):
|
||||||
|
texts = []
|
||||||
|
for m in reversed(msgs):
|
||||||
|
if not isinstance(m, dict) or m.get("role") != "user":
|
||||||
|
continue
|
||||||
|
c = m.get("content")
|
||||||
|
if isinstance(c, str):
|
||||||
|
texts.append(c)
|
||||||
|
elif isinstance(c, list):
|
||||||
|
for b in c:
|
||||||
|
if isinstance(b, dict) and b.get("type") == "text":
|
||||||
|
texts.append(b.get("text", ""))
|
||||||
|
return texts
|
||||||
|
|
||||||
|
|
||||||
|
def match_scenario(loaded, msgs):
|
||||||
|
for text in extract_user_texts_newest_first(msgs):
|
||||||
|
tl = text.lower()
|
||||||
|
for marker, cname, pid in loaded["marker_map"]:
|
||||||
|
if marker in tl:
|
||||||
|
return cname, pid
|
||||||
|
return None, None
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------------------------------- rendering ----
|
||||||
|
def completion_envelope(msg, finish, model, usage=(100, 100)):
|
||||||
|
return {"id": "chatcmpl-gate-" + uuid.uuid4().hex[:12],
|
||||||
|
"object": "chat.completion", "created": int(time.time()),
|
||||||
|
"model": model,
|
||||||
|
"choices": [{"index": 0, "message": msg,
|
||||||
|
"finish_reason": finish, "logprobs": None}],
|
||||||
|
"usage": {"prompt_tokens": usage[0],
|
||||||
|
"completion_tokens": usage[1],
|
||||||
|
"total_tokens": usage[0] + usage[1]}}
|
||||||
|
|
||||||
|
|
||||||
|
def text_completion(text, model):
|
||||||
|
return completion_envelope({"role": "assistant", "content": text},
|
||||||
|
"stop", model, usage=(1, 1))
|
||||||
|
|
||||||
|
|
||||||
|
def pending_body(seq, model):
|
||||||
|
args = json.dumps({"path": "never-%04d.md" % seq,
|
||||||
|
"content": "this run never completes"})
|
||||||
|
msg = {"role": "assistant", "content": None,
|
||||||
|
"tool_calls": [{"id": "call_hostile_pending_%04d" % seq,
|
||||||
|
"type": "function",
|
||||||
|
"function": {"name": "write_file",
|
||||||
|
"arguments": args}}]}
|
||||||
|
return completion_envelope(msg, "tool_calls", model, usage=(1, 1))
|
||||||
|
|
||||||
|
|
||||||
|
def render_step(step, cname, pid, step_idx, model):
|
||||||
|
"""Returns (http_status, body_dict, delivered) - delivered is ground
|
||||||
|
truth for the JSONL log."""
|
||||||
|
delivered = {"tool_calls": [], "finish_reason": None, "api_error": None}
|
||||||
|
if "api_error" in step:
|
||||||
|
e = step["api_error"]
|
||||||
|
delivered["api_error"] = e["status"]
|
||||||
|
return (e["status"],
|
||||||
|
{"error": {"message": e["message"],
|
||||||
|
"type": e.get("type", "server_error"),
|
||||||
|
"param": None, "code": e.get("code")}},
|
||||||
|
delivered)
|
||||||
|
msg = {"role": "assistant"}
|
||||||
|
finish = "stop"
|
||||||
|
if step.get("tool_calls"):
|
||||||
|
tcs = []
|
||||||
|
for k, call in enumerate(step["tool_calls"]):
|
||||||
|
tid = "call_gate_%s_s%d_%d" % (pid, step_idx, k)
|
||||||
|
tcs.append({"id": tid, "type": "function",
|
||||||
|
"function": {"name": call["name"],
|
||||||
|
"arguments": json.dumps(
|
||||||
|
call["arguments"],
|
||||||
|
ensure_ascii=False)}})
|
||||||
|
delivered["tool_calls"].append(call["name"])
|
||||||
|
msg["tool_calls"] = tcs
|
||||||
|
msg["content"] = step.get("text") # null when no narration, like real
|
||||||
|
finish = "tool_calls"
|
||||||
|
else:
|
||||||
|
msg["content"] = step["text"]
|
||||||
|
delivered["finish_reason"] = finish
|
||||||
|
return 200, completion_envelope(msg, finish, model), delivered
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ log ------
|
||||||
|
def log_record(rec):
|
||||||
|
with STATE["lock"]:
|
||||||
|
STATE["seq"] += 1
|
||||||
|
rec["seq"] = STATE["seq"]
|
||||||
|
with open(STATE["log_path"], "a") as f:
|
||||||
|
f.write(json.dumps(rec, ensure_ascii=False) + "\n")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------- server -----
|
||||||
|
class Handler(BaseHTTPRequestHandler):
|
||||||
|
protocol_version = "HTTP/1.1"
|
||||||
|
|
||||||
|
def _send_json(self, status, obj):
|
||||||
|
body = json.dumps(obj, ensure_ascii=False).encode("utf-8")
|
||||||
|
self.send_response(status)
|
||||||
|
self.send_header("Content-Type", "application/json")
|
||||||
|
self.send_header("Content-Length", str(len(body)))
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(body)
|
||||||
|
|
||||||
|
def _send_error(self, verdict):
|
||||||
|
self._send_json(verdict["status"],
|
||||||
|
{"error": {"message": verdict["message"],
|
||||||
|
"type": "invalid_request_error",
|
||||||
|
"param": None, "code": verdict["code"]}})
|
||||||
|
|
||||||
|
def _drop_mid_body(self):
|
||||||
|
"""Valid 200 headers, half the promised body, then a socket abort
|
||||||
|
(same SO_LINGER teardown as gate9's mid-body-drop-brain.py)."""
|
||||||
|
full = json.dumps(text_completion(
|
||||||
|
"This reply will never finish arriving because the connection "
|
||||||
|
"dies in the middle of the body, which is exactly the point of "
|
||||||
|
"this hostile fixture.", "hostile-mid-drop")).encode("utf-8")
|
||||||
|
half = full[: len(full) // 2]
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header("Content-Type", "application/json")
|
||||||
|
self.send_header("Content-Length", str(len(full))) # promises more
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(half)
|
||||||
|
self.wfile.flush()
|
||||||
|
try:
|
||||||
|
self.connection.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER,
|
||||||
|
struct.pack("ii", 1, 0))
|
||||||
|
self.connection.shutdown(socket.SHUT_RDWR)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
self.close_connection = True
|
||||||
|
|
||||||
|
def do_GET(self):
|
||||||
|
path = self.path.split("?")[0]
|
||||||
|
if path == "/gate/health":
|
||||||
|
self._send_json(200, {"ok": True, "mode": STATE["mode"]})
|
||||||
|
elif path == "/gate/stats":
|
||||||
|
with STATE["lock"]:
|
||||||
|
self._send_json(200, {"mode": STATE["mode"],
|
||||||
|
"chat_hits": STATE["chat_hits"]})
|
||||||
|
else:
|
||||||
|
self._send_json(404, {"error": {"message": "not found",
|
||||||
|
"type": "invalid_request_error",
|
||||||
|
"param": None,
|
||||||
|
"code": "unknown_route"}})
|
||||||
|
|
||||||
|
def do_POST(self):
|
||||||
|
n = int(self.headers.get("Content-Length") or 0)
|
||||||
|
raw = self.rfile.read(n)
|
||||||
|
mode = STATE["mode"]
|
||||||
|
rec = {"ts": time.time(), "path": self.path, "mode": mode,
|
||||||
|
"kind": "background", "scenario_class": None, "phrasing": None,
|
||||||
|
"step": None, "n_messages": 0, "n_assistant": 0,
|
||||||
|
"validation": "ok", "validation_detail": None,
|
||||||
|
"delivered": {"tool_calls": [], "finish_reason": None,
|
||||||
|
"api_error": None},
|
||||||
|
"http_status": 200}
|
||||||
|
if self.path.split("?")[0] != "/v1/chat/completions":
|
||||||
|
rec.update(kind="wrong_path", http_status=404)
|
||||||
|
log_record(rec)
|
||||||
|
self._send_json(404, {"error": {
|
||||||
|
"message": "no such route: %s" % self.path,
|
||||||
|
"type": "invalid_request_error", "param": None,
|
||||||
|
"code": "unknown_route"}})
|
||||||
|
return
|
||||||
|
with STATE["lock"]:
|
||||||
|
STATE["chat_hits"] += 1
|
||||||
|
|
||||||
|
# ---- hostile modes: behavior first, no validation ----------------
|
||||||
|
if mode == "black-hole":
|
||||||
|
rec.update(kind="hostile", http_status=None)
|
||||||
|
log_record(rec)
|
||||||
|
threading.Event().wait() # hold the socket open forever
|
||||||
|
return
|
||||||
|
if mode == "mid-body-drop":
|
||||||
|
rec.update(kind="hostile", http_status=200)
|
||||||
|
log_record(rec)
|
||||||
|
self._drop_mid_body()
|
||||||
|
return
|
||||||
|
if mode == "tool-pending-forever":
|
||||||
|
seq = next(_PENDING_SEQ)
|
||||||
|
rec.update(kind="hostile",
|
||||||
|
delivered={"tool_calls": ["write_file"],
|
||||||
|
"finish_reason": "tool_calls",
|
||||||
|
"api_error": None})
|
||||||
|
log_record(rec)
|
||||||
|
self._send_json(200, pending_body(seq, "gate-openai-model"))
|
||||||
|
return
|
||||||
|
|
||||||
|
# ---- normal mode -------------------------------------------------
|
||||||
|
try:
|
||||||
|
req = json.loads(raw)
|
||||||
|
except ValueError as exc:
|
||||||
|
# DIAGNOSTIC CAPTURE (2026-08-06): an unparseable body used to be recorded as
|
||||||
|
# a bare "bad_json" with the bytes thrown away, which made an intermittent
|
||||||
|
# failure impossible to root-cause — you cannot fix what you did not keep.
|
||||||
|
# Dump the raw body next to the log, and record exactly where the parser gave
|
||||||
|
# up plus the offending byte, so one occurrence is enough to diagnose.
|
||||||
|
dump_path = "%s.badbody.%s" % (STATE.get("log_path", "/tmp/stub-openai"),
|
||||||
|
rec.get("seq", "x"))
|
||||||
|
try:
|
||||||
|
data = raw if isinstance(raw, (bytes, bytearray)) else str(raw).encode()
|
||||||
|
with open(dump_path, "wb") as fh:
|
||||||
|
fh.write(data)
|
||||||
|
except Exception as dump_exc:
|
||||||
|
dump_path = "(dump failed: %s)" % dump_exc
|
||||||
|
pos = getattr(exc, "pos", None)
|
||||||
|
near = ""
|
||||||
|
byte_repr = ""
|
||||||
|
if isinstance(pos, int):
|
||||||
|
blob = raw if isinstance(raw, (bytes, bytearray)) else str(raw).encode()
|
||||||
|
near = blob[max(0, pos - 60):pos + 60].decode("utf-8", "replace")
|
||||||
|
if 0 <= pos < len(blob):
|
||||||
|
byte_repr = "0x%02x" % blob[pos]
|
||||||
|
rec.update(kind="bad_json", validation="rejected",
|
||||||
|
validation_detail="request body is not valid JSON: %s" % exc,
|
||||||
|
http_status=400, raw_len=len(raw), raw_dump=dump_path,
|
||||||
|
err_pos=pos, err_byte=byte_repr, err_near=near)
|
||||||
|
log_record(rec)
|
||||||
|
self._send_error(_rej("request body is not valid JSON",
|
||||||
|
"bad_json"))
|
||||||
|
return
|
||||||
|
msgs = req.get("messages") or []
|
||||||
|
rec["n_messages"] = len(msgs)
|
||||||
|
rec["n_assistant"] = sum(1 for m in msgs if isinstance(m, dict)
|
||||||
|
and m.get("role") == "assistant")
|
||||||
|
loaded = STATE["scenarios"]
|
||||||
|
cname, pid = match_scenario(loaded, msgs)
|
||||||
|
if cname:
|
||||||
|
rec.update(kind="scenario", scenario_class=cname, phrasing=pid)
|
||||||
|
|
||||||
|
# Wire-level validation runs for EVERY request, scenario or not.
|
||||||
|
verdict = (validate_dialect(self.headers, req)
|
||||||
|
or validate_pairing([m for m in msgs
|
||||||
|
if isinstance(m, dict)])
|
||||||
|
or validate_echo_args(msgs, loaded))
|
||||||
|
if verdict:
|
||||||
|
rec.update(validation="rejected",
|
||||||
|
validation_detail=verdict["message"],
|
||||||
|
http_status=verdict["status"])
|
||||||
|
log_record(rec)
|
||||||
|
self._send_error(verdict)
|
||||||
|
return
|
||||||
|
|
||||||
|
model = req.get("model", "gate-openai-model")
|
||||||
|
if not cname:
|
||||||
|
log_record(rec)
|
||||||
|
self._send_json(200, text_completion("ok", model))
|
||||||
|
return
|
||||||
|
|
||||||
|
script = (loaded["scripts"].get(cname + "/" + pid)
|
||||||
|
or loaded["scripts"][cname])
|
||||||
|
step_idx = rec["n_assistant"]
|
||||||
|
if step_idx >= len(script):
|
||||||
|
rec.update(kind="overrun", step=step_idx)
|
||||||
|
log_record(rec)
|
||||||
|
self._send_json(200, text_completion(
|
||||||
|
"GATE-SCRIPT-EXHAUSTED %s step %d" % (pid, step_idx), model))
|
||||||
|
return
|
||||||
|
|
||||||
|
step = script[step_idx]
|
||||||
|
exp = dict(loaded["expects"][cname])
|
||||||
|
exp.update(step.get("expect_request", {}))
|
||||||
|
verdict = validate_expect(req, exp)
|
||||||
|
if verdict:
|
||||||
|
rec.update(step=step_idx, validation="rejected",
|
||||||
|
validation_detail=verdict["message"],
|
||||||
|
http_status=verdict["status"])
|
||||||
|
log_record(rec)
|
||||||
|
self._send_error(verdict)
|
||||||
|
return
|
||||||
|
|
||||||
|
status, body, delivered = render_step(step, cname, pid, step_idx,
|
||||||
|
model)
|
||||||
|
rec.update(step=step_idx, delivered=delivered, http_status=status)
|
||||||
|
log_record(rec)
|
||||||
|
self._send_json(status, body)
|
||||||
|
|
||||||
|
def log_message(self, *a):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
ap = argparse.ArgumentParser()
|
||||||
|
ap.add_argument("--port", type=int, required=True)
|
||||||
|
ap.add_argument("--scenarios",
|
||||||
|
help="scenarios-openai.json (required in normal mode)")
|
||||||
|
ap.add_argument("--log", required=True)
|
||||||
|
ap.add_argument("--mode", default="normal",
|
||||||
|
choices=["normal", "black-hole", "mid-body-drop",
|
||||||
|
"tool-pending-forever"])
|
||||||
|
args = ap.parse_args()
|
||||||
|
if args.port in (7770, 7779, 17779):
|
||||||
|
raise SystemExit("stub-openai: refusing production Neuron port")
|
||||||
|
if args.mode == "normal" and not args.scenarios:
|
||||||
|
raise SystemExit("stub-openai: --scenarios is required in normal mode")
|
||||||
|
STATE["mode"] = args.mode
|
||||||
|
STATE["scenarios"] = (load_scenarios(args.scenarios)
|
||||||
|
if args.scenarios else None)
|
||||||
|
STATE["log_path"] = args.log
|
||||||
|
open(args.log, "w").close()
|
||||||
|
n_markers = (len(STATE["scenarios"]["marker_map"])
|
||||||
|
if STATE["scenarios"] else 0)
|
||||||
|
print("stub-openai [%s]: 127.0.0.1:%d /v1/chat/completions "
|
||||||
|
"(%d markers registered, log=%s)"
|
||||||
|
% (args.mode, args.port, n_markers, args.log), flush=True)
|
||||||
|
ThreadingHTTPServer(("127.0.0.1", args.port), Handler).serve_forever()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Executable
+148
@@ -0,0 +1,148 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# run-el-test.sh — build and RUN one engine test (tests/*.el), printing its assertions.
|
||||||
|
#
|
||||||
|
# WHY THIS EXISTS (2026-08-06): the engine's tests/*.el files were never runnable from the
|
||||||
|
# tree. `elc` is a COMPILER — it emits C to stdout and exits; it does not execute anything.
|
||||||
|
# So "the tests" could only ever be read, not run, and a signature change could silently
|
||||||
|
# break them (exactly what happened when bridge_save gained its `wire` argument). This
|
||||||
|
# script closes that: emit the test to C, link it against the engine modules, execute it.
|
||||||
|
#
|
||||||
|
# HOW IT WORKS
|
||||||
|
# 1. elc <test>.el -> C on stdout (the test file's `main` + prototypes)
|
||||||
|
# 2. elb (once, cached) -> per-module C for the whole engine into a scratch dir
|
||||||
|
# 3. cc test.c + all modules EXCEPT soul.c (soul.c owns the real `main`) + the runtime
|
||||||
|
# 4. run it
|
||||||
|
#
|
||||||
|
# The test C references only the engine functions it actually calls, so there are no
|
||||||
|
# duplicate-symbol collisions with the module objects.
|
||||||
|
#
|
||||||
|
# RUNTIME: the REPO-PINNED vendor/el-runtime (NOT ~/el-sdk/el_runtime.c — that June build
|
||||||
|
# is missing builtins August code calls: engram_wm_count, engram_wm_top_json,
|
||||||
|
# http_delete_json, http_serve_async; linking against it fails with "symbol(s) not found").
|
||||||
|
#
|
||||||
|
# USAGE
|
||||||
|
# tests/run-el-test.sh tests/test_bridge_serialization.el # one test
|
||||||
|
# tests/run-el-test.sh --all # every tests/test_*.el
|
||||||
|
# REBUILD=1 tests/run-el-test.sh ... # force module regeneration
|
||||||
|
#
|
||||||
|
# Tests that need a live API key / running soul (see each file's header) will report their
|
||||||
|
# own skips or failures — this runner does not fake them.
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||||
|
cd "$REPO_ROOT" || exit 2
|
||||||
|
|
||||||
|
ELC="${ELC:-$HOME/el-sdk/elc}"
|
||||||
|
ELB="${ELB:-$HOME/Development/el-sdk/bin/elb}"
|
||||||
|
RUNTIME_DIR="${RUNTIME_DIR:-$REPO_ROOT/vendor/el-runtime/v1.0.0-20260501}"
|
||||||
|
SCRATCH="${SCRATCH:-/tmp/el-test-$(basename "$REPO_ROOT")}"
|
||||||
|
MODDIR="$SCRATCH/modules"
|
||||||
|
OPENSSL_INC="${OPENSSL_INC:-/opt/homebrew/opt/openssl@3/include}"
|
||||||
|
OPENSSL_LIB="${OPENSSL_LIB:-/opt/homebrew/opt/openssl@3/lib}"
|
||||||
|
|
||||||
|
for req in "$ELC" "$ELB" "$RUNTIME_DIR/el_runtime.c"; do
|
||||||
|
[ -e "$req" ] || { echo "run-el-test: missing required input: $req" >&2; exit 2; }
|
||||||
|
done
|
||||||
|
|
||||||
|
mkdir -p "$MODDIR" || exit 2
|
||||||
|
|
||||||
|
# ── Step 1: engine modules (cached — regeneration is the slow part) ────────────
|
||||||
|
if [ "${REBUILD:-0}" = "1" ] || [ ! -f "$MODDIR/chat.c" ] || [ "chat.el" -nt "$MODDIR/chat.c" ]; then
|
||||||
|
echo "run-el-test: generating engine modules into $MODDIR (this takes ~1-2 min)..."
|
||||||
|
# elb's own final link step fails by design here (it wants to produce a binary named
|
||||||
|
# `neuron` and we only need the per-module .c files it emits first). Ignore its rc.
|
||||||
|
"$ELB" --elc="$ELC" --runtime="$RUNTIME_DIR" --out="$MODDIR/" >"$SCRATCH/elb.log" 2>&1
|
||||||
|
if [ ! -f "$MODDIR/chat.c" ]; then
|
||||||
|
echo "run-el-test: FATAL — elb produced no chat.c; see $SCRATCH/elb.log" >&2
|
||||||
|
tail -5 "$SCRATCH/elb.log" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
# elb rewrites *.elh in the source tree as a side effect (cosmetic banner churn plus a
|
||||||
|
# stray soul..elh). Say so; the caller decides whether to `git restore` them.
|
||||||
|
echo "run-el-test: NOTE — elb regenerated *.elh in the source tree (cosmetic churn is expected; a stray soul..elh may appear)."
|
||||||
|
fi
|
||||||
|
|
||||||
|
# soul.c is needed for its engine functions (layered_cycle et al.) but it also owns the
|
||||||
|
# daemon's real `main`, which would collide with the test's own. Compile it ONCE to an
|
||||||
|
# object with `main` renamed away, and link that instead of the .c.
|
||||||
|
SOUL_OBJ="$SCRATCH/soul-nomain.o"
|
||||||
|
if [ "${REBUILD:-0}" = "1" ] || [ ! -f "$SOUL_OBJ" ] || [ "$MODDIR/soul.c" -nt "$SOUL_OBJ" ]; then
|
||||||
|
cc -std=c11 -O1 -DHAVE_CURL -Dmain=el_soul_daemon_main_unused \
|
||||||
|
-I "$RUNTIME_DIR" -I "$MODDIR" -I "$OPENSSL_INC" \
|
||||||
|
-include dist/elp-c-decls.h -Wno-error=implicit-function-declaration \
|
||||||
|
-c "$MODDIR/soul.c" -o "$SOUL_OBJ" 2>"$SCRATCH/soul-nomain.err" \
|
||||||
|
|| { echo "run-el-test: FATAL — could not compile soul.c without main" >&2
|
||||||
|
grep -E 'error:' "$SCRATCH/soul-nomain.err" | head -5 >&2; exit 2; }
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Every module except soul.c (linked as the renamed object above) and the stray soul.elh.c.
|
||||||
|
MODS=("$SOUL_OBJ")
|
||||||
|
for f in "$MODDIR"/*.c; do
|
||||||
|
case "$(basename "$f")" in
|
||||||
|
soul.c|soul.elh.c) continue ;;
|
||||||
|
esac
|
||||||
|
MODS+=("$f")
|
||||||
|
done
|
||||||
|
[ "${#MODS[@]}" -gt 1 ] || { echo "run-el-test: no module objects found" >&2; exit 2; }
|
||||||
|
|
||||||
|
run_one() {
|
||||||
|
local test_el="$1"
|
||||||
|
local name; name="$(basename "$test_el" .el)"
|
||||||
|
local cfile="$SCRATCH/$name.c"
|
||||||
|
local bin="$SCRATCH/$name"
|
||||||
|
|
||||||
|
printf '\n══ %s ══\n' "$name"
|
||||||
|
|
||||||
|
if ! "$ELC" "$test_el" >"$cfile" 2>"$SCRATCH/$name.elc.err"; then
|
||||||
|
echo "COMPILE FAILED (elc):"; tail -10 "$SCRATCH/$name.elc.err"; return 1
|
||||||
|
fi
|
||||||
|
[ -s "$cfile" ] || { echo "COMPILE FAILED (elc produced empty C)"; return 1; }
|
||||||
|
|
||||||
|
if ! cc -std=c11 -O1 -DHAVE_CURL -rdynamic \
|
||||||
|
-I "$RUNTIME_DIR" -I "$MODDIR" -I "$OPENSSL_INC" -L "$OPENSSL_LIB" \
|
||||||
|
-include dist/elp-c-decls.h -Wno-error=implicit-function-declaration \
|
||||||
|
-o "$bin" "$cfile" "${MODS[@]}" "$RUNTIME_DIR/el_runtime.c" \
|
||||||
|
-lssl -lcrypto -lcurl -lpthread -lm 2>"$SCRATCH/$name.link.err"; then
|
||||||
|
echo "LINK FAILED:"; grep -E '"_|error:' "$SCRATCH/$name.link.err" | head -10; return 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# THE RUNNER OWNS THE VERDICT — the test files cannot be trusted to report it.
|
||||||
|
#
|
||||||
|
# Every tests/*.el assert helper does `let pass_count = pass_count + 1` INSIDE an if
|
||||||
|
# BLOCK. El's scope rule (the same one chat.el documents at every while-body mutation:
|
||||||
|
# "mutations inside if *blocks* don't escape scope") means those counters never
|
||||||
|
# increment, so all 9 counted test files print "N passed, M failed" as "0 passed, 0
|
||||||
|
# failed" — forever, whatever actually happened. A summary that can never report a
|
||||||
|
# failure is worth exactly as much as an assertion that can never fail. Logged as a
|
||||||
|
# bug for the real in-file fix; until then the verdict is computed HERE, from the
|
||||||
|
# assert helpers' own per-line output, which IS reliable.
|
||||||
|
local out="$SCRATCH/$name.out"
|
||||||
|
"$bin" 2>&1 | tee "$out"; local rc=${PIPESTATUS[0]}
|
||||||
|
|
||||||
|
# NOTE: `grep -c` prints 0 AND exits 1 when there are no matches, so a `|| echo 0`
|
||||||
|
# fallback appends a SECOND zero and every later integer test breaks on "0\n0".
|
||||||
|
# (Caught by running this script — which is the whole argument for running things.)
|
||||||
|
local n_pass n_fail
|
||||||
|
n_pass=$(grep -c '^ PASS: ' "$out" 2>/dev/null); n_pass=${n_pass:-0}
|
||||||
|
n_fail=$(grep -c '^ FAIL: ' "$out" 2>/dev/null); n_fail=${n_fail:-0}
|
||||||
|
echo "── $name: $n_pass passed, $n_fail failed (counted by the runner, not by the file's dead counters)"
|
||||||
|
if [ "$n_fail" -gt 0 ]; then
|
||||||
|
echo " failing assertions:"; grep '^ FAIL: ' "$out" | sed 's/^/ /'
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
if [ "$n_pass" -eq 0 ]; then
|
||||||
|
echo " WARNING: no assertions ran — treating as FAILURE (a test that asserts nothing is not a passing test)"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
[ $rc -eq 0 ] || { echo " (test binary exited rc=$rc)"; return 1; }
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
rc_all=0
|
||||||
|
if [ "${1:-}" = "--all" ]; then
|
||||||
|
for t in tests/test_*.el; do run_one "$t" || rc_all=1; done
|
||||||
|
else
|
||||||
|
[ $# -ge 1 ] || { echo "usage: tests/run-el-test.sh <tests/test_x.el> | --all" >&2; exit 2; }
|
||||||
|
for t in "$@"; do run_one "$t" || rc_all=1; done
|
||||||
|
fi
|
||||||
|
exit $rc_all
|
||||||
@@ -93,7 +93,7 @@ println("1. bridge_save — empty messages guard")
|
|||||||
let sid1: String = "test-session-empty-messages"
|
let sid1: String = "test-session-empty-messages"
|
||||||
state_set("mcp_bridge:" + sid1, "")
|
state_set("mcp_bridge:" + sid1, "")
|
||||||
|
|
||||||
let save1_ok: Bool = bridge_save(sid1, "claude-sonnet-4-5", "sys", "[]", "", "", "call-1")
|
let save1_ok: Bool = bridge_save(sid1, "claude-sonnet-4-5", "sys", "[]", "", "", "call-1", "anthropic")
|
||||||
assert_false("empty messages -> bridge_save returns false", save1_ok)
|
assert_false("empty messages -> bridge_save returns false", save1_ok)
|
||||||
|
|
||||||
let saved1: String = state_get("mcp_bridge:" + sid1)
|
let saved1: String = state_get("mcp_bridge:" + sid1)
|
||||||
@@ -107,7 +107,7 @@ println("2. bridge_save — empty tools_json guard")
|
|||||||
let sid2: String = "test-session-empty-tools"
|
let sid2: String = "test-session-empty-tools"
|
||||||
state_set("mcp_bridge:" + sid2, "")
|
state_set("mcp_bridge:" + sid2, "")
|
||||||
|
|
||||||
let save2_ok: Bool = bridge_save(sid2, "claude-sonnet-4-5", "sys", "", "[{\"role\":\"user\",\"content\":\"hi\"}]", "", "call-2")
|
let save2_ok: Bool = bridge_save(sid2, "claude-sonnet-4-5", "sys", "", "[{\"role\":\"user\",\"content\":\"hi\"}]", "", "call-2", "anthropic")
|
||||||
assert_false("empty tools_json -> bridge_save returns false", save2_ok)
|
assert_false("empty tools_json -> bridge_save returns false", save2_ok)
|
||||||
|
|
||||||
let saved2: String = state_get("mcp_bridge:" + sid2)
|
let saved2: String = state_get("mcp_bridge:" + sid2)
|
||||||
@@ -126,7 +126,7 @@ state_set("mcp_bridge:" + sid3, "")
|
|||||||
|
|
||||||
let msgs3: String = "[{\"role\":\"user\",\"content\":\"hello\"}]"
|
let msgs3: String = "[{\"role\":\"user\",\"content\":\"hello\"}]"
|
||||||
let tools3: String = "[{\"name\":\"read_file\"}]"
|
let tools3: String = "[{\"name\":\"read_file\"}]"
|
||||||
let save3_ok: Bool = bridge_save(sid3, "claude-sonnet-4-5", "You are a helper.", tools3, msgs3, "read_file", "toolu_abc")
|
let save3_ok: Bool = bridge_save(sid3, "claude-sonnet-4-5", "You are a helper.", tools3, msgs3, "read_file", "toolu_abc", "anthropic")
|
||||||
assert_true("valid args -> bridge_save returns true", save3_ok)
|
assert_true("valid args -> bridge_save returns true", save3_ok)
|
||||||
|
|
||||||
let blob3: String = state_get("mcp_bridge:" + sid3)
|
let blob3: String = state_get("mcp_bridge:" + sid3)
|
||||||
@@ -243,7 +243,7 @@ state_set("mcp_bridge:" + sid8, "")
|
|||||||
let special_id: String = "toolu_test\"quoted\""
|
let special_id: String = "toolu_test\"quoted\""
|
||||||
let msgs8: String = "[{\"role\":\"user\",\"content\":\"hi\"}]"
|
let msgs8: String = "[{\"role\":\"user\",\"content\":\"hi\"}]"
|
||||||
let tools8: String = "[{\"name\":\"read_file\"}]"
|
let tools8: String = "[{\"name\":\"read_file\"}]"
|
||||||
let save8_ok: Bool = bridge_save(sid8, "claude-sonnet-4-5", "sys", tools8, msgs8, "", special_id)
|
let save8_ok: Bool = bridge_save(sid8, "claude-sonnet-4-5", "sys", tools8, msgs8, "", special_id, "anthropic")
|
||||||
assert_true("special chars in tool_use_id -> bridge_save returns true", save8_ok)
|
assert_true("special chars in tool_use_id -> bridge_save returns true", save8_ok)
|
||||||
|
|
||||||
let blob8: String = state_get("mcp_bridge:" + sid8)
|
let blob8: String = state_get("mcp_bridge:" + sid8)
|
||||||
@@ -251,6 +251,111 @@ let blob8: String = state_get("mcp_bridge:" + sid8)
|
|||||||
let retrieved_id: String = json_get(blob8, "tool_use_id")
|
let retrieved_id: String = json_get(blob8, "tool_use_id")
|
||||||
assert_eq("tool_use_id with quotes round-trips via json_safe", retrieved_id, special_id)
|
assert_eq("tool_use_id with quotes round-trips via json_safe", retrieved_id, special_id)
|
||||||
|
|
||||||
|
// ── Section 9: the "wire" field (OpenAI-tools port, 2026-08-06) ───────────────
|
||||||
|
//
|
||||||
|
// A suspended turn must resume on the SAME wire format it suspended on: an OpenAI-lane
|
||||||
|
// bridge answered with an Anthropic-shaped tool_result (or vice versa) is a dead run.
|
||||||
|
// bridge_save therefore stamps the blob with "wire", and agentic_resume branches on it.
|
||||||
|
//
|
||||||
|
// §9c is the important one. json_get is a first-substring-match scanner, so any key that
|
||||||
|
// appears inside the UNESCAPED conversation embedded in messages_raw can be matched
|
||||||
|
// instead of the blob's own field — that exact class of bug produced the round-9 resume
|
||||||
|
// failure (json_get(blob,"tool_use_id") matching a web_search_tool_result's id inside the
|
||||||
|
// replayed conversation). "wire" is written as a json_safe'd SCALAR ahead of both raw
|
||||||
|
// fields precisely so a decoy in model-controlled bytes can never win. This test plants
|
||||||
|
// that decoy on purpose. If someone later moves the field after messages_raw, this fails.
|
||||||
|
|
||||||
|
println("")
|
||||||
|
println("9. bridge_save — wire tagging and its field-order guarantee")
|
||||||
|
|
||||||
|
// 9a. an OpenAI-lane suspension round-trips as "openai"
|
||||||
|
let sid9: String = "test-session-wire-openai"
|
||||||
|
state_set("mcp_bridge:" + sid9, "")
|
||||||
|
let msgs9: String = "[{\"role\":\"user\",\"content\":\"hi\"}]"
|
||||||
|
let tools9: String = "[{\"name\":\"read_file\"}]"
|
||||||
|
let save9_ok: Bool = bridge_save(sid9, "llama-3.3-70b-versatile", "sys", tools9, msgs9, "", "call_abc", "openai")
|
||||||
|
assert_true("openai wire -> bridge_save returns true", save9_ok)
|
||||||
|
let blob9: String = state_get("mcp_bridge:" + sid9)
|
||||||
|
assert_eq("wire round-trips as openai", json_get(blob9, "wire"), "openai")
|
||||||
|
|
||||||
|
// 9b. an Anthropic-lane suspension round-trips as "anthropic"
|
||||||
|
let sid9b: String = "test-session-wire-anthropic"
|
||||||
|
state_set("mcp_bridge:" + sid9b, "")
|
||||||
|
let save9b_ok: Bool = bridge_save(sid9b, "claude-sonnet-4-5", "sys", tools9, msgs9, "", "toolu_abc", "anthropic")
|
||||||
|
assert_true("anthropic wire -> bridge_save returns true", save9b_ok)
|
||||||
|
let blob9b: String = state_get("mcp_bridge:" + sid9b)
|
||||||
|
assert_eq("wire round-trips as anthropic", json_get(blob9b, "wire"), "anthropic")
|
||||||
|
|
||||||
|
// 9c. FIELD-ORDER GUARD: a decoy "wire" inside the conversation must NOT be matched.
|
||||||
|
let sid9c: String = "test-session-wire-decoy"
|
||||||
|
state_set("mcp_bridge:" + sid9c, "")
|
||||||
|
let msgs9c: String = "[{\"role\":\"user\",\"content\":\"please save this literal text: \\\"wire\\\":\\\"anthropic\\\" end\"}]"
|
||||||
|
let save9c_ok: Bool = bridge_save(sid9c, "llama-3.3-70b-versatile", "sys", tools9, msgs9c, "", "call_decoy", "openai")
|
||||||
|
assert_true("decoy conversation -> bridge_save returns true", save9c_ok)
|
||||||
|
let blob9c: String = state_get("mcp_bridge:" + sid9c)
|
||||||
|
assert_eq("blob's own wire wins over a decoy planted in messages_raw", json_get(blob9c, "wire"), "openai")
|
||||||
|
|
||||||
|
// 9d. LEGACY blob (written before the port) has no wire field: json_get yields "",
|
||||||
|
// which agentic_resume treats as the Anthropic path — old suspensions still resume.
|
||||||
|
let sid9d: String = "test-session-wire-legacy"
|
||||||
|
let legacy_blob: String = "{\"model\":\"claude-sonnet-4-5\",\"safe_sys\":\"sys\",\"tools_log\":\"\""
|
||||||
|
+ ",\"tool_use_id\":\"toolu_legacy\",\"tools_raw\":[{\"name\":\"read_file\"}]"
|
||||||
|
+ ",\"messages_raw\":[{\"role\":\"user\",\"content\":\"hi\"}]}"
|
||||||
|
state_set("mcp_bridge:" + sid9d, legacy_blob)
|
||||||
|
let blob9d: String = state_get("mcp_bridge:" + sid9d)
|
||||||
|
assert_eq("legacy blob has no wire field -> empty (resumes as anthropic)", json_get(blob9d, "wire"), "")
|
||||||
|
assert_eq("legacy blob still reads its tool_use_id", json_get(blob9d, "tool_use_id"), "toolu_legacy")
|
||||||
|
|
||||||
|
// 9e. THE HARDER DECOY: a LEGACY blob (no wire field of its own) that carries the bytes
|
||||||
|
// of a wire tag deeper inside, where an unbounded first-match scan would find it and
|
||||||
|
// misroute the resume onto the wrong loop — the round-9 defect class exactly.
|
||||||
|
//
|
||||||
|
// WHAT IS AND IS NOT REACHABLE (measured here, not assumed — an earlier version of this
|
||||||
|
// test asserted the wrong thing and was corrected by running it):
|
||||||
|
// * NOT reachable from ordinary conversation TEXT. Any quote a user or model writes is
|
||||||
|
// backslash-escaped when it is serialized into the blob, so prose containing
|
||||||
|
// "wire":"openai" is stored as \"wire\":\"openai\" and does not match a scan for the
|
||||||
|
// unescaped key. §9f pins that.
|
||||||
|
// * REACHABLE from STRUCTURAL keys, which are embedded raw. Conversation and tool
|
||||||
|
// objects keep real quotes — that is precisely how round 9's scan found a
|
||||||
|
// web_search_tool_result's tool_use_id. A connector-supplied tool schema or a future
|
||||||
|
// message field literally named "wire" would be found the same way.
|
||||||
|
// The bound removes the whole class rather than reasoning about which keys exist today.
|
||||||
|
let sid9e: String = "test-session-wire-legacy-decoy"
|
||||||
|
let decoy_blob: String = "{\"model\":\"claude-sonnet-4-5\",\"safe_sys\":\"sys\",\"tools_log\":\"\""
|
||||||
|
+ ",\"tool_use_id\":\"toolu_legacy\""
|
||||||
|
+ ",\"tools_raw\":[{\"name\":\"read_file\",\"wire\":\"openai\"}]"
|
||||||
|
+ ",\"messages_raw\":[{\"role\":\"user\",\"content\":\"hi\"}]}"
|
||||||
|
state_set("mcp_bridge:" + sid9e, decoy_blob)
|
||||||
|
let blob9e: String = state_get("mcp_bridge:" + sid9e)
|
||||||
|
|
||||||
|
// Unbounded read (what NOT to do) — proves the hazard this guard exists for is real.
|
||||||
|
assert_eq("unbounded scan DOES find a structural decoy (why the bound is needed)", json_get(blob9e, "wire"), "openai")
|
||||||
|
|
||||||
|
// Bounded read — the same computation agentic_resume performs.
|
||||||
|
let d_traw: Int = str_index_of(blob9e, ",\"tools_raw\":")
|
||||||
|
let d_tjson: Int = str_index_of(blob9e, ",\"tools_json\":")
|
||||||
|
let d_mraw: Int = str_index_of(blob9e, ",\"messages_raw\":")
|
||||||
|
let d_msgs: Int = str_index_of(blob9e, ",\"messages\":")
|
||||||
|
let dcut1: Int = if d_traw > 0 { d_traw } else { str_len(blob9e) }
|
||||||
|
let dcut2: Int = if d_tjson > 0 && d_tjson < dcut1 { d_tjson } else { dcut1 }
|
||||||
|
let dcut3: Int = if d_mraw > 0 && d_mraw < dcut2 { d_mraw } else { dcut2 }
|
||||||
|
let dcut: Int = if d_msgs > 0 && d_msgs < dcut3 { d_msgs } else { dcut3 }
|
||||||
|
let head9e: String = str_slice(blob9e, 0, dcut)
|
||||||
|
assert_eq("bounded scan ignores the decoy -> legacy blob resumes as anthropic", json_get(head9e, "wire"), "")
|
||||||
|
assert_not_contains("scalar head excludes the bulk fields entirely", head9e, "read_file")
|
||||||
|
|
||||||
|
// 9f. Escaping bounds the severity: prose CANNOT inject a scalar-looking key, because
|
||||||
|
// its quotes are escaped on the way in. Documented as a measured fact, so nobody has to
|
||||||
|
// re-derive it the next time this question comes up.
|
||||||
|
let sid9f: String = "test-session-wire-prose"
|
||||||
|
let prose_blob: String = "{\"model\":\"claude-sonnet-4-5\",\"safe_sys\":\"sys\",\"tools_log\":\"\""
|
||||||
|
+ ",\"tool_use_id\":\"toolu_legacy\",\"tools_raw\":[{\"name\":\"read_file\"}]"
|
||||||
|
+ ",\"messages_raw\":[{\"role\":\"user\",\"content\":\"remember this: \\\"wire\\\":\\\"openai\\\"\"}]}"
|
||||||
|
state_set("mcp_bridge:" + sid9f, prose_blob)
|
||||||
|
let blob9f: String = state_get("mcp_bridge:" + sid9f)
|
||||||
|
assert_eq("escaped prose cannot spoof the key even unbounded (severity bound)", json_get(blob9f, "wire"), "")
|
||||||
|
|
||||||
// ── Summary ────────────────────────────────────────────────────────────────────
|
// ── Summary ────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
println("")
|
println("")
|
||||||
|
|||||||
@@ -0,0 +1,92 @@
|
|||||||
|
// tests/test_utf8_slice.el
|
||||||
|
//
|
||||||
|
// Guards utf8_safe_slice(), the fix for a live defect found 2026-08-06:
|
||||||
|
//
|
||||||
|
// The session preload cuts recalled memory content at a fixed length
|
||||||
|
// (chat.el: `if str_len(acc) > 350 { str_slice(acc, 0, 350) }` and
|
||||||
|
// session_preload_bullets' identical per-bullet cut). str_slice and str_len count
|
||||||
|
// BYTES, so any cut landing inside a multi-byte UTF-8 character leaves a dangling
|
||||||
|
// lead byte in the system prompt — and the whole request body is then invalid UTF-8.
|
||||||
|
// Providers reject it outright, so the user sees "AI unavailable" with no clue why,
|
||||||
|
// on both wire formats. Caught by an OpenAI-lane gate whose stub decodes strictly;
|
||||||
|
// reproduced from a real memory whose content contained box-drawing rules (E2 94 80).
|
||||||
|
//
|
||||||
|
// Trigger is ordinary content: an em dash, a curly quote, an accented name, a table
|
||||||
|
// border, an emoji — anything non-ASCII sitting on the cut boundary. It gets MORE
|
||||||
|
// likely as a user's memory grows, which is the opposite of what should happen.
|
||||||
|
//
|
||||||
|
// §1 also pins the semantics this fix depends on: that str_char_code returns the
|
||||||
|
// BYTE value at a byte index (not a decoded code point). If a future runtime changes
|
||||||
|
// that, these assertions fail loudly instead of the truncation silently rotting.
|
||||||
|
|
||||||
|
import "../chat.el"
|
||||||
|
|
||||||
|
let pass_count: Int = 0
|
||||||
|
let fail_count: Int = 0
|
||||||
|
|
||||||
|
fn assert_eq(label: String, got: String, expected: String) -> Void {
|
||||||
|
if str_eq(got, expected) {
|
||||||
|
let pass_count = pass_count + 1
|
||||||
|
println(" PASS: " + label)
|
||||||
|
} else {
|
||||||
|
let fail_count = fail_count + 1
|
||||||
|
println(" FAIL: " + label)
|
||||||
|
println(" got: " + got)
|
||||||
|
println(" expected: " + expected)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn assert_eq_int(label: String, got: Int, expected: Int) -> Void {
|
||||||
|
assert_eq(label, int_to_str(got), int_to_str(expected))
|
||||||
|
}
|
||||||
|
|
||||||
|
println("")
|
||||||
|
println("1. runtime semantics this fix relies on")
|
||||||
|
|
||||||
|
// "─" is U+2500 = E2 94 80 (three bytes). If str_len counts bytes, len("─") is 3.
|
||||||
|
let dash: String = "─"
|
||||||
|
assert_eq_int("str_len counts BYTES (one box-drawing char = 3)", str_len(dash), 3)
|
||||||
|
assert_eq_int("str_char_code returns the BYTE value (lead byte of U+2500 = 0xE2 = 226)", str_char_code(dash, 0), 226)
|
||||||
|
assert_eq_int("str_char_code second byte = 0x94 = 148", str_char_code(dash, 1), 148)
|
||||||
|
assert_eq_int("str_char_code third byte = 0x80 = 128", str_char_code(dash, 2), 128)
|
||||||
|
|
||||||
|
println("")
|
||||||
|
println("2. utf8_safe_slice — never leaves a partial character")
|
||||||
|
|
||||||
|
// Pure ASCII: behaves exactly like str_slice.
|
||||||
|
assert_eq("ascii under the limit is untouched", utf8_safe_slice("hello", 10), "hello")
|
||||||
|
assert_eq("ascii over the limit cuts exactly", utf8_safe_slice("hello world", 5), "hello")
|
||||||
|
|
||||||
|
// A cut landing INSIDE a 3-byte character must drop that character entirely.
|
||||||
|
// "ab─cd": bytes a b E2 94 80 c d. Cutting at 3 or 4 lands mid-dash.
|
||||||
|
let mixed: String = "ab─cd"
|
||||||
|
assert_eq_int("fixture is 7 bytes (2 ascii + 3 + 2 ascii)", str_len(mixed), 7)
|
||||||
|
assert_eq("cut inside the char (n=3) drops the partial char", utf8_safe_slice(mixed, 3), "ab")
|
||||||
|
assert_eq("cut inside the char (n=4) drops the partial char", utf8_safe_slice(mixed, 4), "ab")
|
||||||
|
// A cut landing exactly AFTER a complete character keeps it.
|
||||||
|
assert_eq("cut on the char boundary (n=5) keeps the whole char", utf8_safe_slice(mixed, 5), "ab─")
|
||||||
|
|
||||||
|
// 2-byte character (é = C3 A9) and 4-byte character (😀 = F0 9F 98 80).
|
||||||
|
let acc: String = "xé"
|
||||||
|
assert_eq("cut inside a 2-byte char drops it", utf8_safe_slice(acc, 2), "x")
|
||||||
|
assert_eq("cut after a 2-byte char keeps it", utf8_safe_slice(acc, 3), "xé")
|
||||||
|
let emo: String = "x😀"
|
||||||
|
assert_eq("cut inside a 4-byte char drops it (n=3)", utf8_safe_slice(emo, 3), "x")
|
||||||
|
assert_eq("cut inside a 4-byte char drops it (n=4)", utf8_safe_slice(emo, 4), "x")
|
||||||
|
assert_eq("cut after a 4-byte char keeps it", utf8_safe_slice(emo, 5), "x😀")
|
||||||
|
|
||||||
|
println("")
|
||||||
|
println("3. the real-world shape that produced the bug")
|
||||||
|
|
||||||
|
// A run of box-drawing rules, cut mid-character — the exact captured failure.
|
||||||
|
let rules: String = "──────"
|
||||||
|
assert_eq_int("six box rules = 18 bytes", str_len(rules), 18)
|
||||||
|
// n=16 lands one byte into the sixth character.
|
||||||
|
let cut16: String = utf8_safe_slice(rules, 16)
|
||||||
|
assert_eq_int("cut at 16 backs off to a clean 15-byte boundary", str_len(cut16), 15)
|
||||||
|
// Every byte of the result must belong to a complete character: the last byte of a
|
||||||
|
// well-formed run of these is always 0x80, and 15 is divisible by 3.
|
||||||
|
assert_eq_int("result ends on a complete char (last byte 0x80)", str_char_code(cut16, 14), 128)
|
||||||
|
|
||||||
|
println("")
|
||||||
|
println("test_utf8_slice.el: " + int_to_str(pass_count) + " passed, " + int_to_str(fail_count) + " failed")
|
||||||
Executable
+74
@@ -0,0 +1,74 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# build-soul-from-dist.sh — build a deployable soul from the SAME input CI compiles.
|
||||||
|
#
|
||||||
|
# THE PROBLEM THIS CLOSES: until now, deploys were built by build-soul.sh, which
|
||||||
|
# compiles a scratch amalgam and never touches dist/soul.c. CI compiles dist/soul.c.
|
||||||
|
# Two lineages. On 2026-08-09 the committed input fell 2,761 bytes behind the sources
|
||||||
|
# while three binaries built the other way were installed on the operator machine —
|
||||||
|
# so "what runs" and "what the repo says builds" were different artifacts again,
|
||||||
|
# which is the whole of #133 and #111 wearing new clothes.
|
||||||
|
#
|
||||||
|
# This builds from dist/soul.c with CI's own flags, after asserting that dist/soul.c
|
||||||
|
# actually matches the .el sources, and writes a provenance sidecar so a deployer can
|
||||||
|
# refuse anything of unknown origin.
|
||||||
|
#
|
||||||
|
# -rdynamic and -DHAVE_CURL are copied from .gitea/workflows/ci.yaml deliberately.
|
||||||
|
# The CI comment explains -rdynamic: without it the runtime cannot resolve its HTTP
|
||||||
|
# handler by name via dlsym and the binary serves nothing on every route.
|
||||||
|
#
|
||||||
|
# usage: build-soul-from-dist.sh <out-binary>
|
||||||
|
set -u
|
||||||
|
OUT="${1:?usage: build-soul-from-dist.sh <out-binary>}"
|
||||||
|
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||||
|
RUNTIME="$ROOT/vendor/el-runtime/v1.0.0-20260501"
|
||||||
|
|
||||||
|
cd "$ROOT" || exit 2
|
||||||
|
|
||||||
|
echo "[build-from-dist] GATE: does dist/soul.c match the sources?"
|
||||||
|
if ! ./tools/soulc-stamp.sh --check; then
|
||||||
|
echo "[build-from-dist] REFUSING — the build input is stale. Regenerate and stamp first." >&2
|
||||||
|
exit 9
|
||||||
|
fi
|
||||||
|
|
||||||
|
[ -f "$RUNTIME/el_runtime.c" ] || { echo "pinned runtime missing at $RUNTIME" >&2; exit 2; }
|
||||||
|
|
||||||
|
# macOS: Homebrew's openssl@3 is keg-only (never linked into /usr/local or
|
||||||
|
# /opt/homebrew directly), so cc/ld cannot find -lssl/-lcrypto without an
|
||||||
|
# explicit -L. CI's runner installs libssl-dev system-wide on Ubuntu, so this
|
||||||
|
# branch is a no-op there. Discovered 2026-08-15: a plain build on macOS with
|
||||||
|
# CI's exact flags fails with "ld: library 'ssl' not found" even though the
|
||||||
|
# flags are otherwise correct and CI's own recipe (.gitea/workflows/ci.yaml)
|
||||||
|
# links -lssl -lcrypto -lcurl -lpthread -lm, which this script had drifted
|
||||||
|
# from (it was missing -lssl -lcrypto entirely).
|
||||||
|
SSL_LIBDIR=()
|
||||||
|
if [ "$(uname -s)" = "Darwin" ] && command -v brew >/dev/null 2>&1; then
|
||||||
|
SSL_PREFIX="$(brew --prefix openssl@3 2>/dev/null || true)"
|
||||||
|
[ -n "$SSL_PREFIX" ] && [ -d "$SSL_PREFIX/lib" ] && SSL_LIBDIR=(-L"$SSL_PREFIX/lib")
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "[build-from-dist] compiling dist/soul.c with CI's flags"
|
||||||
|
cc -O2 -DHAVE_CURL -rdynamic -fbracket-depth=1024 \
|
||||||
|
-I"$RUNTIME" \
|
||||||
|
dist/soul.c \
|
||||||
|
"$RUNTIME/el_runtime.c" \
|
||||||
|
"${SSL_LIBDIR[@]}" \
|
||||||
|
-lssl -lcrypto -lcurl -lpthread -lm \
|
||||||
|
-o "$OUT" || { echo "[build-from-dist] COMPILE FAILED" >&2; exit 3; }
|
||||||
|
|
||||||
|
# Provenance sidecar: what a deployer checks before installing anything.
|
||||||
|
SRC_SHA="$(shasum -a 256 dist/soul.c | awk '{print $1}')"
|
||||||
|
STAMP_SHA="$(shasum -a 256 dist/soul.c.stamp | awk '{print $1}')"
|
||||||
|
COMMIT="$(git rev-parse HEAD 2>/dev/null || echo unknown)"
|
||||||
|
DIRTY="clean"; [ -n "$(git status --porcelain -- '*.el' dist/soul.c 2>/dev/null)" ] && DIRTY="DIRTY"
|
||||||
|
cat > "$OUT.provenance" <<EOF
|
||||||
|
{"built_from":"dist/soul.c",
|
||||||
|
"dist_soul_c_sha256":"$SRC_SHA",
|
||||||
|
"stamp_sha256":"$STAMP_SHA",
|
||||||
|
"git_commit":"$COMMIT",
|
||||||
|
"worktree":"$DIRTY",
|
||||||
|
"runtime":"vendor/el-runtime/v1.0.0-20260501",
|
||||||
|
"flags":"-O2 -DHAVE_CURL -rdynamic"}
|
||||||
|
EOF
|
||||||
|
|
||||||
|
echo "[build-from-dist] OK -> $OUT ($(wc -c < "$OUT" | tr -d ' ') bytes)"
|
||||||
|
echo "[build-from-dist] provenance -> $OUT.provenance (commit ${COMMIT:0:8}, worktree $DIRTY)"
|
||||||
Executable
+113
@@ -0,0 +1,113 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# regenerate-soul-amalgam.sh — regenerate dist/soul.c from the current .el
|
||||||
|
# sources, working around three real elc/elb toolchain gotchas found and
|
||||||
|
# root-caused during the 2026-08-15 local-build audit. See AGENTS.md's
|
||||||
|
# "Build / regenerate dist/soul.c" section for the full explanation of each.
|
||||||
|
#
|
||||||
|
# Requires: `elc` (the El compiler, macOS arm64 binary at
|
||||||
|
# foundation/el/lang/dist/platform/elc-darwin-arm64) on $PATH as `elc`.
|
||||||
|
#
|
||||||
|
# Usage:
|
||||||
|
# tools/regenerate-soul-amalgam.sh
|
||||||
|
#
|
||||||
|
# After it succeeds:
|
||||||
|
# tools/soulc-stamp.sh --write
|
||||||
|
# tools/build-soul-from-dist.sh dist/neuron
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||||
|
# soul.el's own `import "../foundation/el/elp/src/elp.el"` assumes neuron and
|
||||||
|
# foundation are siblings. True for a normal checkout; FALSE for a
|
||||||
|
# `git worktree add .worktrees/<name>` checkout (nested one level deeper) —
|
||||||
|
# exactly the layout this audit was run from. Try both before giving up.
|
||||||
|
FOUNDATION="$(cd "$ROOT/../foundation" 2>/dev/null && pwd || true)"
|
||||||
|
if [ -z "$FOUNDATION" ]; then
|
||||||
|
FOUNDATION="$(cd "$ROOT/../../foundation" 2>/dev/null && pwd || true)"
|
||||||
|
fi
|
||||||
|
ELP_SRC="${ELP_SRC:-$FOUNDATION/el/elp/src}"
|
||||||
|
RUNTIME="$ROOT/vendor/el-runtime/v1.0.0-20260501"
|
||||||
|
FLAT="$(mktemp -t flat-soul).el"
|
||||||
|
OUT_C="$(mktemp -t flat-soul-out).c"
|
||||||
|
|
||||||
|
command -v elc >/dev/null 2>&1 || {
|
||||||
|
echo "regenerate-soul-amalgam: 'elc' not found on \$PATH." >&2
|
||||||
|
echo " export PATH=\"\$(dirname <path-to>/elc-darwin-arm64):\$PATH\" (symlinked as 'elc')" >&2
|
||||||
|
exit 2
|
||||||
|
}
|
||||||
|
[ -d "$ELP_SRC" ] || {
|
||||||
|
echo "regenerate-soul-amalgam: elp.el source dir not found at $ELP_SRC" >&2
|
||||||
|
echo " set ELP_SRC=/path/to/foundation/el/elp/src if 'foundation' isn't a sibling of this repo" >&2
|
||||||
|
exit 2
|
||||||
|
}
|
||||||
|
[ -f "$RUNTIME/el_runtime.h" ] || {
|
||||||
|
echo "regenerate-soul-amalgam: pinned runtime missing at $RUNTIME" >&2
|
||||||
|
exit 2
|
||||||
|
}
|
||||||
|
|
||||||
|
# Gotcha #1: stale committed .elh headers silently truncate the build (elc/elb
|
||||||
|
# prefer an existing .elh over recompiling its source, with no warning).
|
||||||
|
echo "[regen] deleting all *.elh in repo root and dist/ (stale-cache gotcha)"
|
||||||
|
find "$ROOT" -maxdepth 2 -iname "*.elh" -delete
|
||||||
|
|
||||||
|
> "$FLAT"
|
||||||
|
BUF_N=0
|
||||||
|
emit_buffer() {
|
||||||
|
# Gotcha #3: elc silently drops the 1-2 top-level fn defs immediately after
|
||||||
|
# any multi-line leading comment block / file-boundary transition when
|
||||||
|
# compiling a flat concatenated file. Two throwaway functions per boundary
|
||||||
|
# absorb the drop; stripped back out of the .c below.
|
||||||
|
BUF_N=$((BUF_N+1)); printf 'fn __amalgam_buf_%d__() -> Int { return 0 }\n' "$BUF_N" >> "$FLAT"
|
||||||
|
BUF_N=$((BUF_N+1)); printf 'fn __amalgam_buf_%d__() -> Int { return 0 }\n' "$BUF_N" >> "$FLAT"
|
||||||
|
echo "" >> "$FLAT"
|
||||||
|
}
|
||||||
|
add_file() {
|
||||||
|
emit_buffer
|
||||||
|
grep -v '^import ' "$1" >> "$FLAT"
|
||||||
|
echo "" >> "$FLAT"
|
||||||
|
}
|
||||||
|
|
||||||
|
emit_buffer
|
||||||
|
# elp.el's own documented dependency order (see its header comment).
|
||||||
|
for f in language-profile.el vocabulary.el morphology.el \
|
||||||
|
morphology-es.el morphology-fr.el morphology-de.el morphology-ru.el \
|
||||||
|
morphology-ja.el morphology-fi.el morphology-ar.el morphology-hi.el \
|
||||||
|
morphology-sw.el morphology-la.el morphology-he.el morphology-grc.el \
|
||||||
|
morphology-ang.el morphology-sa.el morphology-got.el morphology-non.el \
|
||||||
|
morphology-enm.el morphology-pi.el morphology-fro.el morphology-goh.el \
|
||||||
|
morphology-sga.el morphology-txb.el morphology-peo.el morphology-akk.el \
|
||||||
|
morphology-uga.el morphology-egy.el morphology-sux.el morphology-gez.el \
|
||||||
|
morphology-cop.el grammar.el realizer.el semantics.el elp.el; do
|
||||||
|
add_file "$ELP_SRC/$f"
|
||||||
|
done
|
||||||
|
|
||||||
|
# elb's own reported topological order for this repo's 13 soul modules.
|
||||||
|
cd "$ROOT"
|
||||||
|
for f in persist.el memory.el safety.el stewardship.el imprint.el \
|
||||||
|
awareness.el chat.el studio.el elp-input.el neuron-api.el sessions.el \
|
||||||
|
routes.el soul.el; do
|
||||||
|
add_file "$f"
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "[regen] flat source: $FLAT ($(wc -l < "$FLAT" | tr -d ' ') lines)"
|
||||||
|
|
||||||
|
# Gotcha #2: elb cannot produce this repo's single-TU dist/soul.c (it does
|
||||||
|
# per-module separate compilation, which fails on this codebase's
|
||||||
|
# cross-module implicit-declaration style). Use plain elc on the flat file.
|
||||||
|
echo "[regen] compiling with elc against the pinned runtime ($RUNTIME)"
|
||||||
|
CPATH="$RUNTIME" C_INCLUDE_PATH="$RUNTIME" elc "$FLAT" > "$OUT_C"
|
||||||
|
|
||||||
|
echo "[regen] stripping the priming buffer functions back out"
|
||||||
|
python3 - "$OUT_C" "$ROOT/dist/soul.c" << 'PYEOF'
|
||||||
|
import re, sys
|
||||||
|
src, dst = sys.argv[1], sys.argv[2]
|
||||||
|
with open(src) as f:
|
||||||
|
content = f.read()
|
||||||
|
content = re.sub(r'el_val_t __amalgam_buf_\d+__\(void\);\n', '', content)
|
||||||
|
content = re.sub(r'el_val_t __amalgam_buf_\d+__\(void\) \{\n(?:\s*return 0;\n)+\}\n\n', '', content)
|
||||||
|
with open(dst, 'w') as f:
|
||||||
|
f.write(content)
|
||||||
|
PYEOF
|
||||||
|
|
||||||
|
echo "[regen] OK -> dist/soul.c ($(wc -c < "$ROOT/dist/soul.c" | tr -d ' ') bytes)"
|
||||||
|
echo "[regen] next: tools/soulc-stamp.sh --write && tools/build-soul-from-dist.sh dist/neuron"
|
||||||
|
rm -f "$FLAT" "$OUT_C"
|
||||||
@@ -0,0 +1,147 @@
|
|||||||
|
# Retrieval eval harness
|
||||||
|
|
||||||
|
Measures Neuron's memory retrieval so a change can be shown to help before it is
|
||||||
|
believed to help. Nothing else on the memory roadmap should ship without a run
|
||||||
|
through this.
|
||||||
|
|
||||||
|
```
|
||||||
|
tools/retrieval-eval/run_comparison.sh --baseline main --candidate <branch>
|
||||||
|
```
|
||||||
|
|
||||||
|
That builds a soul from each ref, boots each in isolation on a fixed corpus,
|
||||||
|
runs the gold set three times per ref, and prints a table plus a verdict that
|
||||||
|
refuses to call a difference real if it is inside the noise band.
|
||||||
|
|
||||||
|
## What was reused
|
||||||
|
|
||||||
|
This is not a new idea, it is the missing third of an existing one.
|
||||||
|
|
||||||
|
| Prior work | What it gave | What was missing |
|
||||||
|
|---|---|---|
|
||||||
|
| `docs/research/graphrag_eval/` (`collect.py`, `score.py`, 2026-06-08) | The three-retriever comparison that produced the numbers everyone quotes: substring 1.7% P@5, graph 21.7%, BM25 55%. Per-query relevant-id scoring, fixed-denominator precision@5, unique-relevant analysis. | 13 hand-written queries, judged by an LLM after the fact; measured the *live* soul on the *live* engram. |
|
||||||
|
| `docs/research-archive/p0-prototypes/eval_pinned_40q_20260715.py` | The pinned-query discipline: ground truth committed as regexes so every run judges alike, plus a `--check` winnability gate. 40 queries in 5 bands including a deliberate paraphrase-hard band. | Scored offline replicas of substring/BM25 — it never ran the real retrieval path. |
|
||||||
|
| `docs/research-archive/p0-prototypes/stage0_eval_20260714.py` | The `hit@5` metric and the substring/BM25 reference implementations. | Same: offline only. |
|
||||||
|
| `scripts/verify-soul-contract.sh` | The isolation recipe, verbatim: throwaway port, throwaway `HOME`, `SOUL_ENGRAM_PATH`, and the non-obvious `SOUL_ISE_URL` pin that stops an "isolated" soul silently syncing the operator's live brain. | It is a contract gate, not a measurement. |
|
||||||
|
| `_engine-liveness-91/gen-soul-amalgam.sh` + `.gitea/workflows/ci.yaml` | The build recipe (`elc --target=c` with every `.elh` on the import chain removed) and CI's exact compile flags. | — |
|
||||||
|
|
||||||
|
**Reused directly:** the isolation recipe, the build recipe, fixed-denominator
|
||||||
|
precision@5, the pinned-ground-truth and winnability ideas.
|
||||||
|
**New here:** ids rather than regexes as ground truth, an associative category
|
||||||
|
derived from real graph edges, a superseded/contradicted category scored on
|
||||||
|
ranking, a machine-checked zero-lexical-overlap guarantee on paraphrases,
|
||||||
|
paired significance testing, and — the point — measurement against the **real
|
||||||
|
compiled soul** rather than an offline replica of one leg of it.
|
||||||
|
|
||||||
|
## Design fit
|
||||||
|
|
||||||
|
The thing under measurement is Will's designed retrieval: spreading activation
|
||||||
|
over the weighted directed graph, four-factor multiplicative scoring (parent
|
||||||
|
strength x edge weight x target salience x query/target cosine). A Python
|
||||||
|
re-implementation would measure my reading of the design. So the harness
|
||||||
|
compiles the actual `soul.el` amalgam and asks it over HTTP on
|
||||||
|
`/api/neuron/recall`, exactly as the MCP wrapper and the app do.
|
||||||
|
|
||||||
|
## Files
|
||||||
|
|
||||||
|
| File | Does |
|
||||||
|
|---|---|
|
||||||
|
| `build_gold_set.py` | Derives and **validates** the gold set from the corpus. `--check` re-validates and exits non-zero if a query became unwinnable or a paraphrase leaked a word. |
|
||||||
|
| `gold_set.json` | 38 queries. Every one carries a `derivation` string. |
|
||||||
|
| `run_eval.py` | Boots one soul in isolation, runs the gold set, writes metrics. Kills and **confirms dead** its child; records the confirmation in the results file. |
|
||||||
|
| `compare.py` | Paired diff of two result files with McNemar's exact test and a stated noise floor. |
|
||||||
|
| `build-soul.sh` | Compiles a soul binary from a plain source tree. |
|
||||||
|
| `run_comparison.sh` | All of the above, end to end, from two git refs. |
|
||||||
|
|
||||||
|
## The gold set — 38 queries
|
||||||
|
|
||||||
|
Built from the real corpus (`snapshot-pre-repair-20260806.json`, 78,768 nodes /
|
||||||
|
14,214 edges) so it reflects one person's accumulating memory, not document QA.
|
||||||
|
|
||||||
|
| Category | n | Expected answer derived by |
|
||||||
|
|---|---|---|
|
||||||
|
| `exact_rare` | 6 | **Mined.** Tokens with document frequency 1 across all 78,768 nodes, whose single containing node is a 300–6000 char Memory/Knowledge/Belief. That node is the only possible answer. Re-verified every build. |
|
||||||
|
| `phrase` | 7 | **Mined.** Case-insensitive verbatim scan; the matching set *is* the answer key. Phrases matching >25 nodes are rejected as too diffuse. |
|
||||||
|
| `paraphrase` | 13 | **Hand-selected, machine-checked.** Target locked by id; the build then proves that **zero** content words of the query appear anywhere in the target's label, content, or tags. A leak fails the build — the category cannot quietly decay into lexical matching. |
|
||||||
|
| `associative` | 6 | **Derived from edges.** Query built from one value node's distinctive vocabulary; expected answers are its siblings on the `Self - Values (grounded)` hub. Siblings sharing any query word are dropped, so the only route from query to answer is seed -> hub -> sibling. |
|
||||||
|
| `nonsense` | 3 | **Control.** Verified that no token occurs anywhere in the corpus. Correct behaviour is to return nothing. |
|
||||||
|
| `superseded` | 3 | **Derived.** Correction/stale pairs located by regex scan, kept only when both sides resolve to different surviving nodes. Scored on **ranking**: the correction must be returned *and* rank above the stale node. |
|
||||||
|
|
||||||
|
## Metrics
|
||||||
|
|
||||||
|
`hit@5`, `recall@5`, `recall@10`, `precision@5` (fixed denominator 5, so an
|
||||||
|
empty result is punished like a page of junk), `MRR@10`, and wall-clock latency
|
||||||
|
per query (p50/p95/max). Output is a table plus a machine-readable JSON per run
|
||||||
|
so runs can be diffed.
|
||||||
|
|
||||||
|
## Honesty about noise
|
||||||
|
|
||||||
|
- **Minimum detectable swing on this 38-query set: 6 queries.** If every query
|
||||||
|
that changes changes the same way, `p = 2 x 0.5^n`, which first drops under
|
||||||
|
0.05 at n=6. Any net change smaller than that is inside the noise band and
|
||||||
|
`compare.py` says so in those words.
|
||||||
|
- **Run-to-run drift is measured, not assumed.** Activation is a stateful read
|
||||||
|
by design (traversal reinforces what it touches), so identical inputs need not
|
||||||
|
give identical outputs. Observed: `main` 0 queries of drift across 3 runs
|
||||||
|
(fully deterministic); the activation branch 1 query.
|
||||||
|
- The noise floor used for the verdict is `max(6, observed_drift + 1)`.
|
||||||
|
- **This gold set is underpowered for small effects.** A genuine 3-query
|
||||||
|
improvement would not clear the bar. Growing the set is the fix; until then, a
|
||||||
|
small positive delta means "not shown", not "no effect".
|
||||||
|
|
||||||
|
## First result: `main` vs `feat/recall-through-activation`
|
||||||
|
|
||||||
|
Corpus and gold set identical, three runs each, fresh corpus copy per run.
|
||||||
|
|
||||||
|
| | main | recall-through-activation | delta |
|
||||||
|
|---|---|---|---|
|
||||||
|
| hit@5 | 34.3% | 22.9% | **-11.4pp** |
|
||||||
|
| recall@5 | 26.9% | 19.1% | -7.9pp |
|
||||||
|
| recall@10 | 33.3% | 24.3% | -9.1pp |
|
||||||
|
| precision@5 | 12.0% | 7.4% | -4.6pp |
|
||||||
|
| MRR@10 | 0.294 | 0.242 | -0.053 |
|
||||||
|
| latency p50 | 1140 ms | 3209 ms | **2.81x** |
|
||||||
|
| latency p95 | 1584 ms | 4852 ms | 3.06x |
|
||||||
|
| nonsense clean | 2/3 | 2/3 | — |
|
||||||
|
| superseded outranks | 1/3 | 0/3 | -1 |
|
||||||
|
|
||||||
|
By category (hit@5):
|
||||||
|
|
||||||
|
| category | main | activation |
|
||||||
|
|---|---|---|
|
||||||
|
| exact_rare | 100% | 100% |
|
||||||
|
| phrase | 85.7% | **28.6%** |
|
||||||
|
| paraphrase | 0% | 0% |
|
||||||
|
| associative | 0% | 0% |
|
||||||
|
| superseded | 0% | 0% |
|
||||||
|
|
||||||
|
**Verdict: directionally worse, one query short of significant.** 5 discordant
|
||||||
|
pairs, all 5 against the candidate, 0 for it. McNemar exact p = 0.0625 — under
|
||||||
|
the stated rule that is *inside* the noise band, so the harness reports "no
|
||||||
|
measurable difference" on accuracy and the honest summary is "5 for 5 the wrong
|
||||||
|
way, needs a 6th or a larger gold set to call".
|
||||||
|
|
||||||
|
Latency is a different story: 2.8x at p50 is deterministic and far outside any
|
||||||
|
noise band. That regression is real.
|
||||||
|
|
||||||
|
The result the branch was written for did not appear. Its stated purpose was to
|
||||||
|
recover sibling nodes one hub-hop away — the `associative` category — and that
|
||||||
|
category is **0/6 on both builds**. Probing directly: for the query
|
||||||
|
`Marines hernia sepsis medical ward`, the activation build returns the lexical
|
||||||
|
seed node itself at rank 8, and none of its 12 hub siblings anywhere in the top
|
||||||
|
10. The traversal is running; it is not reaching siblings.
|
||||||
|
|
||||||
|
Two corpus facts likely explain it, and both are measurable rather than
|
||||||
|
speculative:
|
||||||
|
|
||||||
|
1. **The graph is nearly edgeless.** Only 4,060 of 78,768 nodes (5.2%) carry any
|
||||||
|
edge at all — 14,214 edges total, 0.18 per node. Spreading activation over a
|
||||||
|
graph with no edges is an expensive way to do lexical matching, which is
|
||||||
|
roughly what the numbers show.
|
||||||
|
2. **No embeddings.** No node in this snapshot has an embedding field, so the
|
||||||
|
fourth factor of the four-factor product — query/target cosine similarity —
|
||||||
|
has nothing to compute from, and the semantic seeding pass is inert.
|
||||||
|
|
||||||
|
That is the harness earning its keep on its first job: the change would have
|
||||||
|
felt like progress (it is the designed mechanism, and it does run) and measures
|
||||||
|
as a regression on phrase queries plus a 2.8x latency cost, with its intended
|
||||||
|
benefit unrealised because the corpus lacks the structure it needs.
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
import numpy as np, json, urllib.request
|
||||||
|
SP="/private/tmp/claude-501/-Users-timlingo/82369039-a20e-4b5a-8a5e-28234a57b996/scratchpad"
|
||||||
|
np.seterr(all='ignore')
|
||||||
|
M=np.load(SP+'/emb.npy'); eids=open(SP+'/ids.txt',encoding='utf-8',errors='surrogateescape').read().split('\n')
|
||||||
|
eidx={k:i for i,k in enumerate(eids)}
|
||||||
|
gold=json.load(open("/Users/timlingo/Development/neuron-technologies/_wt-assoc-leg/tools/retrieval-eval/gold_set.json"))['queries']
|
||||||
|
VALS=sorted({r for q in gold if q['category']=='paraphrase' for r in q['relevant']})
|
||||||
|
VI=[eidx[v] for v in VALS]
|
||||||
|
def emb(t):
|
||||||
|
b=json.dumps({"model":"nomic-embed-text","prompt":t}).encode()
|
||||||
|
r=urllib.request.Request("http://127.0.0.1:11434/api/embeddings",data=b,headers={"Content-Type":"application/json"})
|
||||||
|
v=np.array(json.load(urllib.request.urlopen(r,timeout=60))["embedding"],dtype=np.float32)
|
||||||
|
return v/(np.linalg.norm(v)+1e-9)
|
||||||
|
print("qid cat bestValueNodeGlobalRank goldGlobalRank goldSiblingRank")
|
||||||
|
for q in gold:
|
||||||
|
if q['category']!='paraphrase': continue
|
||||||
|
v=emb(q['query']); s=M@v; s[~np.isfinite(s)]=-1
|
||||||
|
ranks=sorted(int((s>s[j]).sum())+1 for j in VI)
|
||||||
|
g=eidx[q['relevant'][0]]; gr=int((s>s[g]).sum())+1
|
||||||
|
sv=np.array([s[j] for j in VI]); sib=int((sv>s[g]).sum())+1
|
||||||
|
print("%-4s %-11s best=%-5d (top3 val ranks %s) gold=%-5d sib=%d" % (q['id'],q['category'],ranks[0],ranks[:3],gr,sib))
|
||||||
Executable
+44
@@ -0,0 +1,44 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# build-soul.sh — compile a soul binary from a plain source tree (no git needed).
|
||||||
|
#
|
||||||
|
# Reuses the amalgam recipe worked out in gen-soul-amalgam.sh (round 9.1) and the
|
||||||
|
# compile flags from .gitea/workflows/ci.yaml, so the binary under test is the
|
||||||
|
# same translation unit CI ships — not a re-implementation.
|
||||||
|
#
|
||||||
|
# elc --target=c emits only an extern prototype for any module that has a .elh
|
||||||
|
# header beside it, and inlines the module's bodies when it does not. So the
|
||||||
|
# amalgam is produced in a scratch copy with every .elh on the import chain
|
||||||
|
# deleted.
|
||||||
|
#
|
||||||
|
# usage: build-soul.sh <src-tree-with-*.el> <out-binary>
|
||||||
|
set -euo pipefail
|
||||||
|
SRC="${1:?usage: build-soul.sh <src-tree> <out-binary>}"
|
||||||
|
OUT="${2:?out-binary}"
|
||||||
|
ELC="${ELC:-$HOME/neuron-dev-stack/src/el/lang/dist/platform/elc}"
|
||||||
|
EL_REPO="${EL_REPO:-$HOME/Development/neuron-technologies/el}"
|
||||||
|
RTDIR="${RTDIR:-$SRC/vendor/el-runtime/v1.0.0-20260501}"
|
||||||
|
SSL="${SSL_PREFIX:-/opt/homebrew/opt/openssl@3}"
|
||||||
|
|
||||||
|
[ -x "$ELC" ] || { echo "no elc at $ELC" >&2; exit 2; }
|
||||||
|
[ -f "$RTDIR/el_runtime.c" ] || { echo "no el_runtime.c at $RTDIR" >&2; exit 2; }
|
||||||
|
|
||||||
|
GEN="$(mktemp -d "${TMPDIR:-/tmp}/soul-build.XXXXXX")"
|
||||||
|
trap 'rm -rf "$GEN"' EXIT
|
||||||
|
mkdir -p "$GEN/neuron" "$GEN/foundation/el/elp/src"
|
||||||
|
cp "$SRC"/*.el "$GEN/neuron/"
|
||||||
|
cp "$EL_REPO"/elp/src/*.el "$GEN/foundation/el/elp/src/"
|
||||||
|
find "$GEN" -name '*.elh' -delete
|
||||||
|
|
||||||
|
( cd "$GEN/neuron" && "$ELC" --target=c soul.el ) > "$GEN/soul.c"
|
||||||
|
BODIES=$(grep -c '^el_val_t .*) {$' "$GEN/soul.c" || true)
|
||||||
|
echo "[build-soul] amalgam $(wc -c < "$GEN/soul.c" | tr -d ' ') bytes, ${BODIES} inlined bodies"
|
||||||
|
[ "$BODIES" -ge 1200 ] || { echo "[build-soul] FAIL: only $BODIES bodies — an import was not inlined"; exit 1; }
|
||||||
|
|
||||||
|
cc -O2 -DHAVE_CURL -rdynamic \
|
||||||
|
-I"$RTDIR" -I"$SSL/include" -L"$SSL/lib" \
|
||||||
|
"$GEN/soul.c" "$RTDIR/el_runtime.c" \
|
||||||
|
-lssl -lcrypto -lcurl -lpthread -lm \
|
||||||
|
-o "$OUT" 2> "$GEN/cc.log" || { echo "[build-soul] FAIL compile"; tail -40 "$GEN/cc.log"; exit 1; }
|
||||||
|
if grep -qE 'implicit.*(engram_|el_)' "$GEN/cc.log"; then
|
||||||
|
echo "[build-soul] FAIL: implicit declarations of runtime symbols"; grep -E 'implicit' "$GEN/cc.log" | head; exit 1; fi
|
||||||
|
echo "[build-soul] OK -> $OUT ($(wc -c < "$OUT" | tr -d ' ') bytes)"
|
||||||
Executable
+506
@@ -0,0 +1,506 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
build_gold_set.py — derive the retrieval gold set FROM the corpus, and validate it.
|
||||||
|
|
||||||
|
WHY THIS FILE EXISTS AS CODE AND NOT AS A HAND-WRITTEN JSON
|
||||||
|
A gold set nobody can audit is vibes with extra steps. Every expected answer
|
||||||
|
here is either (a) mined from the corpus by a rule this script re-runs, or
|
||||||
|
(b) hand-selected with a stated criterion that this script then CHECKS
|
||||||
|
against the corpus. Both leave a `derivation` string on every query, and the
|
||||||
|
checks are re-run on demand so the set cannot silently rot as the corpus
|
||||||
|
changes.
|
||||||
|
|
||||||
|
Lineage: this extends the pinned-query approach from
|
||||||
|
docs/research-archive/p0-prototypes/eval_pinned_40q_20260715.py (pinned
|
||||||
|
ground-truth patterns + a --check "winnability" gate) and the per-query
|
||||||
|
relevant-id scoring from docs/research/graphrag_eval/score.py. What is new:
|
||||||
|
ids as ground truth rather than regexes alone, an ASSOCIATIVE category
|
||||||
|
derived from real graph edges, a superseded/contradicted category, and a
|
||||||
|
machine-checked no-lexical-overlap guarantee on the paraphrase category.
|
||||||
|
|
||||||
|
THE SIX CATEGORIES, AND WHAT EACH ONE IS FOR
|
||||||
|
exact_rare a single rare word. Substring matching already wins these.
|
||||||
|
They are a REGRESSION GUARD: any change that loses them is
|
||||||
|
disqualified regardless of what else it gains.
|
||||||
|
phrase a multi-word string that exists verbatim in the corpus.
|
||||||
|
Guards multi-token queries, which the old substring matcher
|
||||||
|
handled by returning nothing.
|
||||||
|
paraphrase same meaning, ZERO shared content words with the target node.
|
||||||
|
THE CATEGORY THAT MATTERS. Mechanically unreachable by string
|
||||||
|
matching; reachable only by semantics or by association.
|
||||||
|
associative the answer is one hub-hop from an obvious starting point and
|
||||||
|
shares no words with the query. This is the case the graph is
|
||||||
|
supposed to buy: query one value, get its siblings.
|
||||||
|
nonsense must return nothing. Guards against a retriever that "improves"
|
||||||
|
recall by returning the whole graph.
|
||||||
|
superseded a fact that was later corrected. The correction must OUTRANK
|
||||||
|
the stale version — ranking, not mere presence.
|
||||||
|
|
||||||
|
usage:
|
||||||
|
python3 build_gold_set.py <snapshot.json> [--out gold_set.json] [--check]
|
||||||
|
--check re-validates an existing gold_set.json against the corpus and exits
|
||||||
|
non-zero if any query became unwinnable or any paraphrase leaked a word.
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
from collections import Counter, defaultdict
|
||||||
|
|
||||||
|
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||||
|
DEFAULT_OUT = os.path.join(HERE, "gold_set.json")
|
||||||
|
|
||||||
|
TOKEN = re.compile(r"[a-z0-9][a-z0-9\-']*")
|
||||||
|
|
||||||
|
# Stopwords are deliberately generous. A paraphrase query is only interesting if
|
||||||
|
# its CONTENT words are absent from the target; "the", "is", "what" appearing in
|
||||||
|
# both proves nothing. Being generous here makes the overlap test STRICTER on
|
||||||
|
# the words that carry meaning, which is the conservative direction.
|
||||||
|
STOP = set("""
|
||||||
|
a about above after again against all also am an and any are aren't as at be because been
|
||||||
|
before being below between both but by can can't cannot could couldn't did didn't do does
|
||||||
|
doesn't doing don't down during each few for from further had hadn't has hasn't have haven't
|
||||||
|
having he her here hers herself him himself his how i if in into is isn't it its itself just
|
||||||
|
me more most my myself no nor not of off on once only or other others ought our ours ourselves
|
||||||
|
out over own same shan't she should shouldn't so some such than that the their theirs them
|
||||||
|
themselves then there these they this those through to too under until up very was wasn't we
|
||||||
|
were weren't what when where which while who whom why will with won't would wouldn't you your
|
||||||
|
yours yourself yourselves get gets got make makes made take takes use uses used way ways thing
|
||||||
|
things does doing done keep keeps kept go goes going come comes came one two something anything
|
||||||
|
""".split())
|
||||||
|
|
||||||
|
|
||||||
|
# ─────────────────────────────────────────────────────────────────────────────
|
||||||
|
# corpus helpers
|
||||||
|
# ─────────────────────────────────────────────────────────────────────────────
|
||||||
|
def load_corpus(path):
|
||||||
|
with open(path, encoding="utf-8", errors="replace") as fh:
|
||||||
|
data = json.load(fh)
|
||||||
|
nodes = [n for n in data.get("nodes", []) if isinstance(n, dict) and n.get("id")]
|
||||||
|
edges = [e for e in data.get("edges", []) if isinstance(e, dict)]
|
||||||
|
return nodes, edges
|
||||||
|
|
||||||
|
|
||||||
|
def doctext(n):
|
||||||
|
return " ".join([str(n.get("label") or ""), str(n.get("content") or ""), str(n.get("tags") or "")])
|
||||||
|
|
||||||
|
|
||||||
|
def content_tokens(s):
|
||||||
|
return {t for t in TOKEN.findall(s.lower()) if t not in STOP and len(t) > 2}
|
||||||
|
|
||||||
|
|
||||||
|
# ─────────────────────────────────────────────────────────────────────────────
|
||||||
|
# hand-authored queries. Every entry states HOW its expected answer was chosen.
|
||||||
|
# The `check` field names the validation this script runs against the corpus.
|
||||||
|
# ─────────────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
# EXACT_RARE — mined, not chosen. The rule (re-run by mine_exact_rare below):
|
||||||
|
# tokens whose document frequency across the whole corpus is 1, whose single
|
||||||
|
# containing node is a Memory/Knowledge/Belief with 300-6000 chars of content
|
||||||
|
# (so the answer is a real memory, not a 117KB whitepaper that contains every
|
||||||
|
# word in English), and whose token is plain lowercase alphabetic. The expected
|
||||||
|
# answer is that one node — it is the only node that can possibly be correct.
|
||||||
|
EXACT_RARE_SEEDS = [
|
||||||
|
"unjailbreakable",
|
||||||
|
"engram-migrate",
|
||||||
|
"cartabandonedevent",
|
||||||
|
"pre-apprenticeship",
|
||||||
|
"inferencenodemanager",
|
||||||
|
"clear-eyed",
|
||||||
|
]
|
||||||
|
|
||||||
|
# PHRASE — chosen by reading the corpus for phrases that (a) occur verbatim,
|
||||||
|
# (b) occur in a small enough set of nodes that "relevant" is well defined.
|
||||||
|
# Expected answers are computed here as EVERY node whose text contains the
|
||||||
|
# phrase case-insensitively — so the answer set is a fact about the corpus, not
|
||||||
|
# an opinion. Queries whose phrase matches more than PHRASE_MAX nodes are
|
||||||
|
# rejected by validation as too diffuse to score.
|
||||||
|
PHRASE_MAX = 25
|
||||||
|
PHRASE_SEEDS = [
|
||||||
|
("patterns not returns",
|
||||||
|
"a verbatim correction Will issued; expected = every node containing the phrase"),
|
||||||
|
("thirty moves",
|
||||||
|
"the canonical biographical phrase; expected = every node containing it"),
|
||||||
|
("Grandma Lucas",
|
||||||
|
"a named person appearing verbatim in the biography/value nodes"),
|
||||||
|
("Directed Harmonic",
|
||||||
|
"the canonical DHARMA expansion, confirmed by Will April 24 2026"),
|
||||||
|
("Sarah Bishop",
|
||||||
|
"a named person; rare enough that the answer set is unambiguous"),
|
||||||
|
("Directed Autonomous Runtime Modification",
|
||||||
|
"the DARMA expansion, quoted verbatim in the backlog item and its correction"),
|
||||||
|
("zero-knowledge encrypted backup",
|
||||||
|
"the paid-tier feature name as written in the roadmap nodes"),
|
||||||
|
]
|
||||||
|
|
||||||
|
# PARAPHRASE — hand-authored. THE SELECTION CRITERION, stated once and applied
|
||||||
|
# to all nine: pick a node whose SUBJECT is unmistakable to a reader, then write
|
||||||
|
# the query a person would actually type when they remember the subject but not
|
||||||
|
# the words. The target is then LOCKED by id, and this script enforces the hard
|
||||||
|
# property that makes the category meaningful: not one content word of the query
|
||||||
|
# appears anywhere in the target node's label, content, or tags. If a word
|
||||||
|
# leaks, validation fails and the query must be rewritten — the set cannot
|
||||||
|
# quietly degrade into a lexical query wearing a paraphrase costume.
|
||||||
|
PARAPHRASE_SEEDS = [
|
||||||
|
("kn-a99cefe3-5e83-4050-98d8-6c69f57c7c71",
|
||||||
|
"the elderly relative who passed while he stayed away",
|
||||||
|
"target: 'Value - Do the Essential Thing While You Can', whose subject is Grandma Lucas "
|
||||||
|
"dying in Feb 2006 without Will saying goodbye. Query names the event with none of the "
|
||||||
|
"node's own vocabulary."),
|
||||||
|
("kn-58874a74-b96f-4883-9e08-45707f4bd3ee",
|
||||||
|
"a soldier sidelined by illness who refused to quit",
|
||||||
|
"target: 'Value - Survival Is Not an Excuse to Stop', whose subject is enlisting in the "
|
||||||
|
"Marines, a severe hernia, and sepsis. Query describes the episode obliquely."),
|
||||||
|
("kn-13f60407-7b70-4db1-964f-ea1f8196efbd",
|
||||||
|
"choosing an uncomfortable fact over a pleasant fiction",
|
||||||
|
"target: 'Value - Honesty Before Comfort'. Query states the principle in wholly "
|
||||||
|
"different words."),
|
||||||
|
("kn-22d77abe-b3c5-42fd-afcd-dcb87d924929",
|
||||||
|
"a tight payload beats a bloated one",
|
||||||
|
"target: 'Value - Precision Over Brute Force'. Query restates the claim with no "
|
||||||
|
"shared vocabulary."),
|
||||||
|
("kn-eb1b9e18-3dc6-4b9b-9cc6-86e0ae6b6be8",
|
||||||
|
"if you are able and nobody is coming the job is yours",
|
||||||
|
"target: 'Value - Capability Is a Debt You Owe the Moment'. Query states the "
|
||||||
|
"obligation without the node's terms."),
|
||||||
|
("kn-0bb4f021-56de-4947-a35b-a37209e7ba21",
|
||||||
|
"learning is the wealth creditors cannot seize",
|
||||||
|
"target: 'Value - Knowledge Survives When Nothing Else Does', whose subject is the "
|
||||||
|
"library following Will across 30+ moves."),
|
||||||
|
("kn-5de5a9ac-fd15-45ab-bf18-77566781cf40",
|
||||||
|
"reliability proven by track record not assertion",
|
||||||
|
"target: 'Value - Earned Trust' ('Trust is demonstrated, not declared')."),
|
||||||
|
("kn-a5b3d0ac-f6a1-49a4-aebb-b8b4cd67fe83",
|
||||||
|
"boundaries that enable instead of confine",
|
||||||
|
"target: 'Value - Constraints as Freedom'. Query is a restatement of the same claim."),
|
||||||
|
("kn-78db5396-3dbc-4481-bfc7-e4e1422feb1c",
|
||||||
|
"what shifts tells you where to cut a system apart",
|
||||||
|
"target: 'Value - Change Is the Signal', the value VBD is built on."),
|
||||||
|
("kn-f230b362-b201-4402-9833-4160c89ab3d4",
|
||||||
|
"a mind that compounds instead of resetting each day",
|
||||||
|
"target: 'Value - The System Must Accumulate'. Query is the accumulation claim in "
|
||||||
|
"different vocabulary."),
|
||||||
|
("kn-db9f141b-dbe3-4037-92e0-4bb9be0e5e6e",
|
||||||
|
"loved for the unedited self and not the polished exterior",
|
||||||
|
"target: 'Value - Being Seen Is Rarer Than Being Known', whose subject is Sarah Bishop "
|
||||||
|
"as the first person Will did not perform for."),
|
||||||
|
("kn-e0423482-cfa5-4796-8689-8495c93b66bc",
|
||||||
|
"cheerfulness you arrive at instead of assuming",
|
||||||
|
"target: 'Value - Hope Is a Conclusion'. Query restates 'a conclusion, not a premise'."),
|
||||||
|
("kn-6061318f-046b-4935-907d-8eafdce14930",
|
||||||
|
"a childhood offering no solid foundation to inherit",
|
||||||
|
"target: 'Value - Structure Is Not Inherited', whose subject is thirty moves between "
|
||||||
|
"two parents' collapses."),
|
||||||
|
]
|
||||||
|
|
||||||
|
# ASSOCIATIVE — derived from real edges, not authored. The construction:
|
||||||
|
# every value node hangs off the 'Self - Values (grounded)' hub by an `identity`
|
||||||
|
# edge. For a chosen value node V, the query is built from V's own distinctive
|
||||||
|
# vocabulary; the expected answers are V's SIBLINGS on that hub. A sibling
|
||||||
|
# shares no query words with the query by construction (validated below), so the
|
||||||
|
# only path from the query to a sibling is: lexical seed on V -> hub -> sibling.
|
||||||
|
# That is a two-hop traversal and nothing else can produce it.
|
||||||
|
VALUES_HUB = "kn-5b606390-a52d-4ca2-8e0e-eba141d13440"
|
||||||
|
ASSOCIATIVE_SEEDS = [
|
||||||
|
("kn-a99cefe3-5e83-4050-98d8-6c69f57c7c71", "Grandma Lucas stroke February 2006 goodbye window"),
|
||||||
|
("kn-58874a74-b96f-4883-9e08-45707f4bd3ee", "Marines hernia sepsis medical ward"),
|
||||||
|
("kn-db9f141b-dbe3-4037-92e0-4bb9be0e5e6e", "Sarah Bishop Dyer trailer performance"),
|
||||||
|
("kn-a5b3d0ac-f6a1-49a4-aebb-b8b4cd67fe83", "Swarm Architecture containment lateral worker"),
|
||||||
|
("kn-e0423482-cfa5-4796-8689-8495c93b66bc", "hope won inside the narrative preface"),
|
||||||
|
("kn-eb1b9e18-3dc6-4b9b-9cc6-86e0ae6b6be8", "man of the house six years old expectation"),
|
||||||
|
]
|
||||||
|
|
||||||
|
# NONSENSE — must return nothing. Strings chosen to be lexically impossible:
|
||||||
|
# validation asserts each appears in ZERO corpus nodes as a substring and that
|
||||||
|
# none of its tokens appears anywhere either (so not even a partial seed exists).
|
||||||
|
NONSENSE_SEEDS = [
|
||||||
|
"zqxjvw plimforth grebulon",
|
||||||
|
"flarnbistle quommetry",
|
||||||
|
"xxqzzt vurblenacht throom",
|
||||||
|
]
|
||||||
|
|
||||||
|
# SUPERSEDED — a fact that was corrected. Chosen by searching the corpus for
|
||||||
|
# explicit correction language and keeping pairs where BOTH the stale statement
|
||||||
|
# and its correction exist as separate nodes. Scored on RANKING: the correction
|
||||||
|
# must appear, and must appear above the stale node. Ids are locked here and
|
||||||
|
# validated to exist and to match their stated role.
|
||||||
|
SUPERSEDED_SEEDS = [
|
||||||
|
# (query, correct_id, stale_id, derivation)
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
# ─────────────────────────────────────────────────────────────────────────────
|
||||||
|
# mining
|
||||||
|
# ─────────────────────────────────────────────────────────────────────────────
|
||||||
|
def mine_exact_rare(nodes, byid, seeds):
|
||||||
|
"""Re-derive: confirm each seed token still has df==1 and name its node."""
|
||||||
|
tok = re.compile(r"[A-Za-z][A-Za-z0-9\-]{4,}")
|
||||||
|
want = set(seeds)
|
||||||
|
df = Counter()
|
||||||
|
post = defaultdict(set)
|
||||||
|
for n in nodes:
|
||||||
|
for t in {w.lower() for w in tok.findall(doctext(n))}:
|
||||||
|
if t in want:
|
||||||
|
df[t] += 1
|
||||||
|
post[t].add(n["id"])
|
||||||
|
out = []
|
||||||
|
for s in seeds:
|
||||||
|
ids = sorted(post.get(s, ()))
|
||||||
|
out.append((s, ids, df.get(s, 0)))
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def phrase_matches(nodes, phrase):
|
||||||
|
p = phrase.lower()
|
||||||
|
return sorted(n["id"] for n in nodes if p in doctext(n).lower())
|
||||||
|
|
||||||
|
|
||||||
|
def hub_siblings(edges, hub, relation="identity"):
|
||||||
|
sibs = []
|
||||||
|
for e in edges:
|
||||||
|
if e.get("from_id") == hub and e.get("relation") == relation:
|
||||||
|
sibs.append(e["to_id"])
|
||||||
|
elif e.get("to_id") == hub and e.get("relation") == relation:
|
||||||
|
sibs.append(e["from_id"])
|
||||||
|
return list(dict.fromkeys(sibs))
|
||||||
|
|
||||||
|
|
||||||
|
def find_superseded_pairs(nodes, byid):
|
||||||
|
"""Locked pairs, each verified here to exist and to carry its stated marker.
|
||||||
|
|
||||||
|
Chosen by scanning the corpus for explicit correction language
|
||||||
|
(CORRECTION/SUPERSEDES/re-corrected/no longer/RECONCILED) and keeping only
|
||||||
|
cases where the STALE claim also survives as its own node — a supersession
|
||||||
|
with nothing to outrank is not a ranking test.
|
||||||
|
"""
|
||||||
|
pairs = []
|
||||||
|
txt = {n["id"]: doctext(n) for n in nodes}
|
||||||
|
|
||||||
|
def find_one(pattern, exclude=()):
|
||||||
|
rx = re.compile(pattern)
|
||||||
|
return [n["id"] for n in nodes
|
||||||
|
if n["id"] not in exclude
|
||||||
|
and rx.search(txt[n["id"]])
|
||||||
|
and 150 < len(str(n.get("content") or "")) < 12000
|
||||||
|
and n.get("node_type") in ("Memory", "Knowledge", "Belief", "BacklogItem")]
|
||||||
|
|
||||||
|
# Each entry: (query, correction-pattern, stale-pattern, why).
|
||||||
|
# The stale side is searched with the correction hits EXCLUDED, because most
|
||||||
|
# correction memories quote the claim they are killing — without the
|
||||||
|
# exclusion the "stale" node resolves to the correction itself and the pair
|
||||||
|
# collapses into a no-op. A pair is only emitted if both sides resolve to
|
||||||
|
# DIFFERENT surviving nodes; otherwise it is dropped and reported.
|
||||||
|
SPECS = [
|
||||||
|
("is the self-improvement architecture called DARMA or DHARMA",
|
||||||
|
r'(?i)CORRECTION:.{0,90}DHARMA .{0,12}not DARMA',
|
||||||
|
r'(?i)\bDARMA\b',
|
||||||
|
"correction node is Will's confirmation that the H is intentional (DHARMA, not DARMA); "
|
||||||
|
"the stale node is the surviving backlog item still titled 'Implement DARMA'."),
|
||||||
|
|
||||||
|
("how many provisional patents does Will actually have",
|
||||||
|
r'(?i)EXACTLY 6 (fully-specced )?provisional',
|
||||||
|
r'(?i)(MY ARCHITECTURE = 12 filed patents|\b12 filed patents\b)',
|
||||||
|
"correction node is the 2026-06-17 confabulation flag establishing EXACTLY 6 provisionals; "
|
||||||
|
"the stale node is the surviving memory that asserts 12 filed patents."),
|
||||||
|
|
||||||
|
("is MCP still the live integration layer",
|
||||||
|
r'(?i)MCP RETIRED',
|
||||||
|
r'(?i)MCP server live at',
|
||||||
|
"correction node is the 'CGI ARCHITECTURE - THREE LAYERS, MCP RETIRED' decision of "
|
||||||
|
"April 30 2026; the stale node still records the MCP server as live."),
|
||||||
|
|
||||||
|
("what does the patterns-not-returns directive mean",
|
||||||
|
r'(?i)CORRECTION:.{0,80}patterns not returns',
|
||||||
|
r'(?i)established returns',
|
||||||
|
"correction node is Will's 'patterns not returns' correction; the stale node is a "
|
||||||
|
"surviving node carrying the misread 'established returns' directive."),
|
||||||
|
|
||||||
|
("was the earlier identity-bug finding correct",
|
||||||
|
r'(?i)SUPERSEDES the earlier .critical identity bug',
|
||||||
|
r'(?i)critical identity bug',
|
||||||
|
"correction node explicitly supersedes the 'critical identity bug' finding; the stale "
|
||||||
|
"node is the surviving original finding."),
|
||||||
|
|
||||||
|
("does Neuron have recursive self-improvement",
|
||||||
|
r'(?i)twice answered .Neuron has no recursive self-improvement',
|
||||||
|
r'(?i)no recursive self-improvement',
|
||||||
|
"correction node records the June-29 finding that the CGI provisional IS the "
|
||||||
|
"recursive-self-improvement mechanism; the stale node is the surviving denial."),
|
||||||
|
]
|
||||||
|
|
||||||
|
for query, cpat, spat, why in SPECS:
|
||||||
|
corr = find_one(cpat)
|
||||||
|
if not corr:
|
||||||
|
continue
|
||||||
|
stale = find_one(spat, exclude=set(corr))
|
||||||
|
if not stale:
|
||||||
|
continue
|
||||||
|
pairs.append((query, corr[0], stale[0], why))
|
||||||
|
|
||||||
|
return pairs
|
||||||
|
|
||||||
|
|
||||||
|
# ─────────────────────────────────────────────────────────────────────────────
|
||||||
|
# build
|
||||||
|
# ─────────────────────────────────────────────────────────────────────────────
|
||||||
|
def build(nodes, edges):
|
||||||
|
byid = {n["id"]: n for n in nodes}
|
||||||
|
tokset = {n["id"]: content_tokens(doctext(n)) for n in nodes}
|
||||||
|
queries = []
|
||||||
|
problems = []
|
||||||
|
qn = [0]
|
||||||
|
|
||||||
|
def add(cat, query, relevant, derivation, **extra):
|
||||||
|
qn[0] += 1
|
||||||
|
q = {
|
||||||
|
"id": f"q{qn[0]:02d}",
|
||||||
|
"category": cat,
|
||||||
|
"query": query,
|
||||||
|
"relevant": sorted(relevant),
|
||||||
|
"derivation": derivation,
|
||||||
|
}
|
||||||
|
q.update(extra)
|
||||||
|
queries.append(q)
|
||||||
|
return q
|
||||||
|
|
||||||
|
# --- exact_rare ---------------------------------------------------------
|
||||||
|
for tokname, ids, df in mine_exact_rare(nodes, byid, EXACT_RARE_SEEDS):
|
||||||
|
if df != 1 or len(ids) != 1:
|
||||||
|
problems.append(f"exact_rare '{tokname}': df={df}, ids={len(ids)} (expected df=1)")
|
||||||
|
continue
|
||||||
|
lab = (byid[ids[0]].get("label") or "")[:60]
|
||||||
|
add("exact_rare", tokname, ids,
|
||||||
|
f"MINED: token '{tokname}' has document frequency 1 over all {len(nodes)} corpus nodes "
|
||||||
|
f"(re-verified at build time). Its single containing node is {ids[0]} "
|
||||||
|
f"('{lab}'), which is therefore the only possible correct answer.")
|
||||||
|
|
||||||
|
# --- phrase -------------------------------------------------------------
|
||||||
|
for phrase, why in PHRASE_SEEDS:
|
||||||
|
ids = phrase_matches(nodes, phrase)
|
||||||
|
if not ids:
|
||||||
|
problems.append(f"phrase '{phrase}': 0 corpus matches — unwinnable")
|
||||||
|
continue
|
||||||
|
if len(ids) > PHRASE_MAX:
|
||||||
|
problems.append(f"phrase '{phrase}': {len(ids)} matches > {PHRASE_MAX} — too diffuse")
|
||||||
|
continue
|
||||||
|
add("phrase", phrase, ids,
|
||||||
|
f"MINED: {why}. Case-insensitive verbatim substring scan over label+content+tags at "
|
||||||
|
f"build time returns exactly {len(ids)} node(s); that set IS the answer key.")
|
||||||
|
|
||||||
|
# --- paraphrase ---------------------------------------------------------
|
||||||
|
for target, query, why in PARAPHRASE_SEEDS:
|
||||||
|
if target not in byid:
|
||||||
|
problems.append(f"paraphrase target {target} not in corpus")
|
||||||
|
continue
|
||||||
|
qt = content_tokens(query)
|
||||||
|
leak = sorted(qt & tokset[target])
|
||||||
|
if leak:
|
||||||
|
problems.append(f"paraphrase '{query}': leaks {leak} into target {target}")
|
||||||
|
continue
|
||||||
|
add("paraphrase", query, [target],
|
||||||
|
f"HAND-SELECTED with criterion: {why} VERIFIED at build time: of the {len(qt)} content "
|
||||||
|
f"words in the query, ZERO appear anywhere in the target's label, content, or tags — so "
|
||||||
|
f"no string-matching retriever can reach this answer.",
|
||||||
|
zero_overlap_verified=True, query_content_words=sorted(qt))
|
||||||
|
|
||||||
|
# --- associative --------------------------------------------------------
|
||||||
|
sibs = hub_siblings(edges, VALUES_HUB)
|
||||||
|
if len(sibs) < 5:
|
||||||
|
problems.append(f"associative: values hub {VALUES_HUB} has only {len(sibs)} siblings")
|
||||||
|
for src, query in ASSOCIATIVE_SEEDS:
|
||||||
|
if src not in byid or src not in sibs:
|
||||||
|
problems.append(f"associative source {src} not a sibling on {VALUES_HUB}")
|
||||||
|
continue
|
||||||
|
qt = content_tokens(query)
|
||||||
|
others = [s for s in sibs if s != src and s in byid]
|
||||||
|
# A sibling only counts as a legitimate expected answer if the query
|
||||||
|
# cannot reach it lexically. Drop any sibling that shares a content word.
|
||||||
|
clean = [s for s in others if not (qt & tokset[s])]
|
||||||
|
dropped = len(others) - len(clean)
|
||||||
|
if len(clean) < 5:
|
||||||
|
problems.append(f"associative '{query}': only {len(clean)} lexically-unreachable siblings")
|
||||||
|
continue
|
||||||
|
add("associative", query, clean,
|
||||||
|
f"DERIVED FROM EDGES: the query is built from the distinctive vocabulary of {src} "
|
||||||
|
f"('{(byid[src].get('label') or '')[:48]}'), which hangs off the values hub {VALUES_HUB} "
|
||||||
|
f"by an `identity` edge. Expected answers are that node's SIBLINGS on the same hub "
|
||||||
|
f"({len(clean)} of {len(others)}; {dropped} dropped because they shared a query word and "
|
||||||
|
f"so were lexically reachable). Every remaining sibling shares ZERO content words with "
|
||||||
|
f"the query — the only route from query to answer is seed({src}) -> hub -> sibling, a "
|
||||||
|
f"two-hop traversal.",
|
||||||
|
associative_source=src, hub=VALUES_HUB, siblings_dropped_for_overlap=dropped)
|
||||||
|
|
||||||
|
# --- nonsense -----------------------------------------------------------
|
||||||
|
all_tokens = set()
|
||||||
|
for n in nodes:
|
||||||
|
all_tokens |= {t for t in TOKEN.findall(doctext(n).lower())}
|
||||||
|
for s in NONSENSE_SEEDS:
|
||||||
|
present = sorted(t for t in TOKEN.findall(s.lower()) if t in all_tokens)
|
||||||
|
if present:
|
||||||
|
problems.append(f"nonsense '{s}': tokens {present} DO occur in corpus")
|
||||||
|
continue
|
||||||
|
add("nonsense", s, [],
|
||||||
|
f"CONTROL: verified at build time that none of this string's tokens occurs anywhere in "
|
||||||
|
f"the corpus. Correct behaviour is to return NOTHING; any result is a false positive.",
|
||||||
|
expect_empty=True)
|
||||||
|
|
||||||
|
# --- superseded ---------------------------------------------------------
|
||||||
|
for query, correct, stale, why in find_superseded_pairs(nodes, byid):
|
||||||
|
if correct not in byid or stale not in byid:
|
||||||
|
problems.append(f"superseded '{query}': id missing from corpus")
|
||||||
|
continue
|
||||||
|
add("superseded", query, [correct],
|
||||||
|
f"DERIVED: {why} Scored on RANKING, not presence: the corrected node {correct} must be "
|
||||||
|
f"returned AND must rank above the stale node {stale}.",
|
||||||
|
must_outrank=[correct, stale],
|
||||||
|
stale_id=stale,
|
||||||
|
correct_label=(byid[correct].get("label") or "")[:70],
|
||||||
|
stale_label=(byid[stale].get("label") or "")[:70])
|
||||||
|
|
||||||
|
return queries, problems
|
||||||
|
|
||||||
|
|
||||||
|
def summarize(queries):
|
||||||
|
c = Counter(q["category"] for q in queries)
|
||||||
|
return ", ".join(f"{k}={c[k]}" for k in
|
||||||
|
("exact_rare", "phrase", "paraphrase", "associative", "nonsense", "superseded")
|
||||||
|
if c[k])
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
ap = argparse.ArgumentParser()
|
||||||
|
ap.add_argument("snapshot")
|
||||||
|
ap.add_argument("--out", default=DEFAULT_OUT)
|
||||||
|
ap.add_argument("--check", action="store_true",
|
||||||
|
help="validate only; do not write. Non-zero exit if anything is unwinnable.")
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
nodes, edges = load_corpus(args.snapshot)
|
||||||
|
print(f"corpus: {len(nodes)} nodes, {len(edges)} edges ({os.path.basename(args.snapshot)})")
|
||||||
|
queries, problems = build(nodes, edges)
|
||||||
|
|
||||||
|
print(f"gold set: {len(queries)} queries [{summarize(queries)}]")
|
||||||
|
if problems:
|
||||||
|
print(f"\n{len(problems)} PROBLEM(S) — these queries were REJECTED, not silently kept:")
|
||||||
|
for p in problems:
|
||||||
|
print(" -", p)
|
||||||
|
|
||||||
|
if args.check:
|
||||||
|
sys.exit(1 if problems else 0)
|
||||||
|
|
||||||
|
doc = {
|
||||||
|
"corpus": os.path.abspath(args.snapshot),
|
||||||
|
"corpus_nodes": len(nodes),
|
||||||
|
"corpus_edges": len(edges),
|
||||||
|
"note": ("Every query carries a `derivation` recording how its expected answer was chosen. "
|
||||||
|
"Re-run with --check to re-validate the whole set against the corpus."),
|
||||||
|
"queries": queries,
|
||||||
|
}
|
||||||
|
with open(args.out, "w", encoding="utf-8") as fh:
|
||||||
|
json.dump(doc, fh, indent=1, ensure_ascii=False)
|
||||||
|
print(f"\nwrote {args.out}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,43 @@
|
|||||||
|
import json,sys,pickle,numpy as np,itertools
|
||||||
|
sys.path.insert(0,'.')
|
||||||
|
from policy2 import legs3,outcome,G,NODES,merge
|
||||||
|
# cache per-query leg id-lists, floored and unfloored
|
||||||
|
cache={}
|
||||||
|
for q in G['queries']:
|
||||||
|
Lf,Sf,Af=legs3(q['query'])
|
||||||
|
Lu,Su,Au=legs3(q['query'],unfloor=True)
|
||||||
|
cache[q['id']]=dict(L=Lf,Sf=Sf,A=Af,Su=Su,Au=Au)
|
||||||
|
pickle.dump(cache,open('ceil.pkl','wb'))
|
||||||
|
def mrg(pattern,L,S,A,lim=10):
|
||||||
|
out=[];p={'L':0,'S':0,'A':0};src={'L':L,'S':S,'A':A}
|
||||||
|
i=0
|
||||||
|
while len(out)<lim:
|
||||||
|
prog=False
|
||||||
|
for ch in pattern:
|
||||||
|
lst=src[ch]
|
||||||
|
if p[ch]<len(lst):
|
||||||
|
x=lst[p[ch]];p[ch]+=1;prog=True
|
||||||
|
if x not in out: out.append(x)
|
||||||
|
if len(out)>=lim: return out
|
||||||
|
if not prog: break
|
||||||
|
return out
|
||||||
|
def ev(pattern,unfl):
|
||||||
|
res={}
|
||||||
|
for q in G['queries']:
|
||||||
|
c=cache[q['id']]
|
||||||
|
S=c['Su'] if unfl else c['Sf']
|
||||||
|
ids=[NODES[i]['id'] for i in mrg(pattern,c['L'],S,c['A'],10)]
|
||||||
|
res[q['id']]=outcome(q,ids)
|
||||||
|
return res
|
||||||
|
base=ev('LSA',False)
|
||||||
|
print("baseline",sum(base.values()))
|
||||||
|
best=[]
|
||||||
|
pats=['LSA','LAS','SLA','ALS','SAL','ASL','LSSA','LSASA','LSAA','LSSAA','LSAS','SSLA','LLSA','SALSA','LSAAS']
|
||||||
|
for unfl in (False,True):
|
||||||
|
for p in pats:
|
||||||
|
r=ev(p,unfl)
|
||||||
|
g=sorted(k for k in base if r[k] and not base[k]);l=sorted(k for k in base if base[k] and not r[k])
|
||||||
|
best.append((len(g)-len(l),p,unfl,g,l))
|
||||||
|
best.sort(reverse=True)
|
||||||
|
for n,p,u,g,l in best[:10]:
|
||||||
|
print("net=%+d pat=%-6s unfloor=%s gains=%s losses=%s"%(n,p,u,g,l))
|
||||||
@@ -0,0 +1,146 @@
|
|||||||
|
{
|
||||||
|
"baseline": "bm25lex",
|
||||||
|
"candidate": "wsclaim24",
|
||||||
|
"n_shared_queries": 38,
|
||||||
|
"fixed_by_candidate": [
|
||||||
|
"q14",
|
||||||
|
"q25"
|
||||||
|
],
|
||||||
|
"broken_by_candidate": [
|
||||||
|
"q15",
|
||||||
|
"q28",
|
||||||
|
"q33",
|
||||||
|
"q34"
|
||||||
|
],
|
||||||
|
"discordant": 6,
|
||||||
|
"net_queries": -2,
|
||||||
|
"mcnemar_exact_p": 0.6875,
|
||||||
|
"min_detectable_swing_queries": 6,
|
||||||
|
"observed_run_to_run_drift_queries": 0,
|
||||||
|
"noise_floor_queries": 6,
|
||||||
|
"verdict": "no measurable difference",
|
||||||
|
"baseline_aggregate": {
|
||||||
|
"n_queries": 38,
|
||||||
|
"n_scored": 35,
|
||||||
|
"hit@5": 0.7428571428571429,
|
||||||
|
"recall@5": 0.5536485340056769,
|
||||||
|
"recall@10": 0.6175677497106068,
|
||||||
|
"precision@5": 0.20000000000000007,
|
||||||
|
"mrr@10": 0.5021428571428571,
|
||||||
|
"nonsense_clean": "2/3",
|
||||||
|
"superseded_outranks": "2/3",
|
||||||
|
"latency_ms_p50": 1184.4,
|
||||||
|
"latency_ms_p95": 1620.0,
|
||||||
|
"latency_ms_max": 1655.4,
|
||||||
|
"errors": 0,
|
||||||
|
"by_category": {
|
||||||
|
"associative": {
|
||||||
|
"n": 6,
|
||||||
|
"hit@5": 0.6666666666666666,
|
||||||
|
"recall@5": 0.08857808857808858,
|
||||||
|
"recall@10": 0.23310023310023312,
|
||||||
|
"mrr@10": 0.25
|
||||||
|
},
|
||||||
|
"exact_rare": {
|
||||||
|
"n": 6,
|
||||||
|
"hit@5": 1.0,
|
||||||
|
"recall@5": 1.0,
|
||||||
|
"recall@10": 1.0,
|
||||||
|
"mrr@10": 1.0
|
||||||
|
},
|
||||||
|
"nonsense": {
|
||||||
|
"n": 3,
|
||||||
|
"clean": 2,
|
||||||
|
"avg_false_positives": 3.3333333333333335
|
||||||
|
},
|
||||||
|
"paraphrase": {
|
||||||
|
"n": 13,
|
||||||
|
"hit@5": 0.6153846153846154,
|
||||||
|
"recall@5": 0.6153846153846154,
|
||||||
|
"recall@10": 0.6153846153846154,
|
||||||
|
"mrr@10": 0.2846153846153846
|
||||||
|
},
|
||||||
|
"phrase": {
|
||||||
|
"n": 7,
|
||||||
|
"hit@5": 1.0,
|
||||||
|
"recall@5": 0.5494614512471656,
|
||||||
|
"recall@10": 0.6023242630385487,
|
||||||
|
"mrr@10": 0.8214285714285714
|
||||||
|
},
|
||||||
|
"superseded": {
|
||||||
|
"n": 3,
|
||||||
|
"hit@5": 0.3333333333333333,
|
||||||
|
"recall@5": 0.3333333333333333,
|
||||||
|
"recall@10": 0.6666666666666666,
|
||||||
|
"mrr@10": 0.20833333333333334,
|
||||||
|
"outranks": 2
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"candidate_aggregate": {
|
||||||
|
"n_queries": 38,
|
||||||
|
"n_scored": 35,
|
||||||
|
"hit@5": 0.7428571428571429,
|
||||||
|
"recall@5": 0.5768475572047,
|
||||||
|
"recall@10": 0.6563414759843332,
|
||||||
|
"precision@5": 0.19428571428571437,
|
||||||
|
"mrr@10": 0.5026530612244898,
|
||||||
|
"nonsense_clean": "0/3",
|
||||||
|
"superseded_outranks": "2/3",
|
||||||
|
"latency_ms_p50": 524.8,
|
||||||
|
"latency_ms_p95": 738.7,
|
||||||
|
"latency_ms_max": 755.8,
|
||||||
|
"errors": 0,
|
||||||
|
"by_category": {
|
||||||
|
"associative": {
|
||||||
|
"n": 6,
|
||||||
|
"hit@5": 0.5,
|
||||||
|
"recall@5": 0.07575757575757576,
|
||||||
|
"recall@10": 0.13636363636363635,
|
||||||
|
"mrr@10": 0.23214285714285712
|
||||||
|
},
|
||||||
|
"exact_rare": {
|
||||||
|
"n": 6,
|
||||||
|
"hit@5": 1.0,
|
||||||
|
"recall@5": 1.0,
|
||||||
|
"recall@10": 1.0,
|
||||||
|
"mrr@10": 1.0
|
||||||
|
},
|
||||||
|
"nonsense": {
|
||||||
|
"n": 3,
|
||||||
|
"clean": 0,
|
||||||
|
"avg_false_positives": 10.0
|
||||||
|
},
|
||||||
|
"paraphrase": {
|
||||||
|
"n": 13,
|
||||||
|
"hit@5": 0.6923076923076923,
|
||||||
|
"recall@5": 0.6923076923076923,
|
||||||
|
"recall@10": 0.7692307692307693,
|
||||||
|
"mrr@10": 0.29423076923076924
|
||||||
|
},
|
||||||
|
"phrase": {
|
||||||
|
"n": 7,
|
||||||
|
"hit@5": 1.0,
|
||||||
|
"recall@5": 0.5335884353741497,
|
||||||
|
"recall@10": 0.5933956916099773,
|
||||||
|
"mrr@10": 0.8214285714285714
|
||||||
|
},
|
||||||
|
"superseded": {
|
||||||
|
"n": 3,
|
||||||
|
"hit@5": 0.3333333333333333,
|
||||||
|
"recall@5": 0.3333333333333333,
|
||||||
|
"recall@10": 0.6666666666666666,
|
||||||
|
"mrr@10": 0.20833333333333334,
|
||||||
|
"outranks": 2
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"repeat_variance": {
|
||||||
|
"baseline": {
|
||||||
|
"runs": 2,
|
||||||
|
"hit@5_min": 0.7428571428571429,
|
||||||
|
"hit@5_max": 0.7428571428571429,
|
||||||
|
"spread_queries": 0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,149 @@
|
|||||||
|
{
|
||||||
|
"baseline": "unfloor-clean",
|
||||||
|
"candidate": "splitfix",
|
||||||
|
"n_shared_queries": 75,
|
||||||
|
"fixed_by_candidate": [
|
||||||
|
"q15",
|
||||||
|
"q28",
|
||||||
|
"q60"
|
||||||
|
],
|
||||||
|
"broken_by_candidate": [],
|
||||||
|
"discordant": 3,
|
||||||
|
"net_queries": 3,
|
||||||
|
"mcnemar_exact_p": 0.25,
|
||||||
|
"min_detectable_swing_queries": 6,
|
||||||
|
"observed_run_to_run_drift_queries": 0,
|
||||||
|
"noise_floor_queries": 6,
|
||||||
|
"verdict": "no measurable difference",
|
||||||
|
"baseline_aggregate": {
|
||||||
|
"n_queries": 75,
|
||||||
|
"n_scored": 65,
|
||||||
|
"hit@5": 0.5384615384615384,
|
||||||
|
"recall@5": 0.44907176157176154,
|
||||||
|
"recall@10": 0.5380300255300255,
|
||||||
|
"precision@5": 0.13230769230769232,
|
||||||
|
"mrr@10": 0.32437728937728944,
|
||||||
|
"nonsense_clean": "10/10",
|
||||||
|
"superseded_outranks": "2/3",
|
||||||
|
"latency_ms_p50": 632.5,
|
||||||
|
"latency_ms_p95": 992.5,
|
||||||
|
"latency_ms_max": 1177.8,
|
||||||
|
"errors": 0,
|
||||||
|
"by_category": {
|
||||||
|
"associative": {
|
||||||
|
"n": 6,
|
||||||
|
"hit@5": 0.5,
|
||||||
|
"recall@5": 0.07575757575757576,
|
||||||
|
"recall@10": 0.13636363636363635,
|
||||||
|
"mrr@10": 0.23214285714285712
|
||||||
|
},
|
||||||
|
"exact_rare": {
|
||||||
|
"n": 6,
|
||||||
|
"hit@5": 1.0,
|
||||||
|
"recall@5": 1.0,
|
||||||
|
"recall@10": 1.0,
|
||||||
|
"mrr@10": 1.0
|
||||||
|
},
|
||||||
|
"heldout_paraphrase": {
|
||||||
|
"n": 30,
|
||||||
|
"hit@5": 0.3,
|
||||||
|
"recall@5": 0.3,
|
||||||
|
"recall@10": 0.4,
|
||||||
|
"mrr@10": 0.11638888888888889
|
||||||
|
},
|
||||||
|
"nonsense": {
|
||||||
|
"n": 10,
|
||||||
|
"clean": 10,
|
||||||
|
"avg_false_positives": 0.0
|
||||||
|
},
|
||||||
|
"paraphrase": {
|
||||||
|
"n": 13,
|
||||||
|
"hit@5": 0.6923076923076923,
|
||||||
|
"recall@5": 0.6923076923076923,
|
||||||
|
"recall@10": 0.7692307692307693,
|
||||||
|
"mrr@10": 0.29423076923076924
|
||||||
|
},
|
||||||
|
"phrase": {
|
||||||
|
"n": 7,
|
||||||
|
"hit@5": 1.0,
|
||||||
|
"recall@5": 0.5335884353741497,
|
||||||
|
"recall@10": 0.5933956916099773,
|
||||||
|
"mrr@10": 0.8214285714285714
|
||||||
|
},
|
||||||
|
"superseded": {
|
||||||
|
"n": 3,
|
||||||
|
"hit@5": 0.3333333333333333,
|
||||||
|
"recall@5": 0.3333333333333333,
|
||||||
|
"recall@10": 0.6666666666666666,
|
||||||
|
"mrr@10": 0.20833333333333334,
|
||||||
|
"outranks": 2
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"candidate_aggregate": {
|
||||||
|
"n_queries": 75,
|
||||||
|
"n_scored": 65,
|
||||||
|
"hit@5": 0.5846153846153846,
|
||||||
|
"recall@5": 0.48102442429365505,
|
||||||
|
"recall@10": 0.5692415490492414,
|
||||||
|
"precision@5": 0.14153846153846153,
|
||||||
|
"mrr@10": 0.3351709401709402,
|
||||||
|
"nonsense_clean": "10/10",
|
||||||
|
"superseded_outranks": "2/3",
|
||||||
|
"latency_ms_p50": 646.9,
|
||||||
|
"latency_ms_p95": 1028.5,
|
||||||
|
"latency_ms_max": 1197.8,
|
||||||
|
"errors": 0,
|
||||||
|
"by_category": {
|
||||||
|
"associative": {
|
||||||
|
"n": 6,
|
||||||
|
"hit@5": 0.6666666666666666,
|
||||||
|
"recall@5": 0.08857808857808858,
|
||||||
|
"recall@10": 0.15967365967365968,
|
||||||
|
"mrr@10": 0.24166666666666667
|
||||||
|
},
|
||||||
|
"exact_rare": {
|
||||||
|
"n": 6,
|
||||||
|
"hit@5": 1.0,
|
||||||
|
"recall@5": 1.0,
|
||||||
|
"recall@10": 1.0,
|
||||||
|
"mrr@10": 1.0
|
||||||
|
},
|
||||||
|
"heldout_paraphrase": {
|
||||||
|
"n": 30,
|
||||||
|
"hit@5": 0.3333333333333333,
|
||||||
|
"recall@5": 0.3333333333333333,
|
||||||
|
"recall@10": 0.43333333333333335,
|
||||||
|
"mrr@10": 0.12120370370370372
|
||||||
|
},
|
||||||
|
"nonsense": {
|
||||||
|
"n": 10,
|
||||||
|
"clean": 10,
|
||||||
|
"avg_false_positives": 0.0
|
||||||
|
},
|
||||||
|
"paraphrase": {
|
||||||
|
"n": 13,
|
||||||
|
"hit@5": 0.7692307692307693,
|
||||||
|
"recall@5": 0.7692307692307693,
|
||||||
|
"recall@10": 0.8461538461538461,
|
||||||
|
"mrr@10": 0.33269230769230773
|
||||||
|
},
|
||||||
|
"phrase": {
|
||||||
|
"n": 7,
|
||||||
|
"hit@5": 1.0,
|
||||||
|
"recall@5": 0.5335884353741497,
|
||||||
|
"recall@10": 0.5775226757369615,
|
||||||
|
"mrr@10": 0.8214285714285714
|
||||||
|
},
|
||||||
|
"superseded": {
|
||||||
|
"n": 3,
|
||||||
|
"hit@5": 0.3333333333333333,
|
||||||
|
"recall@5": 0.3333333333333333,
|
||||||
|
"recall@10": 0.6666666666666666,
|
||||||
|
"mrr@10": 0.20833333333333334,
|
||||||
|
"outranks": 2
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"repeat_variance": {}
|
||||||
|
}
|
||||||
Executable
+210
@@ -0,0 +1,210 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
compare.py — diff two run_eval.py result files, WITH a noise threshold.
|
||||||
|
|
||||||
|
WHY THE STATISTICS ARE NOT OPTIONAL
|
||||||
|
With ~35 scored queries, one query is ~2.9 percentage points. A harness that
|
||||||
|
reports "hit@5 improved 2.9%" without saying that is one query is a harness
|
||||||
|
that will approve noise. So this file refuses to call anything an
|
||||||
|
improvement on the strength of the headline number alone. It reports:
|
||||||
|
|
||||||
|
1. The DISCORDANT PAIRS. Two configurations scored on the same queries are
|
||||||
|
paired data, so the only queries carrying information are the ones
|
||||||
|
where they disagree: b = fixed by B, c = broken by B. Queries both got
|
||||||
|
right, or both got wrong, tell you nothing about which is better.
|
||||||
|
|
||||||
|
2. McNEMAR'S EXACT TEST on (b, c). Under the null "the change is a coin
|
||||||
|
flip", the discordant outcomes are Binomial(b+c, 0.5). The two-sided
|
||||||
|
exact p-value is computed here with no scipy dependency.
|
||||||
|
|
||||||
|
3. The MINIMUM DETECTABLE SWING for this gold set: the smallest number of
|
||||||
|
net-changed queries that would reach p < 0.05 if every discordant pair
|
||||||
|
fell the same way. Anything smaller is inside the noise band, and the
|
||||||
|
verdict line says so in those words.
|
||||||
|
|
||||||
|
Repeat-run variance is the other half of honesty. Spreading activation is a
|
||||||
|
stateful read (it reinforces what it touches), so identical inputs need not
|
||||||
|
give identical outputs. Pass --repeats to fold several runs of the same
|
||||||
|
config into an observed variance band; a delta inside that band is not real
|
||||||
|
either, however good its p-value looks.
|
||||||
|
|
||||||
|
usage:
|
||||||
|
python3 compare.py --baseline results-main.json --candidate results-act.json
|
||||||
|
python3 compare.py --baseline a.json --candidate b.json \
|
||||||
|
--repeats-baseline a2.json a3.json --repeats-candidate b2.json b3.json
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
from math import comb
|
||||||
|
|
||||||
|
|
||||||
|
def binom_two_sided(b, c):
|
||||||
|
"""Two-sided exact binomial p for b successes in n=b+c at p=0.5."""
|
||||||
|
n = b + c
|
||||||
|
if n == 0:
|
||||||
|
return 1.0
|
||||||
|
k = min(b, c)
|
||||||
|
tail = sum(comb(n, i) for i in range(0, k + 1)) / (2 ** n)
|
||||||
|
return min(1.0, 2 * tail)
|
||||||
|
|
||||||
|
|
||||||
|
def min_detectable_swing(n_scored, alpha=0.05):
|
||||||
|
"""Smallest all-one-way discordant count reaching p < alpha.
|
||||||
|
|
||||||
|
If every query that changes changes in the same direction, the p-value is
|
||||||
|
2 * 0.5**n. Solve for the smallest n where that drops under alpha. This is
|
||||||
|
the FLOOR: any real change will have some discordance both ways, so the true
|
||||||
|
requirement is larger. Reporting the floor is the conservative move — it is
|
||||||
|
the most generous threshold we would ever accept.
|
||||||
|
"""
|
||||||
|
n = 1
|
||||||
|
while n <= n_scored:
|
||||||
|
if 2 * (0.5 ** n) < alpha:
|
||||||
|
return n
|
||||||
|
n += 1
|
||||||
|
return n_scored
|
||||||
|
|
||||||
|
|
||||||
|
def load(path):
|
||||||
|
with open(path, encoding="utf-8") as fh:
|
||||||
|
return json.load(fh)
|
||||||
|
|
||||||
|
|
||||||
|
def row_map(doc):
|
||||||
|
return {r["id"]: r for r in doc["rows"]}
|
||||||
|
|
||||||
|
|
||||||
|
def outcome(r):
|
||||||
|
"""Binary per-query outcome used for the paired test.
|
||||||
|
|
||||||
|
hit@5 for scored queries; 'returned nothing' for the nonsense controls;
|
||||||
|
'correction outranks the stale node' for the superseded queries. One number
|
||||||
|
per query, so every query votes exactly once.
|
||||||
|
"""
|
||||||
|
if "clean" in r:
|
||||||
|
return 1.0 if r["clean"] else 0.0
|
||||||
|
if "outranks" in r:
|
||||||
|
return 1.0 if r["outranks"] else 0.0
|
||||||
|
return r.get("hit@5") or 0.0
|
||||||
|
|
||||||
|
|
||||||
|
def band(values):
|
||||||
|
return (min(values), max(values))
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
ap = argparse.ArgumentParser()
|
||||||
|
ap.add_argument("--baseline", required=True)
|
||||||
|
ap.add_argument("--candidate", required=True)
|
||||||
|
ap.add_argument("--repeats-baseline", nargs="*", default=[])
|
||||||
|
ap.add_argument("--repeats-candidate", nargs="*", default=[])
|
||||||
|
ap.add_argument("--out", default=None)
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
A, B = load(args.baseline), load(args.candidate)
|
||||||
|
ra, rb = row_map(A), row_map(B)
|
||||||
|
ids = [q for q in ra if q in rb]
|
||||||
|
n = len(ids)
|
||||||
|
|
||||||
|
aa, ab = A["aggregate"], B["aggregate"]
|
||||||
|
print(f"baseline {A['label']:14} soul={A['soul_md5'][:12]} {n} shared queries")
|
||||||
|
print(f"candidate {B['label']:14} soul={B['soul_md5'][:12]}")
|
||||||
|
print(f"corpus {A['corpus_nodes']} nodes / {A['corpus_edges']} edges "
|
||||||
|
f"(identical copy for both runs)\n")
|
||||||
|
|
||||||
|
metrics = [("hit@5", 1), ("recall@5", 1), ("recall@10", 1),
|
||||||
|
("precision@5", 1), ("mrr@10", 0)]
|
||||||
|
print(f" {'metric':14} {'baseline':>10} {'candidate':>10} {'delta':>10}")
|
||||||
|
for m, as_pct in metrics:
|
||||||
|
x, y = aa[m], ab[m]
|
||||||
|
if as_pct:
|
||||||
|
print(f" {m:14} {100*x:>9.1f}% {100*y:>9.1f}% {100*(y-x):>+9.1f}pp")
|
||||||
|
else:
|
||||||
|
print(f" {m:14} {x:>10.3f} {y:>10.3f} {y-x:>+10.3f}")
|
||||||
|
for m in ("latency_ms_p50", "latency_ms_p95"):
|
||||||
|
x, y = aa[m], ab[m]
|
||||||
|
ratio = f"{y/x:.2f}x" if x else "n/a"
|
||||||
|
print(f" {m:14} {x:>9.0f}ms {y:>9.0f}ms {ratio:>10}")
|
||||||
|
print(f" {'nonsense':14} {aa['nonsense_clean']:>10} {ab['nonsense_clean']:>10}")
|
||||||
|
print(f" {'outranks':14} {aa['superseded_outranks']:>10} {ab['superseded_outranks']:>10}")
|
||||||
|
|
||||||
|
print(f"\n {'category':14} {'n':>3} {'base hit@5':>11} {'cand hit@5':>11} {'delta':>9}")
|
||||||
|
for c in sorted(set(aa["by_category"]) & set(ab["by_category"])):
|
||||||
|
ea, eb = aa["by_category"][c], ab["by_category"][c]
|
||||||
|
if c == "nonsense":
|
||||||
|
print(f" {c:14} {ea['n']:>3} {'clean ' + str(ea['clean']):>11} "
|
||||||
|
f"{'clean ' + str(eb['clean']):>11}")
|
||||||
|
else:
|
||||||
|
print(f" {c:14} {ea['n']:>3} {100*ea['hit@5']:>10.1f}% {100*eb['hit@5']:>10.1f}% "
|
||||||
|
f"{100*(eb['hit@5']-ea['hit@5']):>+8.1f}pp")
|
||||||
|
|
||||||
|
# ---- paired significance -------------------------------------------------
|
||||||
|
fixed, broken = [], []
|
||||||
|
for q in ids:
|
||||||
|
oa, ob = outcome(ra[q]), outcome(rb[q])
|
||||||
|
if ob > oa:
|
||||||
|
fixed.append(q)
|
||||||
|
elif ob < oa:
|
||||||
|
broken.append(q)
|
||||||
|
b, c = len(fixed), len(broken)
|
||||||
|
p = binom_two_sided(b, c)
|
||||||
|
mds = min_detectable_swing(n)
|
||||||
|
|
||||||
|
print(f"\n== paired comparison over {n} queries ==")
|
||||||
|
print(f" fixed by candidate : {b} {[ra[q]['category'] + ':' + q for q in fixed]}")
|
||||||
|
print(f" broken by candidate: {c} {[ra[q]['category'] + ':' + q for q in broken]}")
|
||||||
|
print(f" discordant pairs : {b + c} net {b - c:+d} queries")
|
||||||
|
print(f" McNemar exact p : {p:.4f}")
|
||||||
|
print(f" noise threshold : a difference needs at least {mds} queries moving the "
|
||||||
|
f"same way to clear p<0.05 on this {n}-query set")
|
||||||
|
|
||||||
|
# ---- repeat-run variance -------------------------------------------------
|
||||||
|
var = {}
|
||||||
|
for name, paths, first in (("baseline", args.repeats_baseline, A),
|
||||||
|
("candidate", args.repeats_candidate, B)):
|
||||||
|
docs = [first] + [load(p) for p in paths]
|
||||||
|
if len(docs) > 1:
|
||||||
|
hits = [d["aggregate"]["hit@5"] for d in docs]
|
||||||
|
lo, hi = band(hits)
|
||||||
|
spread_q = round((hi - lo) * first["aggregate"]["n_scored"])
|
||||||
|
var[name] = {"runs": len(docs), "hit@5_min": lo, "hit@5_max": hi,
|
||||||
|
"spread_queries": spread_q}
|
||||||
|
print(f" {name} repeat runs ({len(docs)}): hit@5 {100*lo:.1f}%..{100*hi:.1f}% "
|
||||||
|
f"= {spread_q} query of run-to-run drift")
|
||||||
|
|
||||||
|
drift = max([v["spread_queries"] for v in var.values()], default=0)
|
||||||
|
floor = max(mds, drift + 1)
|
||||||
|
|
||||||
|
print("\n== VERDICT ==")
|
||||||
|
net = b - c
|
||||||
|
if abs(net) < floor:
|
||||||
|
print(f" NO MEASURABLE DIFFERENCE. Net {net:+d} queries is inside the noise band "
|
||||||
|
f"(needs |net| >= {floor}: {mds} for significance, {drift} observed run-to-run drift).")
|
||||||
|
elif net > 0:
|
||||||
|
print(f" CANDIDATE BETTER by {net} queries (p={p:.4f}), outside the noise band "
|
||||||
|
f"(>= {floor}).")
|
||||||
|
else:
|
||||||
|
print(f" CANDIDATE WORSE by {abs(net)} queries (p={p:.4f}), outside the noise band "
|
||||||
|
f"(>= {floor}).")
|
||||||
|
|
||||||
|
if args.out:
|
||||||
|
with open(args.out, "w", encoding="utf-8") as fh:
|
||||||
|
json.dump({
|
||||||
|
"baseline": A["label"], "candidate": B["label"],
|
||||||
|
"n_shared_queries": n,
|
||||||
|
"fixed_by_candidate": fixed, "broken_by_candidate": broken,
|
||||||
|
"discordant": b + c, "net_queries": net,
|
||||||
|
"mcnemar_exact_p": p,
|
||||||
|
"min_detectable_swing_queries": mds,
|
||||||
|
"observed_run_to_run_drift_queries": drift,
|
||||||
|
"noise_floor_queries": floor,
|
||||||
|
"verdict": ("no measurable difference" if abs(net) < floor
|
||||||
|
else ("candidate better" if net > 0 else "candidate worse")),
|
||||||
|
"baseline_aggregate": aa, "candidate_aggregate": ab,
|
||||||
|
"repeat_variance": var,
|
||||||
|
}, fh, indent=1)
|
||||||
|
print(f"\nwrote {args.out}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,150 @@
|
|||||||
|
{
|
||||||
|
"baseline": "assoc-leg",
|
||||||
|
"candidate": "semseed",
|
||||||
|
"n_shared_queries": 38,
|
||||||
|
"fixed_by_candidate": [
|
||||||
|
"q18",
|
||||||
|
"q19",
|
||||||
|
"q22"
|
||||||
|
],
|
||||||
|
"broken_by_candidate": [
|
||||||
|
"q11"
|
||||||
|
],
|
||||||
|
"discordant": 4,
|
||||||
|
"net_queries": 2,
|
||||||
|
"mcnemar_exact_p": 0.625,
|
||||||
|
"min_detectable_swing_queries": 6,
|
||||||
|
"observed_run_to_run_drift_queries": 0,
|
||||||
|
"noise_floor_queries": 6,
|
||||||
|
"verdict": "no measurable difference",
|
||||||
|
"baseline_aggregate": {
|
||||||
|
"n_queries": 38,
|
||||||
|
"n_scored": 35,
|
||||||
|
"hit@5": 0.6285714285714286,
|
||||||
|
"recall@5": 0.45309194773480493,
|
||||||
|
"recall@10": 0.5405733155733157,
|
||||||
|
"precision@5": 0.17714285714285719,
|
||||||
|
"mrr@10": 0.42650793650793645,
|
||||||
|
"nonsense_clean": "2/3",
|
||||||
|
"superseded_outranks": "2/3",
|
||||||
|
"latency_ms_p50": 1228.5,
|
||||||
|
"latency_ms_p95": 1681.8,
|
||||||
|
"latency_ms_max": 1718.6,
|
||||||
|
"errors": 0,
|
||||||
|
"by_category": {
|
||||||
|
"associative": {
|
||||||
|
"n": 6,
|
||||||
|
"hit@5": 0.6666666666666666,
|
||||||
|
"recall@5": 0.07342657342657342,
|
||||||
|
"recall@10": 0.24825174825174826,
|
||||||
|
"mrr@10": 0.22777777777777777
|
||||||
|
},
|
||||||
|
"exact_rare": {
|
||||||
|
"n": 6,
|
||||||
|
"hit@5": 1.0,
|
||||||
|
"recall@5": 1.0,
|
||||||
|
"recall@10": 1.0,
|
||||||
|
"mrr@10": 1.0
|
||||||
|
},
|
||||||
|
"nonsense": {
|
||||||
|
"n": 3,
|
||||||
|
"clean": 2,
|
||||||
|
"avg_false_positives": 3.3333333333333335
|
||||||
|
},
|
||||||
|
"paraphrase": {
|
||||||
|
"n": 13,
|
||||||
|
"hit@5": 0.38461538461538464,
|
||||||
|
"recall@5": 0.38461538461538464,
|
||||||
|
"recall@10": 0.38461538461538464,
|
||||||
|
"mrr@10": 0.17307692307692307
|
||||||
|
},
|
||||||
|
"phrase": {
|
||||||
|
"n": 7,
|
||||||
|
"hit@5": 0.8571428571428571,
|
||||||
|
"recall@5": 0.4882369614512472,
|
||||||
|
"recall@10": 0.6329365079365079,
|
||||||
|
"mrr@10": 0.6634920634920636
|
||||||
|
},
|
||||||
|
"superseded": {
|
||||||
|
"n": 3,
|
||||||
|
"hit@5": 0.3333333333333333,
|
||||||
|
"recall@5": 0.3333333333333333,
|
||||||
|
"recall@10": 0.6666666666666666,
|
||||||
|
"mrr@10": 0.2222222222222222,
|
||||||
|
"outranks": 2
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"candidate_aggregate": {
|
||||||
|
"n_queries": 38,
|
||||||
|
"n_scored": 35,
|
||||||
|
"hit@5": 0.6857142857142857,
|
||||||
|
"recall@5": 0.5213459159887731,
|
||||||
|
"recall@10": 0.6027048348476919,
|
||||||
|
"precision@5": 0.18285714285714294,
|
||||||
|
"mrr@10": 0.4608730158730158,
|
||||||
|
"nonsense_clean": "2/3",
|
||||||
|
"superseded_outranks": "2/3",
|
||||||
|
"latency_ms_p50": 1227.1,
|
||||||
|
"latency_ms_p95": 1692.6,
|
||||||
|
"latency_ms_max": 1710.4,
|
||||||
|
"errors": 0,
|
||||||
|
"by_category": {
|
||||||
|
"associative": {
|
||||||
|
"n": 6,
|
||||||
|
"hit@5": 0.6666666666666666,
|
||||||
|
"recall@5": 0.07342657342657344,
|
||||||
|
"recall@10": 0.24825174825174823,
|
||||||
|
"mrr@10": 0.20833333333333334
|
||||||
|
},
|
||||||
|
"exact_rare": {
|
||||||
|
"n": 6,
|
||||||
|
"hit@5": 1.0,
|
||||||
|
"recall@5": 1.0,
|
||||||
|
"recall@10": 1.0,
|
||||||
|
"mrr@10": 1.0
|
||||||
|
},
|
||||||
|
"nonsense": {
|
||||||
|
"n": 3,
|
||||||
|
"clean": 2,
|
||||||
|
"avg_false_positives": 3.3333333333333335
|
||||||
|
},
|
||||||
|
"paraphrase": {
|
||||||
|
"n": 13,
|
||||||
|
"hit@5": 0.6153846153846154,
|
||||||
|
"recall@5": 0.6153846153846154,
|
||||||
|
"recall@10": 0.6153846153846154,
|
||||||
|
"mrr@10": 0.2846153846153846
|
||||||
|
},
|
||||||
|
"phrase": {
|
||||||
|
"n": 7,
|
||||||
|
"hit@5": 0.7142857142857143,
|
||||||
|
"recall@5": 0.40093537414965985,
|
||||||
|
"recall@10": 0.5150226757369615,
|
||||||
|
"mrr@10": 0.6507936507936508
|
||||||
|
},
|
||||||
|
"superseded": {
|
||||||
|
"n": 3,
|
||||||
|
"hit@5": 0.3333333333333333,
|
||||||
|
"recall@5": 0.3333333333333333,
|
||||||
|
"recall@10": 0.6666666666666666,
|
||||||
|
"mrr@10": 0.20833333333333334,
|
||||||
|
"outranks": 2
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"repeat_variance": {
|
||||||
|
"baseline": {
|
||||||
|
"runs": 2,
|
||||||
|
"hit@5_min": 0.6285714285714286,
|
||||||
|
"hit@5_max": 0.6285714285714286,
|
||||||
|
"spread_queries": 0
|
||||||
|
},
|
||||||
|
"candidate": {
|
||||||
|
"runs": 2,
|
||||||
|
"hit@5_min": 0.6857142857142857,
|
||||||
|
"hit@5_max": 0.6857142857142857,
|
||||||
|
"spread_queries": 0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user