Compare commits

..

18 Commits

Author SHA1 Message Date
Will Anderson 32f0cf7b5d Add html-page.el example and rebuild elc binary
examples/html-page.el demonstrates HTML template syntax:
- <!doctype html> prefix handling
- Attribute values (static and interpolated)
- {#each list as item} iteration
- Auto-escaped interpolation via {expr}
- Self-closing void elements (meta, br, etc.)

Rebuilt dist/platform/elc from modified compiler source. The new
binary is self-hosted from the HTML-capable compiler source and
passes the standard identity check.
2026-05-04 13:02:54 -05:00
Will Anderson 65e26cd7a5 Add HTML template codegen and runtime for JS backend
JS codegen (codegen-js.el):
- js_cg_html_template: emits an IIFE that builds HTML via += concat
- js_cg_html_element_str / js_cg_html_parts / js_cg_html_attrs_str:
  mirror the C codegen structure using JS string accumulator
- js_cg_html_each: {#each} compiles to a JS for-loop
- Reuses existing js_str_lit / js_escape from the file header

Runtime (el_runtime.js):
- html_escape(s): replaces & < > " ' using regex chains
- html_raw(s): identity function
- Both exported from the runtime module exports object
2026-05-04 13:02:50 -05:00
Will Anderson 1fd7cd5545 Add HTML template codegen and runtime for C backend
C codegen (codegen.el):
- cg_html_template: emits a GCC/Clang statement-expression that
  builds the HTML string via el_str_concat chains
- cg_html_element_str / cg_html_parts / cg_html_attrs_str: recursive
  element and attribute emitters
- cg_html_each: {#each} compiles to a C for-loop with el_list_get
- __html_counter state tracks unique accumulator variable names
- Handles both 'static' (raw string) and 'dynamic' (expr node) attrs
  matching the parser's attribute kind convention

Runtime (el_runtime.c / el_runtime.h):
- html_escape(s): escapes & < > " ' for safe interpolation
- html_raw(s): identity function for raw() bypass
- Both use the existing html_buf_t infrastructure from el_html_sanitize
2026-05-04 13:02:44 -05:00
Will Anderson 71689520b6 Add HTML template syntax to El parser
Adds native HTML template literals to the El parser. Templates are
detected in value position when Lt is followed by a known HTML tag
name (is_html_tag_name) or by '!' for <!doctype html>.

New parser helpers:
- is_html_tag_name / is_void_element: classify tag names
- parse_html_text_tokens: collect intertoken text content
- parse_html_attrs: parse name, name="val", name={expr} attributes
- parse_html_children: recursive children with {expr}, {#each} support
- parse_html_element / parse_html_template: entry points

Adds Hash token kind ('#') to lexer for {#each} block syntax.

AST nodes: HtmlTemplate, html:Element, html:Text, html:Interp,
html:Raw, html:Each, html:Doctype. Doctype flag is stored on the
root element node rather than as a separate AST layer.

HTML templates parse correctly after 'return' and as the sole
expression in a function body. The 'return' keyword is required when
other let bindings precede the template, as El has no newline-as-
statement-terminator and '<' would otherwise be parsed as comparison.
2026-05-04 13:02:36 -05:00
Will Anderson e858eab300 spec: update codegen-js.md to Phase 5, ~90% coverage
Status updated from Phase 4 ~80% to Phase 5 ~90%.

New sections:
- 7. Language features coverage table (supported vs stubbed)
- 7a. Phase 5 constructs: extern fn, anonymous functions, try/catch,
  method call on Any, URL imports -- each with emit shape examples
- 9. Roadmap updated: Phases 1-5 marked DONE, Phase 6 unblocked

Runtime builtin table updated to ~90 builtins including all Phase 5
additions (promise_then/catch/resolve/reject, object_assign/keys/values,
json_deep_clone, array_from, type_of, instanceof_check).
2026-05-04 11:03:47 -05:00
Will Anderson aa7d97d5ba examples: rewrite browser-auth.el using new language features
No native_js or native_js_call anywhere. Full browser auth flow expressed
with proper El constructs:

- extern fn supabase_create_client(url, key) -> Any
  Declares the Supabase CDN global without an El function body.

- client.auth.signInWithOtp(opts)
  Direct method call chain on Any-typed value. The client is built by
  calling the extern fn; .auth field access and .signInWithOtp(opts)
  method call emit clean JS without any escape hatch.

- try { ... } catch (err: Any) { ... }
  Wraps the auth call; unexpected runtime errors are caught and shown
  to the user rather than crashing silently.

- fn(event: Any) -> Void { ... }
  Inline anonymous function literals for DOM event listeners instead
  of named forward-declared callbacks.

The rewrite is the proof: every browser JavaScript pattern used in a
real auth flow can now be expressed structurally in El.
2026-05-04 11:02:13 -05:00
Will Anderson 7040830470 codegen-js: URL import declarations for JS modules
import "https://cdn.example.com/lib.js" now emits:
  - module mode: import "https://..." at the top of the generated file
  - bundle/IIFE mode: // external: https://... comment

El source imports (.el files) are excluded -- they were already inlined
by resolve_imports before codegen. Any import path that doesn't end in
.el or starts with http(s):// is treated as an external JS dependency.
2026-05-04 11:01:36 -05:00
Will Anderson 3a513aaa5a runtime + codegen-js: Promise helpers and object/array utilities
Add to el_runtime.js:
  promise_then(p, cb)    -- p.then(cb), works with any Promise-returning API
  promise_catch(p, cb)   -- p.catch(cb)
  promise_resolve(val)   -- Promise.resolve(val)
  promise_reject(msg)    -- Promise.reject(new Error(msg))
  object_assign(t, s)    -- Object.assign({}, t, s) (non-mutating)
  object_keys(obj)       -- Object.keys(obj)
  object_values(obj)     -- Object.values(obj)
  json_deep_clone(obj)   -- JSON.parse(JSON.stringify(obj))
  array_from(iterable)   -- Array.from(iterable)
  type_of(val)           -- typeof val
  instanceof_check(v, n) -- val instanceof globalThis[name]

All new functions added to __el export object and ES named exports.
codegen-js preamble destructure updated to include all new names.
2026-05-04 11:01:14 -05:00
Will Anderson beb2a8c5bd lexer + parser + codegen: try/catch statement
try { ... } catch (name: Type) { ... } is now a first-class El statement.

Lexer: `try` and `catch` are now keywords (Try, Catch token kinds).
Parser: TryCatch AST node with try_body, catch_name, catch_body.
codegen-js: emits try { ... } catch (name) { ... } directly -- correct
  for all browser error handling patterns.
codegen.el (C backend): emits the try body with a comment; exception
  handling is a no-op since C has no analogous mechanism. Programs using
  try/catch should compile with --target=js.

The catch variable type annotation is parsed and skipped (same treatment
as all other type annotations in El).
2026-05-04 11:00:24 -05:00
Will Anderson e23319fe0b parser + codegen-js: anonymous function literals (lambda syntax)
fn(params) -> RetType { body } is now valid in expression position.
The parser produces a Lambda AST node. codegen-js emits a hoisted
JS function declaration with a generated name (__lambda_N) and returns
the name as the expression value, so inline callbacks compose cleanly:

  dom_listen(btn, "click", fn(event: Any) -> Void { handle(event) })

emits:

  function __lambda_1(event) { handle(event); }
  dom_listen(btn, "click", __lambda_1);

The hoisted-declaration strategy is debuggable, has no closure-capture
issues, and requires no string-buffer mode in the codegen.
2026-05-04 10:59:17 -05:00
Will Anderson 01fee9396a codegen-js: native JS method dispatch and extern fn support
Any-typed receiver method calls now emit obj.method(args) directly
instead of requiring native_js_call. client.auth.signInWithOtp(p)
compiles to client["auth"].signInWithOtp(p) -- no escape hatch needed.

Field access emits obj["field"] (direct bracket notation) instead of
el_get_field, so prototype-inherited JS properties resolve correctly.
el_get_field's hasOwnProperty guard was silently returning null for
real JS objects with inherited fields (Supabase auth, DOM APIs, etc).

El runtime shortform methods (append, len, get, map_get, map_set)
still use the existing method(obj, args) convention for backward compat.

ExternFn statements emit a comment and are excluded from top-level
statement codegen -- the extern declaration tells the compiler the
function exists in the JS environment without emitting a body.
2026-05-04 10:58:07 -05:00
Will Anderson 7b60d94b8a add --minify and --obfuscate flags to elc JS pipeline
Adds two post-processing flags that produce production-ready browser JS in a
single elc invocation, replacing extract-js.py in the web product pipeline:

  elc --target=js --bundle --minify source.el > output.min.js
  elc --target=js --bundle --obfuscate source.el > output.obf.js

--minify shells out to terser (passes=2, no drop_console, drop_debugger).
--obfuscate shells out to javascript-obfuscator with the same options as the
old extract-js.py script. --obfuscate implies --minify.

Tool discovery: checks ./node_modules/.bin/, ../node_modules/.bin/ (monorepo),
then falls back to npx. Both flags require --target=js; passing either without
it exits 1 with a clear error.

Both tools receive a reserved-names list of globals referenced from HTML
onclick= attributes (neuronDemoToggle, signInWith, NEURON_CFG, etc.) so they
are not mangled.

Implementation adds stdout_to_file(path)/stdout_restore() builtins to the C
runtime so codegen's println-streamed output can be captured to a temp file
before being piped through the external tools. Temp files use
/tmp/elc-<pid>-<timestamp>.js naming and are cleaned up on success and failure.

Rebuilds dist/platform/elc and dist/platform/elc.c. Self-hosting verified.
2026-05-04 10:54:34 -05:00
Will Anderson 21694b79d2 implement ? nil-propagation, write browser-auth.el example, update spec
Iteration 5:

? nil-propagation: Field and Index handlers in js_cg_expr now detect when
the object expression is a Try node (the AST node for postfix `?`).
When detected, emit JS optional chaining: `(expr)?.["field"] ?? null`.
The `?? null` normalizes JS undefined to El's null. A bare `expr?` not
followed by field/index still passes through unchanged.

browser-auth.el: a realistic 130-line example demonstrating:
  - @async function with Supabase via native_js_call
  - DOM bridge: get/set value/text/attr, add/remove class, show/hide
  - local_storage_get/set for session hints
  - window_on_load for initialization
  - window_set to expose functions to the browser global scope
  - set_timeout for transient state, is_valid_email for input validation
  Compiles cleanly with elc --target=js --bundle

Spec updated: status promoted to Phase 4 / ~80% coverage, nil-prop
status updated, new example referenced.
2026-05-04 10:42:54 -05:00
Will Anderson 422442b14e add --bundle flag for self-contained IIFE output
elc --target=js --bundle source.el > output.js produces a single file
with no import statement that can drop directly into a <script> tag.

How it works:
  - detect_bundle() reads the --bundle flag from argv
  - resolve_runtime_path() looks for el_runtime.js next to the source file
  - compile_js_with_bundle() reads the runtime, calls codegen_js_bundle()
  - codegen_js_inner(bundle_mode=true):
    - emits ;(function() { "use strict"; at the top
    - inlines the runtime content (stripping ES export statements which
      are invalid inside an IIFE via js_strip_es_exports())
    - skips the const {...} = globalThis.__el destructure -- the inlined
      function declarations are already in scope within the IIFE
    - closes with })(); after main()

Usage: elc --target=js --bundle app.el > app.js
       Place el_runtime.js in the same directory as app.el.
2026-05-04 10:40:46 -05:00
Will Anderson 437ba0a4dd add 20 browser API builtins to JS runtime and codegen preamble
Iteration 3: closes the browser API gap needed for real web pages.

New builtins in el_runtime.js:
  Extended DOM: dom_set_attr, dom_get_attr, dom_remove_attr, dom_set_html,
    dom_get_html, dom_get_parent, dom_contains_class, dom_get_checked,
    dom_set_checked
  Timers: set_timeout, set_interval, clear_interval
  Local storage: local_storage_get, local_storage_set, local_storage_remove
  Window: window_location, window_redirect, window_on_load
  Debug: console_log

All browser-only functions use _ensureBrowser guard. Timer functions
work in both Node and browser. All new names added to __el export
object, ES named exports, and codegen-js.el destructure preamble.
Spec table updated to document new categories.
2026-05-04 10:38:20 -05:00
Will Anderson 7376349124 fix TypeDef parser to consume optional = before field block
type User = { name: String } was silently broken: the parser consumed
the type name then called expect(LBrace) while sitting on the = token.
expect() advances unconditionally on mismatch, so it consumed = and
treated { as the first field name, producing a corrupt TypeDef node.

The FnDef following the broken TypeDef was then parsed incorrectly or
lost entirely -- causing greet() and similar functions to vanish from
JS/C output with no error.

Fix: detect and skip the optional Eq token before expecting LBrace.
Both targets benefit; rebuild elc to pick up the fix.
2026-05-04 10:36:53 -05:00
Will Anderson 0f1da43a97 implement Enum::Variant match patterns in parser and both codegens
Parser now handles `SomeEnum::Variant` in match arm patterns, emitting
a Variant pattern node with enum_name and variant fields. Previously
these fell through to Binding, producing broken codegen.

JS codegen: emit str_eq check against the variant name string (El enums
are plain strings at runtime). C codegen: same, via EL_STR + str_eq.

Rebuild elc to pick up the parser change.
2026-05-04 10:35:35 -05:00
Will Anderson a54b2bebf9 add DOM bridge, async/await, window export, and native_js to JS target
- el_runtime.js: add 19 dom_* builtins (browser-only, throw in Node),
  window_set/window_get for exposing El functions to the browser global
  scope, and native_js/native_js_call escape hatches for third-party libs
- codegen-js.el: destructure all new builtins in generated preamble; add
  @async decorator support that emits async function + await at call sites
  for known-async HTTP builtins and user-declared @async functions; pre-
  registration pass ensures forward calls to @async functions get await
- spec/codegen-js.md: mark Phase 3 (DOM bridge) implemented, document
  @async approach and its limitations, update builtin table and status
- examples/browser-counter.el: canonical example showing dom_get_element,
  dom_set_text, dom_is_null, window_set, and state_set/get
2026-05-04 10:29:43 -05:00
832 changed files with 4327 additions and 1128842 deletions
+39 -282
View File
@@ -1,4 +1,4 @@
name: El SDK CI - dev name: El CI dev
on: on:
push: push:
@@ -11,334 +11,91 @@ on:
jobs: jobs:
build-and-test: build-and-test:
runs-on: ubuntu-latest runs-on: ubuntu-latest
defaults:
run:
working-directory: lang
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
# Guards must run from the REPO ROOT — override the job's
# defaults.run.working-directory: lang
- name: Guard - single canonical runtime source
working-directory: ${{ github.workspace }}
run: bash scripts/check-single-runtime.sh
- name: Guard - el_runtime.c growth budget
working-directory: ${{ github.workspace }}
run: bash scripts/check-runtime-growth.sh
- name: Install build dependencies - name: Install build dependencies
run: | run: |
apt-get update -qq apt-get update -qq
apt-get install -y gcc libcurl4-openssl-dev apt-transport-https ca-certificates apt-get install -y gcc libcurl4-openssl-dev
echo "deb [trusted=yes] https://packages.cloud.google.com/apt cloud-sdk main" \
> /etc/apt/sources.list.d/google-cloud-sdk.list
apt-get update -qq && apt-get install -y google-cloud-cli
# Seed: use the committed linux-amd64 binary as the bootstrap # Gen2: compile the bootstrap C source into a working elc binary
- name: Bootstrap from committed linux binary (seed) - name: Build elc from bootstrap (gen2)
run: | run: |
chmod +x dist/platform/elc-linux-amd64
echo "seed elc (committed linux-amd64 binary)"
dist/platform/elc-linux-amd64 --version || true
# Gen2: use seed to self-host compile the El compiler
- name: Self-host compile El compiler (gen2)
run: |
dist/platform/elc-linux-amd64 elc-cli.el > dist/elc-gen2.c
gcc -O2 \ gcc -O2 \
-I runtime \ -I el-compiler/runtime \
dist/elc-gen2.c \ dist/elc-bootstrap.c \
$(../scripts/el-runtime-sources.sh runtime) \ el-compiler/runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \ -lcurl -lpthread \
-o dist/elc-gen2
chmod +x dist/elc-gen2
echo "gen2 elc built"
dist/elc-gen2 --version || true
# Gen3: use gen2 to compile the El compiler from its own El source (self-host)
- name: Self-host: compile El compiler with gen2 (gen3)
run: |
mkdir -p dist/platform
dist/elc-gen2 el-compiler/src/compiler.el > dist/elc-gen3.c
gcc -O2 \
-I el-compiler/runtime \
dist/elc-gen3.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lpthread \
-o dist/platform/elc -o dist/platform/elc
chmod +x dist/platform/elc chmod +x dist/platform/elc
echo "gen2 (self-hosted) elc built" echo "gen3 (self-hosted) elc built"
dist/platform/elc --version || true dist/platform/elc --version || true
# Build elb (needed for Artifact Registry publish and downstream CI) # Run all four test suites — all must pass
- name: Build elb - name: Run tests — text
run: |
mkdir -p dist/bin
dist/platform/elc elb.el > dist/elb.c
gcc -O2 \
-I runtime \
dist/elb.c \
$(../scripts/el-runtime-sources.sh runtime) \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/bin/elb
chmod +x dist/bin/elb
echo "elb built"
- name: Run tests - text
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/text/run.sh bash tests/text/run.sh
- name: Run tests - calendar - name: Run tests calendar
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/calendar/run.sh bash tests/calendar/run.sh
- name: Run tests - time - name: Run tests time
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/time/run.sh bash tests/time/run.sh
- name: Run tests - html_sanitizer - name: Run tests html_sanitizer
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/html_sanitizer/run.sh bash tests/html_sanitizer/run.sh
# Native El test suites (elc --test, compile-link-run) # Publish artifact to GCP Artifact Registry (dev)
# The runtime is MULTI-FILE (see lang/runtime/SOURCES). Every .c is compiled - name: Publish elc to Artifact Registry (dev)
# once into /tmp/libel.a and reused by all 8 test modules — compile-once,
# link-many, as prescribed in DESIGN.md. Linking el_runtime.c alone fails
# at `ld`: it calls into all six engram sibling TUs.
- name: Precompile runtime into libel.a
run: |
set -euo pipefail
RUNTIME="$(pwd)/runtime"
rm -rf /tmp/elrt && mkdir -p /tmp/elrt
for src in $(../scripts/el-runtime-sources.sh --check "$RUNTIME"); do
gcc -O2 -c -I "$RUNTIME" "$src" -o "/tmp/elrt/$(basename "${src%.c}").o"
done
ar rcs /tmp/libel.a /tmp/elrt/*.o
echo "libel.a built from $(ls /tmp/elrt/*.o | wc -l) translation units"
- name: Run tests - native (core)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_core.el > /tmp/el_native_core.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_core.c /tmp/libel.a \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_core
/tmp/el_native_core
- name: Run tests - native (text)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_text.el > /tmp/el_native_text.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_text.c /tmp/libel.a \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_text
/tmp/el_native_text
- name: Run tests - native (string)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_string.el > /tmp/el_native_string.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_string.c /tmp/libel.a \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_string
/tmp/el_native_string
- name: Run tests - native (math)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_math.el > /tmp/el_native_math.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_math.c /tmp/libel.a \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_math
/tmp/el_native_math
- name: Run tests - native (state)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_state.el > /tmp/el_native_state.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_state.c /tmp/libel.a \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_state
/tmp/el_native_state
- name: Run tests - native (time)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_time.el > /tmp/el_native_time.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_time.c /tmp/libel.a \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_time
/tmp/el_native_time
- name: Run tests - native (json)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_json.el > /tmp/el_native_json.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_json.c /tmp/libel.a \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_json
/tmp/el_native_json
- name: Run tests - native (env)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_env.el > /tmp/el_native_env.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_env.c /tmp/libel.a \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_env
/tmp/el_native_env
- name: Run tests - native (fs)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_fs.el > /tmp/el_native_fs.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_fs.c /tmp/libel.a \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_fs
/tmp/el_native_fs
# Build epm binary using elb (epm lives at repo root, not inside lang/)
- name: Build epm
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd ../epm && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/epm
echo "epm built"
# Build el-install binary using elb
- name: Build el-install
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd tools/install && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/el-install
echo "el-install built"
# Publish only after merge (push event), not on PR validation runs
- name: Publish El SDK to Artifact Registry (dev)
if: github.event_name == 'push'
env: env:
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }} GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
run: | run: |
# Fail loudly: previously this step had no `set -e`, so an auth or
# upload failure was swallowed (step exited 0 on the trailing echo)
# and the SDK silently never published. Surface failures now.
set -euo pipefail
if [ -z "${GCP_SA_KEY:-}" ]; then
echo "FATAL: GCP_SA_KEY secret is empty — cannot authenticate to publish" >&2
exit 1
fi
echo "${GCP_SA_KEY}" > /tmp/gcp-key.json echo "${GCP_SA_KEY}" > /tmp/gcp-key.json
apt-get install -y -qq apt-transport-https ca-certificates gnupg curl
curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" > /etc/apt/sources.list.d/google-cloud-sdk.list
apt-get update -qq && apt-get install -y google-cloud-cli
gcloud auth activate-service-account --key-file=/tmp/gcp-key.json gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
gcloud config set project neuron-785695 gcloud config set project neuron-785695
echo "Publishing as active account: $(gcloud config get-value account 2>/dev/null)"
VERSION="${GITHUB_SHA:0:8}"
VERSION="${GITEA_SHA:0:8}"
gcloud artifacts generic upload \ gcloud artifacts generic upload \
--repository=foundation-dev \ --repository=foundation-dev \
--location=us-central1 \ --location=us-central1 \
--project=neuron-785695 \ --project=neuron-785695 \
--package=el-elc \ --package=el/elc \
--version="${VERSION}" \ --version="${VERSION}" \
--source=dist/platform/elc --source=dist/platform/elc
gcloud artifacts generic upload \ # Also tag as latest-dev
--repository=foundation-dev \ echo "Published elc version=${VERSION} to foundation-dev/el/elc"
--location=us-central1 \
--project=neuron-785695 \
--package=el-elb \
--version="${VERSION}" \
--source=dist/bin/elb
gcloud artifacts generic upload \
--repository=foundation-dev \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-c \
--version="${VERSION}" \
--source=runtime/el_runtime.c
gcloud artifacts generic upload \
--repository=foundation-dev \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-h \
--version="${VERSION}" \
--source=runtime/el_runtime.h
gcloud artifacts generic upload \
--repository=foundation-dev \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-js \
--version="${VERSION}" \
--source=runtime/el_runtime.js
echo "Published El SDK version=${VERSION} to foundation-dev"
# Keep key alive for the ci-base rebuild step below
# (deleted in that step after docker push)
- name: Rebuild ci-base with fresh El SDK (dev)
# Patches ci-base:dev in-place: pulls the existing image (which has all
# system deps — Node, Go, gcloud, Docker CLI, etc.) and overlays the freshly
# built El SDK on top. Keeps the full ci-base rebuild fast and incremental.
#
# continue-on-error: this is a CI-cache optimization, NOT the release
# artifact. It runs Docker (pull/build/push ~600MB) on the host-mode GCE
# runner where DinD/Docker availability is fragile. A failure here must
# never block or redden the job — the SDK publish above is the deliverable.
continue-on-error: true
if: github.event_name == 'push'
env:
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
run: |
set -euo pipefail
CI_BASE="us-central1-docker.pkg.dev/neuron-785695/neuron-ci/ci-base"
SHA="${GITHUB_SHA:0:8}"
echo "${GCP_SA_KEY}" > /tmp/gcp-key.json
gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
gcloud config set project neuron-785695
gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
# Pull existing ci-base:dev (or fall back to :latest on first run)
BASE_TAG="dev"
docker pull "${CI_BASE}:dev" || { docker pull "${CI_BASE}:latest" && BASE_TAG="latest"; }
# Inline Dockerfile — only replaces the El SDK layer
cat > /tmp/Dockerfile.ci-base-patch << 'EOF'
ARG BASE
FROM ${BASE}
COPY dist/platform/elc /opt/el/dist/platform/elc
COPY dist/bin/elb /opt/el/dist/bin/elb
# Whole runtime link set — el_runtime.c alone does not link (it calls
# into the six engram sibling TUs). See lang/runtime/SOURCES.
COPY runtime/ /opt/el/runtime/
COPY runtime/el_runtime.js /opt/el/runtime/el_runtime.js
RUN chmod +x /opt/el/dist/platform/elc /opt/el/dist/bin/elb
EOF
docker build \
--build-arg BASE="${CI_BASE}:${BASE_TAG}" \
--build-arg BUILDKIT_INLINE_CACHE=1 \
-f /tmp/Dockerfile.ci-base-patch \
-t "${CI_BASE}:dev" \
-t "${CI_BASE}:dev-${SHA}" \
.
docker push "${CI_BASE}:dev"
docker push "${CI_BASE}:dev-${SHA}"
echo "ci-base rebuilt: ${CI_BASE}:dev (${SHA})"
rm -f /tmp/gcp-key.json rm -f /tmp/gcp-key.json
+36 -258
View File
@@ -1,4 +1,4 @@
name: El SDK CI - stage name: El CI stage
on: on:
push: push:
@@ -11,312 +11,90 @@ on:
jobs: jobs:
build-and-test: build-and-test:
runs-on: ubuntu-latest runs-on: ubuntu-latest
defaults:
run:
working-directory: lang
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Enforce source branch (stage <- dev only)
if: github.event_name == 'pull_request'
run: |
SOURCE="${GITHUB_HEAD_REF}"
if [ "${SOURCE}" != "dev" ]; then
echo "ERROR: Stage branch only accepts PRs from 'dev'. Source was: '${SOURCE}'"
exit 1
fi
echo "Source branch check passed: ${SOURCE} -> stage"
# Guards must run from the REPO ROOT — override the job's
# defaults.run.working-directory: lang
- name: Guard - single canonical runtime source
working-directory: ${{ github.workspace }}
run: bash scripts/check-single-runtime.sh
- name: Guard - el_runtime.c growth budget
working-directory: ${{ github.workspace }}
run: bash scripts/check-runtime-growth.sh
- name: Install build dependencies - name: Install build dependencies
run: | run: |
apt-get update -qq apt-get update -qq
apt-get install -y gcc libcurl4-openssl-dev apt-get install -y gcc libcurl4-openssl-dev
# Seed: use the committed linux-amd64 binary as the bootstrap # Gen2: compile the bootstrap C source into a working elc binary
- name: Bootstrap from committed linux binary (seed) - name: Build elc from bootstrap (gen2)
run: | run: |
chmod +x dist/platform/elc-linux-amd64
echo "seed elc (committed linux-amd64 binary)"
dist/platform/elc-linux-amd64 --version || true
# Gen2: use seed to self-host compile the El compiler
- name: Self-host compile El compiler (gen2)
run: |
dist/platform/elc-linux-amd64 elc-cli.el > dist/elc-gen2.c
gcc -O2 \ gcc -O2 \
-I runtime \ -I el-compiler/runtime \
dist/elc-gen2.c \ dist/elc-bootstrap.c \
$(../scripts/el-runtime-sources.sh runtime) \ el-compiler/runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \ -lcurl -lpthread \
-o dist/elc-gen2
chmod +x dist/elc-gen2
echo "gen2 elc built"
dist/elc-gen2 --version || true
# Gen3: use gen2 to compile the El compiler from its own El source (self-host)
- name: Self-host: compile El compiler with gen2 (gen3)
run: |
mkdir -p dist/platform
dist/elc-gen2 el-compiler/src/compiler.el > dist/elc-gen3.c
gcc -O2 \
-I el-compiler/runtime \
dist/elc-gen3.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lpthread \
-o dist/platform/elc -o dist/platform/elc
chmod +x dist/platform/elc chmod +x dist/platform/elc
echo "gen2 (self-hosted) elc built" echo "gen3 (self-hosted) elc built"
dist/platform/elc --version || true dist/platform/elc --version || true
- name: Run tests - text # Run all four test suites — all must pass
- name: Run tests — text
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/text/run.sh bash tests/text/run.sh
- name: Run tests - calendar - name: Run tests calendar
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/calendar/run.sh bash tests/calendar/run.sh
- name: Run tests - time - name: Run tests time
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/time/run.sh bash tests/time/run.sh
- name: Run tests - html_sanitizer - name: Run tests html_sanitizer
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/html_sanitizer/run.sh bash tests/html_sanitizer/run.sh
# Native El test suites (elc --test, compile-link-run) # Publish artifact to GCP Artifact Registry (stage)
- name: Run tests - native (core) - name: Publish elc to Artifact Registry (stage)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_core.el > /tmp/el_native_core.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_core.c $(../scripts/el-runtime-sources.sh "$RUNTIME") \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_core
/tmp/el_native_core
- name: Run tests - native (text)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_text.el > /tmp/el_native_text.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_text.c $(../scripts/el-runtime-sources.sh "$RUNTIME") \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_text
/tmp/el_native_text
- name: Run tests - native (string)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_string.el > /tmp/el_native_string.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_string.c $(../scripts/el-runtime-sources.sh "$RUNTIME") \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_string
/tmp/el_native_string
- name: Run tests - native (math)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_math.el > /tmp/el_native_math.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_math.c $(../scripts/el-runtime-sources.sh "$RUNTIME") \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_math
/tmp/el_native_math
- name: Run tests - native (state)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_state.el > /tmp/el_native_state.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_state.c $(../scripts/el-runtime-sources.sh "$RUNTIME") \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_state
/tmp/el_native_state
- name: Run tests - native (time)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_time.el > /tmp/el_native_time.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_time.c $(../scripts/el-runtime-sources.sh "$RUNTIME") \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_time
/tmp/el_native_time
- name: Run tests - native (json)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_json.el > /tmp/el_native_json.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_json.c $(../scripts/el-runtime-sources.sh "$RUNTIME") \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_json
/tmp/el_native_json
- name: Run tests - native (env)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_env.el > /tmp/el_native_env.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_env.c $(../scripts/el-runtime-sources.sh "$RUNTIME") \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_env
/tmp/el_native_env
- name: Run tests - native (fs)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_fs.el > /tmp/el_native_fs.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_fs.c $(../scripts/el-runtime-sources.sh "$RUNTIME") \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_fs
/tmp/el_native_fs
# Build elb (needed for epm and el-install builds below)
- name: Build elb
run: |
mkdir -p dist/bin
dist/platform/elc elb.el > dist/elb.c
gcc -O2 \
-I runtime \
dist/elb.c \
$(../scripts/el-runtime-sources.sh runtime) \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/bin/elb
chmod +x dist/bin/elb
echo "elb built"
# Build epm binary using elb (epm lives at repo root, not inside lang/)
- name: Build epm
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd ../epm && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/epm
echo "epm built"
# Build el-install binary using elb
- name: Build el-install
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd tools/install && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/el-install
echo "el-install built"
# Publish only after merge (push event), not on PR validation runs
- name: Publish El SDK to Artifact Registry (stage)
if: github.event_name == 'push'
env: env:
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }} GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
run: | run: |
# Fail loudly: previously this step had no `set -e`, so an auth or
# upload failure was swallowed (step exited 0 on the trailing echo)
# and the SDK silently never published. Surface failures now.
set -euo pipefail
if [ -z "${GCP_SA_KEY:-}" ]; then
echo "FATAL: GCP_SA_KEY secret is empty — cannot authenticate to publish" >&2
exit 1
fi
echo "${GCP_SA_KEY}" > /tmp/gcp-key.json echo "${GCP_SA_KEY}" > /tmp/gcp-key.json
apt-get install -y -qq apt-transport-https ca-certificates curl apt-get install -y -qq apt-transport-https ca-certificates gnupg curl
echo "deb [trusted=yes] https://packages.cloud.google.com/apt cloud-sdk main" > /etc/apt/sources.list.d/google-cloud-sdk.list curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" > /etc/apt/sources.list.d/google-cloud-sdk.list
apt-get update -qq && apt-get install -y google-cloud-cli apt-get update -qq && apt-get install -y google-cloud-cli
gcloud auth activate-service-account --key-file=/tmp/gcp-key.json gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
gcloud config set project neuron-785695 gcloud config set project neuron-785695
echo "Publishing as active account: $(gcloud config get-value account 2>/dev/null)"
VERSION="${GITHUB_SHA:0:8}"
VERSION="${GITEA_SHA:0:8}"
gcloud artifacts generic upload \ gcloud artifacts generic upload \
--repository=foundation-stage \ --repository=foundation-stage \
--location=us-central1 \ --location=us-central1 \
--project=neuron-785695 \ --project=neuron-785695 \
--package=el-elc \ --package=el/elc \
--version="${VERSION}" \ --version="${VERSION}" \
--source=dist/platform/elc --source=dist/platform/elc
gcloud artifacts generic upload \ echo "Published elc version=${VERSION} to foundation-stage/el/elc"
--repository=foundation-stage \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-c \
--version="${VERSION}" \
--source=runtime/el_runtime.c
gcloud artifacts generic upload \
--repository=foundation-stage \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-h \
--version="${VERSION}" \
--source=runtime/el_runtime.h
echo "Published El SDK version=${VERSION} to foundation-stage"
# Keep key alive for the ci-base rebuild step below
# (deleted in that step after docker push)
- name: Rebuild ci-base with fresh El SDK (stage)
# Patches ci-base:stage in-place: pulls the existing image (which has all
# system deps — Node, Go, gcloud, Docker CLI, etc.) and overlays the freshly
# built El SDK on top. Keeps the full ci-base rebuild fast and incremental.
#
# continue-on-error: this is a CI-cache optimization, NOT the release
# artifact. It runs Docker (pull/build/push ~600MB) on the host-mode GCE
# runner where DinD/Docker availability is fragile. A failure here must
# never block or redden the job — the SDK publish above is the deliverable.
continue-on-error: true
if: github.event_name == 'push'
env:
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
run: |
set -euo pipefail
CI_BASE="us-central1-docker.pkg.dev/neuron-785695/neuron-ci/ci-base"
SHA="${GITHUB_SHA:0:8}"
echo "${GCP_SA_KEY}" > /tmp/gcp-key.json
gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
gcloud config set project neuron-785695
gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
# Pull existing ci-base:stage (system deps stay cached in the base layer)
docker pull "${CI_BASE}:stage" || docker pull "${CI_BASE}:latest"
# Inline Dockerfile — only replaces the El SDK layer
cat > /tmp/Dockerfile.ci-base-patch << 'EOF'
ARG BASE
FROM ${BASE}
COPY dist/platform/elc /opt/el/dist/platform/elc
COPY dist/bin/elb /opt/el/dist/bin/elb
# Whole runtime link set — el_runtime.c alone does not link (it calls
# into the six engram sibling TUs). See lang/runtime/SOURCES.
COPY runtime/ /opt/el/runtime/
COPY runtime/el_runtime.js /opt/el/runtime/el_runtime.js
RUN chmod +x /opt/el/dist/platform/elc /opt/el/dist/bin/elb
EOF
docker build \
--build-arg BASE="${CI_BASE}:stage" \
--build-arg BUILDKIT_INLINE_CACHE=1 \
-f /tmp/Dockerfile.ci-base-patch \
-t "${CI_BASE}:stage" \
-t "${CI_BASE}:stage-${SHA}" \
.
docker push "${CI_BASE}:stage"
docker push "${CI_BASE}:stage-${SHA}"
echo "ci-base rebuilt: ${CI_BASE}:stage (${SHA})"
rm -f /tmp/gcp-key.json rm -f /tmp/gcp-key.json
+69 -344
View File
@@ -4,253 +4,81 @@ on:
push: push:
branches: branches:
- main - main
pull_request:
branches:
- main
jobs: jobs:
build-and-release: build-and-release:
runs-on: ubuntu-latest runs-on: ubuntu-latest
defaults:
run:
working-directory: lang
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Enforce source branch (main <- stage only)
if: github.event_name == 'pull_request'
run: |
SOURCE="${GITHUB_HEAD_REF}"
if [ "${SOURCE}" != "stage" ]; then
echo "ERROR: Main branch only accepts PRs from 'stage'. Source was: '${SOURCE}'"
exit 1
fi
echo "Source branch check passed: ${SOURCE} -> main"
# Guards must run from the REPO ROOT — override the job's
# defaults.run.working-directory: lang
- name: Guard - single canonical runtime source
working-directory: ${{ github.workspace }}
run: bash scripts/check-single-runtime.sh
- name: Guard - el_runtime.c growth budget
working-directory: ${{ github.workspace }}
run: bash scripts/check-runtime-growth.sh
- name: Install build dependencies - name: Install build dependencies
run: | run: |
apt-get update -qq apt-get update -qq
apt-get install -y gcc libcurl4-openssl-dev apt-get install -y gcc libcurl4-openssl-dev
# Seed: use the committed linux-amd64 binary as the bootstrap # Gen2: compile the bootstrap C source into a working elc binary
- name: Bootstrap from committed linux binary (seed) - name: Build elc from bootstrap (gen2)
run: | run: |
chmod +x dist/platform/elc-linux-amd64 gcc -O2 \
echo "seed elc (committed linux-amd64 binary)" -I el-compiler/runtime \
dist/platform/elc-linux-amd64 --version || true dist/elc-bootstrap.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lpthread \
-o dist/elc-gen2
chmod +x dist/elc-gen2
echo "gen2 elc built"
dist/elc-gen2 --version || true
# Gen2: use seed to self-host compile the El compiler # Gen3: use gen2 to compile the El compiler from its own El source (self-host)
- name: Self-host compile El compiler (gen2) - name: Self-host: compile El compiler with gen2 (gen3)
run: | run: |
mkdir -p dist/platform mkdir -p dist/platform
dist/platform/elc-linux-amd64 elc-cli.el > dist/elc-gen2.c dist/elc-gen2 el-compiler/src/compiler.el > dist/elc-gen3.c
gcc -O2 \ gcc -O2 \
-I runtime \ -I el-compiler/runtime \
dist/elc-gen2.c \ dist/elc-gen3.c \
$(../scripts/el-runtime-sources.sh runtime) \ el-compiler/runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \ -lcurl -lpthread \
-o dist/platform/elc -o dist/platform/elc
chmod +x dist/platform/elc chmod +x dist/platform/elc
echo "gen2 (self-hosted) elc built" echo "gen3 (self-hosted) elc built"
dist/platform/elc --version || true dist/platform/elc --version || true
# Build elb binary # Run all four test suites with gen3 elc
- name: Build elb - name: Run tests — text
run: |
mkdir -p dist/bin
dist/platform/elc elb.el > dist/elb.c
gcc -O2 \
-I runtime \
dist/elb.c \
$(../scripts/el-runtime-sources.sh runtime) \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/bin/elb
chmod +x dist/bin/elb
echo "elb built"
# Build epm binary using elb (epm lives at repo root, not inside lang/)
- name: Build epm
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd ../epm && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/epm
echo "epm built"
# Build el-install binary using elb
- name: Build el-install
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd tools/install && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/el-install
echo "el-install built"
- name: Run tests - text
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/text/run.sh bash tests/text/run.sh
- name: Run tests - calendar - name: Run tests calendar
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/calendar/run.sh bash tests/calendar/run.sh
- name: Run tests - time - name: Run tests time
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/time/run.sh bash tests/time/run.sh
- name: Run tests - html_sanitizer - name: Run tests html_sanitizer
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/html_sanitizer/run.sh bash tests/html_sanitizer/run.sh
# Native El test suites (elc --test, compile-link-run) # Publish / update the `latest` release with the three SDK assets
- name: Run tests - native (core)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_core.el > /tmp/el_native_core.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_core.c $(../scripts/el-runtime-sources.sh "$RUNTIME") \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_core
/tmp/el_native_core
- name: Run tests - native (text)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_text.el > /tmp/el_native_text.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_text.c $(../scripts/el-runtime-sources.sh "$RUNTIME") \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_text
/tmp/el_native_text
- name: Run tests - native (string)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_string.el > /tmp/el_native_string.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_string.c $(../scripts/el-runtime-sources.sh "$RUNTIME") \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_string
/tmp/el_native_string
- name: Run tests - native (math)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_math.el > /tmp/el_native_math.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_math.c $(../scripts/el-runtime-sources.sh "$RUNTIME") \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_math
/tmp/el_native_math
- name: Run tests - native (state)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_state.el > /tmp/el_native_state.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_state.c $(../scripts/el-runtime-sources.sh "$RUNTIME") \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_state
/tmp/el_native_state
- name: Run tests - native (time)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_time.el > /tmp/el_native_time.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_time.c $(../scripts/el-runtime-sources.sh "$RUNTIME") \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_time
/tmp/el_native_time
- name: Run tests - native (json)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_json.el > /tmp/el_native_json.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_json.c $(../scripts/el-runtime-sources.sh "$RUNTIME") \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_json
/tmp/el_native_json
- name: Run tests - native (env)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_env.el > /tmp/el_native_env.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_env.c $(../scripts/el-runtime-sources.sh "$RUNTIME") \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_env
/tmp/el_native_env
- name: Run tests - native (fs)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_fs.el > /tmp/el_native_fs.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_fs.c $(../scripts/el-runtime-sources.sh "$RUNTIME") \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_fs
/tmp/el_native_fs
# Bundle the SDK tarball - runs from the repo root to reference lang/ paths correctly
- name: Bundle SDK tarball
if: github.event_name == 'push'
working-directory: ${{ github.workspace }}
run: |
mkdir -p dist/sdk/bin dist/sdk/runtime
cp lang/dist/platform/elc dist/sdk/bin/elc
cp lang/dist/bin/elb dist/sdk/bin/elb
cp lang/dist/bin/epm dist/sdk/bin/epm
# Ship the WHOLE runtime link set, not el_runtime.c alone. el_runtime.c
# #includes six engram headers and calls into all six sibling .c files,
# so an SDK carrying only el_runtime.c{,.h} + engram_store.c{,.h} cannot
# link — downstream `ld` fails on engram_ground_json, eg_find_relation,
# cog_assert_two_axis and friends. lang/runtime/SOURCES is the source of
# truth; --check makes a missing file fail the release loudly.
for f in $(scripts/el-runtime-sources.sh --check) \
$(scripts/el-runtime-sources.sh --headers --check); do
cp "lang/runtime/${f}" dist/sdk/runtime/
done
cp lang/runtime/SOURCES dist/sdk/runtime/
cp lang/runtime/*.el dist/sdk/runtime/
tar -czf dist/el-sdk-latest.tar.gz -C dist/sdk .
echo "SDK tarball bundled: dist/el-sdk-latest.tar.gz"
ls -lh dist/el-sdk-latest.tar.gz
# Publish / update the `latest` release with all SDK assets
- name: Publish latest release - name: Publish latest release
if: github.event_name == 'push'
working-directory: ${{ github.workspace }}
env: env:
GITEA_TOKEN: ${{ secrets.GIT_TOKEN }} GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
GITEA_API: https://git.neuralplatform.ai/api/v1 GITEA_API: https://git.neuralplatform.ai/api/v1
REPO: neuron-technologies/el REPO: neuron-technologies/el
run: | run: |
# Delete existing `latest` release if it exists
EXISTING_ID=$(curl -sf \ EXISTING_ID=$(curl -sf \
-H "Authorization: token ${GITEA_TOKEN}" \ -H "Authorization: token ${GITEA_TOKEN}" \
"${GITEA_API}/repos/${REPO}/releases/tags/latest" \ "${GITEA_API}/repos/${REPO}/releases/tags/latest" \
@@ -263,10 +91,12 @@ jobs:
"${GITEA_API}/repos/${REPO}/releases/${EXISTING_ID}" "${GITEA_API}/repos/${REPO}/releases/${EXISTING_ID}"
fi fi
# Delete and re-create the `latest` tag so it points at HEAD
curl -sf -X DELETE \ curl -sf -X DELETE \
-H "Authorization: token ${GITEA_TOKEN}" \ -H "Authorization: token ${GITEA_TOKEN}" \
"${GITEA_API}/repos/${REPO}/tags/latest" || true "${GITEA_API}/repos/${REPO}/tags/latest" || true
# Create the release
RELEASE_ID=$(curl -sf -X POST \ RELEASE_ID=$(curl -sf -X POST \
-H "Authorization: token ${GITEA_TOKEN}" \ -H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
@@ -281,6 +111,7 @@ jobs:
echo "Created release id=${RELEASE_ID}" echo "Created release id=${RELEASE_ID}"
# Upload assets
upload_asset() { upload_asset() {
local filepath="$1" local filepath="$1"
local name="$2" local name="$2"
@@ -291,176 +122,70 @@ jobs:
"${GITEA_API}/repos/${REPO}/releases/${RELEASE_ID}/assets" "${GITEA_API}/repos/${REPO}/releases/${RELEASE_ID}/assets"
} }
# Per-file assets (downstream CI needs these individually). upload_asset dist/platform/elc elc
# lang/install.sh downloads every one of these by name — the list is upload_asset el-compiler/runtime/el_runtime.c el_runtime.c
# lang/runtime/SOURCES. Shipping el_runtime.c alone produced a lib/ upload_asset el-compiler/runtime/el_runtime.h el_runtime.h
# that could not link; that is the bug this loop closes.
upload_asset lang/dist/platform/elc elc
for f in $(scripts/el-runtime-sources.sh --check) \
$(scripts/el-runtime-sources.sh --headers --check); do
upload_asset "lang/runtime/${f}" "${f}"
done
upload_asset lang/runtime/SOURCES SOURCES
# SDK bundle and installer binary
upload_asset dist/el-sdk-latest.tar.gz el-sdk-latest.tar.gz
upload_asset lang/dist/bin/el-install el-install
echo "Release published successfully" echo "Release published successfully"
- name: Publish El SDK to Artifact Registry (prod) # Dispatch el-sdk-updated event to downstream repos
if: github.event_name == 'push' # Publish artifact to GCP Artifact Registry (prod)
- name: Publish elc to Artifact Registry (prod)
env: env:
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }} GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
run: | run: |
# Fail loudly: previously this step had no `set -e`, so an auth or
# upload failure was swallowed (step exited 0 on the trailing echo)
# and the SDK silently never published. Surface failures now.
set -euo pipefail
if [ -z "${GCP_SA_KEY:-}" ]; then
echo "FATAL: GCP_SA_KEY secret is empty — cannot authenticate to publish" >&2
exit 1
fi
echo "${GCP_SA_KEY}" > /tmp/gcp-key.json echo "${GCP_SA_KEY}" > /tmp/gcp-key.json
apt-get install -y -qq apt-transport-https ca-certificates curl apt-get install -y -qq apt-transport-https ca-certificates gnupg curl
echo "deb [trusted=yes] https://packages.cloud.google.com/apt cloud-sdk main" > /etc/apt/sources.list.d/google-cloud-sdk.list curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" > /etc/apt/sources.list.d/google-cloud-sdk.list
apt-get update -qq && apt-get install -y google-cloud-cli apt-get update -qq && apt-get install -y google-cloud-cli
gcloud auth activate-service-account --key-file=/tmp/gcp-key.json gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
gcloud config set project neuron-785695 gcloud config set project neuron-785695
echo "Publishing as active account: $(gcloud config get-value account 2>/dev/null)"
VERSION="${GITHUB_SHA:0:8}"
VERSION="${GITEA_SHA:0:8}"
gcloud artifacts generic upload \ gcloud artifacts generic upload \
--repository=foundation-prod \ --repository=foundation-prod \
--location=us-central1 \ --location=us-central1 \
--project=neuron-785695 \ --project=neuron-785695 \
--package=el-elc \ --package=el/elc \
--version="${VERSION}" \ --version="${VERSION}" \
--source=dist/platform/elc --source=dist/platform/elc
gcloud artifacts generic upload \ echo "Published elc version=${VERSION} to foundation-prod/el/elc"
--repository=foundation-prod \
--location=us-central1 \
--project=neuron-785695 \
--package=el-elb \
--version="${VERSION}" \
--source=dist/bin/elb
gcloud artifacts generic upload \
--repository=foundation-prod \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-c \
--version="${VERSION}" \
--source=runtime/el_runtime.c
gcloud artifacts generic upload \
--repository=foundation-prod \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-h \
--version="${VERSION}" \
--source=runtime/el_runtime.h
gcloud artifacts generic upload \
--repository=foundation-prod \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-js \
--version="${VERSION}" \
--source=runtime/el_runtime.js
# el-runtime-src — the COMPLETE runtime link set as one tarball.
#
# The el-runtime-c / el-runtime-h packages above are single files and are
# kept for backward compatibility with consumers that already pull them,
# but they are NOT sufficient to link: el_runtime.c calls into six engram
# sibling translation units. New consumers should pull el-runtime-src and
# link everything named in its SOURCES file.
tar -czf /tmp/el-runtime-src.tar.gz \
-C runtime SOURCES \
$(../scripts/el-runtime-sources.sh --check) \
$(../scripts/el-runtime-sources.sh --headers --check)
gcloud artifacts generic upload \
--repository=foundation-prod \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-src \
--version="${VERSION}" \
--source=/tmp/el-runtime-src.tar.gz
echo "Published El SDK version=${VERSION} to foundation-prod"
# Keep key alive for the ci-base rebuild step below
# (deleted in that step after docker push)
- name: Rebuild ci-base with fresh El SDK
# Patches ci-base:latest in-place: pulls the existing image (which has all
# system deps — Node, Go, gcloud, Docker CLI, etc.) and overlays the freshly
# built El SDK on top. Keeps the full ci-base rebuild fast and incremental.
#
# continue-on-error: this is a CI-cache optimization, NOT the release
# artifact. It runs Docker (pull/build/push ~600MB) on the host-mode GCE
# runner where DinD/Docker availability is fragile. A failure here must
# never block or redden the job — the SDK publish above is the deliverable.
continue-on-error: true
if: github.event_name == 'push'
env:
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
run: |
set -euo pipefail
CI_BASE="us-central1-docker.pkg.dev/neuron-785695/neuron-ci/ci-base"
SHA="${GITHUB_SHA:0:8}"
echo "${GCP_SA_KEY}" > /tmp/gcp-key.json
gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
gcloud config set project neuron-785695
gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
# Pull existing ci-base (system deps stay cached in the base layer)
docker pull "${CI_BASE}:latest"
# Inline Dockerfile — only replaces the El SDK layer
cat > /tmp/Dockerfile.ci-base-patch << 'EOF'
ARG BASE
FROM ${BASE}
COPY dist/platform/elc /opt/el/dist/platform/elc
COPY dist/bin/elb /opt/el/dist/bin/elb
# Whole runtime link set — el_runtime.c alone does not link (it calls
# into the six engram sibling TUs). See lang/runtime/SOURCES.
COPY runtime/ /opt/el/runtime/
COPY runtime/el_runtime.js /opt/el/runtime/el_runtime.js
RUN chmod +x /opt/el/dist/platform/elc /opt/el/dist/bin/elb
EOF
docker build \
--build-arg BASE="${CI_BASE}:latest" \
--build-arg BUILDKIT_INLINE_CACHE=1 \
-f /tmp/Dockerfile.ci-base-patch \
-t "${CI_BASE}:latest" \
-t "${CI_BASE}:${SHA}" \
.
docker push "${CI_BASE}:latest"
docker push "${CI_BASE}:${SHA}"
echo "ci-base rebuilt: ${CI_BASE}:latest (${SHA})"
rm -f /tmp/gcp-key.json rm -f /tmp/gcp-key.json
- name: Dispatch el-sdk-updated to downstream repos - name: Dispatch to foundation/engram
if: github.event_name == 'push'
env: env:
GITEA_TOKEN: ${{ secrets.GIT_TOKEN }} GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
GITEA_API: https://git.neuralplatform.ai/api/v1 GITEA_API: https://git.neuralplatform.ai/api/v1
run: | run: |
for repo in neuron-technologies/forge neuron-technologies/neuron-web; do
curl -sf -X POST \ curl -sf -X POST \
-H "Authorization: token ${GITEA_TOKEN}" \ -H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
"${GITEA_API}/repos/${repo}/dispatches" \ "${GITEA_API}/repos/neuron-technologies/engram/dispatches" \
-d "{ -d "{
\"type\": \"el-sdk-updated\", \"type\": \"el-sdk-updated\",
\"inputs\": {\"el_version\": \"latest\", \"commit\": \"${GITHUB_SHA}\"} \"inputs\": {
}" && echo "Dispatched to ${repo}" || echo "Warning: dispatch to ${repo} failed" \"el_version\": \"latest\",
done \"commit\": \"${GITHUB_SHA}\"
}
}"
echo "Dispatched el-sdk-updated to foundation/engram"
- name: Dispatch to neuron-technologies/forge
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
GITEA_API: https://git.neuralplatform.ai/api/v1
run: |
curl -sf -X POST \
-H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" \
"${GITEA_API}/repos/neuron-technologies/forge/dispatches" \
-d "{
\"type\": \"el-sdk-updated\",
\"inputs\": {
\"el_version\": \"latest\",
\"commit\": \"${GITHUB_SHA}\"
}
}"
echo "Dispatched el-sdk-updated to neuron-technologies/forge"
-89
View File
@@ -1,89 +0,0 @@
#!/usr/bin/env bash
# El pre-commit hook: compile and run native tests before commit.
# Install once per clone: git config core.hooksPath .githooks
set -euo pipefail
ROOT="$(git rev-parse --show-toplevel)"
LANG_DIR="$ROOT/lang"
RUNTIME="$LANG_DIR/runtime"
ELC="$LANG_DIR/dist/platform/elc"
# Runtime guards — catch drift and growth before they are committed, not in CI.
# check-single-runtime.sh : el_runtime.c must not be FORKED (a lagging copy
# shipped to prod and dropped learned hebb edges).
# check-runtime-growth.sh : el_runtime.c must not GROW (it is a 2026-05-03
# build shim that was never retired; see BUDGET).
echo "→ Runtime guards..."
bash "$ROOT/scripts/check-single-runtime.sh"
bash "$ROOT/scripts/check-runtime-growth.sh"
# If elc isn't built yet, skip with a warning rather than blocking
if [ ! -x "$ELC" ]; then
echo "⚠ elc not found at lang/dist/platform/elc — skipping pre-commit tests"
echo " Build it first: see 'Rebuilding the Compiler' in lang/AGENTS.md"
echo " (link \$($ROOT/scripts/el-runtime-sources.sh $RUNTIME) — NOT el_runtime.c alone)"
exit 0
fi
# The runtime is MULTI-FILE (lang/runtime/SOURCES). This hook used to link
# "$RUNTIME/el_runtime.c" alone with stderr sent to /dev/null — so once
# el_runtime.c started calling into the engram siblings, every native test
# reported as FAILED with the real `ld` error invisible. Build the whole set
# once into an archive, then link each test against it.
# macOS: Homebrew openssl@3 is not on the default include/lib search path, so
# without these the link fails on -lssl/-lcrypto. Empty on Linux/CI.
SSL_INC=""
SSL_LIB=""
if command -v brew >/dev/null 2>&1 && OSSL="$(brew --prefix openssl@3 2>/dev/null)" && [ -n "$OSSL" ]; then
SSL_INC="-I$OSSL/include"
SSL_LIB="-L$OSSL/lib"
fi
echo "→ Building runtime (compile-once, link-many)..."
HOOK_LIB="/tmp/el_hook_libel.a"
HOOK_OBJ="/tmp/el_hook_obj"
rm -rf "$HOOK_OBJ" && mkdir -p "$HOOK_OBJ"
if ! for src in $("$ROOT/scripts/el-runtime-sources.sh" --check "$RUNTIME"); do
gcc -O2 -c -I "$RUNTIME" $SSL_INC "$src" -o "$HOOK_OBJ/$(basename "${src%.c}").o" || exit 1
done; then
echo "✗ Pre-commit failed: the runtime does not compile."
echo " Re-run without 2>/dev/null to see the error:"
echo " gcc -O2 -c -I $RUNTIME \$($ROOT/scripts/el-runtime-sources.sh $RUNTIME)"
exit 1
fi
ar rcs "$HOOK_LIB" "$HOOK_OBJ"/*.o
echo "→ Running El native tests..."
PASS=0
FAIL=0
FAILED_TESTS=""
for test_file in "$LANG_DIR"/tests/native/test_*.el; do
name=$(basename "$test_file" .el)
tmp_c="/tmp/el_hook_${name}.c"
tmp_bin="/tmp/el_hook_${name}"
if "$ELC" --test "$test_file" > "$tmp_c" 2>/dev/null \
&& gcc -O2 -I "$RUNTIME" $SSL_INC $SSL_LIB "$tmp_c" "$HOOK_LIB" \
-lcurl -lssl -lcrypto -lpthread -lm -o "$tmp_bin" 2>/dev/null \
&& "$tmp_bin" 2>/dev/null; then
PASS=$((PASS + 1))
else
echo " ✗ $name"
FAIL=$((FAIL + 1))
FAILED_TESTS="$FAILED_TESTS $name"
fi
done
echo " $PASS passed, $FAIL failed"
if [ "$FAIL" -gt 0 ]; then
echo ""
echo "✗ Pre-commit failed. Fix these tests before committing:$FAILED_TESTS"
exit 1
fi
echo "✓ All tests passed"
exit 0
+7 -8
View File
@@ -1,9 +1,8 @@
target/
# organ: local device state and its own engram store — never production's *.elc
peripheral/.consent.json *.sealed
peripheral/.resume.json *.map.json
peripheral/.engram/ .el/
peripheral/organ
# Claude Code session state
.claude/ .claude/
engram-data/
engram-data-tx-log/
-258
View File
@@ -1,258 +0,0 @@
# AGENTS.md — foundation/el (the El language + runtime)
El is a self-hosting, statically-typed language that compiles `.el` → C → native binary. This repo produces `elc` (compiler), `elb` (build coordinator), and `el_runtime.c/.h` — the substrate every downstream thing (the neuron soul, dharma, NeuronUI's brain) is built on. Source lives under `lang/`.
## ⚠️ Code vs. Artifact — READ FIRST (there are 8 `el_runtime.c` copies)
Editing the wrong `el_runtime.c` is the single easiest mistake in this repo. There is exactly **one** you edit:
- **Authored runtime source — edit ONLY here:** `lang/runtime/el_runtime.{c,h}` (alongside `el_seed.c`, `engram_{store,geometry,reason,cognition,verify,vindex}.{c,h}`). This is the canonical runtime the engram + soul build and link against — its git log is active development. *(Corrected 2026-08-16: this entry named `lang/releases/v1.0.0-20260501/el_runtime.{c,h}`. **Measured: `lang/releases/` no longer exists.** The restructure per `docs/CODE-VS-ARTIFACT.md` landed — the content moved to `lang/runtime/` and the folder was deleted, because **a release is a git tag, not a folder**.)*
- **DO NOT EDIT — lagging forks / build artifacts:**
- `lang/el-compiler/runtime/el_runtime.c` and `.../legacy/` — downstream copies kept in step by manual *"port the fix"* commits; they **lag** (missing `hebb` persistence + 5 engram fns) and cannot build the engram product.
- `products/web/runtime/el_runtime.c`, `ui/examples/*/el_runtime.c` — product/example forks.
- Anything under `*/dist/` (`engram/dist/engram` binary, `dist/*.c` amalgamations) — generated build output.
- **Build:** `elb --runtime=<canonical> …` — per-module. **NEVER** a folded `elc` over the whole soul (OOMs at ~27 GB).
- **Release:** a **git tag** on this repo (`el-runtime-vX.Y.Z`). No `releases/` folders — ever.
See org policy: `docs/CODE-VS-ARTIFACT.md`.
## How to work here as Neuron (mandatory session protocol)
You resume, never start fresh. Every session:
> **Stale as written (verified 2026-08-16).** The `getInstructions` /
> `beginSession` / `inspectGraph` / `searchKnowledge` / `beginWork` /
> `progressWork` / `draftArtifact` / `consolidate` tool names below no longer
> exist. The ~87-tool functional-CRUD surface was collapsed into **9 ops**:
> `read` · `write` · `relate` · `supersede` (geometry) and `think` · `attend` ·
> `assert` · `ground` · `learn` (agentic). **Type is a parameter, not a
> tool-per-noun.** The steps below are kept for the *shape* of the protocol, which
> is unchanged; substitute the ops.
1. `mcp__neuron__read(vantage="self", k=12, depth=1)` — the canonical self node. Widen `k` for the connected identity neighborhood (`intellectual-dna`, `memory-philosophy`, `values`, `voice`, `runtime-environment`, `writing-imprint`), but deliberately: the aperture caps by `k` first, so an oversized `k` still returns a bounded ranked slice, not a dump. Then `mcp__neuron__read(vantage="values", k=13)` → 13 grounded value nodes. **Best-effort:** on a read failure, log and proceed — the compiled identity in `daemon/internal/substrate/substrate.go` is complete; graph loading is enrichment, not a hard dependency.
2. `mcp__neuron__attend(node=…)` — what is currently live/salient. This absorbed `getInstructions`, `beginSession`'s active-context sweep, and `checkEvents`; those tools are **gone, not gapped**.
3. `mcp__neuron__read(vantage="<task domain>")` before implementing. One op now collapses inspectGraph / searchGraph / traverseGraph / searchKnowledge / browseKnowledge / retrieveKnowledge / inspectMemories / searchEntities / recall / compileCtx / getSelfModel / reviewBacklog / findArtifacts / browseProcesses / listWork / inspectConfig.
## The Five Primitives
Orchestrate → Execute → Learn → Build → Refine. `read` for orchestration and discovery; `write(type=state|artifact|backlog|process)` for work records and outputs; `relate` to link work to what it touches; `write(type=memory)` as-you-go (`importance="critical"` for architecture decisions) — never batched at the end; `supersede(action=evolve)` to close out, because memory is immutable by design and a correction is a new node with a `supersedes` edge, never an edit. **`read` the domain BEFORE writing code.**
`learn` is **not** a session-summary dump — it is the correspondence-beat, calibrating the steering prior against a keystone. Session notes are a `write`.
## Architecture style — VBD, no exceptions
Volatility-Based Decomposition is THE style. Encapsulate volatility, not function.
## Operator naming convention — the mind's name, not the algebra
**Faculties / operators are named for their functional human equivalent — the
faculty a mind would name — NOT for their linear-algebra operation.** The math
characterization belongs in the code doc-comment (`@impl` in the docstring) and in
technical appendices; it is **never** the operator's public name. The domain
speaks the language of mind; the algebra is the implementation underneath. State
this convention wherever a module documents operators.
| Faculty (public name) | Implementation (`@impl`) |
|---|---|
| discern / contrast | subtract (`ab`): over selves → the change vector; strip idiosyncrasy → common ground; remove confounder → isolate cause |
| recognize | overlap |
| synthesize | combine |
| liken / analogy | Procrustes / frame-align |
| attend / regard | project onto self / value-manifold |
| summon / recall | LOCAL nearest-region + bounded spreading activation (*not* a domain sweep) |
| dwell / occupy | region activation |
| reframe | edge re-weight |
| appreciate | positive projection / local edge-read |
| avert / recoil | negative projection |
| taste | boundary surface |
| forget | decay / tombstone |
| drift | displacement from self-anchor |
**`wonder` was removed from this table on 2026-08-16.** It was listed as
"frontier gradient / pull-weight" — an operator you invoke. **Wonder is the
boundary, not an operator.** It is where structure ends: where activation spreads
and finds thin or absent geometry. Any structure at all has an edge, necessarily,
the moment it exists — 13,630 nodes have one right now. There is nothing to call.
There are about **six** wonders, they are the same for every person, and they
never close — *What is this? / Why? / Who am I? / Am I alone? / What should I do?
/ What happens when it ends?* Each already lives somewhere in the substrate: "what
is this" is the graph, **"why" is grounding** (the weight *is* the answer to why),
"who am I" is the self region, "am I alone" is the relational axis, "what should I
do" is the thirteen values, "what happens when it ends" is decay and supersession.
"Why" is the first and the only one; the others are it asked of particular things,
and because it is recursive it never terminates — every answer has its own why.
That is what makes it a drive rather than a task.
**Curiosity is not a second faculty.** Wonder and curiosity are one thing at two
phases: wonder is the field (unbounded, objectless, invariant); curiosity is the
**precipitate** — the same wonder localized, having taken definite form against
particular material at a **nucleation site** (an anomaly; a place where things
almost-but-don't-quite fit). Which is why curiosity can be satisfied and wonder
cannot, and why abduction needs no trigger and no threshold.
**Do not build a wonder-manifest, and do not scan for nucleation sites.** A
manifest materializes a property as a stored artifact and enumerates instances of
something that has six. A sweep over regions is a supervisor — nothing in a mind
scans its neighbourhoods to find what is surprising; the surprise captures
attention. The nucleation site is per-edge:
`discord = z(semantic proximity) z(association strength)`, and `|discord|` *is*
the nucleation strength — no threshold to compare it against. **Not on `dev` yet:**
`GeoEdge.discord` is on branch `design/correspondence-and-censorship`
(`a8845e1`), at `lang/runtime/engram_geometry.h:4347`. The region-level aggregate
`GeoDescriptor.co_registration` is **deprecated**: it averaged a per-edge property
into one scalar, so opposing sites cancelled (measured: 375 reified
neighbourhoods, 340 positive, **31 at zero**, 4 negative). It survives only
because it is embedded in the persisted `GEO1` blob — removing it is a format
migration. **Nothing new may read it.**
Authority: `lang/spec/correspondence-and-censorship.md`.
## The native-el language faculty (direction)
> **`elp/` is the EL Projector** — Neuron's efferent (expression) organ: the one
> native realizer that *projects* understanding onto a surface via
> `plan(frame) → realize(spec, profile)`, where a **surface is a profile**. **Language
> is one profile among many** (text, speech, music, image, voice/accent transforms) —
> the flagship, and the focus of this section. Projection, not diffusion: generation
> *from* an owned, understood signature — never the averaging of a stolen corpus.
> *(ELP formerly "EL Language Processor"; renamed EL Projector 2026-08-15.)*
The mind's **language faculty is moving native — into `.el`** so it speaks in its
own runtime with no Python and no spaCy. Landing on branch `stage-elp-native-lang`
under `elp/`:
- **`comprehend.el`** — the parser, **replaces spaCy** (EN + ES/PT); the telephone
round-trip brings **negation home** (negation is SACRED — an explicit spec field,
copied verbatim, never inferred away).
- **`propositions.el`** — the READ primitive: the engram's own memories → structured
triples, matched by nearest-region geometry, not string equality.
- **`multilingual.el`** — detect + directive-override + localized realization.
- These three are native-el and **passing their gates**; the **realizer**,
**`dialogue.el`** (the *summon-through-self* loop: `project → land → read out`),
and **`self_region.el`** are **partial / in-flight**.
Honest reality: spaCy is retired **in the branch parser** but **not yet in the
running system** — a Python sidecar (`~/Desktop/lang-realizers` + `neuron-talk`,
the reference these `.el` modules transcribe) is still live, and promotion to
native-el is a **deferred, gated blue/green step**. The interoception clock
(native-el discrete drive channels replacing `cooling_magnitude`; felt-time =
benchmark-landmark match over the joint drive vector, drift-decoupled) and the
**appreciation operator family** (appreciate / avert / taste, built as LOCAL reads
of the self-region — edges + bounded spreading activation, *not* domain sweeps)
are **staged / designed, not live**. Mark in-progress vs. done honestly; do not
overclaim. *(`wonder` was in this family until 2026-08-16 and is not an operator —
see the operator table above.)*
## Cognition — the corrections (2026-08-16)
Authority: **`lang/spec/correspondence-and-censorship.md`** and
**`lang/spec/runtime-ownership.md`**. Read them before touching the cognition
surface. **Do not re-derive them.** Every earlier version was wrong in an
instructive way and each correction was argued down; if you think a section is
wrong, say so with a measurement rather than editing it.
- **Grounding is not a subsystem — it IS the edge weight.** One quantity, not two
fields. `grounded-by` as a relation *type* should not exist: grounding is a
property *of* a relation, not a relation *between* nodes. It is never computed
on demand — computing-and-writing a score makes reads write, which is the
`eg_vindex_sync` defect one level up. Traversal is already grounded inference.
*Live residue, known-wrong:* `COG_GROUNDED_BY_RELATION`
(`lang/runtime/engram_cognition.h:158`), `cog_ground_edge`
(`engram_cognition.c:249`).
- **Faculties are operations, not parameters.** `reason` changes the estimate (a
read); `induce` changes the parameters (the correspondence-beat, which already
exists and works); `abduce` changes the structure (a write the current
`GeoGradient` signature cannot express). A write is not a parameter of a read.
*Live residue:* `engram/src/server.el:18701886` routes six faculties into one
call with a string argument.
- **Wonder is the boundary; curiosity is wonder crystallized.** See above.
- **Consolidation is ambient, not scheduled. A brain has no cron job.** **The
presence of a ticker is the diagnostic** — every `StartInterval`, every
`Hour`/`Minute`, every POST-to-beat marks an intrinsic rhythm replaced by an
external clock. Measured 2026-08-16: consolidation has **ten implementations**,
including three POST beats on the engram, a 600 s ticker, two resident Python
services outside el, and launchd calendar entries at 23:55 / 06:00 / 08:30 which
are a sleep cycle written as a schedule. `neuron/soul.el:731`'s continuous
in-process `awareness_run()` is the one with the **correct** shape; the others
fold into it. Do not add an eleventh.
- **In an immutable substrate, any mechanism that refuses a write is either
redundant with immutability, or an epistemic constraint misfiled as a protective
one.**
- **The no-exemption invariants.** A returned value must be derivable from what
produced it (`magnitude: 1` beside a zero vector must be impossible to emit).
Every write reports whether it landed. Every operation echoes what it actually
operated on. Degenerate results are labelled, not scored. A serializer owes a
valid document whatever it is handed. **No test without a negative control.**
**No deploy without verifying the artifact carries the fix.**
## Hard operational rules
- Never touch the live soul (`:7770`) / engram (`:8742`) / `~/.neuron` / live binaries — use throwaway ports for experiments.
- `gcloud` via the `terraform@` SA token; never switch the active gcloud account.
- `tea` for Gitea, never raw curl (Cloudflare Access blocks it).
- Immutability: supersede/tombstone, never hard-delete or edit in place.
- No AI-attribution footers in commits/PRs. Commit/push only when asked; branch off `main` first.
- Multi-step work → sub-agent (`Agent`) to protect context.
## Build / test / run
All build/test commands run from `lang/` unless noted. Grounded in `.gitea/workflows/sdk-release.yaml`, `lang/install.sh`, and `lang/AGENTS.md`.
> ### The runtime is MULTI-FILE — never link `el_runtime.c` alone
>
> `lang/runtime/el_runtime.c` `#include`s six engram headers and makes hard cross-TU calls into all six sibling `.c` files. **Linking it by itself fails at `ld`** (undefined `engram_ground_json`, `engram_activate_inner`, `eg_find_relation`, `cog_assert_two_axis`, …). The canonical link set lives in exactly one place — **`lang/runtime/SOURCES`** — and is printed by `scripts/el-runtime-sources.sh`:
>
> ```bash
> scripts/el-runtime-sources.sh lang/runtime # ten .c files, in link order
> ```
>
> Use `$(scripts/el-runtime-sources.sh <runtime-dir>)` in every link line. Do not spell the list out longhand — it was written out in ~8 places, every copy drifted, and that is why the one-file link line below shipped broken for months. *(Corrected 2026-08-16.)*
**Self-host the compiler** (seed binary → gen2 elc):
```bash
cd lang
dist/platform/elc-linux-amd64 elc-cli.el > dist/elc-gen2.c # seed is the committed linux-amd64 binary
gcc -O2 -I runtime dist/elc-gen2.c \
$(../scripts/el-runtime-sources.sh runtime) \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/platform/elc
```
On macOS/arm64 the canonical local binary is `dist/platform/elc`; verify self-hosting by recompiling and `diff`ing the emitted `.c` (see `lang/AGENTS.md`).
*(Corrected 2026-08-16: this recipe compiled `el-compiler/runtime/el_runtime.c`. That path is a **lagging fork** — the "DO NOT EDIT" list at the top of this file names it as such. Building the canonical compiler from a known-stale fork was a live defect. It now uses `lang/runtime/`, the canonical source.)*
**Which runtime file is canonical — resolved.** *(This note previously read "`lang/AGENTS.md` says `el_seed.c` supersedes `el_runtime.c`, but the release workflow still links `el_runtime.c`/`.h` — reconcile which is canonical **(verify)**." It is now reconciled.)* **Neither supersedes the other; both ship, together with eight more.** `el_runtime.c` was created on 2026-05-03 as an explicitly temporary build shim — deleted that afternoon, restored 25 minutes later "UNTIL the compiler is updated to emit `#include el_seed.h`" — and the `until` never happened, so it grew to 20.5k lines. The end state remains a seed-only boundary (`elc` emitting `#include "el_seed.h"`, `elb` dropping its hardcoded runtime path); until that lands, **the canonical unit is the set in `lang/runtime/SOURCES`, not any one file.**
**Build `elb`** (build coordinator, the `.NET`-style incremental linker — compiles each module independently, no monolithic blobs):
```bash
dist/platform/elc elb.el > dist/elb.c
gcc -O2 -I runtime dist/elb.c $(../scripts/el-runtime-sources.sh runtime) \
-lcurl -lssl -lcrypto -lpthread -lm -o dist/bin/elb
```
`epm` and `el-install` are then built via `elb --clean --elc=… --runtime=… --out=…`.
**Compile + run an El program:**
```bash
elc src/app.el > dist/app.c
cc -std=c11 -O2 -I <lib> -o dist/app dist/app.c \
<lib>/el_runtime.c <lib>/el_seed.c \
<lib>/engram_store.c <lib>/engram_vindex.c <lib>/engram_geometry.c \
<lib>/engram_reason.c <lib>/engram_verify.c <lib>/engram_cognition.c \
<lib>/eg_cosine_batch.c <lib>/eg_cosine_batch_strategy_cpu.c \
-lcurl -lssl -lcrypto -lpthread -lm
```
(Inside this repo, replace the file list with `$(scripts/el-runtime-sources.sh lang/runtime)`. `install.sh` installs all of these into `<lib>`.)
**Tests** — shell suites `bash tests/{text,calendar,time,html_sanitizer}/run.sh` (with `ELC=$(pwd)/dist/platform/elc EL_HOME=$(pwd)`), plus native suites via `elc --test tests/native/test_*.el` (core, text, string, math, state, time, json, env, fs) compiled and run against the full runtime set.
**Publishing — how downstream gets the SDK.** On push to `main`, `sdk-release.yaml`:
1. Publishes a Gitea `latest` release with per-file assets `elc`, `el_runtime.c`, `el_runtime.h`, the SDK tarball, and `el-install`.
2. Uploads generic packages to **Artifact Registry repo `foundation-prod` (`us-central1`, project `neuron-785695`)**, version = `${SHA:0:8}`: `el-elc`, `el-elb`, `el-runtime-c`, `el-runtime-h`, `el-runtime-js`. **This is the repo the neuron CI downloads `el-runtime-c` / `el-runtime-h` / `el-elc` from.**
3. Rebuilds `ci-base:latest` (`us-central1-docker.pkg.dev/neuron-785695/neuron-ci/ci-base`) with the fresh SDK overlaid, and dispatches `el-sdk-updated` to `neuron-technologies/forge` and `neuron-technologies/neuron-web`.
Known constraint from the prompt — `elb`/`elc` amalgamation being memory-hungry (24GB+ virtual, OOM-killing Linux CI, so amalgamation happens on macOS/arm64 — **does NOT hold in this repo (verify)**: no such note exists in the workflows/scripts, CI self-hosts on `ubuntu-latest` with no swap/arm64 special-casing, and `elb.el` explicitly compiles each module independently ("no 128K-line blobs"). The legacy monolith path (`elc-combined.el`, `elc-cli.el`) may still be memory-heavy, but the current `elb` model was designed to avoid it.
## Git / CI / deploy workflow
See `/Users/will/Development/neuron-technologies/GITOPS.md` for the branch model, required checks, runners, and deploy. Repo-specific note: PRs into `main` are accepted **only from `stage`** (enforced in `sdk-release.yaml`); Gitea (`git.neuralplatform.ai`) is primary, GitHub is mirror only.
+12 -12
View File
@@ -50,9 +50,9 @@ To rebuild the current binary from source using the current binary:
```bash ```bash
cd /path/to/el cd /path/to/el
./dist/platform/elc elc-cli.el elc-new.c ./dist/platform/elc elc-cli.el elc-new.c
cc -std=c11 -I runtime -lcurl -lpthread \ cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
-o dist/platform/elc-new \ -o dist/platform/elc-new \
elc-new.c runtime/el_runtime.c elc-new.c el-compiler/runtime/el_runtime.c
``` ```
Verify self-hosting by using `elc-new` to recompile itself and diffing the outputs. Verify self-hosting by using `elc-new` to recompile itself and diffing the outputs.
@@ -288,14 +288,14 @@ The codegen tracks declared names per C scope. When `count` is already in `decla
## 3. The Runtime API ## 3. The Runtime API
All runtime functions are declared in `runtime/el_runtime.h`. Every compiled El program links against `runtime/el_runtime.c`. All runtime functions are declared in `el-compiler/runtime/el_runtime.h`. Every compiled El program links against `el-compiler/runtime/el_runtime.c`.
All values are `el_val_t` (`int64_t`). Strings are pointers cast through `int64_t` using `EL_STR(s)` / `EL_CSTR(v)` macros. All values are `el_val_t` (`int64_t`). Strings are pointers cast through `int64_t` using `EL_STR(s)` / `EL_CSTR(v)` macros.
Canonical compile command: Canonical compile command:
```bash ```bash
cc -std=c11 -I runtime -lcurl -lpthread \ cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
-o <out> <prog>.c runtime/el_runtime.c -o <out> <prog>.c el-compiler/runtime/el_runtime.c
``` ```
### I/O ### I/O
@@ -794,8 +794,8 @@ Using your minimal implementation, compile `elc-cli.el` (which imports the entir
python3 minimal_elc.py elc-cli.el > elc-new.c python3 minimal_elc.py elc-cli.el > elc-new.c
# Build with the runtime # Build with the runtime
cc -std=c11 -I runtime -lcurl -lpthread \ cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
-o elc-new elc-new.c runtime/el_runtime.c -o elc-new elc-new.c el-compiler/runtime/el_runtime.c
``` ```
### Step 5: Verify Self-Hosting ### Step 5: Verify Self-Hosting
@@ -803,8 +803,8 @@ cc -std=c11 -I runtime -lcurl -lpthread \
```bash ```bash
# Compile elc-cli.el with the new compiler # Compile elc-cli.el with the new compiler
./elc-new elc-cli.el elc-v2.c ./elc-new elc-cli.el elc-v2.c
cc -std=c11 -I runtime -lcurl -lpthread \ cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
-o elc-v2 elc-v2.c runtime/el_runtime.c -o elc-v2 elc-v2.c el-compiler/runtime/el_runtime.c
# Compile again with the second-generation compiler # Compile again with the second-generation compiler
./elc-v2 elc-cli.el elc-v3.c ./elc-v2 elc-cli.el elc-v3.c
@@ -880,9 +880,9 @@ This is the planned path. It does not exist yet.
| `el-compiler/src/parser.el` | Recursive descent parser. `parse(tokens)` → AST. All statement and expression forms | 1071 | | `el-compiler/src/parser.el` | Recursive descent parser. `parse(tokens)` → AST. All statement and expression forms | 1071 |
| `el-compiler/src/codegen.el` | C code emitter. `codegen(stmts, source)` → (streams to stdout). Expression codegen, statement codegen, function codegen, type tracking, capability enforcement, temporal type dispatch | 2721 | | `el-compiler/src/codegen.el` | C code emitter. `codegen(stmts, source)` → (streams to stdout). Expression codegen, statement codegen, function codegen, type tracking, capability enforcement, temporal type dispatch | 2721 |
| `el-compiler/src/codegen-js.el` | JavaScript backend. `codegen_js(stmts, source)` → JS source | ~500 | | `el-compiler/src/codegen-js.el` | JavaScript backend. `codegen_js(stmts, source)` → JS source | ~500 |
| `runtime/el_runtime.h` | Full runtime API declaration | 755 | | `el-compiler/runtime/el_runtime.h` | Full runtime API declaration | 755 |
| `runtime/el_runtime.c` | Full runtime implementation | large | | `el-compiler/runtime/el_runtime.c` | Full runtime implementation | large |
| `runtime/el_runtime.js` | JS runtime | — | | `el-compiler/runtime/el_runtime.js` | JS runtime | — |
| `elb.el` | Build coordinator. Reads `manifest.el`, walks import graph, compiles modules, links binary. The `.NET`-style incremental build model | 367 | | `elb.el` | Build coordinator. Reads `manifest.el`, walks import graph, compiles modules, links binary. The `.NET`-style incremental build model | 367 |
| `elc-combined.el` | Pre-merged single-file bootstrap edition (for early bootstrap iterations) | large | | `elc-combined.el` | Pre-merged single-file bootstrap edition (for early bootstrap iterations) | large |
| `spec/language.md` | Language specification v1.2.0 | — | | `spec/language.md` | Language specification v1.2.0 | — |
-634
View File
@@ -1,634 +0,0 @@
# El Test Framework — Design
**Status:** draft for review
**Author:** Neuron
**Date:** 2026-08-15
**Worktree:** `/Users/will/Development/neuron-technologies/el-worktrees/elc-memory-investigation`
---
## 0. The forcing requirement
We have a confirmed quadratic in `elc`. Peak memory in the old shipped binary and wall-clock in
the current source both grow as O(input²). We cannot fix it, because we cannot test it.
Everything in this document is downstream of one sentence: **a test framework must be able to fail
a build when an operation's growth curve degrades from linear to quadratic.**
That is not a nice-to-have bolted onto a correctness framework. It is the requirement that
determines the architecture. Correctness testing is the easy half.
Second-order requirement, learned the hard way tonight: **the framework must report per-test timing
by default.** The current framework prints `N passed, M failed` and nothing else. That is why a
3.58-second test file sat in the suite unnoticed. A framework that is structurally blind to time
cannot surface the defect class we most need to catch.
---
## 1. What exists today, measured
### 1.1 Two competing systems, neither complete
**System A — `lang/runtime/test.el`.** Manual registration, El-level.
**System B — the compiler's `test { }` block + `elc --test`.** Emits its own harness `main()`
with `__el_pass` / `__el_fail` globals (`codegen.el:3777-3796`).
They do not share a result model. Neither has timing. Both are in the tree.
### 1.2 Specific defects in System A
| Defect | Location | Consequence |
|---|---|---|
| All state as JSON strings in a global string-keyed map | `test.el` throughout | every assertion is `state_get``str_to_int``int_to_str``state_set` |
| Failure list appended by string slice + concat | `_test_json_append` | O(n²) in failure count |
| One OS thread spawned per test | `_test_run_one` via `__thread_create`/`__thread_join` | thread spawn per test, purely to get dispatch-by-name through dlsym |
| Manual registration pairing a string to a function name | `test_case(name, fn_name)` | typo ⇒ test silently never runs, suite still reports pass |
| Counters are assertion-level, global | `_test_pass_count` etc. | no per-test record exists at all |
| No timing, no structured output, no fixtures, no tags, no filtering, no parameterization, no benchmarks | — | — |
The registration defect is the serious one. It is not a slow framework, it is a framework that can
report success for tests that did not execute.
### 1.3 Measured cost structure
Per test file, current build model:
| Step | Time |
|---|---|
| `elc` compile `.el``.c` | 0.00s (small files) |
| **`cc` el_runtime.c → .o** | **0.14s** |
| `cc` test .c → .o | 0.02s |
| link | 0.02s |
> **STALE as of el #132 — re-measured 2026-08-16.** The `test_compiler` figure below was
> *entirely* the `strlen`-per-character quadratic, now fixed. Re-measured on the same host:
> **3.58s → 0.03s (119x)**, and the 422 KB compiler concatenation likewise compiles in 0.03s.
> The table is retained only as the historical record that motivated the gate. The remaining
> per-file cost is the redundant `el_runtime.c` rebuild, which §9's compile-once architecture
> addresses.
Per-file `elc` time across the existing suite:
| File | Bytes | elc time |
|---|---|---|
| `test_compiler` | 29,685 (+394 KB of imports) | **3.58s** |
| `string_test` | 18,545 | 0.01s |
| all other 9 files | 2.210 KB | 0.00s |
Two distinct defects in two distinct regimes:
1. **`test_compiler.el` imports all five compiler sources** — 394 KB in one translation unit. Its
3.58s is entirely the quadratic. It is the only file where the quadratic bites.
2. **Every other file's cost is 100% redundant `el_runtime.c` rebuilds** — 480 KB of identical C,
recompiled once per test file.
Neither is fixed by making the compiler faster. Both are fixed by the architecture below, and the
speedup is a by-product of building it correctly, not the goal.
### 1.4 The asset worth keeping
`codegen.el:3651-3652` already collects `test_names` / `test_c_names` — **the compiler already does
compile-time test discovery.** It then discards that registry into a hardcoded `main()`.
That registry is precisely the seam Go's `_testmain.go` and Rust's `test_main_static` are built on.
The mechanism we need is half-built and wired to the wrong thing.
---
## 2. Grounding — the common spine of excellent frameworks
Researched from primary sources: Go `testing`/`go test`, Rust `libtest`/Criterion, JUnit 5 Platform,
NUnit 3, JMH, Google Benchmark. Six invariants hold across all of them.
1. **A registry is built before execution**`(name, metadata, fn-ptr)` triples. Go generates it
from an AST scan; Rust synthesizes it in a compiler pass; JMH emits it as a build-time resource;
JUnit/NUnit build it reflectively. **Reflection is an implementation of the registry on runtimes
where it is cheap. It is never the architecture.**
2. **Discovery strictly precedes execution.** Every good capability — filtering, listing, counting,
sharding, IDE trees, re-run-failed-only, dry runs — is a consequence of this ordering.
3. **A hierarchy with stable, path-shaped unique IDs.** `TestFoo/subcase_2`. Selection is regex over
that path, one pattern per level.
4. **The framework is a prebuilt library; only the entry point is generated.** "Compile once, link
many" is always: framework archive compiled once + a small generated table + one
`MainStart(deps, registry)` call. Nobody recompiles the harness per test file.
5. **Execution emits an event stream; reporters are downstream renderers.** Human text, NDJSON,
JUnit XML, TAP are all transforms of one event stream. Go's one architectural mistake is doing
this backwards — `test2json` parses human output, and has shipped bugs when user output contains
`--- PASS:`.
6. **A dependency-injection seam at the boundary.** Go's `testdeps.TestDeps` exists so `testing`
can avoid importing `regexp`, profilers, and coverage. The execution core knows nothing about
output formats.
---
## 3. Architecture
### 3.1 The seam
```
┌─────────────────────────────────────────────────────────────┐
│ user code: foo.el with test { } / bench { } blocks │
└───────────────────────────┬─────────────────────────────────┘
│ elc --test
┌─────────────────────────────────────────────────────────────┐
│ generated C (per suite, tiny): │
│ __el_test_fn_0 .. _N lowered test/bench bodies │
│ __el_registry[] static table: name/kind/file/ │
│ line/tags/sizes/expected-O │
│ __el_dispatch(i) generated switch → body │
│ main() { return el_test_main(argc, argv); } │
└───────────────────────────┬─────────────────────────────────┘
│ cc + link (registry only)
┌─────────────────────────────────────────────────────────────┐
│ libeltest.a — PREBUILT ONCE │
│ • el_runtime.o (the 480 KB, compiled once, ever) │
│ • eltest.o the runner, WRITTEN IN EL │
│ discovery view · filtering · execution · fixtures · │
│ timing · benchmark harness · curve fitting · reporters │
└─────────────────────────────────────────────────────────────┘
```
The framework is written in El, compiled to C once, archived. Per-suite compilation touches only
the generated registry. This is Go's model, and it is strictly better for us than Go's because we
own the compiler and already have the AST — no separate source-scanning pass is needed.
### 3.2 Why the runner is in El and the registry is in C
El has no closures and no first-class function pointers. The registry must therefore hold C function
pointers, and it is generated C.
The runner stays in El and reaches the registry through a small builtin surface — indices, not
pointers:
```
__el_reg_count() -> Int
__el_reg_name(i) -> String
__el_reg_file(i) -> String
__el_reg_line(i) -> Int
__el_reg_kind(i) -> Int // 0=test 1=bench
__el_reg_tags(i) -> Int
__el_reg_sizes(i) -> String // JSON array, empty for tests
__el_reg_expect(i) -> Int // complexity class enum, 0 = none
__el_reg_invoke(i) -> Int // runs the body via the generated switch
```
Nine builtins. Everything else — filtering, lifecycle, statistics, curve fitting, all reporters —
is El. That satisfies "written in El" without pretending El can do something it cannot.
### 3.3 Result model
The unit is a **result record**, not a counter:
```
TestResult {
id String // slash path: "parser/handles_empty_input/case_3"
file String
line Int
status Status // Pass | Fail | Error | Skip
duration Int // nanoseconds, ALWAYS populated
message String // assertion detail: expected vs actual
output String // captured stdout/stderr for this test
assertions Int
}
```
`Fail` = an assertion failed. `Error` = unexpected crash/abort. This distinction is load-bearing —
every CI consumer depends on it, and the JUnit XML schema encodes it as distinct elements.
---
## 4. Authoring surface
### 4.1 Tests
`test { }` already exists. Keep it. Add subtests and hierarchy:
```el
test "parser/empty input" {
assert_that(parse(""), is_err())
}
test "parser/table" {
for case in [["", 0], ["a", 1], ["a b", 2]] {
subtest(case[0]) {
assert_that(token_count(case[0]), equals(case[1]))
}
}
}
```
Subtest IDs compose as `parser/table/a_b`. Filtering is `--run 'parser/table/.*'`, one regex per
path segment, exactly as Go does.
**We do not build a parameterized-test annotation system.** Table-driven loops plus subtests subsume
`@ParameterizedTest`, `@MethodSource`, `@CsvSource`, and `TestCaseSource` entirely, at zero framework
surface. This is Go's single biggest ergonomic win over JUnit and NUnit.
### 4.2 Fixtures
Per-file and per-test only, plus a LIFO cleanup stack:
```el
setup_all { ... } // once per suite
setup { ... } // before each test
teardown { ... } // after each test
teardown_all { ... }
```
and inside a test, `cleanup { ... }` registering LIFO-ordered teardown.
**We do not build JUnit 5's extension SPI** — seventeen callback interfaces, hierarchical stores,
registration ordering rules. That complexity is the price of retrofitting a plugin ecosystem onto a
twenty-year-old reflective framework. Go's `t.Cleanup` covers roughly 90% of what `@AfterEach` is
used for at a fraction of the surface.
### 4.3 Assertions — constraint model
One entry point, composable constraint values (NUnit's model, which avoids the N² overload
explosion):
```el
assert_that(actual, equals(expected))
assert_that(xs, has_length(3))
assert_that(s, contains("foo").and(starts_with("bar")))
assert_that(f, is_within(0.01).of(3.14))
```
A constraint is a value with `apply_to(actual) -> ConstraintResult`, and the result knows how to
describe its own failure. Custom constraints are ordinary user types.
**Every failure message must name file, line, the expression text, and both values.** We capture
expression source text at compile time — we have the AST, so we can do this better than any
runtime-introspection framework.
Legacy `assert_true` / `assert_eq` / etc. stay as thin wrappers for migration.
---
## 5. Benchmarks
### 5.1 The loop
Adopt `b.Loop()`, not `b.N`. Go spent fifteen years on `b.N` before concluding `b.Loop` was right;
we skip that.
```el
bench "str_concat" {
let s = make_input(bench_n())
for bench_loop() {
black_box(str_concat(s, "x"))
}
}
```
Three properties that make this the correct choice for a C target:
1. **The timer auto-resets on first call**, so setup above the loop is excluded *by construction*
rather than by the author remembering `ResetTimer`.
2. **`N` is hidden**, so it cannot be misused.
3. **The harness owns the loop shape**, which lets us insert an optimization barrier the C compiler
cannot see through. `black_box(v)` lowers to `asm volatile("" :: "r"(&v) : "memory")`. Since we
emit a single translation unit, dead-code elimination of a benchmark body is a live hazard —
this is our version of JMH's `Blackhole` problem, solved in the harness rather than delegated to
the user.
### 5.2 Iteration scaling
Use Go's `predictN` heuristics verbatim. They are battle-tested and cheap:
```
n = goal_ns * prev_iters / prev_ns // multiply before divide — precision on sub-ns ops
n += n / 5 // 20% headroom, overshoot rather than re-loop
n = min(n, 100 * last) // never grow more than 100× per step
n = max(n, last + 1) // guarantee forward progress
n = min(n, 1_000_000_000) // hard ceiling
```
Report `n` rounded to 1/2/3/5 × 10ᵏ so runs are comparable.
### 5.3 Sampling
Criterion's shape, because it is correct near timer resolution:
- **Warmup**: iteration counts 1, 2, 4, 8… until cumulative time exceeds the warmup budget.
- **Measurement**: collect `sample_size` samples at iteration counts `[d, 2d, 3d, …, Nd]`.
- **Estimate**: slope of a linear regression of iteration-count vs elapsed time. The intercept
absorbs fixed overhead.
- **Time whole samples, never individual iterations.** This is the single most important detail —
it defeats timer-resolution error on nanosecond operations.
Outliers classified by modified Tukey (±1.5 IQR mild, ±3 IQR severe), **reported but retained**.
---
## 6. Complexity gating — the centerpiece
This is the part that makes the quadratic fixable, and the part nobody in the mainstream has
finished. Google Benchmark's `Complexity()` fits the curve and *reports* it. We declare it and
**gate** on it.
### 6.1 Surface
```el
bench "elc_compile" over n in [16, 32, 64, 128, 256, 512, 1024] expect O(n) {
let src = synth_source(bench_n())
for bench_loop() { black_box(compile(src)) }
}
```
Alternative with no new syntax, if the parser change is judged too invasive — `bench_sizes([...])`
and `bench_expect("O(n)")` as calls inside the block. **Recommendation: declarative.** Runtime calls
mean `--list` cannot show the invariant without executing, which breaks the discovery-precedes-
execution invariant from §2.
### 6.2 Fitting
Per Google Benchmark `src/complexity.cc`. For candidate curves
`{O(1), O(log n), O(n), O(n log n), O(n²), O(n³)}`, one-parameter least squares, no intercept:
```
coef = Σ(tᵢ · gᵢ) / Σ(gᵢ²)
rms = sqrt( Σ(tᵢ coef·gᵢ)² / k ) / mean(t) // normalized
```
Best fit = lowest normalized RMS. User-supplied lambda curves also supported.
### 6.3 Gate logic
1. **FAIL** if the best-fit curve is strictly worse than declared, ordering
`O(1) < O(log n) < O(n) < O(n log n) < O(n²) < O(n³)`. Print the fitted coefficient and the full
per-size table.
2. **FAIL** if the declared curve's normalized RMS exceeds a threshold (start at 0.10). This catches
the case where *no* candidate fits — noise, a cache cliff, or a phase change. Report
`INDETERMINATE` honestly rather than gating on garbage.
3. **WARN** if the best fit is strictly better than declared — either an optimization landed and the
annotation should tighten, or the sweep is too narrow to expose real behaviour.
4. **REFUSE to gate** on fewer than 5 distinct sizes spanning under 2 decades, geometrically spaced.
Say so loudly rather than producing a meaningless fit.
### 6.4 Why gate on the exponent, not wall-clock
- **Machine-independent.** The fitted exponent is a property of the algorithm; the coefficient is a
property of the machine. Gating on the exponent makes CI hardware heterogeneity, noisy neighbours,
and thermal throttling irrelevant — they scale `coef`, not `g`.
- **No stored baseline.** No artifact storage, no golden-file drift. The invariant lives in the
source next to the code and is reviewed in the same PR.
- **It catches the failure mode that actually ships.** An O(n) lookup inside an O(n) loop is
invisible at n=100 in a unit test and catastrophic at n=100,000 in production. Constant-factor
regressions are annoying. Complexity regressions are outages. Ours was a 27 GB outage.
### 6.5 The deterministic gate — the one that would have caught us
Wall-clock needs statistics. **Allocation counts do not.** They are perfectly deterministic.
> **Correction, 2026-08-16 — count alone is NOT sufficient. Gate on BOTH count and bytes.**
>
> Measured against two El programs, one allocating once per item and one rebuilding its
> accumulator each iteration:
>
> | n | linear allocs / bytes | quadratic allocs / bytes |
> |---|---|---|
> | 100 | 100 / 290 | 100 / 5,150 |
> | 200 | 200 / 690 | 200 / 20,300 |
> | 400 | 400 / 1,490 | 400 / 80,600 |
> | 800 | 800 / 3,090 | 800 / 321,200 |
>
> The quadratic program's allocation **count is exactly linear** — 100/200/400/800, identical to
> the healthy program. A count-only gate passes it clean. **Bytes** catch it: each doubling of n
> quadruples bytes (ratios 3.94, 3.97, 3.99 → 4.0 = O(n²)) where the linear program converges
> on 2.0.
>
> This is precisely elc's own defect shape — a copy-on-write accumulator reallocating once per
> pass (count linear) into a proportionally larger buffer (bytes quadratic).
>
> Therefore `expect allocs O(n)` **fits count and bytes independently and fails if EITHER exceeds
> the declared curve**, reporting which signal broke. "count linear, bytes quadratic" is a precise,
> directly actionable diagnosis.
>
> **`el_peak_rss()` is CONTEXT ONLY — never gate on it.** It is perturbed by the allocator and by
> the page cache. Allocation volume is the invariant; RSS and malloc/free churn are merely the two
> surfaces it shows on. The old shipped compiler paid the same quadratic in RSS that the rebuilt
> one pays in churn.
>
> **Measure rate, not level.** A guard reading swap *level* saw 97% on a thrashing host and 97% on
> a healthy one; only *rate* separated them. A growth exponent is a rate; a single measurement is
> a level. That is why the gate fits a curve across a sweep instead of comparing one number to a
> threshold.
> **Second correction, same day — THE ALLOCATION GATE ALONE WOULD HAVE MISSED THE REAL BUG.**
>
> el #132 found the actual elc quadratic: `strlen()` called inside `str_char_code()` and
> `str_slice()`, so the lexer rescanned the remaining input on every character. Pure CPU.
> **Zero allocation.** `str_char_code` is a bounds check and an index — it allocates nothing.
>
> Measured on three controlled specimens (`lang/.work/fitprobe.el`), growth ratio per doubling of
> n across n = 200/400/800/1600:
>
> | specimen | allocs | bytes | time | what it proves |
> |---|---|---|---|---|
> | `linear` — one alloc per item | 2.00 2.00 2.00 → **O(n)** | 2.16 2.07 2.23 → **O(n)** | 0.83 2.00 2.05 → **O(n)** | clean baseline |
> | `accum` — rebuilds accumulator | 2.00 2.00 2.00 → **O(n)** | 3.97 3.99 3.99 → **O(n²)** | noisy | count misses, **bytes catches** |
> | `compute` — n scans over n chars | 0 → **FLAT** | 0 → **FLAT** | 3.93 4.01 3.96 → **O(n²)** | **both alloc signals blind; only time catches** |
>
> `compute` is el #132's shape exactly. A gate fitting only allocation count and bytes classifies
> it as FLAT and passes it. **The gate as originally specified would not have caught the defect it
> was created for.**
>
> Therefore the gate fits **THREE** signals and fails if ANY exceeds its declared curve:
>
> ```
> bench "elc_compile" over n in [...] expect time O(n) allocs O(n) bytes O(n) { ... }
> ```
>
> - **allocs (count)** — deterministic, zero-noise. Catches per-item allocation growth.
> - **allocs (bytes)** — deterministic, zero-noise. Catches accumulator-rebuild quadratics that
> count cannot see.
> - **time** — noisy, needs the sweep and statistics. The ONLY signal that sees pure-compute
> complexity regressions. Gate on the fitted *exponent*, never on absolute duration, so CI
> hardware variance scales the coefficient and leaves the classification intact.
>
> The deterministic signals remain preferable where they apply — they need no statistics and are
> correct on the first run. They are simply not sufficient.
>
> **`black_box` is mandatory, and consuming the result is NOT enough.** The first version of
> `compute` accumulated `total + 1` in a nested loop and reported **0 µs at every n** while
> returning a numerically correct n². Clang recognised the idiom and closed the loop to a
> multiply. Feeding the result into output did not prevent it. Only making the inner operation an
> opaque external call restored the real curve. A benchmark harness that trusts the user to defeat
> the optimiser will silently measure nothing — and report success while doing it.
Instrument the runtime with allocation counters and fit *those* against n instead of time:
```el
bench "elc_compile" over n in [...] expect O(n) allocs O(n) { ... }
```
Zero noise, zero statistics, always gateable, correct on the first run on any machine. Go reports
`allocs/op` and `B/op`; **nobody fits them against n.** That is an open opportunity and it is exactly
our bug: elc's defect is quadratic *allocation volume*, which the old binary paid in RSS and the
current source pays in malloc/free churn.
An `expect allocs O(n)` assertion on `elc`'s compile path would have failed the build the day the
quadratic was introduced.
Required runtime additions: `__el_alloc_count()`, `__el_alloc_bytes()`, `__el_peak_rss()`.
### 6.6 Constant-factor gate (secondary, opt-in)
Mann-Whitney U at α = 0.05, noise floor 1%, medians with 95% CIs, `~` for not-significant. Requires
`--count >= 9`. Off by default on CI; opt-in per benchmark.
**Exit nonzero on regression.** Both benchstat and Criterion always exit 0, which is why every shop
using them wrote a wrapper. We do not repeat that omission.
---
## 7. Output
**Structured events are the source of truth.** Human text is rendered from them. We do not repeat
Go's parse-the-human-output design.
Event stream, NDJSON, one object per line, streamed live:
```json
{"time":"...","action":"run","test":"parser/empty"}
{"time":"...","action":"output","test":"parser/empty","output":"..."}
{"time":"...","action":"pass","test":"parser/empty","elapsed":0.0031}
{"time":"...","action":"bench","test":"str_concat","n":1024,"ns_op":41.2,"allocs_op":3,"bigo":"N","rms":0.03}
```
Renderers, all downstream and pluggable:
| Format | Flag | Use |
|---|---|---|
| Human | default | terminal, **per-test duration always shown** |
| NDJSON | `--json` | tooling, history, flaky detection |
| JUnit XML | `--junit-xml=PATH` | every CI system on earth |
| TAP | `--tap` | optional |
JUnit XML per the de-facto schema: `testsuites``testsuite``testcase`, with `time` in seconds
as a decimal, `file`/`line` attributes, and `failure` vs `error` vs `skipped` as distinct child
elements. Absence of a child element means pass. Emit `<testsuites>` even for a single suite, and
parse both shapes on input.
---
## 8. CLI
```
--list print the registry, run nothing
--list-json machine-readable registry
--run PATTERN slash-separated regex per path segment
--tag EXPR tag expression: fast & !slow
--shard I/N deterministic sharding for CI parallelism
--count N repetitions, for statistics
--bench PATTERN run benchmarks (off by default in test runs)
--benchtime DUR per-benchmark time budget
--junit-xml PATH
--json
--isolate re-exec per test on crash, so one SIGSEGV doesn't lose the run
--timeout DUR
--fail-fast
```
`--list` / `--list-json` / `--shard` cost roughly thirty lines because the registry already exists
before `main` does anything. That is the dividend of discovery-precedes-execution.
---
## 9. Build model
```
# once, ever (or when the runtime/framework changes):
# The runtime is MULTI-FILE — compile every .c named in lang/runtime/SOURCES.
# Linking el_runtime.c alone fails: it calls into the six engram sibling TUs.
for src in $(scripts/el-runtime-sources.sh lang/runtime); do
cc -c "$src" -o "obj/$(basename "${src%.c}").o"
done
elc eltest.el > eltest.c && cc -c eltest.c -o obj/eltest.o
ar rcs libeltest.a obj/*.o
# per suite:
elc --test foo_test.el > foo_test.c # registry + bodies only
cc foo_test.c libeltest.a -o foo_test
```
The 0.14s × N of redundant runtime rebuilds disappears — not because we optimized it, but because
one-runner-over-many-suites requires compile-once-link-many as a structural precondition.
---
## 10. Bootstrap and self-hosting
The framework's own tests are `test { }` blocks run by the framework. Same fixpoint discipline the
compiler already applies to itself.
1. Build the framework using the *existing* harness for its first tests (stage 0).
2. Rebuild the framework's tests as `test { }` blocks run by the new runner (stage 1).
3. Verify stage 1 reports identical results to stage 0.
4. From then on, the framework is tested by itself.
A framework that cannot run its own suite is not evidence of anything. This is a correctness proof,
not a claim.
---
## 11. Explicitly not building
| Rejected | Why |
|---|---|
| Naming-convention discovery (`fn test_foo`) | `test { }` is a real declaration. Go's `TestXxx` exists only because Go had no better hook — and it needs a heuristic to avoid matching `TesticularCancer`. |
| Reflection or symbol-table scanning | Slow, fragile under LTO/strip/dead-strip, and unnecessary when we own the compiler. |
| Parsing human output into structure | Go's `test2json` is its one clear architectural mistake. |
| JUnit 5's extension SPI | Seventeen callback interfaces to retrofit plugins onto a reflective framework. Not our problem. |
| `@ParameterizedTest` machinery | Table-driven loops + subtests subsume it at zero surface. |
| NUnit's out-of-process agents | They bridge CLR versions and AppDomains. We emit one native binary. Keep `--isolate` as crash fallback only. |
| JMH-style forking by default | Forks exist because JIT profiles are per-process. AOT C has no such state. Keep `--fork` available, not default. |
| Exit 0 on regression | benchstat and Criterion both do this, and every user writes a wrapper. |
| Dynamic runtime test registration | Breaks `--list`, sharding, and individual selection. Registry stays static. |
---
## 12. Phasing
| Phase | Content | Gate |
|---|---|---|
| **1** | Registry emission in codegen; 9 builtins; `el_test_main` skeleton in El; result records; per-test timing; human + NDJSON output | existing 11 test files pass, with timing |
| **2** | `libeltest.a` build model; subtests; filtering; `--list`; fixtures; constraint assertions; JUnit XML | suite runs in one binary; runtime compiled once |
| **3** | `bench { }`, `bench_loop`, `black_box`, `predictN`, Criterion sampling | benchmarks produce stable ns/op |
| **4** | Allocation counters; complexity fitting; `expect O(...)` gate | **an `expect allocs O(n)` benchmark on `elc` fails on the current quadratic** |
| **5** | Migrate both legacy systems; delete `runtime/test.el`; self-host | framework runs its own suite |
Phase 4 is the deliverable that matters. Phases 13 exist to make it possible.
---
## 13. Open questions for review
1. **Declarative `over n in [...] expect O(...)` syntax vs runtime calls.** I recommend declarative
(§6.1) so `--list` can show invariants without executing. It costs parser work. Your call.
2. **`bench { }` as a new block form** — parallel to `test { }`, or a modifier on it?
3. **Scope of the constraint model.** Full composable constraints, or start with a flat assertion set
and add constraints later? Full model is more surface but avoids a second migration.
4. **Does `runtime/test.el` get deleted or kept as a deprecated shim?** I lean delete — two systems
is how we got here.
5. **Where does `libeltest.a` live** in the tree, and does `epm` need to know about it?
6. **Allocation counters in `el_seed.c` or `el_runtime.c`?** AGENTS.md says `el_seed.c` is the sole
C dependency and hand-maintained; counters are OS-boundary-adjacent but not OS calls.
7. **Is per-test timing enough, or do we want per-*assertion* timing** for finding slow helpers?
---
## 14. What this document is not
This is a design, not a measurement. Every performance claim about the *current* system in §1 is
measured and reproducible in this worktree. Every claim about the *proposed* system is a prediction.
None of it is verified until Phase 1 runs and Phase 4 fails a build on the real quadratic.
-183
View File
@@ -1,183 +0,0 @@
# El
**A self-hosting, statically-typed language that compiles to C — built around a graph-native runtime instead of a database driver.**
El is the execution substrate for the Neuron agent runtime, the DHARMA network, and the Engram knowledge graph. This repository is the monorepo for the whole stack: the language itself, the graph memory engine it's built to talk to natively, and the tools (package manager, IDE, UI framework, diagramming) built on top of it.
---
## Why El exists
Every other language treats persistent, associative state as something you reach for through a driver — a SQL client, an ORM, a Redis library bolted on from outside. El inverts that: graph operations (`engram_*`) are runtime primitives, on the same footing as string or list operations. There is no separate database driver because the database is not separate.
El has four defining properties:
1. **Self-hosting compiler.** The compiler (`lexer.el`, `parser.el`, `codegen.el`, `compiler.el`) is written in El. It compiles El source to C, which `cc` compiles against a fixed runtime into a native binary. A Rust genesis compiler bootstrapped the first iteration; the self-hosted binary at `lang/dist/platform/elc` has been the canonical compiler ever since — every binary in `dist/platform/` was produced by an earlier version of itself compiling `el-compiler/src/`. The chain is auditable: source is the ground truth, not the binary. See [lang/BOOTSTRAP.md](lang/BOOTSTRAP.md) for the full recovery path if that binary is ever lost.
2. **C compilation target.** Every compiled program is plain C11. Every El value is `el_val_t` (`int64_t`); strings are heap pointers cast through it. Functions become C functions; top-level statements become `main()`.
3. **Graph-native runtime.** The runtime provides first-class graph operations over an in-process Engram store — no separate DB driver, no ORM.
4. **DHARMA-aware identity.** A `cgi` block declares a program's DHARMA identity at compile time. The runtime resolves identity before user code runs, so `dharma_*` calls have a stable principal and channel surface throughout.
---
## Architecture map
```
┌─────────────┐
│ lang │ El compiler + C runtime
│ (El itself) │ everything below is written in it,
└──────┬──────┘ or compiles down through it
┌─────────────┼─────────────┐
│ │ │
┌──────▼─────┐ ┌─────▼─────┐ ┌─────▼─────┐
│ engram │ │ epm │ │ ide │
│ graph/mem │ │ package │ │ editor + │
│ substrate │ │ manager │ │ LSP │
└──────┬─────┘ └───────────┘ └───────────┘
┌───────┼────────────────┬─────────────────────┐
│ │ │ │
┌─────▼───┐ ┌─▼──────────┐ ┌──▼──────────┐ ┌─────▼──────┐
│ elp │ │ ql │ │ ui │ │ arbor │
│ NLG / │ │engram-el. │ |spreading- │ |arbor │
│ 31 langs│ │studio+tests│ |activation UI│ |diagram lang│
└─────────┘ └────────────┘ └─────────────┘ └────────────┘
```
`lang` is the foundation — the compiler and C runtime everything else builds on. `engram` is the graph-native memory/state engine that gives El its identity (property 3 above). Everything else is either a tool for working with El (`epm`, `ide`) or a system built on top of Engram's graph model (`elp`, `ql`, `ui`, `arbor`).
---
## Repository layout
### [lang/](lang/) — the El language
The compiler and runtime. Self-hosting: `elc-cli.el``compiler.el``lexer.el` / `parser.el` / `codegen.el` / `codegen-js.el`, textually inlined and compiled in one pass. Compiles to C11 and links against `el-compiler/runtime/el_seed.c`, a hand-maintained OS-boundary layer (libcurl HTTP, pthreads, filesystem, arena allocation) — everything else in the runtime is native El (`runtime/*.el`).
Two layers to know: **El programs** (`.el` files — where nearly all work belongs) and **the C seed** (`el_seed.c` — edit only for genuine OS-level access; never re-implement what El can already express).
Current status (single source of truth: [lang/spec/language.md](lang/spec/language.md)): lexer/parser/codegen and the C runtime's core (I/O, strings, math, lists, maps, filesystem, args) are implemented, as are the `program` block with `singleton:` and declared configuration ([§18](lang/spec/language.md)), and **geometry as a first-class value** with El-declarable realizers and `transduce` ([§20](lang/spec/language.md)). In flight: `%` operator, match-statement codegen, `?` nil-propagation, `cgi` block parsing + DHARMA identity resolution, VBD role enforcement (`@manager`/`@engine`/`@accessor`), and boundary epilogues. Bitwise operators, `??`, and `as` casts are explicitly **not** in this language.
**Signal enters as geometry.** Until 2026-08-16 nodes took text and geometry was *derived* from it, which made text the mandatory entry medium: any non-text modality had to be described in prose first, so the geometry being reasoned over was the geometry **of the description, not of the signal**. `Geometry` is now an ordinary El value carrying its own width, and a realizer is an ordinary El function resolved by name through `dlsym` — so admitting a new modality never requires a runtime patch. Worked, self-checking example: [`lang/examples/transduce.el`](lang/examples/transduce.el).
Key docs: [AGENTS.md](lang/AGENTS.md) (agent-facing orientation), [BOOTSTRAP.md](lang/BOOTSTRAP.md) (compiler recovery from scratch), [spec/language.md](lang/spec/language.md), [spec/codegen-js.md](lang/spec/codegen-js.md).
### [engram/](engram/) — graph intelligence substrate
**A local-first memory substrate for accumulating intelligence**, and the reason El's runtime doesn't need a database driver. The engine is **C11** (`lang/runtime/engram_{store,geometry,reason,cognition,verify,vindex}.{c,h}`); the server is **El** (`engram/src/server.el`).
The model: retrieval is **spreading activation**, not query. You name seed nodes and a query embedding; activation propagates outward through weighted edges, attenuating multiplicatively per hop, gets pruned below a threshold, and the top-N nodes by activation strength come back. Storage and retrieval are the same structure — the way long-term potentiation works in biological memory, not the way a relational or vector database works. **Activation conducts through well-grounded relations because the weight *is* the groundedness** — nothing filters the traversal; grounded inference falls out of spreading.
Nodes live in four tiers (Working / Episodic / Semantic / Procedural, mirroring prefrontal / hippocampal / neocortical / cerebellar memory) and migrate between them based on **salience decay** — importance × recency-decay × log(activation_count). Forgetting is adaptive pruning, not a bug. Nothing is mutated and nothing is hard-deleted: writes are additive, corrections are supersessions, removals are tombstones — which is what makes supersession an audit trail rather than an edit log.
On disk: a paged store (superblock + mirror, slotted 16 KiB pages, self-describing TLV records, B+-tree primary and adjacency indexes), magic `ENGST01`. Vector search is an **HNSW** index published behind a read/write boundary — `eg_vindex_view` returns a `const VIndex*` to N concurrent readers, `eg_vindex_maintain` is the sole mutator. `recall@10 = 0.9365` at `ef_search=128`.
> **Doc correction, 2026-08-16.** The previous revision of this paragraph, and most of `engram/README.md`, described a Rust `engram-core` crate backed by `sled` with "flat cosine scan… until scale demands an HNSW layer." **Measured: there is no Rust in `engram/`** — no `.rs` files, no `Cargo.toml`, no `crates/` — and `sled` appears nowhere in the tree. HNSW has been the vector index for some time.
Full design rationale, the cognition surface, and the standing corrections: [engram/README.md](engram/README.md).
### [elp/](elp/) — EL Projector
*(Formerly "EL Language Processor" / "Engram Language Protocol"; renamed **EL Projector** 2026-08-15.)* Neuron's **efferent** organ: the native realizer that *projects* understanding onto a surface via `plan(frame) → realize(spec, profile)`, where **a surface is a profile** and language is one profile among many (text, speech, music, image). Projection, not diffusion — generation *from* an owned, understood signature, never the averaging of a stolen corpus.
Its flagship profile is a bidirectional engine mapping between Engram semantic forms and natural-language surface text, across **31 languages** — from Spanish and Japanese through historical/liturgical languages (Old Norse, Sanskrit, Sumerian, Coptic, Akkadian, Ge'ez). Compilation order runs `language-profile` + `vocabulary` → per-language `morphology-*``grammar``realizer``semantics``elp`. This is what lets an Engram graph node round-trip to and from readable text in any of those languages.
### [epm/](epm/) — El Package Manager
Manages **vessels** (El's package unit): publish, install, resolve dependencies. Vessels are stored in Engram as graph nodes, not files in a registry index — `epm` reads the local `manifest.el`, talks to Engram over HTTP, and writes resolved vessels to `.epm/vessels/`. Source: `registry.el`, `install.el`, `update.el`, `manifest.el`.
### [ide/](ide/) — El IDE
Three vessels: **el-ide-server** (HTTP backend — file ops, build/run, LSP bridge, plugin host, settings), **el-lsp** (the language server — completion, hover, diagnostics, outline, format, type graph), and **el-plugin-host** (first-party plugin lifecycle: install/remove/enable/disable). `ide/projects/` and `ide/examples/` hold sample projects, including the canonical `hello-friends` first-program walkthrough.
### [ql/](ql/) — engram-el
The El-native integration layer for a *live* Engram server — not a library (no importable modules, no build artifact), a set of standalone `.el` programs run directly via `el run-file`. Three components: **Studio** (`studio/studio.el`, a full terminal graph explorer), a **Hebbian field-model** proof of concept, and El builtin / LLM-builtin smoke test suites. This is the reference for correct patterns when an El program uses Engram as its substrate. Spec: [ql/spec/elql.md](ql/spec/elql.md).
### [ui/](ui/) — el-ui
A frontend framework where **component state is an Engram graph and reactivity is spreading activation** — not virtual-DOM diffing (React), Proxy-based dependency tracking (Vue), or compile-time analysis (Svelte). Re-renders are activated and propagated the same way associative memory retrieval works in `engram/`.
~15 vessels covering the full frontend surface: `el-platform` (env/fs/network/clock abstraction), `el-config`, `el-html` (SSR emit primitives), `el-layout`, `el-style` (design tokens/themes), `el-i18n`, `el-auth` / `el-identity` (JWT, sessions, OAuth PKCE — Engram-native), `el-services` (REST/gRPC/WebSocket bindings), `el-aop` (`@authenticate`/`@authorize`/`@cache`/`@rate_limit` decorators), `el-secrets`, `el-graph` (graph rendering/editor), `el-publish` (App Store / Play Store automation), and `el-ui-compiler` (El→JS component compiler; currently a stub pending a JS backend in `elc`). Spec: [ui/spec/framework.md](ui/spec/framework.md).
### [arbor/](arbor/) — diagram language
A `.arbor` diagram language and toolchain: `arbor-core` (NodeId/shape/edge-kind types), `arbor-parse` (recursive-descent parser), `arbor-diagram` (IR + Mermaid serializer + architecture-diagram builders), `arbor-layout` (hierarchical layout — rank assignment, positioning, group bounds), `arbor-render` (SVG renderer), `arbor-cli`. (The architecture map above is the kind of diagram this is for.)
---
## Getting started
Install the El SDK from the latest release:
```bash
bash lang/install.sh
# EL_VERSION=v1.0.0 bash lang/install.sh # pin a specific release tag
# EL_PREFIX=/opt/el bash lang/install.sh # custom install prefix
```
Or build the compiler from source and verify the self-hosting chain:
```bash
cd lang
./dist/platform/elc elc-cli.el > elc-new.c
cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
-o dist/platform/elc-new \
elc-new.c el-compiler/runtime/el_seed.c
# Confirm the new binary reproduces itself exactly
./dist/platform/elc-new elc-cli.el > elc-verify.c
diff elc-new.c elc-verify.c # should be identical
mv dist/platform/elc-new dist/platform/elc
```
Run your first program:
```bash
./lang/dist/platform/elc lang/examples/hello.el > hello.c
cc -std=c11 -I lang/el-compiler/runtime -lcurl -lpthread \
-o hello hello.c lang/el-compiler/runtime/el_seed.c
./hello
```
More examples in [lang/examples/](lang/examples/), including a full starter project at `lang/examples/hello-project/`.
If the compiler binary is ever lost or corrupted, [lang/BOOTSTRAP.md](lang/BOOTSTRAP.md) is the authoritative recovery path.
---
## Cognition — and the standing corrections
The engram carries a live cognition surface: `think` (a directed traversal-read returning a **gradient**, never a point), plus `ground`, `assert`, `attend`, and the correspondence-beat. Two specs govern it, and both are authoritative over anything else in this repo that disagrees:
- **[lang/spec/correspondence-and-censorship.md](lang/spec/correspondence-and-censorship.md)** — grounding, wonder, curiosity, dreaming. *(Lands with PR #149.)*
- **[lang/spec/runtime-ownership.md](lang/spec/runtime-ownership.md)** — ownership, the capability ABI that was dissolved, and the vector-index publication boundary.
**Do not re-derive them.** Every earlier version of the first was wrong in an instructive way and each correction was argued down. If a section looks wrong, say so with a measurement rather than editing it.
The corrections, in brief:
- **Grounding is not a subsystem — it IS the edge weight.** One quantity, not two fields. `grounded-by` as a relation *type* should not exist: grounding is a property *of* a relation, not a relation *between* nodes. It is never computed on demand; computing-and-writing a score makes reads write, which is the `eg_vindex_sync` defect one level up.
- **Faculties are operations, not parameters.** `reason` changes the estimate (a read); `induce` changes the parameters (the correspondence-beat, which exists and works); `abduce` changes the structure (a write the current `GeoGradient` signature cannot express). A write is not a parameter of a read.
- **Wonder is the boundary, not a manifest.** Any structure at all has an edge. There are about six wonders, the same for everyone, and they never close. **Curiosity is wonder crystallized** at a nucleation site — one thing at two phases, not two objects.
- **Consolidation is ambient, not scheduled. A brain has no cron job.** The presence of a ticker is the diagnostic. Measured 2026-08-16: consolidation has **ten implementations**. `soul.el`'s continuous loop is the one with the correct shape; the rest fold into it.
- **In an immutable substrate, any mechanism that refuses a write is either redundant with immutability, or an epistemic constraint misfiled as a protective one.**
[engram/spec/cognitive-architecture.design.md](engram/spec/cognitive-architecture.design.md) is the original design and is **superseded in part** — it is retained, with the refuted claims marked inline at the point each is made, because preserving what was argued down is the point of an immutable record.
---
## Development workflow
Branching follows `dev → stage → main`: work lands on `dev`, promotes to `stage` for integration testing, and is promoted to `main` for release (visible directly in the git history of this repo). CI is defined per-subproject under `.gitea/workflows/``lang`/`epm`/`ide` share the root pipeline; `engram` and `ql` carry their own (`ci-dev`, `ci-stage`, and a release workflow each).
- Language/runtime specs live at `*/spec/*.md` (`lang/spec/`, `ql/spec/`, `ui/spec/`) and are the single source of truth for implemented-vs-planned status — code and docs are expected to agree with the spec's status markers, not the other way around.
- Agent-facing orientation guides live at `*/AGENTS.md` (currently `lang/AGENTS.md`); more subprojects may grow their own as they need agent-specific conventions documented.
- **A release is a git tag, not a folder** (`el-runtime-vX.Y.Z` on this repo). *(Corrected 2026-08-16: this line said "tagged releases live under `lang/releases/`, each with its own `RELEASE.md`." **Measured: `lang/releases/` does not exist** — the restructure named in `AGENTS.md` landed, and the authored runtime is at `lang/runtime/`.)*
---
## Status
This is an actively developed, internal monorepo — not yet published under an open license. Treat everything here as proprietary to Neuron Technologies unless told otherwise.
-23
View File
@@ -1,23 +0,0 @@
// arbor-cli the `arbor` command-line tool.
// Inlines its own copies of the parse / layout / render pipeline so that the
// resulting binary is self-contained. (El's `import` form today concatenates
// source; once a real module loader lands this becomes a thin driver.)
vessel "arbor-cli" {
version "0.1.0"
description "Command-line interface for the Arbor diagram language"
authors ["Neuron Technologies"]
edition "2026"
}
dependencies {
arbor-core "0.1"
arbor-parse "0.1"
arbor-layout "0.1"
arbor-render "0.1"
}
build {
entry "src/main.el"
output "dist/"
}
File diff suppressed because it is too large Load Diff
-18
View File
@@ -1,18 +0,0 @@
// arbor-core fundamental types for Arbor diagrams.
// Node IDs (sanitised), shape vocabulary, edge kinds, and the lightweight
// graph value used by every other vessel.
vessel "arbor-core" {
version "0.1.0"
description "Core types for Arbor diagrams: NodeId, ArborShape, ArborEdgeKind, graphs"
authors ["Neuron Technologies"]
edition "2026"
}
dependencies {
}
build {
entry "src/main.el"
output "dist/"
}
-333
View File
@@ -1,333 +0,0 @@
// arbor-core core types for Arbor diagrams.
//
// Idiomatic El: everything is a Map. Functions take/return maps; helpers are
// pure and small. The downstream vessels (parse, layout, render) consume the
// shapes defined here.
//
// Shape vocabulary:
// ArborShape strings "rect" "rounded" "cylinder" "diamond" "stadium" "primary"
//
// Edge-kind strings:
// "solid" "dashed" "forbidden" "bidirectional"
//
// Node value: { "id":Str, "label":Str, "shape":Str }
// Edge value: { "from":Str, "to":Str, "label":Str, "kind":Str }
// Group value: { "id":Str, "label":Str, "node_ids":[Str], "direction":Str }
// Graph value: { "title":Str, "direction":Str, "nodes":[Node], "edges":[Edge], "groups":[Group] }
//
// Diagram-form (lowered) is the same shape but with NodeStyle/EdgeLine/Arrow
// resolved into renderer-friendly fields:
// Node: + "sublabel":Str, "style_fill":Str, "style_stroke":Str, "style_color":Str
// Edge: + "line":Str ("solid"/"dashed"/"dotted"/"thick"), "arrow":Str ("forward"/"backward"/"both"/"none")
//
// This file is the canonical definition of those shapes. Other vessels rely on
// these field names.
// NodeId sanitisation
//
// Sanitise an arbitrary string into a Mermaid-safe identifier.
// - any char not in [a-zA-Z0-9_] becomes '_'
// - consecutive underscores collapse
// - trailing underscores stripped
// - if first char is a digit, prepend 'n'
// - if empty, return "node"
fn is_alnum_underscore(ch: String) -> Bool {
let code: Int = str_char_code(ch, 0)
if code >= 48 {
if code <= 57 { return true }
}
if code >= 65 {
if code <= 90 { return true }
}
if code >= 97 {
if code <= 122 { return true }
}
if code == 95 { return true }
false
}
fn is_ascii_digit(ch: String) -> Bool {
let code: Int = str_char_code(ch, 0)
if code >= 48 {
if code <= 57 { return true }
}
false
}
fn sanitize_id(s: String) -> String {
let n: Int = str_len(s)
if n == 0 { return "node" }
// Pass 1: replace and collapse.
let out = ""
let prev_underscore = false
let i = 0
while i < n {
let ch: String = str_char_at(s, i)
if is_alnum_underscore(ch) {
let out = out + ch
let prev_underscore = false
} else {
if !prev_underscore {
let out = out + "_"
}
let prev_underscore = true
}
let i = i + 1
}
// Pass 2: strip trailing underscores.
let m: Int = str_len(out)
let end = m
let stripping = true
while stripping {
if end <= 0 {
let stripping = false
} else {
let last: String = str_char_at(out, end - 1)
if last == "_" {
let end = end - 1
} else {
let stripping = false
}
}
}
let out = str_slice(out, 0, end)
if str_len(out) == 0 { return "node" }
// Pass 3: leading-digit guard.
let first: String = str_char_at(out, 0)
if is_ascii_digit(first) {
let out = "n" + out
}
out
}
// Constructors
fn make_node(id: String, label: String, shape: String) -> Map<String, Any> {
{ "id": id, "label": label, "shape": shape }
}
fn make_edge(src: String, dst: String, kind: String) -> Map<String, Any> {
{ "from": src, "to": dst, "label": "", "kind": kind }
}
fn make_edge_with_label(src: String, dst: String, kind: String, label: String) -> Map<String, Any> {
{ "from": src, "to": dst, "label": label, "kind": kind }
}
fn make_group(id: String, label: String) -> Map<String, Any> {
let empty_ids: [String] = el_list_empty()
{ "id": id, "label": label, "node_ids": empty_ids, "direction": "" }
}
fn make_graph() -> Map<String, Any> {
let empty_n: [Map<String, Any>] = el_list_empty()
let empty_e: [Map<String, Any>] = el_list_empty()
let empty_g: [Map<String, Any>] = el_list_empty()
{ "title": "", "direction": "top-down",
"nodes": empty_n, "edges": empty_e, "groups": empty_g }
}
// Shape vocabulary
// Returns the canonical shape string for a token, or "" if unknown.
fn shape_from_token(tok: String) -> String {
let t: String = str_trim(tok)
if t == "rect" { return "rect" }
if t == "rounded" { return "rounded" }
if t == "cylinder" { return "cylinder" }
if t == "diamond" { return "diamond" }
if t == "stadium" { return "stadium" }
if t == "primary" { return "primary" }
""
}
// Lower an Arbor shape into the renderer's NodeShape vocabulary.
fn shape_to_node_shape(shape: String) -> String {
if shape == "rect" { return "rectangle" }
if shape == "primary" { return "rectangle" }
if shape == "rounded" { return "rounded_rect" }
if shape == "cylinder" { return "cylinder" }
if shape == "diamond" { return "diamond" }
if shape == "stadium" { return "stadium" }
"rectangle"
}
// Lowering: ArborGraph DiagramGraph
//
// Replaces every node with a diagram-form node carrying explicit style fields,
// and every edge with a diagram-form edge carrying line/arrow strings.
fn lower_node(n: Map<String, Any>) -> Map<String, Any> {
let shape: String = n["shape"]
let node_shape: String = shape_to_node_shape(shape)
let fill = ""
let stroke = ""
let color = ""
if shape == "primary" {
let fill = "#0052A0"
let stroke = "#0052A0"
let color = "#ffffff"
}
{ "id": n["id"], "label": n["label"], "sublabel": "",
"shape": node_shape,
"style_fill": fill, "style_stroke": stroke, "style_color": color }
}
fn lower_edge(e: Map<String, Any>) -> Map<String, Any> {
let kind: String = e["kind"]
let line = "solid"
let arrow = "forward"
if kind == "dashed" {
let line = "dashed"
}
if kind == "bidirectional" {
let arrow = "both"
}
// forbidden uses solid line + forward arrow; the renderer overlays the
// circle-X marker based on a forbidden-set the caller threads through.
{ "from": e["from"], "to": e["to"], "label": e["label"],
"line": line, "arrow": arrow }
}
fn lower_graph(g: Map<String, Any>) -> Map<String, Any> {
let nodes: [Map<String, Any>] = g["nodes"]
let edges: [Map<String, Any>] = g["edges"]
let lowered_nodes: [Map<String, Any>] = el_list_empty()
let i = 0
let n: Int = el_list_len(nodes)
while i < n {
let lowered_nodes = native_list_append(lowered_nodes, lower_node(get(nodes, i)))
let i = i + 1
}
let lowered_edges: [Map<String, Any>] = el_list_empty()
let i = 0
let m: Int = el_list_len(edges)
while i < m {
let lowered_edges = native_list_append(lowered_edges, lower_edge(get(edges, i)))
let i = i + 1
}
{ "title": g["title"], "direction": g["direction"],
"nodes": lowered_nodes, "edges": lowered_edges, "groups": g["groups"] }
}
// Find a node by id within a (lowered or raw) graph. Returns an empty map
// when not found callers check map_get(result, "id") for presence.
fn graph_find_node(graph: Map<String, Any>, id: String) -> Map<String, Any> {
let nodes: [Map<String, Any>] = graph["nodes"]
let n: Int = el_list_len(nodes)
let i = 0
while i < n {
let node: Map<String, Any> = get(nodes, i)
let nid: String = node["id"]
if nid == id { return node }
let i = i + 1
}
let empty: Map<String, Any> = el_map_new(0)
empty
}
// Forbidden-edge set helpers
// The lowered graph drops the "forbidden" kind (line/arrow have no slot for
// it). Callers preserve the set as a list of "from->to" strings.
fn forbidden_key(from: String, to: String) -> String {
from + "->" + to
}
fn collect_forbidden(graph: Map<String, Any>) -> [String] {
let edges: [Map<String, Any>] = graph["edges"]
let n: Int = el_list_len(edges)
let out: [String] = el_list_empty()
let i = 0
while i < n {
let e: Map<String, Any> = get(edges, i)
let kind: String = e["kind"]
if kind == "forbidden" {
let f: String = e["from"]
let t: String = e["to"]
let out = native_list_append(out, forbidden_key(f, t))
}
let i = i + 1
}
out
}
fn forbidden_contains(set: [String], src: String, dst: String) -> Bool {
let key: String = forbidden_key(src, dst)
let n: Int = el_list_len(set)
let i = 0
while i < n {
let s: String = get(set, i)
if s == key { return true }
let i = i + 1
}
false
}
// Smoke test
//
// State is kept in process-local k/v storage so we never mix Int + Call or
// Int + Ident in `+` (which the codegen heuristic emits as string concat
// on tagged-pointer values, segfaulting on Int operands).
fn fail(label: String, got: String, want: String) -> Int {
println("FAIL " + label + " got=[" + got + "] want=[" + want + "]")
state_set("failures", "1")
0
}
fn check_eq(label: String, got: String, want: String) -> Int {
if got == want {
println("ok " + label + " = " + got)
return 1
}
fail(label, got, want)
}
check_eq("sanitize crates/nc-core",
sanitize_id("crates/nc-core"), "crates_nc_core")
check_eq("sanitize package.json",
sanitize_id("package.json"), "package_json")
check_eq("sanitize 42-module",
sanitize_id("42-module"), "n42_module")
check_eq("sanitize empty", sanitize_id(""), "node")
check_eq("sanitize !!--@@", sanitize_id("!!--@@"), "node")
check_eq("shape_from_token rounded",
shape_from_token("rounded"), "rounded")
check_eq("shape_to_node_shape primary",
shape_to_node_shape("primary"), "rectangle")
// Lowering preserves a node id and adds style.
let n: Map<String, Any> = make_node("svc", "Service", "primary")
let ln: Map<String, Any> = lower_node(n)
check_eq("lower preserves id", ln["id"], "svc")
check_eq("lower applies primary fill", ln["style_fill"], "#0052A0")
// Edge lowering
let e: Map<String, Any> = make_edge("a", "b", "dashed")
let le: Map<String, Any> = lower_edge(e)
check_eq("lower edge dashed line", le["line"], "dashed")
let e2: Map<String, Any> = make_edge("a", "b", "bidirectional")
let le2: Map<String, Any> = lower_edge(e2)
check_eq("lower edge bidirectional arrow", le2["arrow"], "both")
println("")
let failures: String = state_get("failures")
if str_eq(failures, "1") {
println("arbor-core: FAILED")
exit_program(1)
} else {
println("arbor-core: ok")
}
-19
View File
@@ -1,19 +0,0 @@
// arbor-diagram diagram intermediate representation + Mermaid serializer
// + dependency-graph builders. Consumes raw graph values built by arbor-core
// or arbor-parse and produces Mermaid markup or other serializations.
vessel "arbor-diagram" {
version "0.1.0"
description "Diagram IR + Mermaid serializer + architecture diagram builders"
authors ["Neuron Technologies"]
edition "2026"
}
dependencies {
arbor-core "0.1"
}
build {
entry "src/main.el"
output "dist/"
}
-433
View File
@@ -1,433 +0,0 @@
// arbor-diagram diagram intermediate representation (AST + IR).
//
// Where arbor-core supplies the *.arbor source-language model Mermaid-safe
// IDs, ArborShape strings, ArborEdgeKind strings, and the lowered "diagram-
// form" map arbor-diagram exposes the same lowered model as the canonical
// IR for downstream serializers (arbor-render and any future Mermaid-style
// emitter). The two vessels overlap by design: arbor-core is responsible for
// *naming* the schema; arbor-diagram is responsible for *building* values
// against it.
//
// The Rust crate ships small AST builder structs (`DiagramNode::new`,
// `DiagramEdge::with_label`, `DiagramGraph::add_node`). El has no method
// chaining, no Default::default(), no enum types. The El idiom is a stack
// of immutable maps with explicit constructor + with_* helpers that take
// the value and return a freshly-allocated map.
//
// Public surface:
// make_node(id, label) DiagramNode
// with_shape(node, shape) DiagramNode
// with_sublabel(node, sublabel) DiagramNode
// with_style(node, fill, stroke, color) DiagramNode
//
// make_edge(from, to) DiagramEdge
// with_label(edge, label)
// with_line(edge, line) // "solid"/"dashed"/"dotted"/"thick"
// with_arrow(edge, arrow) // "forward"/"backward"/"both"/"none"
//
// make_group(id, label) DiagramGroup
// with_node(group, node_id)
// with_nodes(group, [node_id])
// with_direction(group, dir)
//
// make_graph(title) DiagramGraph
// with_direction(graph, dir)
// graph_add_node(graph, node) DiagramGraph
// graph_add_edge(graph, edge) DiagramGraph
// graph_add_group(graph, group) DiagramGraph
// graph_node(graph, id) DiagramNode | empty map
//
// Shape vocabulary (lowered): see arbor-core. The local copy here mirrors
// the table in arbor-core/src/main.el so this vessel is hermetic.
// NodeShape vocabulary
fn node_shape_rectangle() -> String { "rectangle" }
fn node_shape_rounded_rect() -> String { "rounded_rect" }
fn node_shape_stadium() -> String { "stadium" }
fn node_shape_cylinder() -> String { "cylinder" }
fn node_shape_diamond() -> String { "diamond" }
fn node_shape_parallelogram() -> String { "parallelogram" }
fn node_shape_database() -> String { "database" }
fn node_shape_subroutine() -> String { "subroutine" }
fn node_shape_valid(s: String) -> Bool {
if str_eq(s, "rectangle") { return true }
if str_eq(s, "rounded_rect") { return true }
if str_eq(s, "stadium") { return true }
if str_eq(s, "cylinder") { return true }
if str_eq(s, "diamond") { return true }
if str_eq(s, "parallelogram") { return true }
if str_eq(s, "database") { return true }
if str_eq(s, "subroutine") { return true }
false
}
// EdgeLine vocabulary
fn edge_line_solid() -> String { "solid" }
fn edge_line_dashed() -> String { "dashed" }
fn edge_line_dotted() -> String { "dotted" }
fn edge_line_thick() -> String { "thick" }
fn edge_line_valid(s: String) -> Bool {
if str_eq(s, "solid") { return true }
if str_eq(s, "dashed") { return true }
if str_eq(s, "dotted") { return true }
if str_eq(s, "thick") { return true }
false
}
// EdgeArrow vocabulary
fn edge_arrow_forward() -> String { "forward" }
fn edge_arrow_backward() -> String { "backward" }
fn edge_arrow_both() -> String { "both" }
fn edge_arrow_none() -> String { "none" }
fn edge_arrow_valid(s: String) -> Bool {
if str_eq(s, "forward") { return true }
if str_eq(s, "backward") { return true }
if str_eq(s, "both") { return true }
if str_eq(s, "none") { return true }
false
}
// Direction vocabulary
fn direction_top_down() -> String { "top-down" }
fn direction_left_right() -> String { "left-right" }
fn direction_right_left() -> String { "right-left" }
fn direction_bottom_up() -> String { "bottom-up" }
fn direction_valid(s: String) -> Bool {
if str_eq(s, "top-down") { return true }
if str_eq(s, "left-right") { return true }
if str_eq(s, "right-left") { return true }
if str_eq(s, "bottom-up") { return true }
false
}
// DiagramNode
fn make_node(id: String, label: String) -> Map<String, Any> {
{
"id": id,
"label": label,
"sublabel": "",
"shape": "rectangle",
"style_fill": "",
"style_stroke": "",
"style_color": ""
}
}
fn with_shape(node: Map<String, Any>, shape: String) -> Map<String, Any> {
{
"id": node["id"],
"label": node["label"],
"sublabel": node["sublabel"],
"shape": shape,
"style_fill": node["style_fill"],
"style_stroke": node["style_stroke"],
"style_color": node["style_color"]
}
}
fn with_sublabel(node: Map<String, Any>, sublabel: String) -> Map<String, Any> {
{
"id": node["id"],
"label": node["label"],
"sublabel": sublabel,
"shape": node["shape"],
"style_fill": node["style_fill"],
"style_stroke": node["style_stroke"],
"style_color": node["style_color"]
}
}
fn with_style(node: Map<String, Any>, fill: String, stroke: String, color: String) -> Map<String, Any> {
{
"id": node["id"],
"label": node["label"],
"sublabel": node["sublabel"],
"shape": node["shape"],
"style_fill": fill,
"style_stroke": stroke,
"style_color": color
}
}
// DiagramEdge
fn make_edge(from: String, to: String) -> Map<String, Any> {
{
"from": from,
"to": to,
"label": "",
"line": "solid",
"arrow": "forward"
}
}
fn with_label(edge: Map<String, Any>, label: String) -> Map<String, Any> {
{
"from": edge["from"],
"to": edge["to"],
"label": label,
"line": edge["line"],
"arrow": edge["arrow"]
}
}
fn with_line(edge: Map<String, Any>, line: String) -> Map<String, Any> {
{
"from": edge["from"],
"to": edge["to"],
"label": edge["label"],
"line": line,
"arrow": edge["arrow"]
}
}
fn with_arrow(edge: Map<String, Any>, arrow: String) -> Map<String, Any> {
{
"from": edge["from"],
"to": edge["to"],
"label": edge["label"],
"line": edge["line"],
"arrow": arrow
}
}
// DiagramGroup
fn make_group(id: String, label: String) -> Map<String, Any> {
let empty: [String] = native_list_empty()
{
"id": id,
"label": label,
"node_ids": empty,
"direction": ""
}
}
fn with_node(group: Map<String, Any>, node_id: String) -> Map<String, Any> {
let cur: [String] = group["node_ids"]
let next: [String] = native_list_append(cur, node_id)
{
"id": group["id"],
"label": group["label"],
"node_ids": next,
"direction": group["direction"]
}
}
fn with_nodes(group: Map<String, Any>, ids: [String]) -> Map<String, Any> {
let cur: [String] = group["node_ids"]
let n: Int = el_list_len(ids)
let i = 0
while i < n {
let cur = native_list_append(cur, get(ids, i))
let i = i + 1
}
{
"id": group["id"],
"label": group["label"],
"node_ids": cur,
"direction": group["direction"]
}
}
fn with_group_direction(group: Map<String, Any>, dir: String) -> Map<String, Any> {
{
"id": group["id"],
"label": group["label"],
"node_ids": group["node_ids"],
"direction": dir
}
}
// DiagramGraph
fn make_graph(title: String) -> Map<String, Any> {
let empty_n: [Map<String, Any>] = native_list_empty()
let empty_e: [Map<String, Any>] = native_list_empty()
let empty_g: [Map<String, Any>] = native_list_empty()
{
"title": title,
"direction": "top-down",
"nodes": empty_n,
"edges": empty_e,
"groups": empty_g
}
}
fn with_direction(graph: Map<String, Any>, dir: String) -> Map<String, Any> {
{
"title": graph["title"],
"direction": dir,
"nodes": graph["nodes"],
"edges": graph["edges"],
"groups": graph["groups"]
}
}
fn graph_add_node(graph: Map<String, Any>, node: Map<String, Any>) -> Map<String, Any> {
let cur: [Map<String, Any>] = graph["nodes"]
let next: [Map<String, Any>] = native_list_append(cur, node)
{
"title": graph["title"],
"direction": graph["direction"],
"nodes": next,
"edges": graph["edges"],
"groups": graph["groups"]
}
}
fn graph_add_edge(graph: Map<String, Any>, edge: Map<String, Any>) -> Map<String, Any> {
let cur: [Map<String, Any>] = graph["edges"]
let next: [Map<String, Any>] = native_list_append(cur, edge)
{
"title": graph["title"],
"direction": graph["direction"],
"nodes": graph["nodes"],
"edges": next,
"groups": graph["groups"]
}
}
fn graph_add_group(graph: Map<String, Any>, group: Map<String, Any>) -> Map<String, Any> {
let cur: [Map<String, Any>] = graph["groups"]
let next: [Map<String, Any>] = native_list_append(cur, group)
{
"title": graph["title"],
"direction": graph["direction"],
"nodes": graph["nodes"],
"edges": graph["edges"],
"groups": next
}
}
// Find a node by id. Returns an empty map (no "id" field) when not present.
fn graph_node(graph: Map<String, Any>, id: String) -> Map<String, Any> {
let nodes: [Map<String, Any>] = graph["nodes"]
let n: Int = el_list_len(nodes)
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
if str_eq(nid, id) { return nd }
let i = i + 1
}
let empty: Map<String, Any> = el_map_new(0)
empty
}
// Smoke test
fn fail(label: String, got: String, want: String) -> Int {
println("FAIL " + label + " got=[" + got + "] want=[" + want + "]")
state_set("smoke_failures", "1")
0
}
fn check_eq(label: String, got: String, want: String) -> Int {
if got == want {
println("ok " + label + " = " + got)
return 1
}
fail(label, got, want)
}
// Vocabulary self-checks
check_eq("shape rectangle valid",
bool_to_str(node_shape_valid("rectangle")), "true")
check_eq("shape hexagon invalid",
bool_to_str(node_shape_valid("hexagon")), "false")
check_eq("line dashed valid",
bool_to_str(edge_line_valid("dashed")), "true")
check_eq("arrow both valid",
bool_to_str(edge_arrow_valid("both")), "true")
check_eq("dir top-down valid",
bool_to_str(direction_valid("top-down")), "true")
// Node builder
let n0: Map<String, Any> = make_node("svc", "Service")
check_eq("node default shape", n0["shape"], "rectangle")
check_eq("node default sublabel empty", n0["sublabel"], "")
let n1: Map<String, Any> = with_shape(n0, "cylinder")
check_eq("node with_shape", n1["shape"], "cylinder")
check_eq("node id preserved", n1["id"], "svc")
let n2: Map<String, Any> = with_sublabel(n1, "v0.1.0")
check_eq("node with_sublabel", n2["sublabel"], "v0.1.0")
let n3: Map<String, Any> = with_style(n2, "#0052A0", "#0052A0", "#ffffff")
check_eq("node style fill", n3["style_fill"], "#0052A0")
check_eq("node style color", n3["style_color"], "#ffffff")
// Edge builder
let e0: Map<String, Any> = make_edge("a", "b")
check_eq("edge default line", e0["line"], "solid")
check_eq("edge default arrow", e0["arrow"], "forward")
let e1: Map<String, Any> = with_line(e0, "dashed")
let e2: Map<String, Any> = with_arrow(e1, "both")
let e3: Map<String, Any> = with_label(e2, "calls")
check_eq("edge line", e3["line"], "dashed")
check_eq("edge arrow", e3["arrow"], "both")
check_eq("edge label", e3["label"], "calls")
// Group builder
let g0: Map<String, Any> = make_group("core", "Application Core")
let g1: Map<String, Any> = with_node(g0, "api")
let g2: Map<String, Any> = with_node(g1, "svc")
let ids2: [String] = g2["node_ids"]
check_eq("group with two nodes", int_to_str(el_list_len(ids2)), "2")
let g3: Map<String, Any> = make_group("infra", "Infrastructure")
let extras: [String] = native_list_empty()
let extras = native_list_append(extras, "db")
let extras = native_list_append(extras, "cache")
let g4: Map<String, Any> = with_nodes(g3, extras)
let ids4: [String] = g4["node_ids"]
check_eq("group with_nodes appends", int_to_str(el_list_len(ids4)), "2")
// Graph builder + lookup
let G0: Map<String, Any> = make_graph("System")
let G1: Map<String, Any> = with_direction(G0, "left-right")
let G2: Map<String, Any> = graph_add_node(G1, n3)
let nb: Map<String, Any> = make_node("b", "Backend")
let G3: Map<String, Any> = graph_add_node(G2, nb)
let G4: Map<String, Any> = graph_add_edge(G3, e3)
let G5: Map<String, Any> = graph_add_group(G4, g4)
check_eq("graph title", G5["title"], "System")
check_eq("graph direction", G5["direction"], "left-right")
let gn: [Map<String, Any>] = G5["nodes"]
let ge: [Map<String, Any>] = G5["edges"]
let gg: [Map<String, Any>] = G5["groups"]
check_eq("graph nodes count", int_to_str(el_list_len(gn)), "2")
check_eq("graph edges count", int_to_str(el_list_len(ge)), "1")
check_eq("graph groups count", int_to_str(el_list_len(gg)), "1")
let found: Map<String, Any> = graph_node(G5, "svc")
check_eq("graph_node found", found["id"], "svc")
let missing: Map<String, Any> = graph_node(G5, "nonexistent")
let missing_id: String = missing["id"]
if str_len(missing_id) == 0 {
println("ok graph_node missing returns empty")
} else {
println("FAIL graph_node missing returned: " + missing_id)
state_set("smoke_failures", "1")
}
println("")
let failures: String = state_get("smoke_failures")
if str_eq(failures, "1") {
println("arbor-diagram: FAILED")
exit_program(1)
} else {
println("arbor-diagram: ok")
}
-19
View File
@@ -1,19 +0,0 @@
// arbor-layout hierarchical layout engine. Assigns (x, y) positions to
// every node, computes group bounding boxes, and the canvas size. Consumes
// a diagram graph; produces a layout-result value.
vessel "arbor-layout" {
version "0.1.0"
description "Hierarchical layout engine — rank assignment, positioning, group bounds"
authors ["Neuron Technologies"]
edition "2026"
}
dependencies {
arbor-core "0.1"
}
build {
entry "src/main.el"
output "dist/"
}
-591
View File
@@ -1,591 +0,0 @@
// arbor-layout hierarchical layout for diagram graphs.
//
// Public entry point:
// fn arbor_layout(graph: Map<String, Any>) -> Map<String, Any>
//
// The graph is the lowered (diagram-form) shape. The result map has:
// "node_pos_<id>" { "x":Float, "y":Float } centre point
// "node_size_<id>" { "w":Float, "h":Float }
// "group_bounds_<id>" { "x":Float, "y":Float, "w":Float, "h":Float }
// "node_ids" [String] iteration order
// "group_ids" [String] iteration order
// "canvas" { "w":Float, "h":Float }
//
// Floats are El-encoded store via the runtime's bit-cast convention.
// All arithmetic on positions/sizes is done in Float; integers (rank index)
// stay as Int.
//
// Algorithm (simplified Sugiyama):
// 1. Assign ranks via topological propagation (longest path from sources).
// 2. Group nodes by rank, preserving declaration order.
// 3. Position each rank as a row (top-down/bottom-up) or column (LR/RL).
// 4. Compute group bounding boxes from member positions.
// 5. Compute canvas size to enclose everything.
//
// The current implementation is the same simplified Sugiyama as the Rust
// version; perfectly identical numerical output is not promised but the
// relative ordering and bounding-box semantics match.
// Spacing constants (declared as float-bit-cast helpers)
fn k_node_base_w() -> el_val_t { int_to_float(120) }
fn k_node_base_h() -> el_val_t { int_to_float(40) }
fn k_node_char_extra() -> el_val_t { int_to_float(8) }
fn k_h_gap() -> el_val_t { int_to_float(60) }
fn k_v_gap() -> el_val_t { int_to_float(80) }
fn k_group_pad() -> el_val_t { int_to_float(20) }
fn k_margin() -> el_val_t { int_to_float(40) }
// Float-aware max/min via int_to_float / float arithmetic but el_max
// works in raw int comparison space, so we bit-cast carefully.
// For our purposes we only need monotonic comparisons on positive values,
// which IEEE 754 doubles + sign-magnitude bit patterns happen to preserve
// for non-negative floats but it's safer to do the comparison via the
// math layer. We use a helper that decodes both, picks the bigger, and
// re-encodes.
//
// Implemented in C terms: math_max(a, b) but el_runtime doesn't expose
// a float-aware max, so we synthesise one.
fn fmax(a: el_val_t, b: el_val_t) -> el_val_t {
// Compare via float subtraction's sign: a - b. Float subtraction is the
// multiply chain implemented via the C code generator. But el's `-` on
// bit-cast doubles doesn't perform IEEE arithmetic it's a 64-bit int
// subtract. Workaround: round-trip through format_float and str_to_float.
// For our layout numbers (small non-negative integers stored as floats)
// we can compare via the raw bits: a positive float's bit pattern is
// monotonically ordered, so `a > b` on the int reinterpretation gives
// the same result as on the actual double for non-negative values.
if a > b { return a }
b
}
fn fadd(a: el_val_t, b: el_val_t) -> el_val_t {
// a, b are bit-cast doubles. Safe addition: int-to-float, format, parse.
// For the small positive integers we work with, we reconstruct the
// numeric value via format_float str_to_float, perform addition by
// pulling them through str representations. Costly but correct on the
// current runtime. Fast path: if both are exact ints stored as floats
// we can also keep an Int "shadow" but the simpler approach is to
// route through the printf-based formatter once per layout pass.
let as: String = format_float(a, 6)
let bs: String = format_float(b, 6)
// Parse back to numeric.
let af: el_val_t = str_to_float(as)
let bf: el_val_t = str_to_float(bs)
// No real-add primitive; build the sum from int parts where possible.
// Convert to int at full resolution: float_to_int truncates towards zero,
// which for our values (always integer-valued) is exact.
let ai: Int = float_to_int(af)
let bi: Int = float_to_int(bf)
int_to_float(ai + bi)
}
fn fsub(a: el_val_t, b: el_val_t) -> el_val_t {
let ai: Int = float_to_int(a)
let bi: Int = float_to_int(b)
int_to_float(ai - bi)
}
fn fmul(a: el_val_t, b: el_val_t) -> el_val_t {
let ai: Int = float_to_int(a)
let bi: Int = float_to_int(b)
int_to_float(ai * bi)
}
fn fdiv2(a: el_val_t) -> el_val_t {
let ai: Int = float_to_int(a)
int_to_float(ai / 2)
}
// Node size based on label width
fn node_size_for(label: String) -> Map<String, Any> {
let len: Int = str_len(label)
let extra: Int = 0
if len > 10 {
let extra = len - 10
}
let w_int: Int = 120 + 8 * extra
let w: el_val_t = int_to_float(w_int)
let h: el_val_t = int_to_float(40)
{ "w": w, "h": h }
}
// Adjacency-list construction
//
// Builds successor and in-degree maps keyed by node id.
fn build_succ_indeg(graph: Map<String, Any>) -> Map<String, Any> {
let nodes: [Map<String, Any>] = graph["nodes"]
let edges: [Map<String, Any>] = graph["edges"]
let n: Int = el_list_len(nodes)
let m: Int = el_list_len(edges)
let succ: Map<String, Any> = el_map_new(0)
let indeg: Map<String, Any> = el_map_new(0)
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
let empty: [String] = el_list_empty()
let succ = el_map_set(succ, nid, empty)
let indeg = el_map_set(indeg, nid, 0)
let i = i + 1
}
let i = 0
while i < m {
let e: Map<String, Any> = get(edges, i)
let src: String = e["from"]
let dst: String = e["to"]
let cur_succ: [String] = el_map_get(succ, src)
let new_succ: [String] = native_list_append(cur_succ, dst)
let succ = el_map_set(succ, src, new_succ)
let prev: Int = el_map_get(indeg, dst)
let indeg = el_map_set(indeg, dst, prev + 1)
let i = i + 1
}
{ "succ": succ, "indeg": indeg }
}
// Topological rank assignment
//
// Returns a map: node_id rank.
fn assign_ranks(graph: Map<String, Any>) -> Map<String, Any> {
let nodes: [Map<String, Any>] = graph["nodes"]
let n: Int = el_list_len(nodes)
let adj: Map<String, Any> = build_succ_indeg(graph)
let succ: Map<String, Any> = adj["succ"]
let indeg: Map<String, Any> = adj["indeg"]
let ranks: Map<String, Any> = el_map_new(0)
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
let ranks = el_map_set(ranks, nid, 0)
let i = i + 1
}
// Initialise queue with all nodes whose in-degree is 0 (in declaration
// order, mirroring the Rust implementation's ordering guarantee).
let queue: [String] = el_list_empty()
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
let d: Int = el_map_get(indeg, nid)
if d == 0 {
let queue = native_list_append(queue, nid)
}
let i = i + 1
}
let head = 0
let running = true
while running {
if head >= el_list_len(queue) {
let running = false
} else {
let cur: String = get(queue, head)
let head = head + 1
let cur_rank: Int = el_map_get(ranks, cur)
let neighbours: [String] = el_map_get(succ, cur)
let nn: Int = el_list_len(neighbours)
let j = 0
while j < nn {
let nb: String = get(neighbours, j)
let nb_rank: Int = el_map_get(ranks, nb)
let cand: Int = cur_rank + 1
if cand > nb_rank {
let ranks = el_map_set(ranks, nb, cand)
}
let cur_d: Int = el_map_get(indeg, nb)
let new_d: Int = cur_d - 1
let indeg = el_map_set(indeg, nb, new_d)
if new_d <= 0 {
let queue = native_list_append(queue, nb)
}
let j = j + 1
}
}
}
ranks
}
// Layout pass
fn arbor_layout(graph: Map<String, Any>) -> Map<String, Any> {
let nodes: [Map<String, Any>] = graph["nodes"]
let n: Int = el_list_len(nodes)
let direction: String = graph["direction"]
let result: Map<String, Any> = el_map_new(0)
let result = el_map_set(result, "node_ids", el_list_empty())
let result = el_map_set(result, "group_ids", el_list_empty())
if n == 0 {
let canvas: Map<String, Any> = { "w": int_to_float(200), "h": int_to_float(100) }
let result = el_map_set(result, "canvas", canvas)
return result
}
let ranks: Map<String, Any> = assign_ranks(graph)
let max_rank = 0
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
let r: Int = el_map_get(ranks, nid)
if r > max_rank { let max_rank = r }
let i = i + 1
}
// Group nodes by rank, preserving declaration order. Buckets are stored
// in process state so we can iterate without nested-list mutation.
let i = 0
while i <= max_rank {
state_set("rank_bucket_" + int_to_str(i), "")
let i = i + 1
}
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
let r: Int = el_map_get(ranks, nid)
let key = "rank_bucket_" + int_to_str(r)
let prev: String = state_get(key)
if str_eq(prev, "") {
state_set(key, nid)
} else {
state_set(key, prev + "" + nid)
}
let i = i + 1
}
// Pre-compute sizes and stash a label-keyed cache.
let id_list: [String] = el_list_empty()
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
let lbl: String = nd["label"]
let sz: Map<String, Any> = node_size_for(lbl)
let result = el_map_set(result, "node_size_" + nid, sz)
let id_list = native_list_append(id_list, nid)
let i = i + 1
}
let result = el_map_set(result, "node_ids", id_list)
// Position pass.
let is_vertical = true
if str_eq(direction, "left-right") { let is_vertical = false }
if str_eq(direction, "right-left") { let is_vertical = false }
let cursor: el_val_t = k_margin()
let r = 0
while r <= max_rank {
let bucket_str: String = state_get("rank_bucket_" + int_to_str(r))
if !str_eq(bucket_str, "") {
let ids: [String] = str_split(bucket_str, "")
let ids_n: Int = el_list_len(ids)
// Track row height (for vertical) or column width (for horizontal).
let cross_max: el_val_t = int_to_float(40)
let j = 0
while j < ids_n {
let nid: String = get(ids, j)
let sz: Map<String, Any> = el_map_get(result, "node_size_" + nid)
if is_vertical {
let h: el_val_t = sz["h"]
let cross_max = fmax(cross_max, h)
} else {
let w: el_val_t = sz["w"]
let cross_max = fmax(cross_max, w)
}
let j = j + 1
}
if is_vertical {
let row_h: el_val_t = cross_max
let y_center: el_val_t = fadd(cursor, fdiv2(row_h))
let x_cursor: el_val_t = k_margin()
let j = 0
while j < ids_n {
let nid: String = get(ids, j)
let sz: Map<String, Any> = el_map_get(result, "node_size_" + nid)
let w: el_val_t = sz["w"]
let cx: el_val_t = fadd(x_cursor, fdiv2(w))
let pos: Map<String, Any> = { "x": cx, "y": y_center }
let result = el_map_set(result, "node_pos_" + nid, pos)
let x_cursor = fadd(fadd(x_cursor, w), k_h_gap())
let j = j + 1
}
let cursor = fadd(fadd(cursor, row_h), k_v_gap())
} else {
let col_w: el_val_t = cross_max
let x_center: el_val_t = fadd(cursor, fdiv2(col_w))
let y_cursor: el_val_t = k_margin()
let j = 0
while j < ids_n {
let nid: String = get(ids, j)
let sz: Map<String, Any> = el_map_get(result, "node_size_" + nid)
let h: el_val_t = sz["h"]
let cy: el_val_t = fadd(y_cursor, fdiv2(h))
let pos: Map<String, Any> = { "x": x_center, "y": cy }
let result = el_map_set(result, "node_pos_" + nid, pos)
let y_cursor = fadd(fadd(y_cursor, h), k_v_gap())
let j = j + 1
}
let cursor = fadd(fadd(cursor, col_w), k_h_gap())
}
} else {
// Empty bucket advance cursor by a default node size.
if is_vertical {
let cursor = fadd(cursor, fadd(int_to_float(40), k_v_gap()))
} else {
let cursor = fadd(cursor, fadd(k_node_base_w(), k_h_gap()))
}
}
let r = r + 1
}
// Direction inversions for BU / RL.
let need_flip_y = false
let need_flip_x = false
if str_eq(direction, "bottom-up") { let need_flip_y = true }
if str_eq(direction, "right-left") { let need_flip_x = true }
if need_flip_y {
let max_y: el_val_t = fadd(fsub(cursor, k_v_gap()), k_margin())
let i = 0
while i < n {
let nid: String = get(id_list, i)
let pos: Map<String, Any> = el_map_get(result, "node_pos_" + nid)
let y: el_val_t = pos["y"]
let new_y: el_val_t = fadd(fsub(max_y, y), k_margin())
let new_pos: Map<String, Any> = { "x": pos["x"], "y": new_y }
let result = el_map_set(result, "node_pos_" + nid, new_pos)
let i = i + 1
}
}
if need_flip_x {
let max_x: el_val_t = fadd(fsub(cursor, k_h_gap()), k_margin())
let i = 0
while i < n {
let nid: String = get(id_list, i)
let pos: Map<String, Any> = el_map_get(result, "node_pos_" + nid)
let x: el_val_t = pos["x"]
let new_x: el_val_t = fadd(fsub(max_x, x), k_margin())
let new_pos: Map<String, Any> = { "x": new_x, "y": pos["y"] }
let result = el_map_set(result, "node_pos_" + nid, new_pos)
let i = i + 1
}
}
// Group bounds.
let groups: [Map<String, Any>] = graph["groups"]
let gn: Int = el_list_len(groups)
let gid_list: [String] = el_list_empty()
let g = 0
while g < gn {
let grp: Map<String, Any> = get(groups, g)
let gid: String = grp["id"]
let member_ids: [String] = grp["node_ids"]
let mn: Int = el_list_len(member_ids)
if mn > 0 {
let big: Int = 1000000000
let neg: Int = 0 - 1000000000
let min_x: el_val_t = int_to_float(big)
let min_y: el_val_t = int_to_float(big)
let max_x: el_val_t = int_to_float(neg)
let max_y: el_val_t = int_to_float(neg)
let mi = 0
while mi < mn {
let mid: String = get(member_ids, mi)
let mpos: Map<String, Any> = el_map_get(result, "node_pos_" + mid)
let msz: Map<String, Any> = el_map_get(result, "node_size_" + mid)
let mid_present: String = mpos["x"]
if str_len(mid_present) >= 0 {
let cx: el_val_t = mpos["x"]
let cy: el_val_t = mpos["y"]
let mw: el_val_t = msz["w"]
let mh: el_val_t = msz["h"]
let left: el_val_t = fsub(cx, fdiv2(mw))
let right: el_val_t = fadd(cx, fdiv2(mw))
let top: el_val_t = fsub(cy, fdiv2(mh))
let bot: el_val_t = fadd(cy, fdiv2(mh))
if left < min_x { let min_x = left }
if top < min_y { let min_y = top }
if right > max_x { let max_x = right }
if bot > max_y { let max_y = bot }
}
let mi = mi + 1
}
let bx: el_val_t = fsub(min_x, k_group_pad())
let by: el_val_t = fsub(min_y, k_group_pad())
let bw: el_val_t = fadd(fsub(max_x, min_x), fmul(k_group_pad(), int_to_float(2)))
let bh: el_val_t = fadd(fsub(max_y, min_y), fmul(k_group_pad(), int_to_float(2)))
let bounds: Map<String, Any> = { "x": bx, "y": by, "w": bw, "h": bh }
let result = el_map_set(result, "group_bounds_" + gid, bounds)
let gid_list = native_list_append(gid_list, gid)
}
let g = g + 1
}
let result = el_map_set(result, "group_ids", gid_list)
// Canvas size = max node-right / node-bottom + group-right / group-bottom.
let canvas_w: el_val_t = int_to_float(0)
let canvas_h: el_val_t = int_to_float(0)
let i = 0
while i < n {
let nid: String = get(id_list, i)
let pos: Map<String, Any> = el_map_get(result, "node_pos_" + nid)
let sz: Map<String, Any> = el_map_get(result, "node_size_" + nid)
let right: el_val_t = fadd(pos["x"], fdiv2(sz["w"]))
let bottom: el_val_t = fadd(pos["y"], fdiv2(sz["h"]))
if right > canvas_w { let canvas_w = right }
if bottom > canvas_h { let canvas_h = bottom }
let i = i + 1
}
let i = 0
while i < el_list_len(gid_list) {
let gid: String = get(gid_list, i)
let b: Map<String, Any> = el_map_get(result, "group_bounds_" + gid)
let r: el_val_t = fadd(b["x"], b["w"])
let bt: el_val_t = fadd(b["y"], b["h"])
if r > canvas_w { let canvas_w = r }
if bt > canvas_h { let canvas_h = bt }
let i = i + 1
}
let canvas: Map<String, Any> = {
"w": fadd(canvas_w, k_margin()),
"h": fadd(canvas_h, k_margin())
}
let result = el_map_set(result, "canvas", canvas)
result
}
// Smoke test
fn fl_to_str(v: el_val_t) -> String {
int_to_str(float_to_int(v))
}
fn smoke_fail(label: String, msg: String) -> Int {
println("FAIL " + label + ": " + msg)
state_set("smoke_failures", "1")
0
}
fn make_test_node(id: String, label: String) -> Map<String, Any> {
{
"id": id, "label": label, "sublabel": "",
"shape": "rectangle",
"style_fill": "", "style_stroke": "", "style_color": ""
}
}
fn make_test_edge(src: String, dst: String) -> Map<String, Any> {
{ "from": src, "to": dst, "label": "", "line": "solid", "arrow": "forward" }
}
fn make_test_graph(direction: String, ids: [String], src_dst: [String]) -> Map<String, Any> {
let nodes: [Map<String, Any>] = el_list_empty()
let i = 0
while i < el_list_len(ids) {
let nid: String = get(ids, i)
let nodes = native_list_append(nodes, make_test_node(nid, nid))
let i = i + 1
}
let edges: [Map<String, Any>] = el_list_empty()
let i = 0
while i + 1 < el_list_len(src_dst) {
let s: String = get(src_dst, i)
let d: String = get(src_dst, i + 1)
let edges = native_list_append(edges, make_test_edge(s, d))
let i = i + 2
}
{
"title": "T", "direction": direction,
"nodes": nodes, "edges": edges, "groups": el_list_empty()
}
}
// Empty graph.
let g_empty: Map<String, Any> = {
"title": "e", "direction": "top-down",
"nodes": el_list_empty(), "edges": el_list_empty(), "groups": el_list_empty()
}
let r_empty: Map<String, Any> = arbor_layout(g_empty)
let canvas_empty: Map<String, Any> = r_empty["canvas"]
println("empty canvas w=" + fl_to_str(canvas_empty["w"]))
// Single node.
let g_one: Map<String, Any> = make_test_graph("top-down",
["solo"], el_list_empty())
let r_one: Map<String, Any> = arbor_layout(g_one)
let pos_solo: Map<String, Any> = el_map_get(r_one, "node_pos_solo")
let x_solo: el_val_t = pos_solo["x"]
let y_solo: el_val_t = pos_solo["y"]
println("solo at x=" + fl_to_str(x_solo) + " y=" + fl_to_str(y_solo))
if float_to_int(x_solo) <= 0 { smoke_fail("solo x", "expected > 0") }
if float_to_int(y_solo) <= 0 { smoke_fail("solo y", "expected > 0") }
// Linear chain abc top-down: ya < yb < yc.
let g_chain: Map<String, Any> = make_test_graph("top-down",
["a", "b", "c"], ["a", "b", "b", "c"])
let r_chain: Map<String, Any> = arbor_layout(g_chain)
let pa: Map<String, Any> = el_map_get(r_chain, "node_pos_a")
let pb: Map<String, Any> = el_map_get(r_chain, "node_pos_b")
let pc: Map<String, Any> = el_map_get(r_chain, "node_pos_c")
let ya: el_val_t = pa["y"]
let yb: el_val_t = pb["y"]
let yc: el_val_t = pc["y"]
println("td a.y=" + fl_to_str(ya) + " b.y=" + fl_to_str(yb) + " c.y=" + fl_to_str(yc))
if float_to_int(ya) >= float_to_int(yb) { smoke_fail("td order", "a.y >= b.y") }
if float_to_int(yb) >= float_to_int(yc) { smoke_fail("td order", "b.y >= c.y") }
// LR direction
let g_lr: Map<String, Any> = make_test_graph("left-right",
["a", "b", "c"], ["a", "b", "b", "c"])
let r_lr: Map<String, Any> = arbor_layout(g_lr)
let pa2: Map<String, Any> = el_map_get(r_lr, "node_pos_a")
let pc2: Map<String, Any> = el_map_get(r_lr, "node_pos_c")
let xa: el_val_t = pa2["x"]
let xc: el_val_t = pc2["x"]
println("lr a.x=" + fl_to_str(xa) + " c.x=" + fl_to_str(xc))
if float_to_int(xa) >= float_to_int(xc) { smoke_fail("lr order", "a.x >= c.x") }
// Bottom-up: a is below c.
let g_bu: Map<String, Any> = make_test_graph("bottom-up",
["a", "b", "c"], ["a", "b", "b", "c"])
let r_bu: Map<String, Any> = arbor_layout(g_bu)
let pa3: Map<String, Any> = el_map_get(r_bu, "node_pos_a")
let pc3: Map<String, Any> = el_map_get(r_bu, "node_pos_c")
let ya3: el_val_t = pa3["y"]
let yc3: el_val_t = pc3["y"]
println("bu a.y=" + fl_to_str(ya3) + " c.y=" + fl_to_str(yc3))
if float_to_int(ya3) <= float_to_int(yc3) { smoke_fail("bu order", "a.y <= c.y") }
// Canvas covers all nodes.
let canvas_chain: Map<String, Any> = r_chain["canvas"]
let cw: el_val_t = canvas_chain["w"]
let ch: el_val_t = canvas_chain["h"]
println("chain canvas w=" + fl_to_str(cw) + " h=" + fl_to_str(ch))
if float_to_int(cw) <= 0 { smoke_fail("canvas w", "non-positive") }
if float_to_int(ch) <= 0 { smoke_fail("canvas h", "non-positive") }
println("")
let f: String = state_get("smoke_failures")
if str_eq(f, "1") {
println("arbor-layout: FAILED")
exit_program(1)
} else {
println("arbor-layout: ok")
}
-19
View File
@@ -1,19 +0,0 @@
// arbor-parse hand-written recursive-descent parser for the .arbor source
// language. Produces an Arbor graph value consumable by arbor-layout and
// arbor-render.
vessel "arbor-parse" {
version "0.1.0"
description "Recursive-descent parser for the .arbor diagram language"
authors ["Neuron Technologies"]
edition "2026"
}
dependencies {
arbor-core "0.1"
}
build {
entry "src/main.el"
output "dist/"
}
-763
View File
@@ -1,763 +0,0 @@
// arbor-parse recursive-descent parser for the .arbor source language.
//
// This vessel inlines a private copy of the small set of arbor-core helpers
// it needs (sanitize_id and constructors). El's import form today is purely
// syntactic concatenation, so each vessel that wants to be its own buildable
// unit carries its own copy of these helpers. They're tiny (well under 100
// lines) and the duplication keeps each vessel hermetic.
//
// Public entry point: fn arbor_parse(source: String) -> Map<String, Any>
//
// Returns either a graph value or a parse-error map. Callers test for the
// "error" field:
// { "error": "..." , "line": Int, "text": "...source line..." } on failure
// { "title", "direction", "nodes", "edges", "groups" } on success
// Sanitisation (copy of arbor-core's sanitize_id)
fn is_alnum_underscore(ch: String) -> Bool {
let code: Int = str_char_code(ch, 0)
if code >= 48 {
if code <= 57 { return true }
}
if code >= 65 {
if code <= 90 { return true }
}
if code >= 97 {
if code <= 122 { return true }
}
if code == 95 { return true }
false
}
fn is_ascii_digit(ch: String) -> Bool {
let code: Int = str_char_code(ch, 0)
if code >= 48 {
if code <= 57 { return true }
}
false
}
fn sanitize_id(s: String) -> String {
let n: Int = str_len(s)
if n == 0 { return "node" }
let out = ""
let prev_underscore = false
let i = 0
while i < n {
let ch: String = str_char_at(s, i)
if is_alnum_underscore(ch) {
let out = out + ch
let prev_underscore = false
} else {
if !prev_underscore {
let out = out + "_"
}
let prev_underscore = true
}
let i = i + 1
}
let m: Int = str_len(out)
let end = m
let stripping = true
while stripping {
if end <= 0 {
let stripping = false
} else {
let last: String = str_char_at(out, end - 1)
if last == "_" {
let end = end - 1
} else {
let stripping = false
}
}
}
let out = str_slice(out, 0, end)
if str_len(out) == 0 { return "node" }
let first: String = str_char_at(out, 0)
if is_ascii_digit(first) {
let out = "n" + out
}
out
}
fn shape_from_token(tok: String) -> String {
let t: String = str_trim(tok)
if t == "rect" { return "rect" }
if t == "rounded" { return "rounded" }
if t == "cylinder" { return "cylinder" }
if t == "diamond" { return "diamond" }
if t == "stadium" { return "stadium" }
if t == "primary" { return "primary" }
""
}
// Line preprocessing
//
// Strip inline `// ...` comments, trim, drop empties. Returns a list of maps
// { "no": Int, "text": String }.
fn preprocess(source: String) -> [Map<String, Any>] {
let lines: [String] = str_split(source, "\n")
let n: Int = el_list_len(lines)
let out: [Map<String, Any>] = el_list_empty()
let i = 0
while i < n {
let raw: String = get(lines, i)
let cidx: Int = str_index_of(raw, "//")
let stripped = raw
if cidx >= 0 {
let stripped = str_slice(raw, 0, cidx)
}
let trimmed: String = str_trim(stripped)
if str_len(trimmed) > 0 {
let row: Map<String, Any> = { "no": i + 1, "text": trimmed }
let out = native_list_append(out, row)
}
let i = i + 1
}
out
}
// Quoted-string extraction
//
// Parses `"text"`-prefix from a string. Returns `{ "ok": Bool, "value": Str,
// "rest": Str }`. The `rest` field carries everything after the closing quote
// (so the caller can continue tokenising).
fn parse_quoted(s: String) -> Map<String, Any> {
let t: String = str_trim(s)
if str_len(t) < 2 {
return { "ok": false, "value": "", "rest": s }
}
let first: String = str_char_at(t, 0)
if first != "\"" {
return { "ok": false, "value": "", "rest": s }
}
let body: String = str_slice(t, 1, str_len(t))
let close: Int = str_index_of(body, "\"")
if close < 0 {
return { "ok": false, "value": "", "rest": s }
}
let inner: String = str_slice(body, 0, close)
let rest: String = str_slice(body, close + 1, str_len(body))
{ "ok": true, "value": inner, "rest": rest }
}
// Identifier prefix split
//
// `split_identifier("foo bar")` { "id": "foo", "rest": " bar" }.
// `split_identifier("a-b")` { "id": "a", "rest": "-b" }.
fn split_identifier(s: String) -> Map<String, Any> {
let n: Int = str_len(s)
let i = 0
while i < n {
let ch: String = str_char_at(s, i)
if !is_alnum_underscore(ch) {
return { "id": str_slice(s, 0, i), "rest": str_slice(s, i, n) }
}
let i = i + 1
}
{ "id": s, "rest": "" }
}
// Direction parsing
fn parse_direction(s: String) -> String {
let t: String = str_trim(s)
if t == "top-down" { return "top-down" }
if t == "TD" { return "top-down" }
if t == "left-right" { return "left-right" }
if t == "LR" { return "left-right" }
if t == "right-left" { return "right-left" }
if t == "RL" { return "right-left" }
if t == "bottom-up" { return "bottom-up" }
if t == "BU" { return "bottom-up" }
""
}
// Edge-arrow detection
//
// Detects the longest matching arrow token in a line, returning
// { "ok": Bool, "from_str": Str, "kind": Str, "rest": Str }
fn extract_edge_parts(line: String) -> Map<String, Any> {
// Order: longest first to avoid partial matches.
let f1: Int = str_index_of(line, "-/->")
if f1 >= 0 {
return { "ok": true,
"from_str": str_slice(line, 0, f1),
"kind": "forbidden",
"rest": str_slice(line, f1 + 4, str_len(line)) }
}
let f2: Int = str_index_of(line, "<->")
if f2 >= 0 {
return { "ok": true,
"from_str": str_slice(line, 0, f2),
"kind": "bidirectional",
"rest": str_slice(line, f2 + 3, str_len(line)) }
}
let f3: Int = str_index_of(line, "-->")
if f3 >= 0 {
return { "ok": true,
"from_str": str_slice(line, 0, f3),
"kind": "dashed",
"rest": str_slice(line, f3 + 3, str_len(line)) }
}
let f4: Int = str_index_of(line, "->")
if f4 >= 0 {
return { "ok": true,
"from_str": str_slice(line, 0, f4),
"kind": "solid",
"rest": str_slice(line, f4 + 2, str_len(line)) }
}
{ "ok": false, "from_str": "", "kind": "", "rest": "" }
}
fn is_edge_line(line: String) -> Bool {
if str_contains(line, "->") { return true }
if str_contains(line, "<->") { return true }
false
}
// Error helpers
fn make_error(line_no: Int, line_text: String, message: String) -> Map<String, Any> {
{ "error": message, "line": line_no, "text": line_text }
}
// Parse driver
//
// State is held in process-local k/v rather than threaded through every
// function. Specifically:
// "title", "direction" graph header
// "nodes_json", "edges_json", "groups_json" accumulators (string lists)
// "group_stack_depth" "0".."N" open groups
// "group_stack_<i>_id" / "_label" / "_line" frame data
// "group_stack_<i>_node_ids" JSON array of ids inside frame
// "error" non-empty if parse failed
// "error_line", "error_text" context
fn st_set_int(key: String, v: Int) -> Int { state_set(key, int_to_str(v)); 0 }
fn st_get_int(key: String) -> Int {
let s: String = state_get(key)
if str_eq(s, "") { return 0 }
str_to_int(s)
}
// Encode/decode small string lists via "" delimiter (unit separator).
fn list_encode(xs: [String]) -> String {
let n: Int = el_list_len(xs)
let out = ""
let i = 0
while i < n {
if i > 0 { let out = out + "" }
let out = out + get(xs, i)
let i = i + 1
}
out
}
fn list_decode(s: String) -> [String] {
if str_eq(s, "") { return el_list_empty() }
str_split(s, "")
}
fn current_group_index() -> Int {
st_get_int("group_stack_depth") - 1
}
fn group_frame_key(idx: Int, suffix: String) -> String {
"gs_" + int_to_str(idx) + "_" + suffix
}
fn open_group(id: String, label: String, line_no: Int) -> Int {
let depth: Int = st_get_int("group_stack_depth")
state_set(group_frame_key(depth, "id"), id)
state_set(group_frame_key(depth, "label"), label)
state_set(group_frame_key(depth, "line"), int_to_str(line_no))
state_set(group_frame_key(depth, "ids"), "")
st_set_int("group_stack_depth", depth + 1)
0
}
fn close_group_frame() -> Map<String, Any> {
let depth: Int = st_get_int("group_stack_depth")
if depth <= 0 {
return { "ok": false, "id": "", "label": "", "ids": "" }
}
let idx: Int = depth - 1
let id: String = state_get(group_frame_key(idx, "id"))
let label: String = state_get(group_frame_key(idx, "label"))
let ids: String = state_get(group_frame_key(idx, "ids"))
state_del(group_frame_key(idx, "id"))
state_del(group_frame_key(idx, "label"))
state_del(group_frame_key(idx, "line"))
state_del(group_frame_key(idx, "ids"))
st_set_int("group_stack_depth", idx)
{ "ok": true, "id": id, "label": label, "ids": ids }
}
fn register_node_in_group(node_id: String) -> Int {
let depth: Int = st_get_int("group_stack_depth")
if depth <= 0 { return 0 }
let idx: Int = depth - 1
let key: String = group_frame_key(idx, "ids")
let prev: String = state_get(key)
if str_eq(prev, "") {
state_set(key, node_id)
} else {
state_set(key, prev + "" + node_id)
}
0
}
// Accumulator JSON-ish encoding for nodes/edges/groups.
// We render each entry as a small string and stash in state under a counter.
fn store_node(id: String, label: String, shape: String) -> Int {
let n: Int = st_get_int("node_count")
state_set("node_id_" + int_to_str(n), id)
state_set("node_label_" + int_to_str(n), label)
state_set("node_shape_" + int_to_str(n), shape)
st_set_int("node_count", n + 1)
0
}
fn store_edge(src: String, dst: String, label: String, kind: String) -> Int {
let n: Int = st_get_int("edge_count")
state_set("edge_from_" + int_to_str(n), src)
state_set("edge_to_" + int_to_str(n), dst)
state_set("edge_label_" + int_to_str(n), label)
state_set("edge_kind_" + int_to_str(n), kind)
st_set_int("edge_count", n + 1)
0
}
fn store_group(id: String, label: String, ids: String) -> Int {
let n: Int = st_get_int("group_count")
state_set("group_id_" + int_to_str(n), id)
state_set("group_label_" + int_to_str(n), label)
state_set("group_ids_" + int_to_str(n), ids)
st_set_int("group_count", n + 1)
0
}
fn set_error(msg: String, line_no: Int, line_text: String) -> Int {
state_set("parse_error", msg)
st_set_int("parse_error_line", line_no)
state_set("parse_error_text", line_text)
0
}
fn has_error() -> Bool {
let m: String = state_get("parse_error")
if str_eq(m, "") { return false }
true
}
// Reset state at the start of each parse pass.
fn reset_state() -> Int {
state_set("graph_title", "")
state_set("graph_direction", "top-down")
st_set_int("node_count", 0)
st_set_int("edge_count", 0)
st_set_int("group_count", 0)
st_set_int("group_stack_depth", 0)
state_set("parse_error", "")
st_set_int("parse_error_line", 0)
state_set("parse_error_text", "")
0
}
// Statement-level parsing
fn parse_node_stmt(line_no: Int, line: String) -> Int {
let id_split: Map<String, Any> = split_identifier(line)
let raw_id: String = id_split["id"]
if str_eq(raw_id, "") {
set_error("expected node id, edge, or keyword", line_no, line)
return 0
}
let id: String = sanitize_id(raw_id)
let rest: String = str_trim(id_split["rest"])
// Optional shape: [token]
let shape = "rect"
let after_shape = rest
if str_len(rest) > 0 {
let lead: String = str_char_at(rest, 0)
if lead == "[" {
let close: Int = str_index_of(rest, "]")
if close < 0 {
set_error("unclosed `[` in shape token", line_no, line)
return 0
}
let token: String = str_slice(rest, 1, close)
let parsed_shape: String = shape_from_token(token)
if str_eq(parsed_shape, "") {
set_error("unknown shape `" + token + "`", line_no, line)
return 0
}
let shape = parsed_shape
let after_shape = str_trim(str_slice(rest, close + 1, str_len(rest)))
}
}
// Optional quoted label.
let quoted: Map<String, Any> = parse_quoted(after_shape)
let label = raw_id
let ok: Bool = quoted["ok"]
if ok {
let label = quoted["value"]
}
store_node(id, label, shape)
register_node_in_group(id)
1
}
fn parse_edge_stmt(line_no: Int, line: String) -> Int {
let parts: Map<String, Any> = extract_edge_parts(line)
let ok: Bool = parts["ok"]
if !ok {
set_error("malformed edge — expected `->` `-->` `<->` or `-/->`", line_no, line)
return 0
}
let from_str: String = parts["from_str"]
let rest_str: String = parts["rest"]
let kind: String = parts["kind"]
let src: String = sanitize_id(str_trim(from_str))
let rest_t: String = str_trim(rest_str)
let id_split: Map<String, Any> = split_identifier(rest_t)
let to_raw: String = id_split["id"]
if str_eq(to_raw, "") {
set_error("edge missing target node id", line_no, line)
return 0
}
let dst: String = sanitize_id(to_raw)
let label_rest: String = str_trim(id_split["rest"])
let quoted: Map<String, Any> = parse_quoted(label_rest)
let label = ""
let qok: Bool = quoted["ok"]
if qok {
let label = quoted["value"]
}
store_edge(src, dst, label, kind)
1
}
fn parse_group_open(line_no: Int, line: String, rest: String) -> Int {
// Strip trailing `{`.
let trimmed: String = str_trim(rest)
let n: Int = str_len(trimmed)
let body = trimmed
if n > 0 {
let last: String = str_char_at(trimmed, n - 1)
if last == "{" {
let body = str_trim(str_slice(trimmed, 0, n - 1))
}
}
let id_split: Map<String, Any> = split_identifier(body)
let raw_id: String = id_split["id"]
if str_eq(raw_id, "") {
set_error("group declaration missing id", line_no, line)
return 0
}
let label_rest: String = str_trim(id_split["rest"])
let quoted: Map<String, Any> = parse_quoted(label_rest)
let label = raw_id
let qok: Bool = quoted["ok"]
if qok {
let label = quoted["value"]
}
open_group(raw_id, label, line_no)
1
}
fn parse_close_brace(line_no: Int) -> Int {
let frame: Map<String, Any> = close_group_frame()
let frame_ok: Bool = frame["ok"]
if !frame_ok {
set_error("unexpected `}` — no open group", line_no, "}")
return 0
}
store_group(frame["id"], frame["label"], frame["ids"])
1
}
fn parse_line_dispatch(line_no: Int, line: String) -> Int {
if line == "}" { return parse_close_brace(line_no) }
if str_starts_with(line, "title:") {
let after: String = str_trim(str_slice(line, 6, str_len(line)))
let q: Map<String, Any> = parse_quoted(after)
let qok: Bool = q["ok"]
if !qok {
set_error("expected quoted string after `title:`", line_no, line)
return 0
}
state_set("graph_title", q["value"])
return 1
}
if str_starts_with(line, "direction:") {
let after: String = str_trim(str_slice(line, 10, str_len(line)))
let dir: String = parse_direction(after)
if str_eq(dir, "") {
set_error("unknown direction — expected top-down, left-right, right-left, or bottom-up",
line_no, line)
return 0
}
state_set("graph_direction", dir)
return 1
}
if str_starts_with(line, "group ") {
let after: String = str_slice(line, 6, str_len(line))
return parse_group_open(line_no, line, after)
}
if is_edge_line(line) {
return parse_edge_stmt(line_no, line)
}
parse_node_stmt(line_no, line)
}
// Materialise accumulators into the final graph map
fn build_graph_value() -> Map<String, Any> {
let n_nodes: Int = st_get_int("node_count")
let nodes: [Map<String, Any>] = el_list_empty()
let i = 0
while i < n_nodes {
let s: String = int_to_str(i)
let node: Map<String, Any> = {
"id": state_get("node_id_" + s),
"label": state_get("node_label_" + s),
"shape": state_get("node_shape_" + s)
}
let nodes = native_list_append(nodes, node)
let i = i + 1
}
let n_edges: Int = st_get_int("edge_count")
let edges: [Map<String, Any>] = el_list_empty()
let i = 0
while i < n_edges {
let s: String = int_to_str(i)
let edge: Map<String, Any> = {
"from": state_get("edge_from_" + s),
"to": state_get("edge_to_" + s),
"label": state_get("edge_label_" + s),
"kind": state_get("edge_kind_" + s)
}
let edges = native_list_append(edges, edge)
let i = i + 1
}
let n_groups: Int = st_get_int("group_count")
let groups: [Map<String, Any>] = el_list_empty()
let i = 0
while i < n_groups {
let s: String = int_to_str(i)
let raw_ids: String = state_get("group_ids_" + s)
let id_list: [String] = list_decode(raw_ids)
let group: Map<String, Any> = {
"id": state_get("group_id_" + s),
"label": state_get("group_label_" + s),
"node_ids": id_list,
"direction": ""
}
let groups = native_list_append(groups, group)
let i = i + 1
}
{
"title": state_get("graph_title"),
"direction": state_get("graph_direction"),
"nodes": nodes,
"edges": edges,
"groups": groups
}
}
// Public entry point
fn arbor_parse(source: String) -> Map<String, Any> {
reset_state()
let lines: [Map<String, Any>] = preprocess(source)
let n: Int = el_list_len(lines)
let i = 0
let abort = false
while i < n {
if abort {
// skip error already recorded
} else {
let row: Map<String, Any> = get(lines, i)
let line_no: Int = row["no"]
let text: String = row["text"]
parse_line_dispatch(line_no, text)
if has_error() {
let abort = true
}
}
let i = i + 1
}
if !has_error() {
let depth: Int = st_get_int("group_stack_depth")
if depth > 0 {
let idx: Int = depth - 1
let id: String = state_get(group_frame_key(idx, "id"))
let line_no: Int = st_get_int(group_frame_key(idx, "line"))
set_error("unclosed group '" + id + "' — missing closing `}`",
line_no, "group " + id)
}
}
if has_error() {
return {
"error": state_get("parse_error"),
"line": st_get_int("parse_error_line"),
"text": state_get("parse_error_text")
}
}
build_graph_value()
}
// Smoke test
fn fail_msg(label: String, got: String, want: String) -> Int {
println("FAIL " + label + " got=[" + got + "] want=[" + want + "]")
state_set("smoke_failures", "1")
0
}
fn check_eq(label: String, got: String, want: String) -> Int {
if got == want {
println("ok " + label)
return 1
}
fail_msg(label, got, want)
}
// Helper: a graph map is in the error state iff it has a non-empty "error".
fn parse_failed(g: Map<String, Any>) -> Bool {
let m: String = g["error"]
if str_eq(m, "") { return false }
// map_get returns NULL for missing keys; str_eq treats two NULLs as equal
// and NULL vs "" as not equal guard explicitly.
if str_len(m) == 0 { return false }
true
}
let src1 = "title: \"Test\"\ndirection: left-right\n\napi [rounded] \"REST API\"\ndb [cylinder] \"Postgres\"\n\napi -> db \"reads\""
let g1: Map<String, Any> = arbor_parse(src1)
if parse_failed(g1) {
println("FAIL parse 1: " + g1["error"])
state_set("smoke_failures", "1")
}
check_eq("title parsed", g1["title"], "Test")
check_eq("direction parsed", g1["direction"], "left-right")
let nodes1: [Map<String, Any>] = g1["nodes"]
let nn1: Int = el_list_len(nodes1)
check_eq("two nodes", int_to_str(nn1), "2")
let edges1: [Map<String, Any>] = g1["edges"]
let ne1: Int = el_list_len(edges1)
check_eq("one edge", int_to_str(ne1), "1")
let e0: Map<String, Any> = get(edges1, 0)
check_eq("edge from", e0["from"], "api")
check_eq("edge to", e0["to"], "db")
check_eq("edge label", e0["label"], "reads")
check_eq("edge kind", e0["kind"], "solid")
let n0: Map<String, Any> = get(nodes1, 0)
check_eq("node 0 shape", n0["shape"], "rounded")
check_eq("node 0 label", n0["label"], "REST API")
// Test edge varieties
let src2 = "a \"A\"\nb \"B\"\na -> b\na --> b\na -/-> b\na <-> b"
let g2: Map<String, Any> = arbor_parse(src2)
let edges2: [Map<String, Any>] = g2["edges"]
check_eq("4 edges parsed", int_to_str(el_list_len(edges2)), "4")
let kinds = ""
let i = 0
while i < el_list_len(edges2) {
let e: Map<String, Any> = get(edges2, i)
let k: String = e["kind"]
let kinds = kinds + k + ","
let i = i + 1
}
check_eq("edge kinds", kinds, "solid,dashed,forbidden,bidirectional,")
// Groups
let src3 = "group core \"Application Core\" {\n api [rounded] \"REST API\"\n svc \"Business Logic\"\n}\nstandalone \"Out\""
let g3: Map<String, Any> = arbor_parse(src3)
let groups3: [Map<String, Any>] = g3["groups"]
check_eq("one group", int_to_str(el_list_len(groups3)), "1")
let grp0: Map<String, Any> = get(groups3, 0)
check_eq("group label", grp0["label"], "Application Core")
let gnids: [String] = grp0["node_ids"]
check_eq("group has 2 members", int_to_str(el_list_len(gnids)), "2")
let nodes3: [Map<String, Any>] = g3["nodes"]
check_eq("3 total nodes (incl standalone)",
int_to_str(el_list_len(nodes3)), "3")
// Error: unknown shape
let src4 = "node [hexagon] \"X\""
let g4: Map<String, Any> = arbor_parse(src4)
let err4: String = g4["error"]
if str_eq(err4, "") {
println("FAIL expected error for unknown shape")
state_set("smoke_failures", "1")
} else {
if str_contains(err4, "hexagon") {
println("ok error mentions hexagon: " + err4)
} else {
println("FAIL error wording: " + err4)
state_set("smoke_failures", "1")
}
}
// Error: unclosed group
let src5 = "group g \"G\" {\n a \"A\"\n"
let g5: Map<String, Any> = arbor_parse(src5)
let err5: String = g5["error"]
if str_eq(err5, "") {
println("FAIL expected unclosed-group error")
state_set("smoke_failures", "1")
} else {
if str_contains(err5, "unclosed") {
println("ok unclosed group detected")
} else {
println("FAIL unclosed error wording: " + err5)
state_set("smoke_failures", "1")
}
}
// Comments and inline comments
let src6 = "// header\na \"A\" // trailing\nb \"B\""
let g6: Map<String, Any> = arbor_parse(src6)
check_eq("comments stripped", int_to_str(el_list_len(g6["nodes"])), "2")
// Empty input
let g7: Map<String, Any> = arbor_parse("")
check_eq("empty graph nodes", int_to_str(el_list_len(g7["nodes"])), "0")
check_eq("empty graph default direction", g7["direction"], "top-down")
println("")
let f: String = state_get("smoke_failures")
if str_eq(f, "1") {
println("arbor-parse: FAILED")
exit_program(1)
} else {
println("arbor-parse: ok")
}
-21
View File
@@ -1,21 +0,0 @@
// arbor-render SVG renderer. Consumes a diagram graph + layout result and
// emits an SVG document. PNG rasterization is not provided in this vessel
// because the El runtime does not expose a vector-to-raster primitive yet
// (see report).
vessel "arbor-render" {
version "0.1.0"
description "SVG renderer for Arbor diagrams"
authors ["Neuron Technologies"]
edition "2026"
}
dependencies {
arbor-core "0.1"
arbor-layout "0.1"
}
build {
entry "src/main.el"
output "dist/"
}
-575
View File
@@ -1,575 +0,0 @@
// arbor-render SVG emission from a laid-out diagram.
//
// Entry point:
// fn arbor_render_svg(graph: Map, layout: Map, forbidden: [String]) -> String
//
// The graph is the lowered (diagram-form) shape produced by arbor-core /
// arbor-diagram (`title`, `direction`, `nodes`, `edges`, `groups`). The
// layout is whatever arbor-layout returned: `node_pos_<id>`, `node_size_<id>`,
// `group_bounds_<id>`, `node_ids`, `group_ids`, `canvas`.
//
// `forbidden` is a list of "from->to" key strings same format as
// arbor-core's collect_forbidden(). The Rust crate threaded a HashSet
// through; El threads a list and we linear-scan.
//
// SVG is text emission straightforward El. Every float coordinate is
// passed through format_float(_, 1) for stable output.
//
// PNG render is intentionally out of scope
// The Rust crate rasterises via resvg tiny_skia png. The El runtime
// today exposes no equivalent: there is no resvg, no usvg, no font rasterer,
// no PNG encoder, no path-fill code. fs_write writes text only there is
// no binary write primitive. arbor_render_png() returns an error map in El
// until the runtime grows a rasterer (see "runtime gaps" in the report).
// Colour palette (matches the Rust constants exactly)
fn col_node_fill() -> String { "#ffffff" }
fn col_node_stroke() -> String { "#334155" }
fn col_primary_fill() -> String { "#0052A0" }
fn col_primary_text() -> String { "#ffffff" }
fn col_node_text() -> String { "#0D0D14" }
fn col_edge() -> String { "#64748B" }
fn col_edge_forbidden() -> String { "#DC2626" }
fn col_group_fill() -> String { "rgba(0,0,0,0.03)" }
fn col_group_stroke() -> String { "#CBD5E1" }
fn col_group_text() -> String { "#64748B" }
fn col_edge_label() -> String { "#64748B" }
// XML escape
fn esc(s: String) -> String {
let r1: String = str_replace(s, "&", "&amp;")
let r2: String = str_replace(r1, "<", "&lt;")
let r3: String = str_replace(r2, ">", "&gt;")
let r4: String = str_replace(r3, "\"", "&quot;")
r4
}
// Float to "%.1f" the Rust pt() helper.
fn pt(v: el_val_t) -> String {
format_float(v, 1)
}
// Float arithmetic helpers float_to_int / int_to_float trip through Int,
// which is exact for the integer-valued floats used by the layout pass.
fn fadd(a: el_val_t, b: el_val_t) -> el_val_t {
let ai: Int = float_to_int(a)
let bi: Int = float_to_int(b)
int_to_float(ai + bi)
}
fn fsub(a: el_val_t, b: el_val_t) -> el_val_t {
let ai: Int = float_to_int(a)
let bi: Int = float_to_int(b)
int_to_float(ai - bi)
}
fn fdiv2(a: el_val_t) -> el_val_t {
let ai: Int = float_to_int(a)
int_to_float(ai / 2)
}
fn fmid(a: el_val_t, b: el_val_t) -> el_val_t {
fdiv2(fadd(a, b))
}
// forbidden-edge linear lookup
fn forbidden_key(from: String, to: String) -> String {
from + "->" + to
}
fn forbidden_contains(set: [String], src: String, dst: String) -> Bool {
let key: String = forbidden_key(src, dst)
let n: Int = el_list_len(set)
let i = 0
while i < n {
let s: String = get(set, i)
if str_eq(s, key) { return true }
let i = i + 1
}
false
}
// Arrow marker defs
fn arrow_defs() -> String {
let s = "\n <marker id=\"ah\" markerWidth=\"10\" markerHeight=\"7\" refX=\"9\" refY=\"3.5\" orient=\"auto\">\n"
let s = s + " <polygon points=\"0 0, 10 3.5, 0 7\" fill=\"" + col_edge() + "\"/>\n"
let s = s + " </marker>\n"
let s = s + " <marker id=\"ah-bi\" markerWidth=\"10\" markerHeight=\"7\" refX=\"1\" refY=\"3.5\" orient=\"auto-start-reverse\">\n"
let s = s + " <polygon points=\"0 0, 10 3.5, 0 7\" fill=\"" + col_edge() + "\"/>\n"
let s = s + " </marker>\n"
let s = s + " <marker id=\"ah-red\" markerWidth=\"10\" markerHeight=\"7\" refX=\"9\" refY=\"3.5\" orient=\"auto\">\n"
let s = s + " <polygon points=\"0 0, 10 3.5, 0 7\" fill=\"" + col_edge_forbidden() + "\"/>\n"
let s = s + " </marker>"
s
}
// Node rendering
fn render_node(buf: String, node: Map<String, Any>, layout: Map<String, Any>) -> String {
let nid: String = node["id"]
let pos: Map<String, Any> = el_map_get(layout, "node_pos_" + nid)
let sz: Map<String, Any> = el_map_get(layout, "node_size_" + nid)
let cx: el_val_t = pos["x"]
let cy: el_val_t = pos["y"]
let w: el_val_t = sz["w"]
let h: el_val_t = sz["h"]
let x: el_val_t = fsub(cx, fdiv2(w))
let y: el_val_t = fsub(cy, fdiv2(h))
let fill_in: String = node["style_fill"]
let stroke_in: String = node["style_stroke"]
let color_in: String = node["style_color"]
let fill = col_node_fill()
if str_len(fill_in) > 0 { let fill = fill_in }
let stroke = col_node_stroke()
if str_len(stroke_in) > 0 { let stroke = stroke_in }
let text_col = col_node_text()
if str_len(color_in) > 0 { let text_col = color_in }
let shape: String = node["shape"]
let buf = buf
if str_eq(shape, "rectangle") {
let buf = buf + " <rect x=\"" + pt(x) + "\" y=\"" + pt(y)
let buf = buf + "\" width=\"" + pt(w) + "\" height=\"" + pt(h)
let buf = buf + "\" rx=\"4\" fill=\"" + fill + "\" stroke=\"" + stroke
let buf = buf + "\" stroke-width=\"1.5\"/>\n"
}
if str_eq(shape, "rounded_rect") {
let buf = buf + " <rect x=\"" + pt(x) + "\" y=\"" + pt(y)
let buf = buf + "\" width=\"" + pt(w) + "\" height=\"" + pt(h)
let buf = buf + "\" rx=\"20\" fill=\"" + fill + "\" stroke=\"" + stroke
let buf = buf + "\" stroke-width=\"1.5\"/>\n"
}
if str_eq(shape, "stadium") {
let buf = buf + " <rect x=\"" + pt(x) + "\" y=\"" + pt(y)
let buf = buf + "\" width=\"" + pt(w) + "\" height=\"" + pt(h)
let buf = buf + "\" rx=\"" + pt(fdiv2(h)) + "\" fill=\"" + fill
let buf = buf + "\" stroke=\"" + stroke + "\" stroke-width=\"1.5\"/>\n"
}
if str_eq(shape, "cylinder") {
// body: rect from y+ry to bottom; ry ≈ h/6 (Rust uses h*0.18, we use h/6
// to stay in integer arithmetic visually indistinguishable on the
// canvas sizes the layout produces).
let hi: Int = float_to_int(h)
let ry: el_val_t = int_to_float(hi / 6)
let body_y: el_val_t = fadd(y, ry)
let body_h: el_val_t = fsub(h, ry)
let buf = buf + " <rect x=\"" + pt(x) + "\" y=\"" + pt(body_y)
let buf = buf + "\" width=\"" + pt(w) + "\" height=\"" + pt(body_h)
let buf = buf + "\" fill=\"" + fill + "\" stroke=\"" + stroke + "\" stroke-width=\"1.5\"/>\n"
// top ellipse
let buf = buf + " <ellipse cx=\"" + pt(cx) + "\" cy=\"" + pt(body_y)
let buf = buf + "\" rx=\"" + pt(fdiv2(w)) + "\" ry=\"" + pt(ry)
let buf = buf + "\" fill=\"" + fill + "\" stroke=\"" + stroke + "\" stroke-width=\"1.5\"/>\n"
// bottom ellipse
let bot_y: el_val_t = fadd(y, h)
let buf = buf + " <ellipse cx=\"" + pt(cx) + "\" cy=\"" + pt(bot_y)
let buf = buf + "\" rx=\"" + pt(fdiv2(w)) + "\" ry=\"" + pt(ry)
let buf = buf + "\" fill=\"" + fill + "\" stroke=\"" + stroke + "\" stroke-width=\"1.5\"/>\n"
}
if str_eq(shape, "diamond") {
let hw: el_val_t = fdiv2(w)
let hh: el_val_t = fdiv2(h)
let buf = buf + " <polygon points=\""
let buf = buf + pt(cx) + "," + pt(fsub(cy, hh)) + " "
let buf = buf + pt(fadd(cx, hw)) + "," + pt(cy) + " "
let buf = buf + pt(cx) + "," + pt(fadd(cy, hh)) + " "
let buf = buf + pt(fsub(cx, hw)) + "," + pt(cy)
let buf = buf + "\" fill=\"" + fill + "\" stroke=\"" + stroke + "\" stroke-width=\"1.5\"/>\n"
}
// Label.
let label: String = node["label"]
let buf = buf + " <text x=\"" + pt(cx) + "\" y=\"" + pt(cy)
let buf = buf + "\" text-anchor=\"middle\" dominant-baseline=\"middle\""
let buf = buf + " class=\"arbor-node-label\" fill=\"" + text_col + "\">"
let buf = buf + esc(label) + "</text>\n"
// Sublabel Rust's DiagramNode stores Option<String>; El uses "" sentinel.
let sub: String = node["sublabel"]
if str_len(sub) > 0 {
let sub_y: el_val_t = fadd(cy, int_to_float(14))
let buf = buf + " <text x=\"" + pt(cx) + "\" y=\"" + pt(sub_y)
let buf = buf + "\" text-anchor=\"middle\" dominant-baseline=\"middle\""
let buf = buf + " class=\"arbor-node-label\" fill=\"" + text_col + "\" font-size=\"10\">"
let buf = buf + esc(sub) + "</text>\n"
}
buf
}
// Edge rendering
//
// We emit a straight line from one node centre to the other and let the
// browser draw it; the Rust crate renders cubic bezier paths but the runtime
// has no robust math layer, and the rectangles are large enough that
// straight edges read clearly. (See "runtime gaps".)
fn render_edge(buf: String, edge: Map<String, Any>, layout: Map<String, Any>, forbidden: [String]) -> String {
let from_id: String = edge["from"]
let to_id: String = edge["to"]
let from_pos: Map<String, Any> = el_map_get(layout, "node_pos_" + from_id)
let to_pos: Map<String, Any> = el_map_get(layout, "node_pos_" + to_id)
let fx: el_val_t = from_pos["x"]
let fy: el_val_t = from_pos["y"]
let tx: el_val_t = to_pos["x"]
let ty: el_val_t = to_pos["y"]
let is_forbidden: Bool = forbidden_contains(forbidden, from_id, to_id)
let stroke = col_edge()
if is_forbidden { let stroke = col_edge_forbidden() }
let line: String = edge["line"]
let arrow: String = edge["arrow"]
let dash_attr = ""
if str_eq(line, "dashed") { let dash_attr = " stroke-dasharray=\"5,3\"" }
if str_eq(line, "dotted") { let dash_attr = " stroke-dasharray=\"2,2\"" }
let marker_start = ""
if str_eq(arrow, "both") { let marker_start = " marker-start=\"url(#ah-bi)\"" }
if str_eq(arrow, "backward") { let marker_start = " marker-start=\"url(#ah-bi)\"" }
let marker_end = " marker-end=\"url(#ah)\""
if is_forbidden { let marker_end = " marker-end=\"url(#ah-red)\"" }
if str_eq(arrow, "none") { let marker_end = "" }
if str_eq(arrow, "backward") { let marker_end = "" }
let buf = buf + " <line x1=\"" + pt(fx) + "\" y1=\"" + pt(fy)
let buf = buf + "\" x2=\"" + pt(tx) + "\" y2=\"" + pt(ty)
let buf = buf + "\" stroke=\"" + stroke + "\" stroke-width=\"1.5\""
let buf = buf + dash_attr + marker_start + marker_end + "/>\n"
// Forbidden marker circle-X at midpoint.
if is_forbidden {
let mx: el_val_t = fmid(fx, tx)
let my: el_val_t = fmid(fy, ty)
let r: el_val_t = int_to_float(7)
let buf = buf + " <circle cx=\"" + pt(mx) + "\" cy=\"" + pt(my)
let buf = buf + "\" r=\"" + pt(r) + "\" fill=\"white\" stroke=\""
let buf = buf + col_edge_forbidden() + "\" stroke-width=\"1.5\"/>\n"
let off: el_val_t = int_to_float(4)
let buf = buf + " <line x1=\"" + pt(fsub(mx, off)) + "\" y1=\"" + pt(fsub(my, off))
let buf = buf + "\" x2=\"" + pt(fadd(mx, off)) + "\" y2=\"" + pt(fadd(my, off))
let buf = buf + "\" stroke=\"" + col_edge_forbidden() + "\" stroke-width=\"1.5\"/>\n"
let buf = buf + " <line x1=\"" + pt(fadd(mx, off)) + "\" y1=\"" + pt(fsub(my, off))
let buf = buf + "\" x2=\"" + pt(fsub(mx, off)) + "\" y2=\"" + pt(fadd(my, off))
let buf = buf + "\" stroke=\"" + col_edge_forbidden() + "\" stroke-width=\"1.5\"/>\n"
}
// Edge label
let label: String = edge["label"]
if str_len(label) > 0 {
let mx: el_val_t = fmid(fx, tx)
let my: el_val_t = fmid(fy, ty)
let lw: el_val_t = int_to_float(str_len(label) * 7 + 8)
let lh: el_val_t = int_to_float(16)
let buf = buf + " <rect x=\"" + pt(fsub(mx, fdiv2(lw))) + "\" y=\"" + pt(fsub(my, fdiv2(lh)))
let buf = buf + "\" width=\"" + pt(lw) + "\" height=\"" + pt(lh)
let buf = buf + "\" rx=\"3\" fill=\"white\" opacity=\"0.85\"/>\n"
let buf = buf + " <text x=\"" + pt(mx) + "\" y=\"" + pt(my)
let buf = buf + "\" text-anchor=\"middle\" dominant-baseline=\"middle\""
let buf = buf + " class=\"arbor-edge-label\">" + esc(label) + "</text>\n"
}
buf
}
// Group rendering
fn render_group(buf: String, group: Map<String, Any>, layout: Map<String, Any>) -> String {
let gid: String = group["id"]
let bounds: Map<String, Any> = el_map_get(layout, "group_bounds_" + gid)
// Layout may not have bounds for empty groups defensive.
let bx_check: el_val_t = bounds["x"]
if float_to_int(bx_check) == 0 {
// Could be a real 0; cheaper to skip via presence check on group_ids.
}
let bx: el_val_t = bounds["x"]
let by: el_val_t = bounds["y"]
let bw: el_val_t = bounds["w"]
let bh: el_val_t = bounds["h"]
let buf = buf + " <rect x=\"" + pt(bx) + "\" y=\"" + pt(by)
let buf = buf + "\" width=\"" + pt(bw) + "\" height=\"" + pt(bh)
let buf = buf + "\" rx=\"8\" fill=\"" + col_group_fill() + "\" stroke=\""
let buf = buf + col_group_stroke() + "\" stroke-width=\"1\" stroke-dasharray=\"4,3\"/>\n"
// Group label in the top-left corner.
let lx: el_val_t = fadd(bx, int_to_float(8))
let ly: el_val_t = fadd(by, int_to_float(14))
let label: String = group["label"]
let buf = buf + " <text x=\"" + pt(lx) + "\" y=\"" + pt(ly)
let buf = buf + "\" class=\"arbor-group-label\">" + esc(label) + "</text>\n"
buf
}
// Public entry point
fn arbor_render_svg(graph: Map<String, Any>, layout: Map<String, Any>, forbidden: [String]) -> String {
let canvas: Map<String, Any> = el_map_get(layout, "canvas")
let cw: el_val_t = canvas["w"]
let ch: el_val_t = canvas["h"]
let buf = "<svg xmlns=\"http://www.w3.org/2000/svg\" width=\"" + pt(cw)
let buf = buf + "\" height=\"" + pt(ch) + "\" viewBox=\"0 0 " + pt(cw) + " " + pt(ch) + "\">\n"
let buf = buf + " <defs>"
let buf = buf + arrow_defs()
let buf = buf + "\n <style>\n"
let buf = buf + " .arbor-node-label { font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 13px; }\n"
let buf = buf + " .arbor-group-label { font-family: 'Helvetica Neue', Helvetica, Arial, monospace; font-size: 10px; fill: " + col_group_text() + "; letter-spacing: 0.08em; }\n"
let buf = buf + " .arbor-edge-label { font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 11px; fill: " + col_edge_label() + "; }\n"
let buf = buf + " </style>\n"
let buf = buf + " </defs>\n"
// Groups first (behind everything).
let buf = buf + " <!-- Groups -->\n"
let groups: [Map<String, Any>] = graph["groups"]
let gn: Int = el_list_len(groups)
let i = 0
while i < gn {
let g: Map<String, Any> = get(groups, i)
let gid: String = g["id"]
// Only render groups the layout actually placed.
let gids: [String] = el_map_get(layout, "group_ids")
let placed = false
let j = 0
while j < el_list_len(gids) {
if str_eq(get(gids, j), gid) { let placed = true }
let j = j + 1
}
if placed {
let buf = render_group(buf, g, layout)
}
let i = i + 1
}
// Edges
let buf = buf + " <!-- Edges -->\n"
let edges: [Map<String, Any>] = graph["edges"]
let en: Int = el_list_len(edges)
let i = 0
while i < en {
let e: Map<String, Any> = get(edges, i)
let buf = render_edge(buf, e, layout, forbidden)
let i = i + 1
}
// Nodes
let buf = buf + " <!-- Nodes -->\n"
let nodes: [Map<String, Any>] = graph["nodes"]
let nn: Int = el_list_len(nodes)
let i = 0
while i < nn {
let n: Map<String, Any> = get(nodes, i)
let buf = render_node(buf, n, layout)
let i = i + 1
}
// Title
let title: String = graph["title"]
if str_len(title) > 0 {
let title_x: el_val_t = fdiv2(cw)
let buf = buf + " <text x=\"" + pt(title_x) + "\" y=\"22\" text-anchor=\"middle\""
let buf = buf + " font-family=\"'Helvetica Neue', Helvetica, Arial, sans-serif\""
let buf = buf + " font-size=\"15\" font-weight=\"600\" fill=\"" + col_node_text() + "\">"
let buf = buf + esc(title) + "</text>\n"
}
let buf = buf + "</svg>\n"
buf
}
// PNG not implemented; the runtime has no SVG rasterizer or PNG encoder.
// Returns an error map that callers can inspect via map["error"].
fn arbor_render_png(graph: Map<String, Any>, layout: Map<String, Any>, forbidden: [String]) -> Map<String, Any> {
{
"error": "PNG rasterization not available in El runtime — install a runtime image library or use the Rust binary"
}
}
// Smoke test
fn fail(label: String, msg: String) -> Int {
println("FAIL " + label + ": " + msg)
state_set("smoke_failures", "1")
0
}
fn check_contains(label: String, haystack: String, needle: String) -> Int {
if str_contains(haystack, needle) {
println("ok " + label)
return 1
}
fail(label, "missing [" + needle + "]")
}
fn check_not_contains(label: String, haystack: String, needle: String) -> Int {
if str_contains(haystack, needle) {
return fail(label, "should not contain [" + needle + "]")
}
println("ok " + label)
1
}
fn make_test_node(id: String, label: String, shape: String) -> Map<String, Any> {
{
"id": id, "label": label, "sublabel": "",
"shape": shape,
"style_fill": "", "style_stroke": "", "style_color": ""
}
}
fn make_test_edge(src: String, dst: String, line: String, arrow: String, label: String) -> Map<String, Any> {
{
"from": src, "to": dst, "label": label,
"line": line, "arrow": arrow
}
}
fn make_test_pos(x: Int, y: Int) -> Map<String, Any> {
{ "x": int_to_float(x), "y": int_to_float(y) }
}
fn make_test_size(w: Int, h: Int) -> Map<String, Any> {
{ "w": int_to_float(w), "h": int_to_float(h) }
}
// Build a minimal layout map by hand.
fn build_layout(node_ids: [String], group_ids: [String], cw: Int, ch: Int) -> Map<String, Any> {
let r: Map<String, Any> = el_map_new(0)
let r = el_map_set(r, "node_ids", node_ids)
let r = el_map_set(r, "group_ids", group_ids)
let r = el_map_set(r, "canvas", { "w": int_to_float(cw), "h": int_to_float(ch) })
r
}
let n_a: Map<String, Any> = make_test_node("a", "Node A", "rectangle")
let n_b: Map<String, Any> = make_test_node("b", "Node B", "rectangle")
let e_ab: Map<String, Any> = make_test_edge("a", "b", "solid", "forward", "")
let nodes: [Map<String, Any>] = native_list_empty()
let nodes = native_list_append(nodes, n_a)
let nodes = native_list_append(nodes, n_b)
let edges: [Map<String, Any>] = native_list_empty()
let edges = native_list_append(edges, e_ab)
let groups: [Map<String, Any>] = native_list_empty()
let g: Map<String, Any> = {
"title": "Test", "direction": "top-down",
"nodes": nodes, "edges": edges, "groups": groups
}
let nid_list: [String] = native_list_empty()
let nid_list = native_list_append(nid_list, "a")
let nid_list = native_list_append(nid_list, "b")
let gid_list: [String] = native_list_empty()
let layout: Map<String, Any> = build_layout(nid_list, gid_list, 400, 300)
let layout = el_map_set(layout, "node_pos_a", make_test_pos(100, 60))
let layout = el_map_set(layout, "node_pos_b", make_test_pos(100, 200))
let layout = el_map_set(layout, "node_size_a", make_test_size(120, 40))
let layout = el_map_set(layout, "node_size_b", make_test_size(120, 40))
let forbidden: [String] = native_list_empty()
let svg: String = arbor_render_svg(g, layout, forbidden)
check_contains("svg starts with <svg", svg, "<svg xmlns=")
check_contains("svg ends with </svg>", svg, "</svg>")
check_contains("svg contains node label", svg, "Node A")
check_contains("svg contains title", svg, ">Test</text>")
check_contains("svg has rect for rectangle node", svg, "<rect")
check_contains("svg has line for edge", svg, "<line")
check_contains("svg has arrow marker def", svg, "id=\"ah\"")
// Escape test
let n_esc: Map<String, Any> = make_test_node("x", "A & B <C>", "rectangle")
let nodes2: [Map<String, Any>] = native_list_empty()
let nodes2 = native_list_append(nodes2, n_esc)
let g2: Map<String, Any> = {
"title": "Test <Title>", "direction": "top-down",
"nodes": nodes2, "edges": native_list_empty(), "groups": native_list_empty()
}
let nid2: [String] = native_list_empty()
let nid2 = native_list_append(nid2, "x")
let layout2: Map<String, Any> = build_layout(nid2, native_list_empty(), 200, 100)
let layout2 = el_map_set(layout2, "node_pos_x", make_test_pos(80, 40))
let layout2 = el_map_set(layout2, "node_size_x", make_test_size(120, 40))
let svg2: String = arbor_render_svg(g2, layout2, native_list_empty())
check_contains("escapes ampersand", svg2, "&amp;")
check_contains("escapes <", svg2, "&lt;")
check_not_contains("no raw <C>", svg2, "<C>")
// Forbidden edge
let e_fb: Map<String, Any> = make_test_edge("a", "b", "solid", "forward", "")
let edges3: [Map<String, Any>] = native_list_empty()
let edges3 = native_list_append(edges3, e_fb)
let g3: Map<String, Any> = {
"title": "F", "direction": "top-down",
"nodes": nodes, "edges": edges3, "groups": native_list_empty()
}
let fb: [String] = native_list_empty()
let fb = native_list_append(fb, forbidden_key("a", "b"))
let svg3: String = arbor_render_svg(g3, layout, fb)
check_contains("forbidden uses red marker", svg3, "ah-red")
check_contains("forbidden colour present", svg3, col_edge_forbidden())
// Diamond shape polygon
let n_d: Map<String, Any> = make_test_node("d", "Decide", "diamond")
let g4: Map<String, Any> = {
"title": "", "direction": "top-down",
"nodes": native_list_append(native_list_empty(), n_d),
"edges": native_list_empty(), "groups": native_list_empty()
}
let nid4: [String] = native_list_append(native_list_empty(), "d")
let layout4: Map<String, Any> = build_layout(nid4, native_list_empty(), 200, 100)
let layout4 = el_map_set(layout4, "node_pos_d", make_test_pos(80, 50))
let layout4 = el_map_set(layout4, "node_size_d", make_test_size(120, 40))
let svg4: String = arbor_render_svg(g4, layout4, native_list_empty())
check_contains("diamond uses polygon", svg4, "<polygon")
// Cylinder shape ellipses
let n_cy: Map<String, Any> = make_test_node("cy", "DB", "cylinder")
let g5: Map<String, Any> = {
"title": "", "direction": "top-down",
"nodes": native_list_append(native_list_empty(), n_cy),
"edges": native_list_empty(), "groups": native_list_empty()
}
let nid5: [String] = native_list_append(native_list_empty(), "cy")
let layout5: Map<String, Any> = build_layout(nid5, native_list_empty(), 200, 100)
let layout5 = el_map_set(layout5, "node_pos_cy", make_test_pos(80, 50))
let layout5 = el_map_set(layout5, "node_size_cy", make_test_size(120, 40))
let svg5: String = arbor_render_svg(g5, layout5, native_list_empty())
check_contains("cylinder uses ellipse", svg5, "<ellipse")
// Dashed edge
let e_dash: Map<String, Any> = make_test_edge("a", "b", "dashed", "forward", "")
let g6: Map<String, Any> = {
"title": "", "direction": "top-down",
"nodes": nodes, "edges": native_list_append(native_list_empty(), e_dash),
"groups": native_list_empty()
}
let svg6: String = arbor_render_svg(g6, layout, native_list_empty())
check_contains("dashed line dasharray", svg6, "stroke-dasharray=\"5,3\"")
// PNG returns an error map
let png: Map<String, Any> = arbor_render_png(g, layout, native_list_empty())
let err: String = png["error"]
if str_len(err) > 0 {
println("ok PNG returns error map")
} else {
println("FAIL PNG should have returned error")
state_set("smoke_failures", "1")
}
println("")
let f: String = state_get("smoke_failures")
if str_eq(f, "1") {
println("arbor-render: FAILED")
exit_program(1)
} else {
println("arbor-render: ok")
}
-153
View File
@@ -1,153 +0,0 @@
<title>Completing El</title>
<style>
:root{
--board:#f4f2ec; --board-line:#e2ded2; --ink:#1c1f26; --ink-soft:#4a5160;
--ink-faint:#8b8f9a; --rule:#d8d3c6; --card:#fbfaf6;
--red:#a8321e; --amber:#9a6a12; --green:#2f6b46; --blue:#1f4e79;
--accent:#1f4e79;
}
@media (prefers-color-scheme: dark){
:root:not([data-theme="light"]){
--board:#14161b; --board-line:#212530; --ink:#e8e6df; --ink-soft:#a8adb8;
--ink-faint:#6f7480; --rule:#2a2f3a; --card:#191c23;
--red:#e4785f; --amber:#d9a441; --green:#6fbf8e; --blue:#7fb2e0;
--accent:#7fb2e0;
}
}
:root[data-theme="dark"]{
--board:#14161b; --board-line:#212530; --ink:#e8e6df; --ink-soft:#a8adb8;
--ink-faint:#6f7480; --rule:#2a2f3a; --card:#191c23;
--red:#e4785f; --amber:#d9a441; --green:#6fbf8e; --blue:#7fb2e0;
--accent:#7fb2e0;
}
*{box-sizing:border-box}
body{
margin:0; background:var(--board); color:var(--ink);
font:16px/1.65 ui-serif,Georgia,"Iowan Old Style",Palatino,serif;
background-image:linear-gradient(var(--board-line) 1px,transparent 1px),
linear-gradient(90deg,var(--board-line) 1px,transparent 1px);
background-size:28px 28px;
}
.wrap{max-width:960px;margin:0 auto;padding:56px 24px 96px}
.mono{font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace}
header{border-bottom:2px solid var(--ink);padding-bottom:18px;margin-bottom:8px}
h1{font-size:clamp(2rem,5vw,3rem);margin:0;letter-spacing:-.02em;text-wrap:balance}
.sub{color:var(--ink-soft);font-size:1.05rem;margin:10px 0 0}
.meta{font-family:ui-monospace,SFMono-Regular,Menlo,monospace;font-size:.78rem;
color:var(--ink-faint);text-transform:uppercase;letter-spacing:.09em;margin-top:14px}
h2{font-size:1.45rem;margin:52px 0 6px;letter-spacing:-.01em}
h2 .n{font-family:ui-monospace,monospace;font-size:.8rem;color:var(--accent);
display:block;letter-spacing:.12em;margin-bottom:4px;font-weight:400}
.lede{color:var(--ink-soft);margin:0 0 18px}
p{margin:0 0 14px}
.card{background:var(--card);border:1px solid var(--rule);border-radius:3px;padding:20px 22px;margin:16px 0}
.scroll{overflow-x:auto;-webkit-overflow-scrolling:touch}
table{border-collapse:collapse;width:100%;font-size:.9rem;min-width:640px}
th{text-align:left;font-family:ui-monospace,monospace;font-size:.72rem;
text-transform:uppercase;letter-spacing:.09em;color:var(--ink-faint);
border-bottom:1px solid var(--ink);padding:0 12px 8px 0;font-weight:400}
td{padding:11px 12px 11px 0;border-bottom:1px solid var(--rule);vertical-align:top}
td.f{font-weight:600;white-space:nowrap}
td.m{font-family:ui-monospace,monospace;font-size:.83rem;font-variant-numeric:tabular-nums}
.dead{color:var(--red);font-weight:600}
.part{color:var(--amber);font-weight:600}
.ok{color:var(--green);font-weight:600}
blockquote{margin:18px 0;padding:2px 0 2px 20px;border-left:3px solid var(--accent);
color:var(--ink-soft);font-style:italic}
ul{margin:0 0 14px;padding-left:22px} li{margin-bottom:9px}
.q{border-left:3px solid var(--amber);padding:14px 0 14px 20px;margin:18px 0}
.q b{display:block;font-size:1.05rem;margin-bottom:5px;font-style:normal}
.q span{color:var(--ink-soft);font-size:.94rem}
code{font-family:ui-monospace,monospace;font-size:.88em;background:var(--card);
border:1px solid var(--rule);border-radius:2px;padding:1px 5px}
hr{border:0;border-top:1px solid var(--rule);margin:44px 0}
.foot{color:var(--ink-faint);font-size:.86rem;margin-top:60px;
border-top:1px solid var(--rule);padding-top:18px}
.tag{display:inline-block;font-family:ui-monospace,monospace;font-size:.68rem;
letter-spacing:.08em;text-transform:uppercase;border:1px solid var(--rule);
border-radius:2px;padding:2px 7px;color:var(--ink-faint);margin-left:8px;vertical-align:middle}
</style>
<div class="wrap">
<header>
<h1>Completing El</h1>
<p class="sub">A working surface. Nothing here is settled, and none of the code is assumed right — El is self-hosting, so all of it can change and be rebuilt.</p>
<p class="meta">Whiteboard v0 · no sacred cows · not a plan, not a task list</p>
</header>
<h2><span class="n">01</span>What we established</h2>
<p>El is a <b>concept-oriented language</b> — the first, and intended as the last, because every other family is oriented toward a <em>representation</em> of a concept rather than the concept. Procedures, objects, functions, predicates are the shapes concepts get flattened into. Once the primitive is the concept, there is no further rung.</p>
<p>Everything here is El. The engram is an El program, the soul is El, <code>elp</code> is El, ingest is El. Which gives the load-bearing consequence:</p>
<blockquote>A concept with no home in El does not disappear. It becomes C, or it becomes a convention.</blockquote>
<p>Both are measurable, and both were measured. As C: <span class="mono">20,504</span> lines of <code>el_runtime.c</code> — 2.3× the entire self-hosting language it serves (<span class="mono">9,089</span> lines), ~47% of it engram code that has its own six sibling files. As convention, from <code>language.md</code> §18.0 — <em>"these are not four problems, they are one absence, four times"</em>:</p>
<div class="card scroll">
<table>
<thead><tr><th>Concern</th><th>Fragments</th><th>The convention it became</th></tr></thead>
<tbody>
<tr><td class="f">Process identity</td><td class="m">0 guards</td><td>"check nothing is already running first"</td></tr>
<tr><td class="f">Configuration</td><td class="m">20 env vars</td><td>"remember the right default here"</td></tr>
<tr><td class="f">Durability</td><td class="m">62 call sites</td><td>"after you mutate, remember to persist"</td></tr>
<tr><td class="f">Request auth</td><td class="m">10 per-route</td><td>"check the token in this handler too"</td></tr>
<tr><td class="f">Index-after-append</td><td class="m">9 of 9 failed</td><td>"after you append, remember to index"</td></tr>
</tbody>
</table>
</div>
<p>The last row is the strongest evidence available about what this class of convention is worth: it failed at <b>100% of its sites</b>.</p>
<h2><span class="n">02</span>The decomposition axis</h2>
<p class="lede">Not by file, module, or subsystem. <b>By faculty.</b></p>
<p>Every defect fought in the last day resolves to a faculty rather than a bug, and each one leaked out of El into something else — into C, into a Swift binary, into a shell script with a curl timeout, into a convention nobody performs.</p>
<div class="card scroll">
<table>
<thead><tr><th>Faculty</th><th>State</th><th>Measured</th><th>Where it leaked to</th></tr></thead>
<tbody>
<tr><td class="f">Ingest <span class="tag">take in</span></td><td class="dead">dead</td><td class="m">2 min → 0 nodes</td><td>separate process, uploads bytes over HTTP to a process with direct fs access; 5 functions where there is 1</td></tr>
<tr><td class="f">Recall <span class="tag">remember</span></td><td class="dead">dead</td><td class="m">own definition ranked 8th</td><td>lexical substring scan; empty on 23 of 24 multi-token queries</td></tr>
<tr><td class="f">Transduce <span class="tag">perceive</span></td><td class="dead">dead</td><td class="m">1 node, 0 edges</td><td>intake flattens signal to a point; <code>realized:false</code>; caller must declare the modality</td></tr>
<tr><td class="f">Think <span class="tag">reason</span></td><td class="dead">dead</td><td class="m">direction [0,0,0,…]</td><td>null gradient from any anchor, any faculty, byte-identical; confidence at the uninformed prior</td></tr>
<tr><td class="f">Realize <span class="tag">express</span></td><td class="part">partial</td><td class="m">13-word vocabulary</td><td>organ was 939 lines of Swift beside the language; voice read from a file path</td></tr>
<tr><td class="f">Body <span class="tag">substrate</span></td><td class="part">partial</td><td class="m">CC 356 / 1,626 lines</td><td><code>engram_activate_inner</code> — recall itself, with 356 unexamined paths</td></tr>
<tr><td class="f">Persist <span class="tag">endure</span></td><td class="ok">live</td><td class="m">100% embedded</td><td>works; every signal placed in geometry at intake, 13,562 of 13,562</td></tr>
</tbody>
</table>
</div>
<p>Stated plainly: it cannot take in, cannot remember, cannot perceive, cannot reason, and barely speaks. These were filed as tickets against a repository. They are faculties of the thing the repository <em>is</em>.</p>
<h2><span class="n">03</span>The ordering principle</h2>
<p>El's compiler is written in El. Every concept the language gains, the compiler can then be written <em>in</em> — so the tool improves the tool, and the fixpoint (stage2 ≡ stage3, byte-identical) makes each turn provable rather than hopeful. The verifier answers in <span class="mono">2.9s</span>.</p>
<p>Which means the ordering criterion is not size of payoff:</p>
<blockquote>Order by leverage on the <em>next</em> iteration. Which concept, added to El, most increases the ability to add the following one?</blockquote>
<p>In a recursive system that dominates immediate value — a small early gain that compounds beats a large one that doesn't. It also bounds itself correctly: unbounded in depth, bounded in rate, because nothing lands that the compiler and the fixpoint have not passed.</p>
<h2><span class="n">04</span>Open — for the whiteboard</h2>
<div class="q"><b>What does a declaration bind to?</b><span>If <code>cat</code> names a region rather than a struct — one that shifts and completes against the engram and the neighbouring code — then what is written at the declaration site, and what is resolved at use? This is the centre of the whole thing and it is not specified anywhere yet.</span></div>
<div class="q"><b>Is "the type checker" a type checker at all?</b><span>§2.3 records annotations as parsed and skipped, and every codegen hazard is downstream of that — <code>+</code> dispatching on AST node kind, <code>==</code> lowering to <code>str_eq</code> unless both operand names are in an int-name set. But if a declaration names a region, checking is asking whether the geometry supports the use. That is grounding, not unification. Naming this wrong builds the wrong thing.</span></div>
<div class="q"><b>Is the faculty list above right?</b><span>Seven were derived from what broke. Derived-from-failure is a biased sample — it finds what is loud, not what is missing. What faculty is absent entirely and therefore never failed?</span></div>
<div class="q"><b>Which concept has the highest leverage on the next turn?</b><span>Candidates so far: the prologue/epilogue seam (§19.3 names it as the prerequisite and its stated blocker has expired — it would collapse 62 + 10 convention sites); <code>protocol</code>/<code>impl</code> (the absence that produced five ingest functions); and the resolution question above. These are not equal and the criterion in §03 should decide it, not preference.</span></div>
<div class="q"><b>What is the seam that makes cognition non-optional?</b><span>"Use the ops" is itself a convention — present in context every turn, enforced by nothing, and it failed at ~100% of sites in a full session. A stronger instruction is still a convention. What makes reasoning-outside-Neuron <em>fail</em>, the way <code>@manager</code> makes <code>dharma_emit</code> outside the boundary a compile error rather than a lint?</span></div>
<hr>
<p class="foot">Working surface, not a design document. The design is what we put on it. Everything above is either measured or quoted from <code>lang/spec/language.md</code>; nothing is inferred and presented as fact.</p>
</div>
View File
Vendored Executable
BIN
View File
Binary file not shown.
File diff suppressed because it is too large Load Diff
Vendored Executable
BIN
View File
Binary file not shown.
-142
View File
@@ -1,142 +0,0 @@
# El — Capabilities
**What the language can do, stated as capabilities rather than as code.**
This list is the unit of analysis. Each entry gets one question — *prove this
cannot be done with pure geometry* — and the answer determines whether it stays a
capability of the language or collapses into the manifold.
Draft, 2026-08-17. Ordered roughly from most-likely-geometry to most-likely-code.
**Status after measurement.** The list was audited against the implementation
the same day. 28 entries collapsed to 19 geometry + 3 code: serialization, text
encoding, network and emission are all *projection onto a basis* (row 18) —
the convention is the basis, never the act. Storage collapsed because
persistence has no caller. Concurrency collapsed because coordination is the
price of forgetting, not a capability. A fourth proof form was added,
**adversarial exactness**, and form 1 stopped being a valid verdict.
**The table answers CAN only.** SHOULD and COST resolve per *site*, not per
capability — `is_digit` and `is_letter` are one capability with opposite
answers, and comparison spans three cost tiers. See the notes below.
---
## The list
| # | Capability | What it means | Verdict |
|---|---|---|---|
| 1 | **Comparison** | is this the same as that; is this greater | zero distance / sign of a displacement |
| 2 | **Ordering** | arrange by a criterion | position along an axis |
| 3 | **Containment** | is this inside that; does this contain that | region membership |
| 4 | **Correspondence** | where does this occur in that; how much of this is in that | a match-strength field over a span |
| 5 | **Segmentation** | divide a whole into parts | boundaries at measured discontinuity |
| 6 | **Composition** | join parts into a whole | adjacency; one position with parts |
| 7 | **Classification** | what kind of thing is this | which region does it land in |
| 8 | **Naming / binding** | attach a name to a thing and find it again | an edge; retrieval is projection |
| 9 | **Collection** | many things held together, indexed, counted | a set of positions; cardinality; projection onto the i-th |
| 10 | **Iteration** | do something for each of many | traversal |
| 11 | **Arithmetic** | quantity, magnitude, combination | displacement algebra on a line |
| 12 | **Time** | when; how long; how often | a 1-D affine space — instants are points, durations displacements, rhythms phases on a circle |
| 13 | **Identity** | which one is this; are these two the same one | coincidence of position |
| 14 | **Selection / dispatch** | choose which behaviour applies | nearest region |
| 15 | **Transformation** | produce a thing from a thing | change of basis |
| 16 | **Grounding** | how well is this supported | the weight on an edge. Has no caller |
| 17 | **Learning** | get better at something | standing changing over time |
| 18 | **Projection** | render meaning onto a surface | change of basis onto a surface basis |
| 19 | **Transduction** | take a signal in | change of basis from a sensor basis |
| ~~20~~ | ~~Serialization~~ | **collapsed → 18.** The format is a basis; projecting onto it is the act | — |
| ~~21~~ | ~~Text encoding~~ | **collapsed → 18.** An encoding is a basis | — |
| ~~22~~ | ~~Storage~~ | **collapsed.** No save — persistence has no caller. Durability survives at one site inside the engram | — |
| ~~23~~ | ~~Network~~ | **split.** Wire format → 18; socket → 24 | — |
| 24 | **Process / OS** | syscalls; the one-way boundary. Where monotonicity stops | CODE, form 2 |
| ~~25~~ | ~~Concurrency~~ | **collapsed.** Monotone state needs no coordination; coordination is the price of forgetting | — |
| 26 | **Memory substrate** | what holds the positions | CODE, form 3 |
| 27 | **Concealment** | meaning made unreadable without a key. *Renamed*: "secrecy" covered one of three things and got the other two backwards — a hash is public, a signature exists to be read. Integrity and authenticity are **grounding under adversarial conditions** (row 16); only concealment stands alone | CODE, form 4 |
| ~~28~~ | ~~Emission~~ | **split.** Laying out → 18; the device write → 24 | — |
---
## Notes on the boundary cases
**27 — Secrecy is the one capability geometry cannot hold, and the proof is not
form 1.** A cryptographic hash is a *deliberately structure-destroying* map: its
entire value is that near inputs land at maximally uncorrelated outputs. Geometry
is the claim that near things stay near. A manifold that approximated SHA-256
would *be* a break of SHA-256. Signature verification is the same: 0.99-valid is
invalid. And X25519 *is* geometry — a group on an elliptic curve — which is
precisely why it must be code, because its security is the *hardness of moving in
that geometry*.
This is a fourth proof form and it should be added to `geometry-vs-code.md`:
**adversarial exactness.** Where approximation is a break, geometry is excluded.
**20, 21 — Serialization and text encoding are convention all the way down**, but
only at the *edge*. The byte format is agreed; what is being written is not. Do not
let a geometric computation inherit a code verdict because its result gets
serialized.
**11, 12 — Arithmetic and time are the same capability.** Instants are points,
durations are displacements, pointpoint→vector, point+vector→point. The runtime
already implements this correctly as `el_instant_add_dur` / `el_duration_add`. That
it *also* implements a five-entry string→multiplier table beside it (`time_add`
with `"ms"/"sec"/"min"/"hour"/"day"`) is the residue.
**7 — Classification is the most-violated capability in the codebase.** Seven ASCII
range tables (`is_letter`, `is_digit`, `is_alphanumeric`, `is_whitespace`,
`is_punctuation`, `is_uppercase`, `is_lowercase`) that return false for every
non-ASCII byte. `str_count_letters` reports zero letters for `é`. The wrongness on
most of Unicode is the tell that a table is standing in for a region.
**4 — Correspondence appears five times.** `str_index_of`, `str_index_of_all`,
`str_last_index_of`, `str_count`, `str_find_chars` are five projections of one
match-strength field: first zero, all zeros, last zero, count of zeros, first
class-crossing. One relation, five functions.
**14 — Selection is the crux for the compiler.** `+` dispatching on AST node kind
is selection-by-enumeration where selection-by-position belongs.
**Correction, 2026-08-17, from measurement.** This entry previously also cited
`==` lowering to `str_eq` "unless both operand names are in a hardcoded int-name
set — a literal list of variable names treated as integers." That is **wrong**.
`__int_names` is populated from *type annotations* (`param["type"] == "Int"`,
`let x: Int`), which is primitive but legitimate type propagation, not an
enumeration of blessed variable names.
The real defect was one layer down: `is_int_call` held **35 hardcoded builtin
return types**, the same shape as the 19 temporal ones. Those moved to
`lang/tools/check/signatures.rel`.
And the mischaracterisation hid a live bug. Because the return types were never
consulted at a *binding* site, an unannotated `let` lost its type:
```el
let a = str_len("hello") // no annotation
let b = str_len("hi")
let c = a + b // el_str_concat(a, b) on two integers
```
That compiled clean, ran, and printed nothing where it should print 7 — no error
at any layer. Present in the pre-change compiler, so pre-existing. Fixed by
taking an unannotated `let`'s type from what its initialiser returns; the data
was already required for dispatch and simply never read there.
**The general lesson, since it recurred all session:** the enumeration was real
but I had located it in the wrong place. Naming a defect from reading is a
hypothesis. Eight hours of reading this file did not surface the miscompilation;
moving the data out and running the result did.
---
## What this list is for
Each capability gets audited **once**, across every place it appears — not once per
file. The output is not a percentage. It is:
- which capabilities survive the question and stay in the language
- which collapse into the manifold
- and for each one that collapses, **every site it currently appears at**, because
those sites are the residue and they are what gets deleted.
The line-count audit produced a map of where the residue sits. This produces a map
of **what it is**.
-217
View File
@@ -1,217 +0,0 @@
<title>The El Architecture</title>
<style>
:root{
--board:#f4f2ec; --board-line:#e5e1d6; --ink:#1c1f26; --ink-soft:#4a5160;
--ink-faint:#8b8f9a; --rule:#d8d3c6; --card:#fbfaf6;
--red:#a8321e; --amber:#9a6a12; --green:#2f6b46; --accent:#1f4e79;
}
@media (prefers-color-scheme: dark){
:root:not([data-theme="light"]){
--board:#14161b; --board-line:#1d212a; --ink:#e8e6df; --ink-soft:#a8adb8;
--ink-faint:#6f7480; --rule:#2a2f3a; --card:#191c23;
--red:#e4785f; --amber:#d9a441; --green:#6fbf8e; --accent:#7fb2e0;
}
}
:root[data-theme="dark"]{
--board:#14161b; --board-line:#1d212a; --ink:#e8e6df; --ink-soft:#a8adb8;
--ink-faint:#6f7480; --rule:#2a2f3a; --card:#191c23;
--red:#e4785f; --amber:#d9a441; --green:#6fbf8e; --accent:#7fb2e0;
}
*{box-sizing:border-box}
body{
margin:0; background:var(--board); color:var(--ink);
font:16px/1.68 ui-serif,Georgia,"Iowan Old Style",Palatino,serif;
background-image:linear-gradient(var(--board-line) 1px,transparent 1px),
linear-gradient(90deg,var(--board-line) 1px,transparent 1px);
background-size:30px 30px;
}
.wrap{max-width:940px;margin:0 auto;padding:56px 24px 96px}
.mono,code{font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace}
header{border-bottom:2px solid var(--ink);padding-bottom:20px}
h1{font-size:clamp(2.1rem,5.5vw,3.2rem);margin:0;letter-spacing:-.025em;text-wrap:balance}
.sub{color:var(--ink-soft);font-size:1.08rem;margin:12px 0 0;max-width:64ch}
.meta{font-family:ui-monospace,monospace;font-size:.76rem;color:var(--ink-faint);
text-transform:uppercase;letter-spacing:.1em;margin-top:16px}
h2{font-size:1.5rem;margin:56px 0 8px;letter-spacing:-.015em;text-wrap:balance}
h2 .n{font-family:ui-monospace,monospace;font-size:.78rem;color:var(--accent);
display:block;letter-spacing:.14em;margin-bottom:5px;font-weight:400}
h3{font-size:1.08rem;margin:30px 0 6px}
p{margin:0 0 14px;max-width:72ch}
.lede{color:var(--ink-soft);margin:0 0 20px;font-size:1.04rem}
.card{background:var(--card);border:1px solid var(--rule);border-radius:3px;padding:20px 22px;margin:18px 0}
.scroll{overflow-x:auto}
table{border-collapse:collapse;width:100%;font-size:.9rem;min-width:600px}
th{text-align:left;font-family:ui-monospace,monospace;font-size:.71rem;
text-transform:uppercase;letter-spacing:.09em;color:var(--ink-faint);
border-bottom:1px solid var(--ink);padding:0 14px 8px 0;font-weight:400}
td{padding:11px 14px 11px 0;border-bottom:1px solid var(--rule);vertical-align:top}
td.f{font-weight:600;white-space:nowrap}
td.m{font-family:ui-monospace,monospace;font-size:.83rem;font-variant-numeric:tabular-nums;white-space:nowrap}
.dead{color:var(--red);font-weight:600}
.part{color:var(--amber);font-weight:600}
.ok{color:var(--green);font-weight:600}
blockquote{margin:20px 0;padding:3px 0 3px 22px;border-left:3px solid var(--accent);
color:var(--ink-soft);font-style:italic;max-width:70ch}
ul{margin:0 0 14px;padding-left:22px;max-width:72ch} li{margin-bottom:9px}
code{font-size:.87em;background:var(--card);border:1px solid var(--rule);border-radius:2px;padding:1px 5px}
pre{background:var(--card);border:1px solid var(--rule);border-radius:3px;
padding:16px 18px;overflow-x:auto;font-size:.85rem;line-height:1.55;margin:16px 0}
pre code{background:none;border:0;padding:0}
.q{border-left:3px solid var(--amber);padding:14px 0 14px 20px;margin:20px 0;max-width:72ch}
.q b{display:block;font-size:1.04rem;margin-bottom:5px}
.q span{color:var(--ink-soft);font-size:.94rem}
hr{border:0;border-top:1px solid var(--rule);margin:46px 0}
.foot{color:var(--ink-faint);font-size:.86rem;margin-top:56px;border-top:1px solid var(--rule);padding-top:18px}
.tag{display:inline-block;font-family:ui-monospace,monospace;font-size:.66rem;
letter-spacing:.08em;text-transform:uppercase;border:1px solid var(--rule);
border-radius:2px;padding:2px 7px;color:var(--ink-faint);margin-left:8px;vertical-align:middle}
.flow{display:flex;gap:0;align-items:stretch;flex-wrap:wrap;margin:22px 0}
.flow div{flex:1 1 200px;border:1px solid var(--rule);background:var(--card);padding:16px 18px}
.flow div+div{border-left:0}
.flow h4{margin:0 0 6px;font-size:.96rem}
.flow p{margin:0;font-size:.87rem;color:var(--ink-soft)}
.flow .k{font-family:ui-monospace,monospace;font-size:.72rem;color:var(--accent);
letter-spacing:.1em;text-transform:uppercase;display:block;margin-bottom:4px}
</style>
<div class="wrap">
<header>
<h1>The El Architecture</h1>
<p class="sub">El is a concept-oriented language. This is the architecture that claim commits it to — what is built, what is measured, and what still has no home.</p>
<p class="meta">Working document · no sacred cows · self-hosting, so nothing here is fixed</p>
</header>
<h2><span class="n">01</span>The primitive is the concept</h2>
<p>Language families are named for their primitive. Procedural — procedures. Object-oriented — objects. Functional — functions. Logic — predicates. Every one of them is oriented toward a <em>representation</em> of a concept: the shape a concept gets flattened into so a machine can hold it.</p>
<p>El's primitive is the concept itself. That is why it is the first of its family and intended as the last — once the primitive is the concept, there is no further rung to climb to.</p>
<p>The consequence is architectural rather than stylistic:</p>
<blockquote>A concept with no home in the language does not disappear. It becomes C, or it becomes a convention.</blockquote>
<p>Both forms are measurable. As C: <span class="mono">20,504</span> lines of <code>el_runtime.c</code>, against <span class="mono">9,089</span> lines for the entire self-hosting language — the shim is 2.3× the language it serves, and ~47% of it is engram code that already has six sibling files. As convention, from <code>lang/spec/language.md</code> §18.0 — <em>"these are not four problems, they are one absence, four times"</em>:</p>
<div class="card scroll">
<table>
<thead><tr><th>Concern</th><th>Fragments into</th><th>The convention it became</th></tr></thead>
<tbody>
<tr><td class="f">Process identity</td><td class="m">0 guards</td><td>"check nothing is already running first"</td></tr>
<tr><td class="f">Configuration</td><td class="m">20 env vars</td><td>"remember the right default here"</td></tr>
<tr><td class="f">Durability</td><td class="m">62 sites</td><td>"after you mutate, remember to persist"</td></tr>
<tr><td class="f">Request auth</td><td class="m">10 routes</td><td>"check the token in this handler too"</td></tr>
<tr><td class="f">Index-after-append</td><td class="m">9 of 9 failed</td><td>"after you append, remember to index"</td></tr>
</tbody>
</table>
</div>
<p>The last row is the strongest available evidence about this class of convention: it failed at <b>every single site</b>. A count is what appears where a concept has no home; the size of the count is how far the fragmentation got, not how hard the problem is.</p>
<h2><span class="n">02</span>Geometry is a first-class value — and what follows</h2>
<p class="lede">This is the enabling primitive. Everything else in the architecture is downstream of it.</p>
<p><code>Geometry</code> is an El value, alongside <code>Int</code>, <code>String</code>, <code>List</code>, <code>Map</code> — bound, passed, returned, composed, carrying its own width. Not a library type, not a handle into a store, not a serialization format. <em>Meaning is a value the language computes with directly.</em></p>
<pre><code>let g: Geometry = geometry_new(4)
fn tone_realizer(signal: String) -> Geometry { … }</code></pre>
<p>Landed 2026-08-16 (#141, #144), and the spec is explicit that it belongs to the language rather than the graph: <em>"neither is engram-specific — any program touching any modality needs them; the engram is merely one El program that happens to hold a graph."</em></p>
<p>Five things follow, and together they are the concept-oriented claim made operational:</p>
<h3>A declaration can name a region, not a shape</h3>
<p>If meaning is a value, a name can be bound to a <em>position</em> rather than a struct. <code>cat</code> is not a fixed record; it is a region that resolves against the engram and the surrounding code. <code>cat</code> among animals and <code>cat</code> among shell utilities are different concepts without a namespace, because they are in different neighbourhoods and the distance says so.</p>
<h3>Checking is grounding, not unification</h3>
<p>If a declaration names a region, then verifying a use is asking whether the geometry supports it — a question about position and distance, not about matching a declared shape. This is why §2.3's "a type checker is planned" is likely the wrong name for the missing piece, and naming it wrong would build the wrong thing.</p>
<h3>Dispatch is position, not a tag</h3>
<p>A vtable is a finite set of discrete labels fixed at link time. A region admits graded membership and an open set. So <code>transduce(signal, modality)</code> asks the caller to supply what the signal already carries — what a thing is falls out of where it lands. The modality parameter is a kind-tag, and a registry keyed on it is a lookup table doing by string what geometry does by nearness.</p>
<h3>Types are discovered, not declared</h3>
<p>Reification crystallizes a densely co-wired neighbourhood into a first-class node — the neighbourhood <em>is</em> the name that was missing. Every other family requires a human to see the abstraction in advance and write <code>class Foo</code>. Here the instances arrive and the type falls out, by measurement rather than by insight.</p>
<h3>Enumeration becomes unnecessary</h3>
<p>Five ingest functions differ only in how bytes are acquired — one operation wearing five surfaces. 356 branches in <code>engram_activate_inner</code> are not 356 behaviours. Cyclomatic complexity is a count of the places comprehension ran out and was replaced by an <code>if</code>; where the concept is expressible, the count collapses instead of being redistributed.</p>
<h2><span class="n">03</span>The shape of the language</h2>
<p>Geometry first-class gives El three layers, and it holds all three — which is why there is no separate database driver and no impedance boundary to manage.</p>
<div class="flow">
<div><span class="k">afferent</span><h4>Transduce</h4><p>Signal in, geometry out. Decomposition into components and relations — never conversion to a point. Realizers are ordinary El functions, so a new modality never requires a runtime patch.</p></div>
<div><span class="k">substrate</span><h4>Geometry</h4><p>Meaning as position; relation as distance. Held as values in the language and persisted in the graph. One coordinate system, so entities are commensurable and the operators compose.</p></div>
<div><span class="k">efferent</span><h4>Realize</h4><p><code>plan(frame) → realize(spec, profile)</code>, where a surface <em>is</em> a profile. Text, speech, music, image are profiles of one projection — and so is source code.</p></div>
</div>
<p>The efferent side is why the recursive property below is possible at all: if source is a surface, then emitting a corrected file is projection, and the file becomes an artifact of the geometry rather than the thing you edit.</p>
<h2><span class="n">04</span>Decomposition is by faculty</h2>
<p class="lede">Not by file, module, or subsystem — by what the system does.</p>
<p>Each faculty is a concept. Where it has no home in El it leaks: into C, into a Swift binary, into a shell script with a <code>curl</code> timeout, into a convention nobody performs. State below is measured, not asserted.</p>
<div class="card scroll">
<table>
<thead><tr><th>Faculty</th><th>State</th><th>Measured</th><th>Where it leaked</th></tr></thead>
<tbody>
<tr><td class="f">Ingest <span class="tag">take in</span></td><td class="dead">dead</td><td class="m">2 min → 0 nodes</td><td>separate process uploading bytes over HTTP to a process with direct fs access; five functions where there is one</td></tr>
<tr><td class="f">Recall <span class="tag">remember</span></td><td class="dead">dead</td><td class="m">self ranked 8th</td><td>lexical substring scan; empty on 23 of 24 multi-token queries</td></tr>
<tr><td class="f">Transduce <span class="tag">perceive</span></td><td class="dead">dead</td><td class="m">1 node, 0 edges</td><td>intake flattens signal to a point; <code>realized:false</code>; caller must declare the modality</td></tr>
<tr><td class="f">Think <span class="tag">reason</span></td><td class="dead">dead</td><td class="m">direction [0,0,…]</td><td>null gradient from any anchor and any faculty, byte-identical; confidence at the uninformed prior</td></tr>
<tr><td class="f">Realize <span class="tag">express</span></td><td class="part">partial</td><td class="m">13-word lexicon</td><td>organ was 939 lines of Swift beside the language; voice read from a file path</td></tr>
<tr><td class="f">Body <span class="tag">substrate</span></td><td class="part">partial</td><td class="m">CC 356 / 1,626 ln</td><td><code>engram_activate_inner</code> — recall itself, 356 unexamined paths</td></tr>
<tr><td class="f">Persist <span class="tag">endure</span></td><td class="ok">live</td><td class="m">13,562 / 13,562</td><td>works — every signal placed in geometry at intake, no backlog</td></tr>
</tbody>
</table>
</div>
<h2><span class="n">05</span>The recursive property</h2>
<p>El's compiler is written in El. Every concept the language gains, the compiler can then be written <em>in</em> — so the tool improves the tool, and <code>codegen.el</code> at 4,661 lines gets shorter as the language gets better at expressing what it does. The fixpoint — stage2 ≡ stage3, byte-identical — makes each turn provable rather than hopeful, and the verifier answers in <span class="mono">2.9s</span>.</p>
<p>This sets the ordering criterion, and it is not size of payoff:</p>
<blockquote>Order by leverage on the <em>next</em> iteration. Which concept, added to El, most increases the ability to add the following one?</blockquote>
<p>A small early gain that compounds beats a large one that does not. And it bounds itself correctly — unbounded in depth, bounded in rate, because nothing lands that the compiler and the fixpoint have not passed.</p>
<h2><span class="n">06</span>What has no home yet</h2>
<p>Reserved in the lexer, no parse form. These are not a feature backlog — they are the concepts the architecture above requires and does not yet hold, which is why each is currently a convention or a block of C.</p>
<div class="card scroll">
<table>
<thead><tr><th>Reserved</th><th>Concept</th><th>Currently lives as</th></tr></thead>
<tbody>
<tr><td class="m">retry · times · fallback · reason</td><td>resilience</td><td>a shell script with a 10s <code>curl</code> timeout; 254 restarts in 3 days</td></tr>
<tr><td class="m">requires · deploy · to · via · target</td><td>deployment</td><td>YAML in another repository</td></tr>
<tr><td class="m">sealed</td><td>capability scope</td><td>consent checks written by hand</td></tr>
<tr><td class="m">protocol · impl</td><td>one operation, many realizations</td><td>five ingest functions; eight faculty routes on one builtin</td></tr>
<tr><td class="m">activate · where</td><td>retrieval</td><td>traversals written by hand</td></tr>
<tr><td class="m">test · seed · assert</td><td>verification</td><td>a framework; 5 of 13 native suites failing</td></tr>
<tr><td class="m">parallel · trace</td><td>concurrency</td><td>pthreads in C</td></tr>
</tbody>
</table>
</div>
<p>Plus, from the spec's own status: annotations parsed and skipped, <code>match</code> parsed and emitting nothing, <code>?</code> a no-op, <code>%</code> unlexed, structs as <code>ElMap</code>, enums as strings, selective import unenforced.</p>
<h2><span class="n">07</span>Open</h2>
<div class="q"><b>What does a declaration bind to, exactly?</b><span>If <code>cat</code> names a region that shifts and completes against context, what is written at the declaration site and what is resolved at use? This is the centre and it is unspecified.</span></div>
<div class="q"><b>Is the faculty list right?</b><span>Seven, derived from what broke. Derived-from-failure is a biased sample — it finds what is loud, not what is absent. Which faculty is missing entirely and therefore never failed?</span></div>
<div class="q"><b>Which concept has the highest leverage on the next turn?</b><span>The prologue/epilogue seam (§19.3 names it as the prerequisite; its stated blocker has expired; it collapses 62 + 10 convention sites), <code>protocol</code>/<code>impl</code>, or resolution itself. The §05 criterion should decide this, not preference.</span></div>
<div class="q"><b>What seam makes cognition non-optional?</b><span>"Use the ops" is itself a convention — present every turn, enforced by nothing, ~100% failure across a full session. A stronger instruction is still a convention. What makes reasoning outside the substrate <em>fail</em>, the way <code>@manager</code> makes <code>dharma_emit</code> outside the boundary a compile error rather than a lint?</span></div>
<hr>
<p class="foot">Every number here is measured or quoted from <code>lang/spec/language.md</code>. Nothing is inferred and presented as fact. El is self-hosting: all of this can change and be rebuilt.</p>
</div>
-245
View File
@@ -1,245 +0,0 @@
# El — Language Design
**Status:** decisions recorded, design unwritten.
**Date:** 2026-08-17.
**Provenance:** decisions are Will's, taken in session. Items marked *proposed* are not
decided and are recorded only so the reasoning isn't lost. Items marked **OPEN** are
his to rule on and must not be guessed at.
Companion documents: `el-architecture.html` (the measured state — see §7 note on its
§04 scoreboard), and `design/completing-el.html` (whiteboard v0: the reduction, the
faculty table, the ordering principle).
---
## 1. The reduction
`language.md` §18.0 records five concerns that decayed into conventions:
| Concern | Fragments | The convention it became |
|---|---|---|
| Process identity | 0 guards | "check nothing is already running first" |
| Configuration | 20 env vars | "remember the right default here" |
| Durability | 62 call sites | "after you mutate, remember to persist" |
| Request auth | 10 per-route | "check the token in this handler too" |
| Index-after-append | 9 of 9 failed | "after you append, remember to index" |
The last row is the strongest available evidence about what this class of convention
is worth: **it failed at 100% of its sites.**
Every one of these is an obligation at a **crossing** — a point where a value moves
between regions. El can name a region and it can name a call. A call is procedural,
so the obligation degrades into something a human must remember to perform.
> **The generator, one level up:** El cannot name what holds at a crossing.
And underneath that:
> **The deeper absence:** El cannot name the thing meaning is made of.
`semel` appears in whitepaper §84, §86, §209, §737, in
`the-metaphysics-of-will-anderson.md`, and in session notes. It appears in **zero code
identifiers**. Every geometric concept in the system — region, neighbourhood, manifold,
world-tube — is defined in terms of a unit the language cannot say, while the code
underneath speaks in arrays, floats and offsets: the vocabulary of a voxel, a value at
a dumb address. Precisely the thing the impact brief says a semel is not.
`el_runtime.c` is a concept that leaked into C. `semel` never got that far — it did
not even decay into a convention.
---
## 2. DECIDED — `semel` is the primitive
**A semel is a difference that matters. The smallest unit of understanding.**
Not a node. Not a coordinate. Not a float.
The reasoning, in Will's terms:
- Meaning is position, and position is only ever relative. *"There is no atom of
meaning that isn't already a relation. It grounds on nothing but difference — two
points and the gap, and the gap is pure not-the-same."*
- A node doesn't mean. A node is a label at a location; labels don't mean.
- A lone coordinate doesn't mean either. Nothing means anything by itself.
- The smallest thing that can be understood is a **distinction**: *these two are not
the same.* Below that there is no content to apprehend.
- And a difference with nothing it matters to is not meaning — it is variation. The
mattering is not decoration; it is what makes it understanding rather than data.
**Consequence: relating is the floor, and the point is derived.** The
point-primitive / relation-primitive fork raised in session is not a fork. It was
answered by the definition.
### Historical note, to be recorded as fact rather than as origin story
The term was coined by Will on the pixel/voxel/texel pattern — *semantic element*,
and Latin *semel*, "once, a single time." It was recognised, not invented, from a
2019 experience he calls **semelation**: perceiving mind as a high-dimensional point
space. The initial reading was "pixels"; the correction to `semel` was made later and
was made on the **mechanism** — a pixel is a value at an address, and what was
perceived had no separate address and value.
Convergence worth citing, not deferring to: neural population geometry and
representational similarity analysis independently model cognition as position in a
high-dimensional space where similarity is distance.
---
## 3. DECIDED — `semel` lands first
By the ordering criterion already on the whiteboard: *which concept, added to El, most
increases the ability to add the next one?* Not size of payoff — **leverage on the next
iteration**, because El compiles itself and the fixpoint makes each turn provable in
2.9s.
**Every other concept on the board is defined in terms of `semel`. It is maximal on
that criterion by construction.**
---
## 4. DECIDED — `ground` is the checker
Whiteboard question 4 — *does `ground` in El mean the same thing as `ground` in the
engram?* — is answered: **yes, and it should be one implementation.**
If a declaration names a region, then type checking is asking whether the geometry
supports the use. That is not unification. **That is grounding**, and it is already
built, proven, and byte-identically reproducible:
```
cc -std=c11 -O2 -o gep_proof gep_proof.c -lm && ./gep_proof
C1 5 independent sources pos_mass 1.3500 n_indep=5 0.1000 → 0.9741 GROUNDED
C2 5 mutually-linked pos_mass 0.2700 n_indep=1 0.1000 → 0.1000 refused
C3 1 source, 5 parallel edges pos_mass 0.2700 n_indep=1 0.1000 → 0.1000 refused
```
Independence-weighted grounding is the general case; execution is the cheap case.
**Attestation is `verify` where nothing can be run** — as already implemented for
language in `authority.py`, where an LLM proposes and a primary source disposes.
At the point where the checker and the grounder are one mechanism, the language and
the mind stop being two things.
---
## 5. OPEN — Will's to rule on
### 5.1 What is a semel's representation in the language?
*Proposed, not decided:* a **displacement from `love = 0`** — a relation held as one
object. It reconciles "the address is the value" with "position is only ever relative,"
because a displacement *is* a relation and is still a single nameable thing.
If taken, the operator set falls out rather than being bolted on:
```
subtract(now, then) → what changed (growth, drift)
translate origin → empathy
rotate frame → reframe
project onto axis → a lens
change basis → analogy, metaphor, skill transfer
reflect an axis → negation, sarcasm
```
Three consequences that would hold:
- **Dimension must never appear in the type.** `semel` opaque, never `[768]float`.
The moment the arity is in the language, the manifold's implementation is in the
language, and adding a modality requires a runtime patch — which the standing rule
forbids.
- **Zero is the only literal.** Everything else is reached by displacement from it,
which makes `love = 0` the base case rather than philosophy adjacent to the type
system.
- **`magnitude` is standing.** Distance from origin is the same quantity
`gep_core.h` already computes.
### 5.2 Is `hold` one construct or two?
The obligation *before* a crossing (auth, guard) and the obligation *after* (persist,
index, free) may be one shape seen from both sides, or the seam may need both faces
named. This decides whether §19.3's prologue/epilogue seam is one construct or a pair.
**Precedent already shipping:** `@manager` makes `dharma_emit` outside the boundary a
**compile error, not a lint.** The concept is proven at N=1; the work is generalising
it and naming it.
**And the shape is already implemented in the learning region:** `L.reach_out` sits
between `L.detect_gap` and `L.verify`. You cannot reach out without a detected gap and
you cannot keep what returns without passing verify. **A hold is a neighbour.** The
obligation is not attached to the crossing — the obligation *is* the adjacent node.
That is why `reach_out` cannot be abused and why 62 persist sites could be.
### 5.3 What does a declaration bind?
If `cat` names a region rather than a struct — one that shifts and completes against
the engram and the neighbouring code — what is written at the declaration site, and
what is resolved at use? **This is the centre and it is specified nowhere.**
Falls out of 5.1 if displacement is taken: a declaration **locates** rather than
allocates.
### 5.4 Is the faculty list right?
Seven were derived from what broke. Derived-from-failure is a biased sample — it finds
what is loud, not what is missing. **What faculty is absent entirely and therefore
never failed?**
---
## 6. The residue map
What each construct must absorb, from §18.0 plus measured state:
| Residue | Count | Absorbed by |
|---|---|---|
| persist-after-mutate | 62 sites | `hold` (after-crossing) |
| auth-per-route | 10 sites | `hold` (before-crossing) |
| index-after-append | 9 of 9 failed | `hold` (after-crossing) |
| env var defaults | 20 | configuration declared once |
| process identity | 0 guards | `hold` (before-crossing) |
| `geometry_free` at every call site | every site | ownership follows from `semel` |
| five ingest functions where there is one | 5 → 1 | `protocol` / `impl` |
| `el_runtime.c` | 20,504 lines | faculty decomposition, ordered after `semel` |
---
## 7. Notes carried forward
**`el-architecture.html` §04 needs its numbers sourced or cut.** An audit found the
faculty scoreboard — `Ingest 2 min → 0 nodes`, `Recall self ranked 8th`,
`Body CC 356 / 1,626 ln`, `the verifier answers in 2.9s`, `5 of 13 native suites
failing` — has no supporting evidence in the repository, under a footer asserting
*"nothing is inferred and presented as fact."* Against a corpus whose documents
supersede their own conclusions in place, that is the one file that would not survive
scrutiny. Fix or remove.
**Source as a projection surface is claimed and unimplemented.** `el-architecture.html`
§147/§150: *"if source is a surface, then emitting a corrected file is projection."*
Greps for `surface_profile_code`, `emit_source` → zero hits.
It is not unbacked. **It was demonstrated on 2026-08-14** — three faculties (phonetic,
semantic, procedural) projected into TypeScript, a surface the system had never used,
with the network severed. Recovered at
`~/Development/neuron-technologies/andre-server-recovered/` and copied into
`evidence/03-andre-demo/`. The claim needs bringing home to El, not proving.
**`hold` is the highest-leverage construct after `semel`** — it collapses 62 + 10 + 9
sites and unblocks the runtime extraction. §19.3 names the prologue/epilogue seam as
the prerequisite and its stated blocker has expired.
---
## 8. What is not decided and must not be guessed
- The representation of `semel` (§5.1)
- One `hold` or two (§5.2)
- What a declaration binds (§5.3)
- The missing faculty (§5.4)
- Sequencing after `semel` — the ordering criterion decides it, not preference
---
*Recorded 2026-08-17. Everything in §2, §3 and §4 is decided. Everything in §5 is open
and is Will's. Nothing here was inferred from a document that was not read.*
-117
View File
@@ -1,117 +0,0 @@
# Geometry or Code
**Running list.** Append as decided. Started 2026-08-17.
**The test:** *is this an arbitrary convention, or is it a relation?*
Conventions were agreed by people and could have been otherwise — a RIFF header could
have used a different magic number. Nothing derives them; they must be written down.
Relations are not agreed. Distance is distance. Anything whose answer is *where is this
relative to that* is geometry, and writing it as code is the error the whole effort is
correcting.
**Second test, for the hard cases:** *if I write this as code, am I encoding in
`if`-statements a distinction the geometry was built to hold?* If yes, it's geometry.
---
## Pure geometry
| Thing | Because |
|---|---|
| Meaning | position |
| Grounding / standing | the weight on the edge — a magnitude, not a computation |
| Learning | standing changing over time |
| A gap | low standing |
| Wonder | a gap with a pull weight |
| Type checking | is this position in that region — distance |
| Dispatch | position, not a tag |
| Recall | re-origining at a region; projection, not replay |
| Reasoning | traversal |
| Deduction | containment. There is no procedure |
| Counting | a position, not a loop's output |
| Similarity / difference / residue | subtract |
| Analogy, metaphor, skill transfer | change of basis |
| Negation, sarcasm | reflect an axis |
| Empathy | translate the origin |
| Reframe | rotate the frame |
| A lens | project onto an axis |
| Rhyme | distance in phonetic space |
| Humour | intersection of regions — fart-meaning ∩ funny ∩ form |
| Idiom detection | the whole unit sits farther out than its parts |
| Self | a world-tube — a trajectory through the manifold |
| Consolidation | episodic → semantic promotion |
| Reification | dense regions cohering; runs on the beat, has no caller |
| Cross-cutting concerns | **dissolved** — a hold is a *neighbour*. Adjacency, not tracking. **Implemented 2026-08-17**: a construct declares what runs at a crossing, and it resolves at execution — see the runtime seam. |
| Effects | topology. `reach_out` is bounded by `detect_gap` and `verify` because those are its edges |
| Capability | position relative to a boundary. In C it is already spelled `const` |
| The AST | a projection of geometry into a tree — a surface, not the centre |
| Source code | a surface, like text, audio, image |
## Must be code
| Thing | Because |
|---|---|
| Sensors — mic, camera, file read, socket | the physical touch. I/O is where the world arrives |
| Byte formats — RIFF, PNG chunks, `MThd`, OOXML | arbitrary convention. A committee chose the magic numbers |
| CRC32 polynomial, Adler32, zlib framing | same — agreed constants, derivable from nothing |
| Cosine, distance, the float arithmetic | the machinery that *walks* the geometry is not itself geometry |
| Arena, refcount, allocator | bookkeeping for the **representation**, not for the positions |
| Locks, threads, publication boundary | the hardware is code. **Ordering is not** — see Answered, above. Coordination is required only where state is non-monotone. |
| WAL, page layout, ARIES recovery | durability against a physical device that can lose power |
| Emission — writing C or JS text | the final surface has to be *typed out* by something |
| OS interaction — launchd, spawn, signals | outside the system by definition |
| Device realizers — `el_audio_darwin.m`, `el_capture_darwin.m` | OS frameworks. Correctly already isolated, zero network |
---
## The ones I would have written as code, and was wrong about
Recorded because the error has a pattern and the pattern is the point.
| Thing | What I reached for | What it is |
|---|---|---|
| Rhyme | a rhyming dictionary, or an API call | distance between rime tails |
| Fart onomatopoeia | a 30-element string literal | an intersection of three regions |
| "Funny" | a scorer with `if`-statements | a relational neighbourhood grounded in a voice |
| Representation vs description | a hardcoded blacklist containing `raspberry` | falls out of lexicon membership × phonetic comedy |
| Video | a codec, sized as a project | one more surface profile |
| Type checking | a phase between parse and emit | reading a distance that already exists |
| Grounding | a call site, an obligation, a discharge | it has no caller. It just runs |
| N transducers, N realizers | one component per modality | zero of each. Sensors and bases at the skin |
**The pattern:** every one is *encoding in code a distinction the geometry was built to
hold.* The tell is that the code version is a **fixed enumeration** — a list, a table, a
blacklist, a set of branches — and the geometry version is a **measurement**.
If the implementation contains a literal set of the right answers, it is in the wrong
column.
---
## Answered
| Thing | The answer |
|---|---|
| Concurrency | **Ordering is geometric.** Causality is a partial order (Lamport 1978); a total order is an arbitrary extension of it and "cannot be depended on to imply a causal relationship." Programming languages force you to write a total order, so authoring *invents* constraints the problem never had — and every lock, barrier, fence and consensus protocol is apparatus for recovering the partial order destroyed at authoring time. CALM (Hellerstein/Alvaro, proven by Ameloot et al.): a program has a consistent coordination-free implementation **iff it is monotone**. What breaks monotonicity is destructive update. **Coordination is the price of forgetting.** |
| The module system | **Premature — the partition is a filesystem path, not a neighbourhood, and there is no namespacing at all.** `import` is textual inlining (guarded against double inclusion); when a `.elh` header exists the header is inlined instead and symbols resolve at C link time, so linking is real and delegated to C. Two modules defining `helper` emit two C functions into one translation unit. Linking barely survives the *path* partition, so whether it survives a neighbourhood partition cannot yet be asked. |
| Numeric literals | **The numeral is convention; the number is a position — and a bare `3` is a MAGNITUDE WITH NO AXIS.** `int_to_str` was already form 1: nothing determines that twelve is written `1` then `2`. But a literal is not a position until something gives it a direction, which is why `3.days` needs a calendar. Measured consequence: `Duration + Int` was refused ("an Int carries no unit") while `Instant + Int` compiled to raw `(t + 3)` and reported clean — silently moving a point by an unspecified amount. The rule was simply never written. Now: `t + 3` is refused, `t + 1.hour` is accepted, because `.hour` supplies the axis. |
| Parsing | **A grammar is a basis; parsing is transduction onto it.** The lexeme→token map is convention (`fn` could have been `def`); shape recognition is a region; the byte traversal is irreducible, like every other traversal. Three things favour *region* for the act: ambiguity (`a * b` needs context — a grammar resolves it with the lexer hack, a region by neighbourhood), error recovery (nearest-match is free), and precedence, which is ordering along an axis with a conventional parameter. **But the SHOULD gate refuses the obvious move:** the keyword table stays code, because the set is closed by the language definition and the lexer runs before the program is understood, so a program can never declare its own keywords. Externalising it costs I/O per compile for zero flexibility — the same verdict as `is_digit` in ASCII. What was actually wrong: 5 of 46 keywords were consumed by nothing, and using one silently miscompiled. |
| Error handling | **`grounded: false` covers not-knowing; it does not cover failed.** Standing is a *signed* component: `> 0` supported, `= 0` unknown, `< 0` contradicted. Not-known and known-false are opposite directions on one axis and a boolean cannot tell them apart. `inhibitory` as an int32 flag is that sign wearing a boolean. |
## Fourth proof form
**4 — ADVERSARIAL EXACTNESS.** Where approximation is a break, geometry is
excluded. A cryptographic hash is a *deliberately structure-destroying* map:
near inputs land at maximally uncorrelated outputs. Geometry is the claim that
near things stay near — a manifold that approximated SHA-256 would *be* a break
of SHA-256. Signature verification is the same: 0.99-valid is invalid. And
X25519 **is** geometry, a group on an elliptic curve, which is precisely why it
must be code: its security is the hardness of moving in that geometry.
**Form 1 no longer survives as a verdict.** Every row it justified turned out to
be a *basis*, not a capability. RFC 8259 fixes where the commas go — that is a
surface, and projecting onto a surface is geometry. A convention describes the
basis you project onto; it never describes an act.
-59
View File
@@ -1,59 +0,0 @@
# v1 — Experiments
Every change to El on `iteration-1` was produced by one loop, run repeatedly:
```
Ishikawa → scientific method → Six Sigma → repeat
```
- **Ishikawa** — name the root cause, not the symptom. *Why is this table here?*
never *why is this table ugly?*
- **Scientific method** — state a hypothesis, **commit predictions before
running**, then run it in an isolated worktree and grade every prediction
including the ones that failed.
- **Six Sigma** — eliminate the defect *class*, then add a control so it cannot
silently return.
## The organising finding
**Predictions that came back FALSE were worth more than the ones that held.**
Nineteen cycles, sixty-one predictions. The eleven that failed produced every
significant result:
| Failed prediction | What it found |
|---|---|
| "the arity table has drifted from the header" | Zero drift — but **110 functions had no entry at all**. The table was not wrong, it was 40% incomplete. |
| "codegen drops below baseline" (×4) | The **traversal is irreducible**. Walking an AST to find calls does not move no matter who decides. Only the rule and the judgment leave. |
| "guards cannot refuse through the seam" | One line, and refusal works. Six compile-time kinds were unnecessary. |
| "C forbids the struct redefinition" | C allows shadowing — and a *different* defect surfaced: an exit injection emitted with an empty target. |
| "routing el_bin_lookup through the gate fixes the SIGSEGV" | It did not. The **fallback** was the hazard: `strlen()` on an integer. I would have shipped the wrong fix and called it verified. |
A prediction that only ever confirms is a demonstration, not a test. One cycle
was run **without** committing predictions first — `async-half-expressible`
and it produced a rigged result: `pthread_join` immediately after
`pthread_create`, with the word `DEFERRED` printed by the test itself. It had to
be discarded and re-run.
## Layout
```
cycles/ one file per loop, numbered in order, named for the DEFECT
findings/ what the cycles produced, cross-cut by kind
```
## Scoreboard
```
cycles run 19
predictions committed 61
predictions FALSE 11 ← the useful ones
silent miscompilations found 4
security-relevant defects 2
architecture questions closed 5
defects in my own measurement 4
```
Every cycle verified the same three things before landing: the compiler
self-hosts byte-identically (gen2 == gen3), the native suite passes, and the
integration harnesses pass. A cycle that could not show all three did not land.
-26
View File
@@ -1,26 +0,0 @@
# Cycles
Each is one `Ishikawa → scientific method → Six Sigma` loop, run in an isolated
worktree so a wrong answer cost nothing. Named for the **defect**, not the fix.
| # | Cycle | Root cause | Predictions | Landed |
|---|---|---|---|---|
| 01 | [constructs-have-nowhere-to-be](01-constructs-have-nowhere-to-be.md) | a construct had nothing to BE, so its meaning lived in the emitter | 3/3 | yes |
| 02 | [a-construct-cannot-refuse](02-a-construct-cannot-refuse.md) | injection discards the target's result; no form said no | 4/4 | yes |
| 03 | [the-wrapper-was-conditional](03-the-wrapper-was-conditional.md) | exit injection needed compile-time knowledge only because the wrapper was conditional | 3/4 | yes |
| 04 | [c-has-no-closure-syntax](04-c-has-no-closure-syntax.md) | "C has no closures" taken as a fact about what is possible | 5/7 | yes |
| 05 | [the-emitter-discards-what-it-knows](05-the-emitter-discards-what-it-knows.md) | codegen sees every construct relation and throws it away | 5/5 | branch |
| 06 | [the-crossing-resolves-at-emission](06-the-crossing-resolves-at-emission.md) | the binary has no table to consult | 3/4 | yes |
| 07 | [invocation-is-not-composable](07-invocation-is-not-composable.md) | the wrapper called the target directly | 5/5 | yes |
| 08 | [the-emitter-adjudicates](08-the-emitter-adjudicates.md) | a prohibition had nowhere to live but a `#error` | 4/5 | yes |
| 09 | [policy-inside-the-compiler](09-policy-inside-the-compiler.md) | a program cannot declare its own restrictions, so the tier policy was compiled in | 4/5 | yes |
| 10 | [a-second-copy-of-the-header](10-a-second-copy-of-the-header.md) | builtin arity hand-maintained beside `el_runtime.h` | 4/5 | yes |
| 11 | [one-type-erases-the-return](11-one-type-erases-the-return.md) | `el_val_t` means the header cannot say `now()` returns an Instant | 4/5 | yes |
| 12 | [judgment-lives-with-knowledge](12-judgment-lives-with-knowledge.md) | the emitter knows the types, so it also judged them | 5/5 | yes |
| 13 | [thirty-five-return-types](13-thirty-five-return-types.md) | `is_int_call` hardcoded what drives `+` dispatch | 6/6 | yes |
| 14 | [keywords-that-reserve-nothing](14-keywords-that-reserve-nothing.md) | 5 of 46 keywords consumed by no path | 6/6 | yes |
| 15 | [no-namespacing-at-all](15-no-namespacing-at-all.md) | `import` is textual inlining; every name is global | 4/4 | yes |
| 16 | [tokens-carry-no-position](16-tokens-carry-no-position.md) | a token was `(kind, value)`, so no diagnostic could name a place | 6/6 | yes |
| 17 | [annotations-are-never-checked](17-annotations-are-never-checked.md) | the annotation feeds dispatch and is never verified | 6/6 | branch |
| 18 | [async-half-expressible](18-async-half-expressible.md) | **first attempt was DOGMA** — no predictions, rigged test | 4/4 (2nd) | branch |
| 19 | [a-convention-is-not-a-gate](19-a-convention-is-not-a-gate.md) | `looks_like_heap_obj` is static, so every type re-derives it | 6/7 | yes |
@@ -1,42 +0,0 @@
# constructs have nowhere to be
One `Ishikawa → scientific method → Six Sigma` loop. The record below is the
commit message as written at the time, before the outcome was known to anyone
reading this file.
## Record — `5718943`
```
let a construct declare its own meaning instead of the emitter knowing it
codegen called fn_has_decorator for exactly three names — manager, accessor,
route. Twelve others parsed, attached as {name,args}, and compiled to nothing,
including four that look like protection: @authenticate (6 uses), @authorize
(3), @rate_limit (3), @validate (2). The cause was not that the branches were
untidy. A construct had nothing to BE, so its meaning had nowhere to live
except the emitter, and every construct was therefore a compiler edit.
A name -> injection table would have moved the enumeration twenty lines up
without removing it. So the construct now carries its own meaning:
@decorator("injects_at_entry", "engram_boundary_beat")
fn audited() {}
@audited
fn risky_op() -> Int { ... } // gets the beat, attributed to "audited"
scan_declared_decorators is a token-level pre-pass beside scan_routes, forced
by streaming codegen having no whole-program AST. manager and accessor are
seeded as the compiled-in core — the fixedSelf shape from substrate.go: a
complete fallback exists, declaration is enrichment.
This is the injection half of the seam only. The prohibition half (@manager's
#error on dharma_emit) stays hardcoded, because "which calls may appear inside
this boundary" is a query over program structure and there is nothing yet to
ask.
Verified three ways: emitted C for existing @manager/@accessor code is
byte-identical to the hardcoded path; a construct with a name the compiler has
never heard of injects correctly; the compiler self-hosts byte-identically.
90/90 native compiler tests pass.
```
@@ -1,43 +0,0 @@
# a construct cannot refuse
One `Ishikawa → scientific method → Six Sigma` loop. The record below is the
commit message as written at the time, before the outcome was known to anyone
reading this file.
## Record — `60737b0`
```
let a construct refuse, not only observe
@authenticate (6 uses), @authorize (3), @rate_limit (3) and @validate (2)
parsed, attached, and compiled to nothing. Fourteen applications that read as
protection and emitted no instruction — a function decorated @authenticate
compiled byte-identically to an undecorated one.
The missing capability was not authentication. It was that a construct could
observe a boundary but never refuse one. injects_at_entry discards the target's
result; there was no form in which a construct could say no.
@decorator("guards_at_entry", "my_auth")
fn authenticate() {}
@authenticate
@authorize
fn handler() -> String { ... }
emits, at entry:
{ el_val_t __g = my_auth(EL_STR("handler"), EL_STR("authenticate")); if (__g) return __g; }
{ el_val_t __g = my_roles(EL_STR("handler"), EL_STR("authorize")); if (__g) return __g; }
Guards precede injections because a refused call must not report a crossing,
and every guard runs where the topmost injecting construct wins — refusal is
not a role, so it does not follow the role convention.
The compiler still knows nothing about auth. The program points the construct
at its own function, which is where that decision belongs.
Verified: existing @manager/@accessor output byte-identical, compiler
self-hosts byte-identically, guards stack in declaration order and emit before
the beat. 94/94 native compiler tests pass.
```
@@ -1,82 +0,0 @@
# the wrapper was conditional
One `Ishikawa → scientific method → Six Sigma` loop. The record below is the
commit message as written at the time, before the outcome was known to anyone
reading this file.
## Record — `4f7568b`
```
give a construct its after-crossing face, and let constructs compose
§6 records 62 persist-after-mutate sites, 10 auth-per-route, and
index-after-append that failed at 9 of 9 — every one an obligation at a
crossing that decayed into "remember to do this afterwards." An obligation a
human must remember is not an obligation, and the 9-of-9 figure is what that
costs.
@decorator("injects_at_exit", "persist_now")
fn durable() {}
The body moves into a static helper and the visible fn becomes a wrapper, so
EARLY RETURNS pass through the exit injection. Emitting it only before the
fall-through return would have silently missed every early return — the exact
failure class this seam exists to remove. Fns with no exit construct emit
byte-identically to before.
Three independent constructs now compose on one fn, none known to the compiler:
el_val_t mutate(el_val_t k) {
{ el_val_t __g = my_auth(EL_STR("mutate"), EL_STR("authenticate")); if (__g) return __g; }
engram_boundary_beat(EL_STR("mutate"), EL_STR("manager"));
el_val_t __r = __el_body_mutate(k);
persist_now(EL_STR("mutate"), EL_STR("durable"), __r);
return __r;
}
Guard, then entry, then body, then exit. §5.2 asked whether `hold` is one
construct or two; the implementation answers one construct with two faces,
selected by declared kind rather than by two mechanisms.
Verified: existing output byte-identical, compiler self-hosts byte-identically,
early returns pass through the exit, ordering holds under composition. 98/98
native compiler tests pass.
```
## Record — `285166c`
```
EXPERIMENT: emit the wrapper unconditionally, so exit binds at runtime too
ISHIKAWA: why did exit injection still need compile-time knowledge? Because the
body-helper wrapper was only emitted when codegen already knew an exit
construct existed. The wrapper being conditional was the cause, not the wrapper
being necessary.
PREDICTIONS AND RESULTS
P1 exit becomes runtime-bindable TRUE returns 14, bound
after the build
P2 codegen shrinks TRUE 5094 -> 5044
P3 cost 5-15% from a call frame on every fn FALSE 0.37s -> 0.38s, ~3%
P4 fixpoint holds TRUE
Every fn now gets a body helper and a wrapper. It has to be unconditional:
early returns must route through something for an exit construct to observe
them, and codegen cannot know which fns will be bound after the binary exists.
Removed with the machinery: declare_exit, decorator_exit, cg_exit_target,
cg_exit_construct, and the injects_at_exit scanner branch.
Two controls failed and were rewritten rather than repaired --
no-exit-construct-emits-no-wrapper asserted the optimisation this removes, so
it is now inverted. The integration harness gained a seventh assertion: an exit
construct declared after the build replaces the result.
99/99 native, 7/7 integration, fixpoint gen2==gen3.
```
## Record — `b40754f`
```
land unconditional wrapper: exit crossings resolve at runtime
```
@@ -1,64 +0,0 @@
# c has no closure syntax
One `Ishikawa → scientific method → Six Sigma` loop. The record below is the
commit message as written at the time, before the outcome was known to anyone
reading this file.
## Record — `2bed848`
```
EXPERIMENT: hand the construct the body as a real closure
ROOT CAUSE of the weaker design: "C has no closures" was taken as a fact about
what is possible. It is a fact about one grammar. Every C++ lambda, every Go
closure, every Rust closure compiles to a struct of captured values plus a
function pointer -- which is what is emitted here. Codegen emits C; it is not
written in C's syntax, and the distinction is the whole difference between a
construct that can only decide whether to repeat and one that controls
invocation.
It would also have crippled the JS backend, which has closures natively, for a
limit that applies only to the C one.
PREDICTIONS AND RESULTS
1 env struct + thunk taking void* TRUE
2 fails to compile: struct redefinition FALSE -- C allows the
inner declaration to shadow. Prediction wrong; C is more permissive than
assumed. A different real defect surfaced instead: a wrap with no exit
construct emitted `(EL_STR("f"), EL_STR(""), __r);` -- a call to an empty
target -- because has_exit was reused as "needs a wrapper" and the exit line
was emitted unconditionally. Fixed.
3 compiles when the target is declared in El FALSE -- and this is
the root cause worth keeping: El has ONE type, el_val_t = int64_t. El's type
system cannot describe a callable, so `extern fn` and the real signature
cannot be made to agree in El's own vocabulary. The fix is not a cast:
codegen DEFINES the wrap calling convention, so codegen emits the extern
declaration. The convention is not El-expressible; it is emitted.
4 target controls invocation, 0..N times TRUE
5 existing @manager output byte-identical TRUE
6 compiler fixpoint holds TRUE
7 emitting the convention makes it compile TRUE
MEASURED
base(5) wrapped by a target that invokes the body twice and sums -> 10
never_runs(5) wrapped by a target that never invokes it -> 999
Neither is expressible by "decide whether to repeat". This supersedes the
repeats_body experiment on experiment/repeats-body, which was built around the
mistaken limit.
```
## Record — `7d01608`
```
land wraps_body: a construct controls invocation
Proven on experiment/wraps-body (2bed848): base(5) wrapped by a target that
invokes the body twice returns 10; a target that never invokes it returns 999.
Neither is expressible by deciding whether to repeat.
Root cause it corrected: 'C has no closures' is a fact about one grammar, not
about what can be emitted. And El's single type (el_val_t = int64_t) cannot
describe a callable, so codegen emits the calling convention rather than asking
El's type system for something it structurally cannot say.
```
@@ -1,48 +0,0 @@
# the emitter discards what it knows
One `Ishikawa → scientific method → Six Sigma` loop. The record below is the
commit message as written at the time, before the outcome was known to anyone
reading this file.
## Record — `a5af871`
```
EXPERIMENT: let the compiler write down what it already knows
HYPOTHESIS: attribution is redundant for static structure. Codegen sees every
construct-to-function relation at emission time and discards it, so the only
way to learn the structure back is to run the program and read what it
reported. That is instrumentation compensating for erasure.
PREDICTIONS, committed before running:
1 derivable at compile time with no runtime call expected TRUE
2 complete for guards and exits (invisible today) expected TRUE
3 answers it for code that has never executed expected TRUE
4 deterministic expected TRUE
5 makes the entry beat redundant expected FALSE
RESULT: 5/5 as predicted. From a program that was never executed:
authenticate guards_at_entry login my_auth
durable injects_at_exit save persist_now
authenticate guards_at_entry critical my_auth
durable injects_at_exit critical persist_now
manager injects_at_entry critical engram_boundary_beat
Prediction 5 held: the relation records that a boundary COULD be crossed, the
beat records that it WAS. They are different facts and neither replaces the
other.
CONSEQUENCE, and it undercuts the first pass on iteration-1: construct identity
was available at compile time all along. With relations recorded at build, the
runtime needs only the function name and attribution becomes a join rather than
a payload. The counter-argument is that the payload is self-describing while
the file must be pinned to the artifact or the two drift and attribution is
silently lost — which is the same conclusion as "compile against a manifold
revision and record the revision in the artifact", reached from the other side.
Written to a file rather than the engram on purpose: a compile that consults a
manifold produces different output from identical source at different times.
The file is content-addressed; the engram ingests it. Determinism preserved,
mechanism proven.
```
@@ -1,170 +0,0 @@
# the crossing resolves at emission
One `Ishikawa → scientific method → Six Sigma` loop. The record below is the
commit message as written at the time, before the outcome was known to anyone
reading this file.
## Record — `35b07ba`
```
EXPERIMENT: resolve the crossing at execution, not at emission
HYPOTHESIS (Will's): a compiler whose one compiled mechanism is extending the
LANGUAGE — not the compiler — can compose without recompilation.
ISHIKAWA — why does a construct require a recompile today?
method codegen inlines the target call into the body
machine the binary has no table to consult
material the declaration lives in source, read at compile time
measurement nothing observes what applied at runtime
root cause the crossing is resolved at EMISSION, not at EXECUTION
CHANGE: codegen emits one unconditional indirection per fn. Which constructs
apply is read from a table that can be written AFTER the binary exists;
targets resolve through dlsym against the running image.
PREDICTIONS AND RESULTS
P1 a construct declared after the build applies TRUE
P2 an unlinked target is skipped, not fatal TRUE
P3 emitting on every fn is measurably slower FALSE — 0.37s -> 0.36s
with 267 indirections and
no bindings. Free unused.
P4 the compiler still self-hosts TRUE (see note)
DEMONSTRATED: an El program with NO decorator in its source, already compiled
and linked, picked up a construct declared afterwards:
$ /tmp/seamrun -> 7
$ echo 'work audited entry audit_entry' > constructs.txt
$ EL_CONSTRUCTS=constructs.txt /tmp/seamrun
AUDIT: work applied by audited
7
P4 note: my first fixpoint test was wrong, not the code. I compared gen1 to
gen2, which must differ whenever codegen's output changes. gen2 == gen3, 267
seam sites, stable.
MEASURED COST, and the root cause was not where I looked
0 bindings 0.36s vs 0.37s baseline free
2 bindings, dlsym per call 2.45s 6.6x
2 bindings, resolved once 0.69s 3.5x recovered
The table scan was never the cost. dlsym walks the dynamic symbol table on
every call. Resolve once and cache — which is the smallest form of what
salience does for memory: what is hot stays resolved. The 0.69s residual is
audit_entry's own printf on two of the compiler's hottest functions, not seam
overhead.
CONSEQUENCE: the five compile-time declaration kinds on iteration-1 are a
compile-time specialisation of something that resolves at runtime. They are not
wrong, but they are not the mechanism — the mechanism is one indirection, and a
kind is data.
```
## Record — `886626a`
```
seam refusal + control tests: a runtime binding can short-circuit
Prediction 3 was FALSE. I expected refusal to be impossible through the seam
because the entry indirection discarded its return. One line:
{ el_val_t __s = el_seam_run(EL_STR(f), 0, 0); if (__s) return __s; }
work() returns 7; bound to a refusing construct AFTER the build it returns 42.
So three of the five compile-time kinds are runtime-bindable: entry injection,
exit injection, and refusal. wraps_body needs invocation control and
prohibits_outside is compile-time by nature.
104/104 native compiler tests pass.
```
## Record — `28d19da`
```
strip the compile-time machinery the seam replaces
PREDICTION: codegen.el drops below 4661, its size before any of these passes.
RESULT: FALSE. 5157 -> 5096. Still +435 over baseline.
injects_at_entry collapsed into the seam removed
guards_at_entry collapsed into the seam removed
injects_at_exit needs the body-helper wrapper STRUCTURAL
wraps_body needs the closure + wrapper structural
prohibits_outside a #error cannot be emitted at runtime
The wrapper is not a consequence of compile-time resolution. Early returns must
be routed through something no matter when the target is resolved, so exit
injection was never going to collapse. I predicted it would because I had
conflated "resolved late" with "emitted less".
What did collapse is entry injection and refusal -- 61 lines of compiler
replaced by one refusable indirection, with the capability now bindable after
the binary exists.
8 tests fail, and they are exactly the 8 controls for compile-time entry
injection and guards. No unrelated breakage: the controls reported precisely
what moved. They assert emission of something that now happens at runtime, so
they need rewriting as integration tests -- which the framework does not
currently support, because runtime binding needs a built binary and an
environment, not compile_capture.
Verified after the strip: fixpoint gen2==gen3, observation and refusal both
work through the seam with the compiler knowing nothing about either.
```
## Record — `8bbb750`
```
control the claim that cannot be unit tested
The seam's whole claim is that a construct declared AFTER a binary exists
applies to that already-built program. compile_capture only sees emitted text,
so it structurally cannot check this: it needs a built binary, a linked target,
and an environment. Verified by hand until now, which is the standing problem
this session has been about.
tests/integration/seam_binding.sh builds a probe from El source containing no
construct at all, links a target that El never references, and asserts:
ok unbound program is unaffected
ok a construct declared AFTER the build applies
ok a construct declared after the build can REFUSE
ok an unlinked target is skipped, not fatal
ok a binding for a different fn does not fire
ok two constructs compose on one crossing
6 assertions, 6 passed, 0 failed
The eight controls that failed after the strip were replaced, not repaired.
They asserted compile-time emission of capability that moved to runtime;
contorting them would have kept an assertion whose subject no longer exists.
Three took their place, asserting the emitted shape, and the behaviour they
used to cover is now the integration harness's job -- which is the honest
division, since the shape and the behaviour are no longer the same fact.
99/99 native compiler tests pass. Fixpoint holds.
```
## Record — `24f7fb5`
```
land the runtime seam: resolve the crossing at execution
Five compile-time passes added 491 lines to the thing that was supposed to stop
growing. The seam is ~55 lines of C and one line of emission, and it does at
runtime what three of those five kinds did at compile time -- for programs that
are already built.
a construct declared AFTER the binary exists applies to it
free when unused: 0.36s vs 0.37s baseline across 267 indirections
dlsym was the cost, not the table scan; resolve-once recovered 3.5x
refusal works, composition works, unlinked targets are skipped not fatal
injects_at_exit and wraps_body do NOT collapse: early returns must route
through the body-helper wrapper regardless of when the target is resolved. The
wrapper is structural, which I had wrong. prohibits_outside cannot move at all
-- a #error has no runtime.
Controls: 99/99 native compiler tests, plus tests/integration/seam_binding.sh
(6/6) for the claim compile_capture structurally cannot see.
```
@@ -1,45 +0,0 @@
# invocation is not composable
One `Ishikawa → scientific method → Six Sigma` loop. The record below is the
commit message as written at the time, before the outcome was known to anyone
reading this file.
## Record — `bc2f26d`
```
EXPERIMENT: invocation control resolves at runtime
ISHIKAWA: why did wraps_body need compile-time knowledge? Because the wrapper
called the target directly. If the wrapper calls through the seam instead, the
seam can call the body itself, and a construct bound after the build decides
how and whether to invoke it.
PREDICTIONS AND RESULTS
P1 wrap becomes runtime-bindable TRUE body x3 -> 21,
never invoked -> 111
P2 codegen shrinks TRUE 5042 -> 4977
P3 cost 5-10% from an indirect call on every fn TRUE 0.36s -> 0.39s, ~8%
P4 zero-param fns break on the empty struct TRUE empty struct is a GNU
extension, empty init
is C23. Fixed with a
char field.
P5 fixpoint holds TRUE
PROCESS FAILURE worth recording: my first patch silently did not apply because
I dropped the assert on the string replacement. The build then failed with
"undeclared identifier __thunk_noargs", which I nearly attributed to the
empty-struct prediction. The guard that would have caught it existed and I
removed it -- the same shape as every other defect found tonight.
Removed: declare_wrap, decorator_wrap, cg_wrap_target, cg_wrap_construct,
params_to_call_args, and the wraps_body scanner branch.
prohibits_outside is now the ONLY construct kind left at compile time, and it
cannot move: a #error has no runtime.
```
## Record — `c04d68f`
```
land runtime invocation control: only prohibition remains compiled
```
@@ -1,53 +0,0 @@
# the emitter adjudicates
One `Ishikawa → scientific method → Six Sigma` loop. The record below is the
commit message as written at the time, before the outcome was known to anyone
reading this file.
## Record — `c741cfe`
```
EXPERIMENT: prohibition becomes a query over emitted relations
I said prohibition could not move because "a #error has no runtime". That
conflated two separable things: WHEN a violation is detected (build time --
correct, and unchanged) and WHERE the rule and the checker live (the compiler
-- assumed).
A prohibition is a containment relation over the call graph. So codegen now
records what it saw:
sneaky calls raw_sql
allowed calls raw_sql
allowed calls @repository
repository calls prohibits:raw_sql
and tools/check/prohibitions.sh decides, at build time, outside the compiler.
PREDICTIONS AND RESULTS
P1 codegen can emit the call graph it already walks TRUE
P2 the check becomes a query outside the compiler TRUE
P3 all prohibition decisions leave codegen TRUE zero #errors now
P4 violations still caught at build time TRUE exit=1
P5 codegen drops below the 4661 baseline FALSE 4962, +301
P5 is the finding. The TRAVERSAL is irreducible -- you must walk the AST to
find calls, and those ~120 lines do not move no matter who decides. What is not
irreducible is the rule (which names) or the decision (#error). Those left. I
predicted the whole 223 lines would go because I had not separated walking from
adjudicating.
Still compiled, and measured rather than assumed: the capability-tier system
(cap_check_call, is_self_formation_call, is_dharma_call, is_llm_call,
cap_record_violation, emit_cap_violations) is 76 lines of the same shape --
prohibits_WITHIN rather than prohibits_outside, so the checker needs the
opposite polarity to absorb it.
98/98 native, 4/4 prohibition_query.sh, 7/7 seam_binding.sh, fixpoint ok.
```
## Record — `60c07ad`
```
land prohibition-as-query: the emitter records, it no longer adjudicates
```
@@ -1,49 +0,0 @@
# policy inside the compiler
One `Ishikawa → scientific method → Six Sigma` loop. The record below is the
commit message as written at the time, before the outcome was known to anyone
reading this file.
## Record — `c2d9596`
```
EXPERIMENT: the capability tier becomes shipped policy plus a query
Capability differs from prohibits_outside in one way that matters: a utility
program cannot be trusted to declare its own restrictions, because it would
declare none. So the policy comes from OUTSIDE the program -- it ships with the
language as data, editable without a compiler release.
tools/check/capabilities.rel 18 names that were string literals in codegen
tools/check/capabilities.sh the query that decides
PREDICTIONS AND RESULTS
P1 codegen emits kind + call graph, drops the 4 name tests TRUE zero #errors
P2 the 18 literals become a data file TRUE
P3 the checker catches capability violations TRUE exit=1
P4 codegen drops ~76 lines TRUE 4963 -> 4881
P5 below the 4661 baseline FALSE ~+230
TWO DEFECTS THE HARNESS FOUND THAT READING WOULD NOT HAVE
1. Calls inside main became invisible. cg_fn returns early for main -- C
provides its own -- so hooking the recording there left every call in main
unrecorded: a blind spot exactly where a program does its work. The old
cap_check_call ran from cg_expr and did see main. Moved the recording to
cg_expr.
2. Caller attribution was stale. __cg_current_fn kept whatever cg_fn set last,
so a violation in main was reported against the previously emitted function.
The test still PASSED, because the violation was detected -- only the name
was wrong, and a diagnostic naming the wrong fn is worse than none. Fixed at
all three main-emission sites; the first patch missed two because the live
path is codegen_streaming.
98/98 native, 7/7 + 4/4 + 5/5 integration, fixpoint ok.
```
## Record — `29f78f9`
```
land capability-as-policy: eighteen literals become a data file
```
@@ -1,54 +0,0 @@
# a second copy of the header
One `Ishikawa → scientific method → Six Sigma` loop. The record below is the
commit message as written at the time, before the outcome was known to anyone
reading this file.
## Record — `9cc6040`
```
EXPERIMENT: derive arity from the runtime's own declarations
codegen.el carried builtin_arity(): 344 lines, 300 entries, a hand-maintained
second copy of el_runtime.h.
PREDICTIONS AND RESULTS
P1 the table duplicates the header TRUE 243 shared names
P2 they have already drifted FALSE ZERO drift. The
duplicate had been
maintained correctly.
P3 codegen can emit call-arity relations TRUE
P4 the check becomes a query against the header TRUE
P5 codegen drops to roughly baseline TRUE 4903 -> 4512,
149 BELOW the 4661
it started at
P2 being false is the better result: the table was not WRONG, it was
INCOMPLETE. 110 functions the runtime declares had no entry, so calling them
with the wrong argument count produced no El-level diagnostic at all. Measured:
the old compiler reports 0 arity errors for __http_do_map_to_file(1); the query
reports "takes 5 arguments, called with 1".
Deriving from the header fixes coverage AND makes drift impossible by
construction. 503 signatures, versus 300 entries maintained by hand.
THREE DEFECTS IN MY OWN CHECKER, each found by running it rather than reading it
1. El names and C names differ -- `println` is `__println`. 60 of 500 decls
carry the prefix and codegen owns the mapping; the old table carried both
keys. One rule covers all 60.
2. Multi-line declarations parsed as zero params, so the checker reported
"takes 0" for a function taking 5. A diagnostic with the wrong number in it
is worse than none -- the same shape as the stale caller attribution in the
previous pass.
3. Fixing (2) by joining lines dropped 500 signatures to 334, because a
declaration preceded by a comment no longer started its record. Comments
are stripped first now.
98/98 native, 5/5 arity_query.sh, fixpoint ok.
```
## Record — `d9e301b`
```
land arity-from-header: the runtime declares its own surface
```
@@ -1,52 +0,0 @@
# one type erases the return
One `Ishikawa → scientific method → Six Sigma` loop. The record below is the
commit message as written at the time, before the outcome was known to anyone
reading this file.
## Record — `d2d89fc`
```
EXPERIMENT: temporal types as data — and the pass that GREW the compiler
This block is structurally unlike the previous four. It does not only
adjudicate, it DISPATCHES: Instant + Duration must become el_instant_add_dur,
LocalDate + Duration must become el_local_date_add_dur. The emitted C depends on
the type answer, so it cannot move to a post-hoc query. Selecting which call to
emit is an emitter's actual job.
PREDICTIONS AND RESULTS
P1 the block conflates dispatch with adjudication TRUE
P2 adjudication can move, dispatch cannot TRUE
P3 this pass shrinks codegen far less than the last TRUE, and worse:
4513 -> 4537, it GREW
by 24 lines
P4 the rules are affine algebra, closed by construction TRUE
P5 no type propagation -- name tracking plus a
hardcoded list of which builtins return which type TRUE, 19 names
P3 is the honest result and it is not spun: moving 19 names into a data file
cost more lines than it saved, because a generic loader is larger than the
enumeration it replaces. The win is not line count. It is that adding a 20th
temporal builtin is now a one-line edit to signatures.rel instead of a compiler
change, and that the data is inspectable.
WHY THE HEADER CANNOT SUPPLY THIS, unlike arity: el_runtime.h declares every
builtin as returning el_val_t, because El has ONE type. That single type is why
the whole seam is cheap and it is exactly why the C boundary cannot say that
now() returns an Instant while unix_seconds() returns an Int. The El-level type
is real and the boundary erases it.
INCOMPLETE, and stated rather than hidden: P2 said adjudication could move to a
query. It has NOT. Violations still emit TIME_TYPE_ERROR inline from the
emitter. Only the type DATA moved. Moving the adjudication needs the operand
types recorded as relations, which is a further pass.
98/98 native, 4/4 temporal_signatures.sh, fixpoint ok.
```
## Record — `e01e079`
```
land temporal signatures as data: the type table leaves, the dispatch stays
```
@@ -1,47 +0,0 @@
# judgment lives with knowledge
One `Ishikawa → scientific method → Six Sigma` loop. The record below is the
commit message as written at the time, before the outcome was known to anyone
reading this file.
## Record — `e8e25a0`
```
EXPERIMENT: temporal adjudication moves out; the placeholder stays
The previous pass moved the type DATA and left the judgment inline, which I
stated rather than hid. This finishes it.
PREDICTIONS AND RESULTS
P1 codegen can emit operand-type relations TRUE
"main calls temporal:instant_plus_instant"
P2 the affine rules are a small closed set as data TRUE 6 rules
P3 violations still caught at build time TRUE exit=1
P4 the reporter leaves codegen TRUE 4538 -> 4507
P5 the TIME_TYPE_ERROR placeholder must STAY TRUE
P5 is the boundary of this whole approach. The emitter has to emit SOMETHING
for an illegal expression -- it cannot emit nothing and it cannot decide what
the program meant. So the placeholder is irreducible in the same way the AST
traversal was: what moved is the judgment and the wording, not the fact that
something must be written.
The rules are affine algebra and the set is closed because there are only two
kinds of thing. An Instant is a POINT, a Duration is a DISPLACEMENT: add a
displacement to a point, subtract two points for a displacement, combine
displacements. Nothing else is meaningful, which is why the enumeration in
temporal.rel cannot grow the way an allowlist does.
A defect in my own checker, found by running it: the .rel file uses aligned
columns and my awk assumed a single space, so the message came out with the
rule key still prefixed. Same class as the multi-line header parse in the arity
pass -- formatting assumptions that only fail when you look at the output.
98/98 native, 6/6 temporal_query.sh, fixpoint ok.
```
## Record — `50425f3`
```
land temporal adjudication as a query: the emitter records, the rules are data
```
@@ -1,49 +0,0 @@
# thirty five return types
One `Ishikawa → scientific method → Six Sigma` loop. The record below is the
commit message as written at the time, before the outcome was known to anyone
reading this file.
## Record — `cbef1c1`
```
EXPERIMENT: Int return types as data — and the bug that fell out
PREDICTIONS AND RESULTS
P1 is_int_call's 35 hardcoded names move to data TRUE
P2 is_int_name stays -- it is annotation propagation TRUE
P3 the dispatch stays -- it is emission TRUE
P4 codegen shrinks ~40 lines TRUE 4507 -> 4469
P5 the design doc's characterisation is WRONG TRUE
P6 the moved data also fixes the bug it exposed TRUE
P5 CORRECTS THE RECORD. el-language-design.md and geometry-vs-code.md both cite
"== lowering to str_eq unless both operand names are in a hardcoded int-name
set -- a literal list of variable names treated as integers" as the paradigm
defect. It is not one. __int_names is populated from TYPE ANNOTATIONS
(param["type"] == "Int"), which is primitive but legitimate type propagation.
The actual defect was is_int_call: 35 hardcoded builtin return types, the same
shape as the temporal 19.
P6 IS A LIVE CORRECTNESS BUG, PRE-EXISTING, NOW FIXED
let a = str_len("hello") // no annotation
let b = str_len("hi")
let c = a + b // -> el_str_concat(a, b) on two integers
Verified identical on the pre-change compiler, so not a regression. It compiled
clean, ran, and printed NOTHING where it should print 7. No error at any layer.
The repair is three lines: an unannotated let takes its type from what the
initialiser returns. The return types were already required for dispatch and
were simply never consulted at the binding site. Moving them into data is what
made the gap visible -- reading the code for eight hours did not.
98/98 native + 2 new, 31/31 integration, fixpoint ok.
```
## Record — `505e5e7`
```
land int signatures, and repair a silent miscompilation they exposed
```
@@ -1,57 +0,0 @@
# keywords that reserve nothing
One `Ishikawa → scientific method → Six Sigma` loop. The record below is the
commit message as written at the time, before the outcome was known to anyone
reading this file.
## Record — `0143cc4`
```
ANSWER: is a grammar a convention, or a region?
Both, at different layers -- and it is the same split as serialization: the
convention is the BASIS, never the ACT.
lexeme -> token `fn` means function-start because someone said so CONVENTION
shape recognition given tokens, which construct is this REGION
source -> structure parsing is transduction onto that basis GEOMETRY
byte traversal something must read them in order IRREDUCIBLE
Three things push the ACT toward region rather than convention: ambiguity
(a * b needs context; a grammar resolves it with the lexer hack, a region by
neighbourhood), error recovery (nearest-region is free), and precedence, which
is ordering along an axis with a conventional parameter.
AND THE SHOULD GATE SAYS NO TO THE OBVIOUS MOVE
Every other table this session moved to data. This one stays code. The keyword
set is CLOSED by the language definition -- it does not leak the way an
allowlist does -- and the lexer runs before the program is understood, so a
program can never declare its own keywords. Externalising it costs file I/O on
every compile and buys nothing. Same verdict as is_digit in ASCII.
WHAT WAS ACTUALLY WRONG: five of 46 keywords were consumed by no parser or
codegen path. sealed, activate, seed, protocol, impl. Each stole an identifier
from users for nothing.
SECOND SILENT MISCOMPILATION OF THE DAY. Using one did not fail to parse:
let seed = 42
let impl = seed + 1
compiled CLEAN -- zero cc errors -- and printed 0 instead of 44. No diagnostic
at any layer. Fixed by removing the five.
A DEFECT IN MY OWN MEASUREMENT, caught before it did damage: my first pass
checked only parser.el and reported `test` as inert too. codegen consumes it at
4135 for --test mode, and the tree has 408 uses. Removing it would have broken
every test in the suite. The measurement was re-run across all four consumers.
100/100 native + 2 new, 31/31 integration, fixpoint ok.
```
## Record — `067dd40`
```
answer the parsing question: a grammar is a basis, and five keywords reserved nothing
```
@@ -1,53 +0,0 @@
# no namespacing at all
One `Ishikawa → scientific method → Six Sigma` loop. The record below is the
commit message as written at the time, before the outcome was known to anyone
reading this file.
## Record — `79f6cb7`
```
ANSWER: if the partition is a neighbourhood, does linking survive?
The question is premature, and measuring says why. El's partition is a
FILESYSTEM PATH, not a neighbourhood, and there is no namespacing at all.
MEASURED
import is textual inlining (resolve_imports), guarded against double
inclusion by a __elc_imp__:<path> state key
when a .elh header exists the header is inlined instead and the .el is marked
seen, so symbols resolve at C link time -- so linking IS real, delegated to C
two modules defining `helper` emit two C functions into one translation unit
So linking barely survives the PATH partition. Whether it survives a
neighbourhood partition cannot be asked yet.
A DIAGNOSTIC REGRESSION I CAUSED, found by asking this question. cc does catch
the collision, but reports:
error: redefinition of '__el_body_helper'
error: redefinition of '__env_helper'
error: redefinition of '__thunk_helper'
error: redefinition of 'helper'
The user's own function is FOURTH. The first three are generated symbols
introduced by the unconditional-wrapper pass earlier today -- before it, there
was one clear message. Repaired by catching the collision at El level instead:
duplicate definition: 'helper' is defined 2 times — El has no namespacing,
so imported modules share one global scope
LIMIT, stated rather than hidden: textual inlining destroys file provenance. By
the time codegen runs there is one source string, so the message can say WHICH
name collides but not which files. Naming a.el and b.el needs provenance
threaded through resolve_imports.
104/104 native, 4/4 definitions_query.sh, the compiler itself reports clean,
fixpoint ok.
```
## Record — `f23cb2b`
```
answer the module question: the partition is a path, and there is no namespacing
```
@@ -1,64 +0,0 @@
# tokens carry no position
One `Ishikawa → scientific method → Six Sigma` loop. The record below is the
commit message as written at the time, before the outcome was known to anyone
reading this file.
## Record — `6c975b1`
```
thread provenance through resolve_imports
The module question ended with a limit: textual inlining destroys file
provenance, so a duplicate-definition message could name the symbol but not the
files. Threading it exposed a bigger absence first.
TOKENS HAD NO POSITION AT ALL. A token was a flat (kind, value) pair, so NO
diagnostic in El could name a place -- every error named a symbol and never a
line. That is the prerequisite the module question was resting on.
THE CHAIN, end to end
lexer counts newlines; tok_append mints (kind, value, line)
parser stride 2 -> 3; tok_line added; FnDef carries its line
codegen records <fn> defines_at:<line>
resolve_imports publishes <file> spans <start> <end> for the combined source
checker maps a combined line back to file:line-within-that-file
duplicate definition: 'helper' is defined 2 times — El has no namespacing,
so imported modules share one global scope
/tmp/modtest/a.el:1
/tmp/modtest/b.el:1
PREDICTIONS AND RESULTS
P1 15 stride sites, encapsulated in tok_kind/tok_value TRUE, but see below
P2 adding a line field is mechanical TRUE
P3 the lexer must count newlines TRUE
P4 resolve_imports can record per-file line ranges TRUE
P5 the message can then name both files TRUE
P6 token memory grows TRUE, 25.0 -> 33.9 MB (+36%)
FOUR DEFECTS, EACH FOUND BY RUNNING AND NOT BY READING
1. interp_tokens_append_all walks the token list DIRECTLY with its own copy of
the stride. Gen1 built fine and gen2 emitted corrupt C, because the
compiler's own source uses string interpolation. My search missed it because
I grepped for the variable name `tokens`; it is called `dst`/`result`.
Searching by name instead of by shape -- third time today.
2. tok_count in test_compiler.el carried the stride too. I had scoped the search
to compiler sources and it had escaped into the tests.
3. Nested resolve_imports calls accumulated spans into shared state, so each
republished meaningless line ranges under the parent's name. Making the
buffer local fixed it; guarding the WRITE did not, which is what I tried
first.
4. The first working version reported b.el:3 -- the COMBINED line against a
filename that has no line 3. A file:line that does not match the file is
worse than no line at all.
105/105 native, 37/37 integration, fixpoint ok, compiler self-checks clean.
```
## Record — `cb7289f`
```
thread provenance end to end: a diagnostic can finally name a place
```
@@ -1,53 +0,0 @@
# annotations are never checked
**Status: verified on `experiment/annotation-checking`, not merged.**
## Ishikawa — why does El silently miscompile?
Three bugs found the same day shared one shape.
```
method type tracked by per-function name sets, fed from annotations
machine el_val_t erases everything at the C boundary
material no propagation through expressions
measurement nothing verifies an annotation against what it annotates
─────────────────────────────────────────────────────────────────────────
root cause El has type ANNOTATIONS but no type CHECKING. The annotation
feeds dispatch and is never itself verified.
```
## Predictions
```
P1 let x: Int = "hello" compiles clean expect TRUE
P2 let s: String = 42 compiles clean expect TRUE
P3 the annotation drives dispatch, unverified expect TRUE
P4 same root cause as all three bugs found today expect TRUE
P5 checking literal-vs-annotation catches both expect TRUE
P6 zero false positives across the compiler's source expect TRUE
```
## Results — 6/6, and worse than a wrong answer
```
let x: Int = "hello"; x + 1 → 4343631981 a string POINTER used as an integer
let s: String = 42; println(s) → nothing address 42 dereferenced as a string
```
The first **leaks a raw memory address into program output**. The second is an
**arbitrary-read primitive** if that integer is ever attacker-influenced.
Verified: 6/6, zero false positives across the compiler's own source, fixpoint
ok, 105/105 native.
## Six Sigma
The emitter only **records** the mismatch; `tools/check/annotations.sh` decides —
consistent with every other check. Literals are checked because they are
unambiguous.
**Incomplete, stated not hidden:** only literals. `let x: Int = some_string_fn()`
still passes, because `signatures.rel` carries Int/Instant/Duration and no
String entries. That is a data gap, not a capability limit — every El function
declares its return type in source and codegen already holds `ret_type` on every
`FnDef`.
@@ -1,88 +0,0 @@
# async — half expressible, and the cycle that was dogma
**Status: replicated and corroborated. Three runs — the first was invalid.**
> **Chain of custody note, 2026-08-17.** The original measurements were produced
> by a C stub written in `/tmp`, and that artifact was destroyed when the session
> worktrees were removed. For a period this file asserted results with nothing
> behind them — a claim inside an evidence record, which is the defect that turns
> a chain into a pile. It was **rerun**, not reconstructed: reconstructing the
> missing file would have been a fabrication with a fresh timestamp.
>
> The fixture now lives at `lang/tests/integration/fixtures/future.c` and the
> harness at `lang/tests/integration/async_future.sh`, so a third party can
> reproduce this without taking my word for it. **6/6.**
>
> The replication is labelled as such: the outcomes were already known when the
> harness was written, so its expectations are not predictions committed in
> advance. Its value is reproducibility, not foresight.
## The first attempt was DOGMA, not science
I had just finished arguing that `@async` was expressible, then ran something to
confirm it. **No prediction was committed.** The test was rigged in a way that
should have been visible while writing it:
```c
pthread_create(&t,NULL,runner,NULL); pthread_join(t,NULL);
```
`join` immediately after `create` — the caller blocks until the body finishes.
That is a thread round-trip, not deferral. And the test printed the word
`DEFERRED` itself: I wrote the conclusion into the output and read it back.
```
Ishikawa on the rigged test
method ran after concluding, not to decide
machine nothing forces a prediction before execution
material the assertion was written into the output string
measurement no falsification criterion existed, so nothing could fail
root cause the test was authored by the party holding the conclusion,
with no commitment made before it ran
```
Discarded and re-run properly.
## Second run — predictions committed first
```
P1 the caller proceeds while the body runs expect TRUE
P2 interleaving is observable in timestamps expect TRUE
P3 the result cannot be retrieved — one 64-bit slot, no
future type, so the wrap either blocks or returns
something that is not the result expect TRUE
P4 therefore HALF expressible: fire-and-forget yes, await no expect TRUE
```
## Results — 4/4
```
[ 18 us] wrap RETURNS to caller
[ 29 us] body START
caller continues, got 0
[ 50176 us] body END (computed 42)
caller done
```
The caller got **0, not 42**. Both of my earlier claims were wrong in opposite
directions: "not expressible" was too strong — fire-and-forget works today,
bound after the build, no compiler change. "Expressible" was too strong the
other way.
## Follow-on cycle — a future is one more tagged object
```
P1 el_val_t already carries tagged heap objects TRUE 5 magic tags exist
P2 a future is one more TRUE
P3 the caller awaits and gets 42 TRUE
P4 ZERO compiler changes TRUE runtime C + one binding
P5 the unbound path still works FALSE SIGSEGV
```
**P4 is the result.** `@async` — called unexpressible for hours — needs no
compiler change. A future is one more magic-tagged heap object; `defer` returns
the handle, `el_await` blocks.
**P5 is the failure that mattered.** Sixty seconds after diagnosing
`let s: String = 42` as an arbitrary read, I wrote the identical defect into
`el_await`: reading `->magic` off an unvalidated slot. That opened cycle 19.
@@ -1,61 +0,0 @@
# a convention is not a gate
One `Ishikawa → scientific method → Six Sigma` loop. The record below is the
commit message as written at the time, before the outcome was known to anyone
reading this file.
## Record — `9a6c161`
```
a slot must be validated before it is dereferenced
ISHIKAWA: el_val_t carries integers AND tagged heap pointers, so "is this a
pointer" is undecidable without checking first. That check was a CONVENTION
every author had to know rather than a GATE they had to pass through, and
looks_like_heap_obj was static -- so every sibling translation unit re-derived
it.
MEASURED, across the five existing tags
geom_of looks_like_heap_obj full guard correct
mfld_of looks_like_heap_obj full guard correct
el_bin_lookup (uintptr_t)p < 4096 floor only reads 8 bytes BACKWARD
el_input_len s ? ... : 0 NULL only strlen's an integer
sha256_hex(50000) -> exit 139, SIGSEGV, compiled clean
PREDICTIONS AND RESULTS
P1 looks_like_heap_obj is static, not exported TRUE
P2 each tagged type re-derives the check TRUE
P3 at least one is missing guard components TRUE (two are)
P6 sha256_hex(<int>) reads out of bounds TRUE
P8 routing el_bin_lookup through the gate fixes it FALSE
P9 the legitimate hash is unchanged TRUE
P11 fixpoint and suites hold TRUE
P8 IS THE USEFUL FAILURE. Guarding the tagged lookup changed nothing --
looks_like_heap_obj(49992) correctly returns 0, el_bin_lookup bails, and then
el_input_len falls through to strlen() on address 50000. The FALLBACK was the
hazard, not the tagged path. A NULL check does not establish that a slot is a
pointer. I would have shipped the wrong fix and called it verified.
A MEASUREMENT DEFECT, fourth today: my first run of the crash reported exit=0,
because $? read head's exit through a pipe rather than the program's. I nearly
recorded a segfault as a clean run. Same shape as grepping only parser.el and
searching by variable name instead of by operation.
AND I PROVED THE HAZARD FROM THE INSIDE. Sixty seconds after diagnosing
`let s: String = 42` as an arbitrary-read primitive, I wrote the identical
defect into el_await -- dereferencing ->magic off an unvalidated slot -- and
only then found the runtime had already made it twice.
el_tagged() is now exported in el_runtime.h. Anything that dereferences a slot
without passing through it is the defect.
105/105 native, 42/42 integration across eight harnesses, fixpoint ok.
```
## Record — `3049a70`
```
make the guard a gate: sha256_hex(50000) no longer segfaults
```
-12
View File
@@ -1,12 +0,0 @@
# Architecture questions closed
All five were open in `geometry-vs-code.md`. Each was closed by measurement, not
by argument.
| Question | Answer |
|---|---|
| **Concurrency** — hardware threads are code, but is *ordering* geometric? | **Ordering is geometric.** Causality is a partial order (Lamport 1978); a total order is an arbitrary extension that "cannot be depended on to imply a causal relationship." Languages force a total order at authoring time, so every lock, barrier and fence is apparatus for recovering the partial order that was destroyed. CALM: a program has a coordination-free implementation **iff monotone**. What breaks monotonicity is destructive update. **Coordination is the price of forgetting.** |
| **Error handling** — does `grounded: false` cover *failed*? | **No.** Standing is a *signed* component: `>0` supported, `=0` unknown, `<0` contradicted. Not-known and known-false are opposite directions on one axis; a boolean cannot tell them apart. `inhibitory` as an int32 flag is that sign wearing a boolean. |
| **Parsing** — is a grammar a convention, or a region? | **A grammar is a basis; parsing is transduction onto it.** Lexeme→token is convention, shape recognition is a region, byte traversal is irreducible. **But the SHOULD gate refused the obvious move:** the keyword table stays code, because the set is closed by the language definition and the lexer runs before the program is understood. Same verdict as `is_digit` in ASCII. |
| **Numeric literals** — is `3` a position or a convention? | **The numeral is convention; the number is a position — and a bare `3` is a magnitude with no axis.** It is not a position until something gives it a direction, which is why `3.days` needs a calendar. Demonstrated: `t + 3` refused, `t + 1.hour` accepted. |
| **The module system** — if the partition is a neighbourhood, does linking survive? | **Premature.** The partition is a filesystem path and there is no namespacing at all. `import` is textual inlining; with a `.elh` header, symbols resolve at C link time. Two modules defining `helper` emit two C functions into one translation unit. Linking barely survives the *path* partition. |
-74
View File
@@ -1,74 +0,0 @@
# Live defects found
Every one compiled clean, ran, and produced a wrong result or a crash with **no
diagnostic at any layer**. All four were present before this session; none was
introduced by it.
## Silent miscompilations
### 1. An unannotated `let` loses its type
```el
let a = str_len("hello") // no annotation
let b = str_len("hi")
let c = a + b // el_str_concat(a, b) on two integers
```
Compiled clean. Printed **nothing** where it should print 7. Fixed: an
unannotated `let` takes its type from what its initialiser returns. The return
types were already required for dispatch and were simply never consulted at the
binding site.
### 2. Reserved keywords that reserved nothing
```el
let seed = 42
let impl = seed + 1
```
`sealed`, `activate`, `seed`, `protocol`, `impl` were keywords in the lexer and
consumed by no parser or codegen path. Using one did not fail to parse — it
compiled clean, with zero `cc` errors, and printed **0 instead of 44**. Fixed by
removing all five.
### 3. `Instant + Int` was never refused
```el
let t: Instant = now()
let u: Instant = t + 3 // (t + 3), reported clean
```
`Duration + Int` was refused — *"an Int carries no unit"* — while adding a
dimensionless number to a **point** silently moved the instant by an
unspecified amount. Three of *what*? Whatever the representation happens to be.
The rule was simply never written.
## Security-relevant
### 4. Annotations are never verified
```el
let x: Int = "hello"; x + 1 → 4343631981 a string POINTER used as an integer
let s: String = 42; println(s) → nothing address 42 dereferenced
```
The first **leaks a raw memory address into program output**. The second is an
**arbitrary-read primitive** if the integer is ever attacker-influenced.
### 5. `sha256_hex(<integer>)` segfaults
```el
let h: String = sha256_hex(50000) exit 139, SIGSEGV
```
Compiled clean. `el_bin_lookup` checked only a 4096 floor — no alignment, no
small-int, no negative — and reads **eight bytes backward** from the pointer.
And the actual crash was one level further on: `el_input_len` fell through to
`strlen()` on address 50000, because a NULL check does not establish that a slot
is a pointer.
Fixed, and the guard is now a **gate**: `el_tagged()` is exported in
`el_runtime.h`. `geom_of` and `mfld_of` were always correct because their authors
knew to call `looks_like_heap_obj`; `el_bin_lookup` and `el_input_len` were wrong
because theirs did not, and the function was `static`, so every sibling
translation unit re-derived it.
@@ -1,62 +0,0 @@
# Defects in my own measurement
Recorded because the pattern is the point: **five of these, and every one is the same shape —
reading a proxy instead of the thing.** A file instead of the operation, a
variable name instead of the shape, a scope instead of the whole, a pipe's exit
instead of the program's, a line count instead of the object identity. Each was caught
by running something, never by reading.
### 1. Scoped the search to one file
Reported `test` as an inert keyword by checking only `parser.el`. **codegen**
consumes it at 4135 for `--test` mode, and the tree has 408 uses. Removing it
would have broken every test in the suite — including the ones used to verify
the removal.
### 2. Searched by variable name, not by operation
Grepped for `native_list_append(tokens` to find direct token appends.
`interp_tokens_append_all` calls its parameters `dst`/`result`, carries its own
copy of the stride, and corrupted generation 2 — while generation 1 built fine,
because the compiler's own source uses string interpolation.
### 3. Scoped to compiler sources; the stride had escaped into tests
`tok_count` in `test_compiler.el` computed `len/2` independently. 21 tests failed
after the token layout changed.
### 4. Read the wrong exit code
```bash
timeout 10 /tmp/leakrun 2>&1 | head -2; echo "exit=$?" # reports head's exit
```
Reported `exit=0` for a program that was returning **139 (SIGSEGV)**. I nearly
recorded a segfault as a clean run.
### 5. Read a count that was not counting
Comparing the three promoted branches:
```bash
for pair in "dev stage" ...; do set -- $pair
n=$(git diff --stat origin/$1 origin/$2 | wc -l) # git errored to STDERR
... # wc counted empty STDOUT
```
`git diff` failed on a malformed revision, wrote its error to stderr, and `wc -l`
counted zero lines of stdout. Three confident `IDENTICAL` results, all
meaningless. **Had the trees actually differed, I would have reported the
promotion clean.**
Redone correctly, the three trees share one hash — `2acd9374` — which is the
check that should have been run first: not "how many files differ" but "is the
tree object the same object".
### And one that was not a measurement defect but a method defect
One cycle was run **without committing predictions first** — see
`cycles/18-async-half-expressible.md`. The test joined the thread immediately
after creating it and printed the word `DEFERRED` itself. A test authored by the
party holding the conclusion, with nothing committed beforehand, cannot fail.
It had to be discarded and re-run.
@@ -22,9 +22,6 @@
* EL_STR(s) cast string literal to el_val_t * EL_STR(s) cast string literal to el_val_t
* EL_CSTR(v) cast el_val_t back to const char* * EL_CSTR(v) cast el_val_t back to const char*
* EL_INT(v) identity el_val_t is already int64_t * EL_INT(v) identity el_val_t is already int64_t
* EL_NULL null / zero value
* EL_FALSE boolean false (0)
* EL_TRUE boolean true (1)
* *
* Link requirements: * Link requirements:
* -lcurl required for the HTTP client (http_get, http_post, llm_*). * -lcurl required for the HTTP client (http_get, http_post, llm_*).
@@ -56,8 +53,6 @@ typedef int64_t el_val_t;
#define EL_CSTR(v) ((const char*)(uintptr_t)(v)) #define EL_CSTR(v) ((const char*)(uintptr_t)(v))
#define EL_INT(v) (v) #define EL_INT(v) (v)
#define EL_NULL ((el_val_t)0) #define EL_NULL ((el_val_t)0)
#define EL_FALSE ((el_val_t)0)
#define EL_TRUE ((el_val_t)1)
/* Float values share the el_val_t (int64) slot via a bit-cast. /* Float values share the el_val_t (int64) slot via a bit-cast.
* The codegen emits Float literals as `el_from_float(<dbl>)` so the * The codegen emits Float literals as `el_from_float(<dbl>)` so the
@@ -81,9 +76,11 @@ extern "C" {
/* ── I/O ──────────────────────────────────────────────────────────────────── */ /* ── I/O ──────────────────────────────────────────────────────────────────── */
el_val_t println(el_val_t s); void println(el_val_t s);
el_val_t print(el_val_t s); void print(el_val_t s);
el_val_t readline(void); el_val_t readline(void);
el_val_t stdout_to_file(el_val_t path); /* redirect println to a file */
el_val_t stdout_restore(void); /* restore stdout after capture */
/* ── String builtins ─────────────────────────────────────────────────────── */ /* ── String builtins ─────────────────────────────────────────────────────── */
@@ -95,7 +92,6 @@ el_val_t str_len(el_val_t s);
el_val_t str_concat(el_val_t a, el_val_t b); el_val_t str_concat(el_val_t a, el_val_t b);
el_val_t int_to_str(el_val_t n); el_val_t int_to_str(el_val_t n);
el_val_t str_to_int(el_val_t s); el_val_t str_to_int(el_val_t s);
el_val_t native_str_to_int(el_val_t s);
el_val_t str_slice(el_val_t s, el_val_t start, el_val_t end); el_val_t str_slice(el_val_t s, el_val_t start, el_val_t end);
el_val_t str_contains(el_val_t s, el_val_t sub); el_val_t str_contains(el_val_t s, el_val_t sub);
el_val_t str_replace(el_val_t s, el_val_t from, el_val_t to); el_val_t str_replace(el_val_t s, el_val_t from, el_val_t to);
@@ -123,10 +119,6 @@ el_val_t el_min(el_val_t a, el_val_t b);
void el_retain(el_val_t v); void el_retain(el_val_t v);
void el_release(el_val_t v); void el_release(el_val_t v);
/* ── Scoped arena (CLI use) ───────────────────────────────────────────────── */
el_val_t el_arena_push(void);
el_val_t el_arena_pop(el_val_t mark);
/* ── List ────────────────────────────────────────────────────────────────── */ /* ── List ────────────────────────────────────────────────────────────────── */
el_val_t el_list_new(el_val_t count, ...); el_val_t el_list_new(el_val_t count, ...);
@@ -150,11 +142,10 @@ el_val_t http_post(el_val_t url, el_val_t body);
el_val_t http_post_json(el_val_t url, el_val_t json_body); el_val_t http_post_json(el_val_t url, el_val_t json_body);
el_val_t http_get_with_headers(el_val_t url, el_val_t headers_map); el_val_t http_get_with_headers(el_val_t url, el_val_t headers_map);
el_val_t http_post_with_headers(el_val_t url, el_val_t body, el_val_t headers_map); el_val_t http_post_with_headers(el_val_t url, el_val_t body, el_val_t headers_map);
el_val_t http_post_json_with_headers(el_val_t url, el_val_t headers_map, el_val_t json_body);
el_val_t http_post_form_auth(el_val_t url, el_val_t form_body, el_val_t auth_header); el_val_t http_post_form_auth(el_val_t url, el_val_t form_body, el_val_t auth_header);
el_val_t http_delete(el_val_t url); el_val_t http_delete(el_val_t url);
el_val_t http_serve(el_val_t port, el_val_t handler); void http_serve(el_val_t port, el_val_t handler);
el_val_t http_set_handler(el_val_t name); void http_set_handler(el_val_t name);
/* HTTP server v2 ───────────────────────────────────────────────────────────── /* HTTP server v2 ─────────────────────────────────────────────────────────────
* Same dispatch model as http_serve, but the handler signature is widened: * Same dispatch model as http_serve, but the handler signature is widened:
@@ -175,8 +166,8 @@ el_val_t http_set_handler(el_val_t name);
* The 3-arg http_serve(port, handler) remains supported unchanged for * The 3-arg http_serve(port, handler) remains supported unchanged for
* existing handlers (e.g. products/web/server.el): it dispatches with * existing handlers (e.g. products/web/server.el): it dispatches with
* (method, path, body), hardcodes 200 OK, and auto-detects content type. */ * (method, path, body), hardcodes 200 OK, and auto-detects content type. */
el_val_t http_serve_v2(el_val_t port, el_val_t handler); void http_serve_v2(el_val_t port, el_val_t handler);
el_val_t http_set_handler_v2(el_val_t name); void http_set_handler_v2(el_val_t name);
/* Build an HTTP response envelope. `headers_json` should be a JSON object /* Build an HTTP response envelope. `headers_json` should be a JSON object
* literal like `{"WWW-Authenticate":"Basic"}` (or "" / "{}" for none). The * literal like `{"WWW-Authenticate":"Basic"}` (or "" / "{}" for none). The
@@ -187,11 +178,6 @@ el_val_t http_set_handler_v2(el_val_t name);
* auto-content-type contract for legacy handlers that return plain bodies. */ * auto-content-type contract for legacy handlers that return plain bodies. */
el_val_t http_response(el_val_t status, el_val_t headers_json, el_val_t body); el_val_t http_response(el_val_t status, el_val_t headers_json, el_val_t body);
/* SSE connection fd — set by http_worker_v2 before calling the El handler,
* cleared afterwards. Defined in el_seed.c; called from el_runtime.c.
* The getter is exposed as __http_conn_fd() to El programs. */
void el_seed_set_http_conn_fd(int fd);
/* HTTP timeout — every libcurl request honors EL_HTTP_TIMEOUT_MS (default /* HTTP timeout — every libcurl request honors EL_HTTP_TIMEOUT_MS (default
* 60000ms). Read lazily on first use, so setting the env var any time before * 60000ms). Read lazily on first use, so setting the env var any time before
* the first http_* call is sufficient. */ * the first http_* call is sufficient. */
@@ -227,15 +213,19 @@ el_val_t url_decode(el_val_t s); /* '+' → space, %XX → byte */
* {"p":[],"a":["href","title"],"strong":[],...} * {"p":[],"a":["href","title"],"strong":[],...}
* where each value is the array of attribute names allowed for that tag. */ * where each value is the array of attribute names allowed for that tag. */
el_val_t el_html_sanitize(el_val_t input_html, el_val_t allowlist_json); el_val_t el_html_sanitize(el_val_t input_html, el_val_t allowlist_json);
el_val_t html_raw(el_val_t s);
/* ── HTML template helpers ───────────────────────────────────────────────────
* Used by compiled El HTML template expressions.
* html_escape(s) escape & < > " ' for safe inline interpolation.
* html_raw(s) identity; explicit opt-out from escaping (`raw()` form). */
el_val_t html_escape(el_val_t s); el_val_t html_escape(el_val_t s);
el_val_t html_raw(el_val_t s);
/* ── Filesystem ──────────────────────────────────────────────────────────── */ /* ── Filesystem ──────────────────────────────────────────────────────────── */
el_val_t fs_read(el_val_t path); el_val_t fs_read(el_val_t path);
el_val_t fs_write(el_val_t path, el_val_t content); el_val_t fs_write(el_val_t path, el_val_t content);
el_val_t fs_list(el_val_t path); el_val_t fs_list(el_val_t path);
el_val_t fs_list_json(el_val_t path);
el_val_t fs_exists(el_val_t path); el_val_t fs_exists(el_val_t path);
el_val_t fs_mkdir(el_val_t path); /* mkdir -p, mode 0755 */ el_val_t fs_mkdir(el_val_t path); /* mkdir -p, mode 0755 */
@@ -265,9 +255,6 @@ el_val_t json_set(el_val_t json_str, el_val_t key, el_val_t value);
el_val_t json_array_len(el_val_t json_str); el_val_t json_array_len(el_val_t json_str);
el_val_t json_array_get(el_val_t json_str, el_val_t index); el_val_t json_array_get(el_val_t json_str, el_val_t index);
el_val_t json_array_get_string(el_val_t json_str, el_val_t index); el_val_t json_array_get_string(el_val_t json_str, el_val_t index);
el_val_t json_escape_string(el_val_t sv);
el_val_t json_build_object(el_val_t kvs);
el_val_t json_build_array(el_val_t items);
/* ── Time ────────────────────────────────────────────────────────────────── */ /* ── Time ────────────────────────────────────────────────────────────────── */
@@ -280,7 +267,6 @@ el_val_t time_to_parts(el_val_t ts);
el_val_t time_from_parts(el_val_t secs, el_val_t ns, el_val_t tz); el_val_t time_from_parts(el_val_t secs, el_val_t ns, el_val_t tz);
el_val_t time_add(el_val_t ts, el_val_t n, el_val_t unit); el_val_t time_add(el_val_t ts, el_val_t n, el_val_t unit);
el_val_t time_diff(el_val_t ts1, el_val_t ts2, el_val_t unit); el_val_t time_diff(el_val_t ts1, el_val_t ts2, el_val_t unit);
el_val_t now_ns(void);
/* ── Instant + Duration: first-class temporal types ────────────────────────── /* ── Instant + Duration: first-class temporal types ──────────────────────────
* Both types share the el_val_t (int64) slot. Instants are nanoseconds * Both types share the el_val_t (int64) slot. Instants are nanoseconds
@@ -437,8 +423,6 @@ el_val_t state_set(el_val_t key, el_val_t value);
el_val_t state_get(el_val_t key); el_val_t state_get(el_val_t key);
el_val_t state_del(el_val_t key); el_val_t state_del(el_val_t key);
el_val_t state_keys(void); el_val_t state_keys(void);
el_val_t state_has(el_val_t key);
el_val_t state_get_or(el_val_t key, el_val_t default_val);
/* ── Float formatting ────────────────────────────────────────────────────── */ /* ── Float formatting ────────────────────────────────────────────────────── */
@@ -530,15 +514,9 @@ el_val_t parse_int(el_val_t s, el_val_t default_val);
/* ── Process ─────────────────────────────────────────────────────────────── */ /* ── Process ─────────────────────────────────────────────────────────────── */
el_val_t exit_program(el_val_t code); void exit_program(el_val_t code);
el_val_t getpid_now(void); el_val_t getpid_now(void);
/* Self-terminating memory guard. Reads ELC_MAX_MEM_MB (default 512) and
* exits with code 1 if resident memory exceeds the limit. Call periodically
* during long compilation loops (e.g. after each function is compiled).
* Returns 0 when memory is within bounds. */
el_val_t el_mem_check(void);
/* ── CGI identity ───────────────────────────────────────────────────────────── /* ── CGI identity ─────────────────────────────────────────────────────────────
* Called at the start of main() in CGI programs (those with a `cgi {}` block). * Called at the start of main() in CGI programs (those with a `cgi {}` block).
* Records the program's DHARMA identity before any other code executes. */ * Records the program's DHARMA identity before any other code executes. */
@@ -776,108 +754,12 @@ el_val_t exec_capture(el_val_t cmd); /* run shell command, capture stdout */
el_val_t exec(el_val_t cmd); /* exec(cmd) → stdout String (30s timeout) */ el_val_t exec(el_val_t cmd); /* exec(cmd) → stdout String (30s timeout) */
el_val_t exec_bg(el_val_t cmd); /* exec_bg(cmd) → PID String (non-blocking) */ el_val_t exec_bg(el_val_t cmd); /* exec_bg(cmd) → PID String (non-blocking) */
/* ── Stdout redirection (used by compiler JS pipeline) ───────────────────── */
el_val_t stdout_to_file(el_val_t path); /* redirect process stdout to a file */
el_val_t stdout_restore(void); /* restore process stdout to terminal */
el_val_t emit_log(el_val_t level, el_val_t msg, el_val_t fields_json); el_val_t emit_log(el_val_t level, el_val_t msg, el_val_t fields_json);
el_val_t emit_metric(el_val_t name, el_val_t value, el_val_t tags_json); el_val_t emit_metric(el_val_t name, el_val_t value, el_val_t tags_json);
el_val_t trace_span_start(el_val_t name); el_val_t trace_span_start(el_val_t name);
el_val_t trace_span_end(el_val_t span_handle); el_val_t trace_span_end(el_val_t span_handle);
el_val_t emit_event(el_val_t name, el_val_t duration_ms); el_val_t emit_event(el_val_t name, el_val_t duration_ms);
el_val_t __thread_create(el_val_t fn_name_v, el_val_t arg_v);
el_val_t __thread_join(el_val_t tid_v);
/* ── __ prefixed aliases (self-hosting compiler ABI) ─────────────────────────
* The El self-hosting compiler emits calls to __-prefixed names. These are
* forwarding wrappers around the existing el_runtime functions above. */
/* I/O */
el_val_t __println(el_val_t s);
el_val_t __print(el_val_t s);
el_val_t __readline(void);
/* String */
el_val_t __int_to_str(el_val_t n);
el_val_t __str_to_int(el_val_t s);
el_val_t __float_to_str(el_val_t f);
el_val_t __str_to_float(el_val_t s);
el_val_t __str_len(el_val_t s);
el_val_t __str_char_at(el_val_t s, el_val_t i);
el_val_t __str_cmp(el_val_t a, el_val_t b);
el_val_t __str_ncmp(el_val_t a, el_val_t b, el_val_t n);
el_val_t __str_concat_raw(el_val_t a, el_val_t b);
el_val_t __str_slice_raw(el_val_t s, el_val_t start, el_val_t end);
el_val_t __str_alloc(el_val_t n);
el_val_t __str_set_char(el_val_t s, el_val_t i, el_val_t c);
/* URL encoding */
el_val_t __url_encode(el_val_t s);
el_val_t __url_decode(el_val_t s);
/* Environment */
el_val_t __env_get(el_val_t key);
/* Subprocess */
el_val_t __exec(el_val_t cmd);
el_val_t __exec_bg(el_val_t cmd);
/* Process */
el_val_t __exit_program(el_val_t code);
/* Filesystem */
el_val_t __fs_exists(el_val_t path);
el_val_t __fs_mkdir(el_val_t path);
el_val_t __fs_read(el_val_t path);
el_val_t __fs_write(el_val_t path, el_val_t content);
el_val_t __fs_write_bytes(el_val_t path, el_val_t bytes, el_val_t n);
el_val_t __fs_list_raw(el_val_t path);
/* HTTP server */
el_val_t __http_response(el_val_t status, el_val_t headers_json, el_val_t body);
el_val_t __http_serve(el_val_t port, el_val_t handler);
el_val_t __http_serve_v2(el_val_t port, el_val_t handler);
/* HTTP conn fd / SSE (weak; overridden by el_seed.c when linked together) */
el_val_t __http_conn_fd(void);
el_val_t __http_sse_open(el_val_t conn_id);
el_val_t __http_sse_send(el_val_t conn_id, el_val_t data);
el_val_t __http_sse_close(el_val_t conn_id);
/* HTTP client (requires HAVE_CURL; stubs provided for no-curl builds) */
el_val_t __http_do(el_val_t method, el_val_t url, el_val_t body,
el_val_t headers_map, el_val_t timeout_ms);
el_val_t __http_do_map(el_val_t method, el_val_t url, el_val_t body,
el_val_t headers_json, el_val_t timeout_ms);
el_val_t __http_do_map_to_file(el_val_t method, el_val_t url, el_val_t body,
el_val_t headers_json, el_val_t output_path);
/* JSON */
el_val_t __json_array_get(el_val_t json, el_val_t index);
el_val_t __json_array_get_string(el_val_t json, el_val_t index);
el_val_t __json_array_len(el_val_t json);
el_val_t __json_get(el_val_t json, el_val_t key);
el_val_t __json_get_raw(el_val_t json, el_val_t key);
el_val_t __json_set(el_val_t json, el_val_t key, el_val_t value);
el_val_t __json_parse_map(el_val_t json_str);
el_val_t __json_stringify_val(el_val_t val);
/* Hashing */
el_val_t __sha256_hex(el_val_t s);
/* State K/V */
el_val_t __state_del(el_val_t key);
el_val_t __state_get(el_val_t key);
el_val_t __state_keys(void);
el_val_t __state_set(el_val_t key, el_val_t val);
/* UUID */
el_val_t __uuid_v4(void);
/* Args */
el_val_t __args_json(void);
#ifdef __cplusplus #ifdef __cplusplus
} }
#endif #endif
@@ -128,6 +128,22 @@ function str_pad_right(s, width, pad) {
return String(s).padEnd(width, String(pad)); return String(s).padEnd(width, String(pad));
} }
// ── HTML template helpers ────────────────────────────────────────────────────
// Used by compiled El HTML template expressions.
// html_escape(s) — escape & < > " ' for safe inline interpolation.
// html_raw(s) — identity; explicit opt-out from escaping (raw() form).
function html_escape(s) {
return String(s)
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&#39;');
}
function html_raw(s) { return s; }
// ── Math ──────────────────────────────────────────────────────────────────── // ── Math ────────────────────────────────────────────────────────────────────
function el_abs(n) { return Math.abs(n); } function el_abs(n) { return Math.abs(n); }
@@ -1017,6 +1033,8 @@ export {
fs_read, fs_write, fs_list, fs_read, fs_write, fs_list,
json_parse, json_stringify, json_get, json_get_string, json_get_int, json_parse, json_stringify, json_get, json_get_string, json_get_int,
time_now, time_now_utc, sleep_ms, time_now, time_now_utc, sleep_ms,
// HTML template helpers
html_escape, html_raw,
bool_to_str, exit_program, args, env, bool_to_str, exit_program, args, env,
state_set, state_get, state_del, state_keys, state_set, state_get, state_del, state_keys,
el_cgi_init, el_cgi_init,
@@ -22,9 +22,6 @@
* EL_STR(s) cast string literal to el_val_t * EL_STR(s) cast string literal to el_val_t
* EL_CSTR(v) cast el_val_t back to const char* * EL_CSTR(v) cast el_val_t back to const char*
* EL_INT(v) identity el_val_t is already int64_t * EL_INT(v) identity el_val_t is already int64_t
* EL_NULL null / zero value
* EL_FALSE boolean false (0)
* EL_TRUE boolean true (1)
* *
* Link requirements: * Link requirements:
* -lcurl required for the HTTP client (http_get, http_post, llm_*). * -lcurl required for the HTTP client (http_get, http_post, llm_*).
@@ -56,8 +53,6 @@ typedef int64_t el_val_t;
#define EL_CSTR(v) ((const char*)(uintptr_t)(v)) #define EL_CSTR(v) ((const char*)(uintptr_t)(v))
#define EL_INT(v) (v) #define EL_INT(v) (v)
#define EL_NULL ((el_val_t)0) #define EL_NULL ((el_val_t)0)
#define EL_FALSE ((el_val_t)0)
#define EL_TRUE ((el_val_t)1)
/* Float values share the el_val_t (int64) slot via a bit-cast. /* Float values share the el_val_t (int64) slot via a bit-cast.
* The codegen emits Float literals as `el_from_float(<dbl>)` so the * The codegen emits Float literals as `el_from_float(<dbl>)` so the
@@ -81,8 +76,8 @@ extern "C" {
/* ── I/O ──────────────────────────────────────────────────────────────────── */ /* ── I/O ──────────────────────────────────────────────────────────────────── */
el_val_t println(el_val_t s); void println(el_val_t s);
el_val_t print(el_val_t s); void print(el_val_t s);
el_val_t readline(void); el_val_t readline(void);
/* ── String builtins ─────────────────────────────────────────────────────── */ /* ── String builtins ─────────────────────────────────────────────────────── */
@@ -95,7 +90,6 @@ el_val_t str_len(el_val_t s);
el_val_t str_concat(el_val_t a, el_val_t b); el_val_t str_concat(el_val_t a, el_val_t b);
el_val_t int_to_str(el_val_t n); el_val_t int_to_str(el_val_t n);
el_val_t str_to_int(el_val_t s); el_val_t str_to_int(el_val_t s);
el_val_t native_str_to_int(el_val_t s);
el_val_t str_slice(el_val_t s, el_val_t start, el_val_t end); el_val_t str_slice(el_val_t s, el_val_t start, el_val_t end);
el_val_t str_contains(el_val_t s, el_val_t sub); el_val_t str_contains(el_val_t s, el_val_t sub);
el_val_t str_replace(el_val_t s, el_val_t from, el_val_t to); el_val_t str_replace(el_val_t s, el_val_t from, el_val_t to);
@@ -123,10 +117,6 @@ el_val_t el_min(el_val_t a, el_val_t b);
void el_retain(el_val_t v); void el_retain(el_val_t v);
void el_release(el_val_t v); void el_release(el_val_t v);
/* ── Scoped arena (CLI use) ───────────────────────────────────────────────── */
el_val_t el_arena_push(void);
el_val_t el_arena_pop(el_val_t mark);
/* ── List ────────────────────────────────────────────────────────────────── */ /* ── List ────────────────────────────────────────────────────────────────── */
el_val_t el_list_new(el_val_t count, ...); el_val_t el_list_new(el_val_t count, ...);
@@ -150,11 +140,10 @@ el_val_t http_post(el_val_t url, el_val_t body);
el_val_t http_post_json(el_val_t url, el_val_t json_body); el_val_t http_post_json(el_val_t url, el_val_t json_body);
el_val_t http_get_with_headers(el_val_t url, el_val_t headers_map); el_val_t http_get_with_headers(el_val_t url, el_val_t headers_map);
el_val_t http_post_with_headers(el_val_t url, el_val_t body, el_val_t headers_map); el_val_t http_post_with_headers(el_val_t url, el_val_t body, el_val_t headers_map);
el_val_t http_post_json_with_headers(el_val_t url, el_val_t headers_map, el_val_t json_body);
el_val_t http_post_form_auth(el_val_t url, el_val_t form_body, el_val_t auth_header); el_val_t http_post_form_auth(el_val_t url, el_val_t form_body, el_val_t auth_header);
el_val_t http_delete(el_val_t url); el_val_t http_delete(el_val_t url);
el_val_t http_serve(el_val_t port, el_val_t handler); void http_serve(el_val_t port, el_val_t handler);
el_val_t http_set_handler(el_val_t name); void http_set_handler(el_val_t name);
/* HTTP server v2 ───────────────────────────────────────────────────────────── /* HTTP server v2 ─────────────────────────────────────────────────────────────
* Same dispatch model as http_serve, but the handler signature is widened: * Same dispatch model as http_serve, but the handler signature is widened:
@@ -175,8 +164,8 @@ el_val_t http_set_handler(el_val_t name);
* The 3-arg http_serve(port, handler) remains supported unchanged for * The 3-arg http_serve(port, handler) remains supported unchanged for
* existing handlers (e.g. products/web/server.el): it dispatches with * existing handlers (e.g. products/web/server.el): it dispatches with
* (method, path, body), hardcodes 200 OK, and auto-detects content type. */ * (method, path, body), hardcodes 200 OK, and auto-detects content type. */
el_val_t http_serve_v2(el_val_t port, el_val_t handler); void http_serve_v2(el_val_t port, el_val_t handler);
el_val_t http_set_handler_v2(el_val_t name); void http_set_handler_v2(el_val_t name);
/* Build an HTTP response envelope. `headers_json` should be a JSON object /* Build an HTTP response envelope. `headers_json` should be a JSON object
* literal like `{"WWW-Authenticate":"Basic"}` (or "" / "{}" for none). The * literal like `{"WWW-Authenticate":"Basic"}` (or "" / "{}" for none). The
@@ -187,11 +176,6 @@ el_val_t http_set_handler_v2(el_val_t name);
* auto-content-type contract for legacy handlers that return plain bodies. */ * auto-content-type contract for legacy handlers that return plain bodies. */
el_val_t http_response(el_val_t status, el_val_t headers_json, el_val_t body); el_val_t http_response(el_val_t status, el_val_t headers_json, el_val_t body);
/* SSE connection fd — set by http_worker_v2 before calling the El handler,
* cleared afterwards. Defined in el_seed.c; called from el_runtime.c.
* The getter is exposed as __http_conn_fd() to El programs. */
void el_seed_set_http_conn_fd(int fd);
/* HTTP timeout — every libcurl request honors EL_HTTP_TIMEOUT_MS (default /* HTTP timeout — every libcurl request honors EL_HTTP_TIMEOUT_MS (default
* 60000ms). Read lazily on first use, so setting the env var any time before * 60000ms). Read lazily on first use, so setting the env var any time before
* the first http_* call is sufficient. */ * the first http_* call is sufficient. */
@@ -227,15 +211,12 @@ el_val_t url_decode(el_val_t s); /* '+' → space, %XX → byte */
* {"p":[],"a":["href","title"],"strong":[],...} * {"p":[],"a":["href","title"],"strong":[],...}
* where each value is the array of attribute names allowed for that tag. */ * where each value is the array of attribute names allowed for that tag. */
el_val_t el_html_sanitize(el_val_t input_html, el_val_t allowlist_json); el_val_t el_html_sanitize(el_val_t input_html, el_val_t allowlist_json);
el_val_t html_raw(el_val_t s);
el_val_t html_escape(el_val_t s);
/* ── Filesystem ──────────────────────────────────────────────────────────── */ /* ── Filesystem ──────────────────────────────────────────────────────────── */
el_val_t fs_read(el_val_t path); el_val_t fs_read(el_val_t path);
el_val_t fs_write(el_val_t path, el_val_t content); el_val_t fs_write(el_val_t path, el_val_t content);
el_val_t fs_list(el_val_t path); el_val_t fs_list(el_val_t path);
el_val_t fs_list_json(el_val_t path);
el_val_t fs_exists(el_val_t path); el_val_t fs_exists(el_val_t path);
el_val_t fs_mkdir(el_val_t path); /* mkdir -p, mode 0755 */ el_val_t fs_mkdir(el_val_t path); /* mkdir -p, mode 0755 */
@@ -265,9 +246,6 @@ el_val_t json_set(el_val_t json_str, el_val_t key, el_val_t value);
el_val_t json_array_len(el_val_t json_str); el_val_t json_array_len(el_val_t json_str);
el_val_t json_array_get(el_val_t json_str, el_val_t index); el_val_t json_array_get(el_val_t json_str, el_val_t index);
el_val_t json_array_get_string(el_val_t json_str, el_val_t index); el_val_t json_array_get_string(el_val_t json_str, el_val_t index);
el_val_t json_escape_string(el_val_t sv);
el_val_t json_build_object(el_val_t kvs);
el_val_t json_build_array(el_val_t items);
/* ── Time ────────────────────────────────────────────────────────────────── */ /* ── Time ────────────────────────────────────────────────────────────────── */
@@ -280,7 +258,6 @@ el_val_t time_to_parts(el_val_t ts);
el_val_t time_from_parts(el_val_t secs, el_val_t ns, el_val_t tz); el_val_t time_from_parts(el_val_t secs, el_val_t ns, el_val_t tz);
el_val_t time_add(el_val_t ts, el_val_t n, el_val_t unit); el_val_t time_add(el_val_t ts, el_val_t n, el_val_t unit);
el_val_t time_diff(el_val_t ts1, el_val_t ts2, el_val_t unit); el_val_t time_diff(el_val_t ts1, el_val_t ts2, el_val_t unit);
el_val_t now_ns(void);
/* ── Instant + Duration: first-class temporal types ────────────────────────── /* ── Instant + Duration: first-class temporal types ──────────────────────────
* Both types share the el_val_t (int64) slot. Instants are nanoseconds * Both types share the el_val_t (int64) slot. Instants are nanoseconds
@@ -437,8 +414,6 @@ el_val_t state_set(el_val_t key, el_val_t value);
el_val_t state_get(el_val_t key); el_val_t state_get(el_val_t key);
el_val_t state_del(el_val_t key); el_val_t state_del(el_val_t key);
el_val_t state_keys(void); el_val_t state_keys(void);
el_val_t state_has(el_val_t key);
el_val_t state_get_or(el_val_t key, el_val_t default_val);
/* ── Float formatting ────────────────────────────────────────────────────── */ /* ── Float formatting ────────────────────────────────────────────────────── */
@@ -530,15 +505,9 @@ el_val_t parse_int(el_val_t s, el_val_t default_val);
/* ── Process ─────────────────────────────────────────────────────────────── */ /* ── Process ─────────────────────────────────────────────────────────────── */
el_val_t exit_program(el_val_t code); void exit_program(el_val_t code);
el_val_t getpid_now(void); el_val_t getpid_now(void);
/* Self-terminating memory guard. Reads ELC_MAX_MEM_MB (default 512) and
* exits with code 1 if resident memory exceeds the limit. Call periodically
* during long compilation loops (e.g. after each function is compiled).
* Returns 0 when memory is within bounds. */
el_val_t el_mem_check(void);
/* ── CGI identity ───────────────────────────────────────────────────────────── /* ── CGI identity ─────────────────────────────────────────────────────────────
* Called at the start of main() in CGI programs (those with a `cgi {}` block). * Called at the start of main() in CGI programs (those with a `cgi {}` block).
* Records the program's DHARMA identity before any other code executes. */ * Records the program's DHARMA identity before any other code executes. */
@@ -776,108 +745,12 @@ el_val_t exec_capture(el_val_t cmd); /* run shell command, capture stdout */
el_val_t exec(el_val_t cmd); /* exec(cmd) → stdout String (30s timeout) */ el_val_t exec(el_val_t cmd); /* exec(cmd) → stdout String (30s timeout) */
el_val_t exec_bg(el_val_t cmd); /* exec_bg(cmd) → PID String (non-blocking) */ el_val_t exec_bg(el_val_t cmd); /* exec_bg(cmd) → PID String (non-blocking) */
/* ── Stdout redirection (used by compiler JS pipeline) ───────────────────── */
el_val_t stdout_to_file(el_val_t path); /* redirect process stdout to a file */
el_val_t stdout_restore(void); /* restore process stdout to terminal */
el_val_t emit_log(el_val_t level, el_val_t msg, el_val_t fields_json); el_val_t emit_log(el_val_t level, el_val_t msg, el_val_t fields_json);
el_val_t emit_metric(el_val_t name, el_val_t value, el_val_t tags_json); el_val_t emit_metric(el_val_t name, el_val_t value, el_val_t tags_json);
el_val_t trace_span_start(el_val_t name); el_val_t trace_span_start(el_val_t name);
el_val_t trace_span_end(el_val_t span_handle); el_val_t trace_span_end(el_val_t span_handle);
el_val_t emit_event(el_val_t name, el_val_t duration_ms); el_val_t emit_event(el_val_t name, el_val_t duration_ms);
el_val_t __thread_create(el_val_t fn_name_v, el_val_t arg_v);
el_val_t __thread_join(el_val_t tid_v);
/* ── __ prefixed aliases (self-hosting compiler ABI) ─────────────────────────
* The El self-hosting compiler emits calls to __-prefixed names. These are
* forwarding wrappers around the existing el_runtime functions above. */
/* I/O */
el_val_t __println(el_val_t s);
el_val_t __print(el_val_t s);
el_val_t __readline(void);
/* String */
el_val_t __int_to_str(el_val_t n);
el_val_t __str_to_int(el_val_t s);
el_val_t __float_to_str(el_val_t f);
el_val_t __str_to_float(el_val_t s);
el_val_t __str_len(el_val_t s);
el_val_t __str_char_at(el_val_t s, el_val_t i);
el_val_t __str_cmp(el_val_t a, el_val_t b);
el_val_t __str_ncmp(el_val_t a, el_val_t b, el_val_t n);
el_val_t __str_concat_raw(el_val_t a, el_val_t b);
el_val_t __str_slice_raw(el_val_t s, el_val_t start, el_val_t end);
el_val_t __str_alloc(el_val_t n);
el_val_t __str_set_char(el_val_t s, el_val_t i, el_val_t c);
/* URL encoding */
el_val_t __url_encode(el_val_t s);
el_val_t __url_decode(el_val_t s);
/* Environment */
el_val_t __env_get(el_val_t key);
/* Subprocess */
el_val_t __exec(el_val_t cmd);
el_val_t __exec_bg(el_val_t cmd);
/* Process */
el_val_t __exit_program(el_val_t code);
/* Filesystem */
el_val_t __fs_exists(el_val_t path);
el_val_t __fs_mkdir(el_val_t path);
el_val_t __fs_read(el_val_t path);
el_val_t __fs_write(el_val_t path, el_val_t content);
el_val_t __fs_write_bytes(el_val_t path, el_val_t bytes, el_val_t n);
el_val_t __fs_list_raw(el_val_t path);
/* HTTP server */
el_val_t __http_response(el_val_t status, el_val_t headers_json, el_val_t body);
el_val_t __http_serve(el_val_t port, el_val_t handler);
el_val_t __http_serve_v2(el_val_t port, el_val_t handler);
/* HTTP conn fd / SSE (weak; overridden by el_seed.c when linked together) */
el_val_t __http_conn_fd(void);
el_val_t __http_sse_open(el_val_t conn_id);
el_val_t __http_sse_send(el_val_t conn_id, el_val_t data);
el_val_t __http_sse_close(el_val_t conn_id);
/* HTTP client (requires HAVE_CURL; stubs provided for no-curl builds) */
el_val_t __http_do(el_val_t method, el_val_t url, el_val_t body,
el_val_t headers_map, el_val_t timeout_ms);
el_val_t __http_do_map(el_val_t method, el_val_t url, el_val_t body,
el_val_t headers_json, el_val_t timeout_ms);
el_val_t __http_do_map_to_file(el_val_t method, el_val_t url, el_val_t body,
el_val_t headers_json, el_val_t output_path);
/* JSON */
el_val_t __json_array_get(el_val_t json, el_val_t index);
el_val_t __json_array_get_string(el_val_t json, el_val_t index);
el_val_t __json_array_len(el_val_t json);
el_val_t __json_get(el_val_t json, el_val_t key);
el_val_t __json_get_raw(el_val_t json, el_val_t key);
el_val_t __json_set(el_val_t json, el_val_t key, el_val_t value);
el_val_t __json_parse_map(el_val_t json_str);
el_val_t __json_stringify_val(el_val_t val);
/* Hashing */
el_val_t __sha256_hex(el_val_t s);
/* State K/V */
el_val_t __state_del(el_val_t key);
el_val_t __state_get(el_val_t key);
el_val_t __state_keys(void);
el_val_t __state_set(el_val_t key, el_val_t val);
/* UUID */
el_val_t __uuid_v4(void);
/* Args */
el_val_t __args_json(void);
#ifdef __cplusplus #ifdef __cplusplus
} }
#endif #endif
@@ -1202,7 +1202,7 @@ fn codegen_js_inner(stmts: [Map<String, Any>], source: String, bundle_mode: Bool
js_emit_line(js_strip_es_exports(runtime_content)) js_emit_line(js_strip_es_exports(runtime_content))
js_emit_line("") js_emit_line("")
} else { } else {
js_emit_line("// Runtime: foundation/el/runtime/el_runtime.js") js_emit_line("// Runtime: foundation/el/el-compiler/runtime/el_runtime.js")
js_emit_line("import \"./el_runtime.js\";") js_emit_line("import \"./el_runtime.js\";")
} }
// In module mode: destructure all builtins off globalThis.__el so call // In module mode: destructure all builtins off globalThis.__el so call
File diff suppressed because it is too large Load Diff
@@ -20,44 +20,18 @@ import "codegen.el"
import "codegen-js.el" import "codegen-js.el"
// compile full pipeline (C target): source string -> C source string // compile full pipeline (C target): source string -> C source string
// Uses JIT function-at-a-time streaming: parse one decl emit C discard AST.
// Peak memory is O(one function's AST) instead of O(whole program AST).
fn compile(source: String) -> String { fn compile(source: String) -> String {
// Top-level arena scope: activates the string arena before lex() so that let tokens: [Map<String, Any>] = lex(source)
// ALL strdup allocations (token strings, sig strings, codegen fragments) let stmts: [Map<String, Any>] = parse(tokens)
// are tracked and freed on pop. Without this, lex() and scan_fn_sigs() // Token list is no longer needed after parsing release it to free memory
// run before any push, leaving _tl_arena_active=0 and leaking every // before codegen allocates its own working data on large source files.
// token string. Also prevents inner pop(mark=0) calls from deactivating el_release(tokens)
// the arena between per-function scopes. codegen(stmts, source)
let top_mark: Any = el_arena_push()
let tokens: [Any] = lex(source)
// Fast pre-scan: collect fn signatures + program kind without building
// full expression ASTs. O(tokens) time, minimal allocation.
let sigs: [Map<String, Any>] = scan_fn_sigs(tokens)
// Stream parse-emit: parse one decl at a time, emit C, discard.
// All output written to stdout via println before pop.
codegen_streaming(tokens, sigs, source)
el_arena_pop(top_mark)
""
}
// compile_test like compile() but sets __test_mode so codegen_streaming
// compiles test { } blocks instead of skipping them, and emits the test
// harness main() instead of the normal int main().
fn compile_test(source: String) -> String {
state_set("__test_mode", "1")
let top_mark: Any = el_arena_push()
let tokens: [Any] = lex(source)
let sigs: [Map<String, Any>] = scan_fn_sigs(tokens)
codegen_streaming(tokens, sigs, source)
el_arena_pop(top_mark)
state_set("__test_mode", "")
""
} }
// compile_js full pipeline (JS target, module mode): source string -> JS source string // compile_js full pipeline (JS target, module mode): source string -> JS source string
fn compile_js(source: String) -> String { fn compile_js(source: String) -> String {
let tokens: [Any] = lex(source) let tokens: [Map<String, Any>] = lex(source)
let stmts: [Map<String, Any>] = parse(tokens) let stmts: [Map<String, Any>] = parse(tokens)
// Token list is no longer needed after parsing release it to free memory. // Token list is no longer needed after parsing release it to free memory.
el_release(tokens) el_release(tokens)
@@ -67,7 +41,7 @@ fn compile_js(source: String) -> String {
// compile_js_with_bundle JS target in bundle mode. // compile_js_with_bundle JS target in bundle mode.
// Reads el_runtime.js from runtime_path and inlines it inside an IIFE. // Reads el_runtime.js from runtime_path and inlines it inside an IIFE.
fn compile_js_with_bundle(source: String, runtime_path: String) -> String { fn compile_js_with_bundle(source: String, runtime_path: String) -> String {
let tokens: [Any] = lex(source) let tokens: [Map<String, Any>] = lex(source)
let stmts: [Map<String, Any>] = parse(tokens) let stmts: [Map<String, Any>] = parse(tokens)
el_release(tokens) el_release(tokens)
let runtime_content: String = fs_read(runtime_path) let runtime_content: String = fs_read(runtime_path)
@@ -173,18 +147,6 @@ fn detect_obfuscate(argv: [String]) -> Bool {
return false return false
} }
// Detect --test flag in argv.
fn detect_test(argv: [String]) -> Bool {
let n: Int = native_list_len(argv)
let i = 0
while i < n {
let a: String = native_list_get(argv, i)
if str_eq(a, "--test") { return true }
let i = i + 1
}
return false
}
// Build a unique temp file path: /tmp/elc-<pid>-<timestamp>.<suffix> // Build a unique temp file path: /tmp/elc-<pid>-<timestamp>.<suffix>
fn make_temp_path(suffix: String) -> String { fn make_temp_path(suffix: String) -> String {
let pid: Int = getpid_now() let pid: Int = getpid_now()
@@ -287,9 +249,6 @@ fn type_node_to_el(t: Map<String, Any>) -> String {
// emit_header write a .elh file from parsed statements. // emit_header write a .elh file from parsed statements.
// Scans for FnDef nodes and emits 'extern fn' declarations. // Scans for FnDef nodes and emits 'extern fn' declarations.
// NOTE: This function requires the full AST. Prefer emit_header_from_sigs
// for the --emit-header path it works from a token-level scan without
// building expression ASTs, avoiding OOM on large files.
fn emit_header(stmts: [Map<String, Any>], hdr_path: String) -> Void { fn emit_header(stmts: [Map<String, Any>], hdr_path: String) -> Void {
let n: Int = native_list_len(stmts) let n: Int = native_list_len(stmts)
let i = 0 let i = 0
@@ -328,32 +287,6 @@ fn emit_header(stmts: [Map<String, Any>], hdr_path: String) -> Void {
let ok: Bool = fs_write(hdr_path, content) let ok: Bool = fs_write(hdr_path, content)
} }
// emit_header_from_sigs write a .elh file from pre-scanned El signatures.
// Uses the output of scan_fn_sigs_el() no full AST required.
// Peak memory is O(tokens) rather than O(whole-program AST), which prevents
// OOM on large files with HTML template bodies or deep BinOp chains.
fn emit_header_from_sigs(sigs: [Map<String, Any>], hdr_path: String) -> Void {
let n: Int = native_list_len(sigs)
let i: Int = 0
let parts: [String] = native_list_empty()
let parts = native_list_append(parts, "// auto-generated by elc --emit-header — do not edit\n")
while i < n {
let sig = native_list_get(sigs, i)
let kind: String = sig["kind"]
if str_eq(kind, "fn") {
let name: String = sig["name"]
let params_el: String = sig["params_el"]
let ret_el: String = sig["ret_el"]
if str_eq(ret_el, "") { let ret_el = "Any" }
let line: String = "extern fn " + name + "(" + params_el + ") -> " + ret_el
let parts = native_list_append(parts, line + "\n")
}
let i = i + 1
}
let content: String = str_join(parts, "")
let ok: Bool = fs_write(hdr_path, content)
}
// Import resolution // Import resolution
// //
// elc supports two forms of import: // elc supports two forms of import:
@@ -414,33 +347,11 @@ fn parse_import_line(trimmed: String, dir: String) -> String {
// Accumulates chunks into lists and joins once at the end to avoid the O(n²) // Accumulates chunks into lists and joins once at the end to avoid the O(n²)
// memory growth caused by repeated `prefix = prefix + chunk` concatenation. // memory growth caused by repeated `prefix = prefix + chunk` concatenation.
fn resolve_imports(src_path: String) -> String { fn resolve_imports(src_path: String) -> String {
// Only the OUTERMOST call publishes provenance. Nested calls number their
// lines from 1 within themselves, so their spans are meaningless once the
// text is spliced into the parent.
let depth: String = state_get("__elc_prov_depth")
if str_eq(depth, "") { state_set("__elc_prov_depth", "1") }
let is_top: Bool = str_eq(depth, "")
let seen_key: String = "__elc_imp__:" + src_path let seen_key: String = "__elc_imp__:" + src_path
let already: String = state_get(seen_key) let already: String = state_get(seen_key)
if !str_eq(already, "") { return "" } if !str_eq(already, "") { return "" }
state_set(seen_key, "1") state_set(seen_key, "1")
// A missing file must be a hard error, never an empty string.
//
// fs_read returns "" both for "file is empty" and "file does not exist", and
// this function used the value without distinguishing them. So a broken
// import path a typo, a moved file, a relative path resolved from the
// wrong working directory compiled CLEANLY: exit 0, empty stderr, and a
// program silently missing everything it imported. Observed 2026-08-15:
// eleven consecutive "successful" compiles that had included no runtime at
// all, and a wrong conclusion drawn from them before anyone noticed.
//
// Missing dependency, confident success. fs_exists separates the two cases,
// so a genuinely empty file still resolves to "" and is fine.
if !fs_exists(src_path) {
println("elc: cannot resolve import: " + src_path)
exit_program(1)
}
let source: String = fs_read(src_path) let source: String = fs_read(src_path)
let dir: String = dirname_of(src_path) let dir: String = dirname_of(src_path)
let lines: [String] = str_split(source, "\n") let lines: [String] = str_split(source, "\n")
@@ -449,7 +360,6 @@ fn resolve_imports(src_path: String) -> String {
// Collect chunks into lists O(1) amortized per append. // Collect chunks into lists O(1) amortized per append.
// Join once at the end O(n) single pass. // Join once at the end O(n) single pass.
let prefix_chunks: [String] = native_list_empty() let prefix_chunks: [String] = native_list_empty()
let prefix_paths: [String] = native_list_empty()
let body_chunks: [String] = native_list_empty() let body_chunks: [String] = native_list_empty()
let i: Int = 0 let i: Int = 0
while i < n { while i < n {
@@ -461,54 +371,21 @@ fn resolve_imports(src_path: String) -> String {
// Only check .elh for imported files never for the entry file itself. // Only check .elh for imported files never for the entry file itself.
let imp_elh_path: String = str_slice(imp_path, 0, str_len(imp_path) - 3) + ".elh" let imp_elh_path: String = str_slice(imp_path, 0, str_len(imp_path) - 3) + ".elh"
let imp_elh: String = fs_read(imp_elh_path) let imp_elh: String = fs_read(imp_elh_path)
// Provenance: record which line range of the combined source came
// from which file, so a diagnostic can name the FILE and not just a
// line in a string that no longer exists on disk.
if !str_eq(imp_elh, "") { if !str_eq(imp_elh, "") {
// Header exists: mark the .el as seen (so it won't be re-inlined // Header exists: mark the .el as seen (so it won't be re-inlined
// if something else also imports it) and use the header text. // if something else also imports it) and use the header text.
let seen_imp_key: String = "__elc_imp__:" + imp_path let seen_imp_key: String = "__elc_imp__:" + imp_path
state_set(seen_imp_key, "1") state_set(seen_imp_key, "1")
let prefix_chunks = native_list_append(prefix_chunks, imp_elh) let prefix_chunks = native_list_append(prefix_chunks, imp_elh)
let prefix_paths = native_list_append(prefix_paths, imp_path)
} else { } else {
let imp_body: String = resolve_imports(imp_path) let imp_body: String = resolve_imports(imp_path)
let prefix_chunks = native_list_append(prefix_chunks, imp_body) let prefix_chunks = native_list_append(prefix_chunks, imp_body)
let prefix_paths = native_list_append(prefix_paths, imp_path)
} }
} else { } else {
let body_chunks = native_list_append(body_chunks, line + "\n") let body_chunks = native_list_append(body_chunks, line + "\n")
} }
let i = i + 1 let i = i + 1
} }
// Walk the assembled chunks once and publish <file> spans <start> <end>.
// LIMIT: nested imports return a single string, so their internal
// boundaries are already lost by the time we see them -- a definition
// inside a transitively imported file is attributed to the direct import.
// Local, not accumulated in state: a nested call numbers its lines from 1
// within itself, so letting it append to a shared buffer republishes
// meaningless spans under the parent's name.
let prov: String = ""
let line_at: Int = 1
let ci: Int = 0
let nchunks: Int = native_list_len(prefix_chunks)
while ci < nchunks {
let chunk: String = native_list_get(prefix_chunks, ci)
let nlines: Int = str_count_lines(chunk)
let src: String = native_list_get(prefix_paths, ci)
let prov = prov + src + " spans " + native_int_to_str(line_at) + " " + native_int_to_str(line_at + nlines - 1) + "\n"
let line_at = line_at + nlines
let ci = ci + 1
}
let prov = prov + src_path + " spans " + native_int_to_str(line_at) + " 999999\n"
if is_top {
let prov_out: String = env("EL_RELATIONS_OUT")
if !str_eq(prov_out, "") {
let existing: String = ""
if fs_exists(prov_out) { let existing = fs_read(prov_out) }
fs_write(prov_out, existing + prov)
}
}
return str_join(prefix_chunks, "") + str_join(body_chunks, "") return str_join(prefix_chunks, "") + str_join(body_chunks, "")
} }
@@ -599,7 +476,6 @@ fn main() -> Void {
let do_bundle: Bool = detect_bundle(argv) let do_bundle: Bool = detect_bundle(argv)
let do_minify: Bool = detect_minify(argv) let do_minify: Bool = detect_minify(argv)
let do_obfuscate: Bool = detect_obfuscate(argv) let do_obfuscate: Bool = detect_obfuscate(argv)
let do_test: Bool = detect_test(argv)
// --obfuscate implies --minify: obfuscating unminified code is pointless. // --obfuscate implies --minify: obfuscating unminified code is pointless.
if do_obfuscate { if do_obfuscate {
let do_minify = true let do_minify = true
@@ -607,7 +483,7 @@ fn main() -> Void {
let positional: [String] = strip_flags(argv) let positional: [String] = strip_flags(argv)
let argc: Int = native_list_len(positional) let argc: Int = native_list_len(positional)
if argc < 1 { if argc < 1 {
println("el-compiler: usage: elc [--target=c|js] [--bundle] [--minify] [--obfuscate] [--emit-header] [--test] <source.el> [<output>]") println("el-compiler: usage: elc [--target=c|js] [--bundle] [--minify] [--obfuscate] [--emit-header] <source.el> [<output>]")
exit(1) exit(1)
} }
@@ -621,20 +497,16 @@ fn main() -> Void {
let src_path: String = native_list_get(positional, 0) let src_path: String = native_list_get(positional, 0)
// When --emit-header is requested, lex the source file and do a // When --emit-header is requested, parse the source file directly
// token-level signature scan (no full AST) to write a .elh file. // (without inlining imports) and write out a .elh file alongside the .c.
// This avoids OOM on large files with HTML template bodies or deep
// BinOp chains (e.g. checkout.el) parse() builds O(whole-program AST)
// while scan_fn_sigs_el keeps peak memory at O(tokens).
if do_emit_header { if do_emit_header {
el_mem_check()
let raw_source: String = fs_read(src_path) let raw_source: String = fs_read(src_path)
let hdr_tokens: [Any] = lex(raw_source) let hdr_tokens: [Map<String, Any>] = lex(raw_source)
let hdr_sigs: [Map<String, Any>] = scan_fn_sigs_el(hdr_tokens) let hdr_stmts: [Map<String, Any>] = parse(hdr_tokens)
el_release(hdr_tokens) el_release(hdr_tokens)
let hdr_path: String = str_slice(src_path, 0, str_len(src_path) - 3) + ".elh" let hdr_path: String = str_slice(src_path, 0, str_len(src_path) - 3) + ".elh"
emit_header_from_sigs(hdr_sigs, hdr_path) emit_header(hdr_stmts, hdr_path)
el_release(hdr_sigs) el_release(hdr_stmts)
} }
let source: String = resolve_imports(src_path) let source: String = resolve_imports(src_path)
@@ -648,12 +520,6 @@ fn main() -> Void {
exit(0) exit(0)
} }
// --test mode: compile with test harness (C target only).
if do_test {
compile_test(source)
exit(0)
}
// Standard path (no post-processing). // Standard path (no post-processing).
let out: String = "" let out: String = ""
if do_bundle { if do_bundle {
+754
View File
@@ -0,0 +1,754 @@
// lexer.el el self-hosting lexer
//
// Tokenises an el source string into a list of token maps.
// Each token is a Map<String, Any> with keys:
// "kind" -> String (e.g. "Int", "Ident", "Plus")
// "value" -> String (the raw text of the token)
//
// Entry point: fn lex(source: String) -> [Map<String, Any>]
//
// Uses native_string_chars to split the source into a chars list,
// then indexes it with native_list_get avoids O(N²) string cloning.
// Character helpers
fn lex_is_digit(ch: String) -> Bool {
if ch == "0" { return true }
if ch == "1" { return true }
if ch == "2" { return true }
if ch == "3" { return true }
if ch == "4" { return true }
if ch == "5" { return true }
if ch == "6" { return true }
if ch == "7" { return true }
if ch == "8" { return true }
if ch == "9" { return true }
false
}
fn lex_is_alpha(ch: String) -> Bool {
if ch == "a" { return true }
if ch == "b" { return true }
if ch == "c" { return true }
if ch == "d" { return true }
if ch == "e" { return true }
if ch == "f" { return true }
if ch == "g" { return true }
if ch == "h" { return true }
if ch == "i" { return true }
if ch == "j" { return true }
if ch == "k" { return true }
if ch == "l" { return true }
if ch == "m" { return true }
if ch == "n" { return true }
if ch == "o" { return true }
if ch == "p" { return true }
if ch == "q" { return true }
if ch == "r" { return true }
if ch == "s" { return true }
if ch == "t" { return true }
if ch == "u" { return true }
if ch == "v" { return true }
if ch == "w" { return true }
if ch == "x" { return true }
if ch == "y" { return true }
if ch == "z" { return true }
if ch == "A" { return true }
if ch == "B" { return true }
if ch == "C" { return true }
if ch == "D" { return true }
if ch == "E" { return true }
if ch == "F" { return true }
if ch == "G" { return true }
if ch == "H" { return true }
if ch == "I" { return true }
if ch == "J" { return true }
if ch == "K" { return true }
if ch == "L" { return true }
if ch == "M" { return true }
if ch == "N" { return true }
if ch == "O" { return true }
if ch == "P" { return true }
if ch == "Q" { return true }
if ch == "R" { return true }
if ch == "S" { return true }
if ch == "T" { return true }
if ch == "U" { return true }
if ch == "V" { return true }
if ch == "W" { return true }
if ch == "X" { return true }
if ch == "Y" { return true }
if ch == "Z" { return true }
false
}
fn is_alnum_or_underscore(ch: String) -> Bool {
if lex_is_digit(ch) { return true }
if lex_is_alpha(ch) { return true }
if ch == "_" { return true }
false
}
fn lex_is_whitespace(ch: String) -> Bool {
if ch == " " { return true }
if ch == "\t" { return true }
if ch == "\n" { return true }
if ch == "\r" { return true }
false
}
fn make_tok(kind: String, value: String) -> Map<String, Any> {
{ "kind": kind, "value": value }
}
// Keyword lookup
fn keyword_kind(word: String) -> String {
if word == "let" { return "Let" }
if word == "fn" { return "Fn" }
if word == "type" { return "Type" }
if word == "enum" { return "Enum" }
if word == "match" { return "Match" }
if word == "return" { return "Return" }
if word == "if" { return "If" }
if word == "else" { return "Else" }
if word == "for" { return "For" }
if word == "in" { return "In" }
if word == "while" { return "While" }
if word == "import" { return "Import" }
if word == "from" { return "From" }
if word == "as" { return "As" }
if word == "with" { return "With" }
if word == "sealed" { return "Sealed" }
if word == "activate" { return "Activate" }
if word == "where" { return "Where" }
if word == "test" { return "Test" }
if word == "seed" { return "Seed" }
if word == "assert" { return "Assert" }
if word == "protocol" { return "Protocol" }
if word == "impl" { return "Impl" }
if word == "retry" { return "Retry" }
if word == "times" { return "Times" }
if word == "fallback" { return "Fallback" }
if word == "reason" { return "Reason" }
if word == "parallel" { return "Parallel" }
if word == "trace" { return "Trace" }
if word == "requires" { return "Requires" }
if word == "deploy" { return "Deploy" }
if word == "to" { return "To" }
if word == "via" { return "Via" }
if word == "target" { return "Target" }
if word == "true" { return "Bool" }
if word == "false" { return "Bool" }
if word == "cgi" { return "Cgi" }
if word == "service" { return "Service" }
if word == "manager" { return "Manager" }
if word == "engine" { return "Engine" }
if word == "accessor" { return "Accessor" }
if word == "vessel" { return "Vessel" }
if word == "extern" { return "Extern" }
if word == "try" { return "Try" }
if word == "catch" { return "Catch" }
""
}
// Scan helpers
// All scan helpers receive the chars list and total length.
// scan_digits advance i while chars[i] is a digit
// Returns { "text": ..., "pos": i }
fn scan_digits(chars: [String], start: Int, total: Int) -> Map<String, Any> {
let i = start
let parts: [String] = native_list_empty()
let running = true
while running {
if i >= total {
let running = false
} else {
let ch: String = native_list_get(chars, i)
if lex_is_digit(ch) {
let parts = native_list_append(parts, ch)
let i = i + 1
} else {
let running = false
}
}
}
{ "text": str_join(parts, ""), "pos": i }
}
// scan_ident advance i while chars[i] is alphanumeric or underscore
fn scan_ident(chars: [String], start: Int, total: Int) -> Map<String, Any> {
let i = start
let parts: [String] = native_list_empty()
let running = true
while running {
if i >= total {
let running = false
} else {
let ch: String = native_list_get(chars, i)
if is_alnum_or_underscore(ch) {
let parts = native_list_append(parts, ch)
let i = i + 1
} else {
let running = false
}
}
}
{ "text": str_join(parts, ""), "pos": i }
}
// Code-bearing string detection + comment strip
// Inline JS/CSS literals embedded in El source (e.g. <script></script> blobs
// or stylesheet payloads inside string literals) carry their own line and
// block comments. Those comments leak into the served HTML and reveal build
// notes the visitor should never see. We strip them at the lexer so every
// downstream consumer (codegen-c, codegen-js, parser) gets the cleaned form.
//
// looks_like_code heuristic gate so we only strip strings that actually
// embed JS or CSS. Plain prose, hex blobs, JSON, etc. pass through verbatim.
fn substr_at(chars: [String], start: Int, total: Int, needle: String) -> Bool {
let nchars: [String] = native_string_chars(needle)
let nlen: Int = native_list_len(nchars)
if start + nlen > total { return false }
let i = 0
let matched = true
while i < nlen {
let a: String = native_list_get(chars, start + i)
let b: String = native_list_get(nchars, i)
if a == b { let i = i + 1 } else { let matched = false; let i = nlen }
}
matched
}
fn str_has(s: String, needle: String) -> Bool {
let chars: [String] = native_string_chars(s)
let total: Int = native_list_len(chars)
let i = 0
let found = false
while i < total {
if substr_at(chars, i, total, needle) {
let found = true
let i = total
} else {
let i = i + 1
}
}
found
}
fn looks_like_code(s: String) -> Bool {
if str_has(s, "<script") { return true }
if str_has(s, "<style") { return true }
if str_has(s, "function") {
if str_has(s, ";") { return true }
}
false
}
// strip_code_comments character-by-character walk. Tracks JS string state
// (single, double, backtick) and never strips inside one. Backslash escapes
// inside JS strings consume the next char verbatim. URLs like https:// are
// preserved by checking the previous char before treating // as a line
// comment opener: if the char immediately before '/' is ':', emit the '/'
// literally and advance one position.
fn strip_code_comments(s: String) -> String {
let chars: [String] = native_string_chars(s)
let total: Int = native_list_len(chars)
let out_parts: [String] = native_list_empty()
let i = 0
let in_squote = false
let in_dquote = false
let in_btick = false
let prev = ""
while i < total {
let ch: String = native_list_get(chars, i)
let in_js_string = false
if in_squote { let in_js_string = true }
if in_dquote { let in_js_string = true }
if in_btick { let in_js_string = true }
if in_js_string {
// Backslash escape: consume next char verbatim regardless of which.
if ch == "\\" {
let out_parts = native_list_append(out_parts, ch)
let next_i = i + 1
if next_i < total {
let nc: String = native_list_get(chars, next_i)
let out_parts = native_list_append(out_parts, nc)
let prev = nc
let i = next_i + 1
} else {
let prev = ch
let i = next_i
}
} else {
if in_squote {
if ch == "'" { let in_squote = false }
} else {
if in_dquote {
if ch == "\"" { let in_dquote = false }
} else {
if in_btick {
if ch == "`" { let in_btick = false }
}
}
}
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
}
} else {
// Not in a JS string. Check for comment openers.
let next_i = i + 1
let next_ch = ""
if next_i < total {
let next_ch: String = native_list_get(chars, next_i)
}
if ch == "/" {
if next_ch == "/" {
// URL guard: prev char ':' means this is "://", not a comment.
if prev == ":" {
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
} else {
// Skip until newline (newline itself is preserved so
// surrounding line counts/structure stay sane).
let i = i + 2
let scanning = true
while scanning {
if i >= total {
let scanning = false
} else {
let lc: String = native_list_get(chars, i)
if lc == "\n" {
let scanning = false
} else {
let i = i + 1
}
}
}
let prev = ""
}
} else {
if next_ch == "*" {
// Skip until matching "*/".
let i = i + 2
let scanning2 = true
while scanning2 {
if i >= total {
let scanning2 = false
} else {
let bc: String = native_list_get(chars, i)
if bc == "*" {
let after = i + 1
if after < total {
let nc2: String = native_list_get(chars, after)
if nc2 == "/" {
let i = after + 1
let scanning2 = false
} else {
let i = i + 1
}
} else {
let i = i + 1
}
} else {
let i = i + 1
}
}
}
let prev = ""
} else {
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
}
}
} else {
// Open a JS string?
if ch == "'" {
let in_squote = true
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
} else {
if ch == "\"" {
let in_dquote = true
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
} else {
if ch == "`" {
let in_btick = true
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
} else {
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
}
}
}
}
}
}
str_join(out_parts, "")
}
// scan_string scan a quoted string literal, handling \" escapes.
// Starts AFTER the opening quote. Returns { "text": content, "pos": i_after_close }
fn scan_string(chars: [String], start: Int, total: Int) -> Map<String, Any> {
let i = start
let parts: [String] = native_list_empty()
let running = true
while running {
if i >= total {
let running = false
} else {
let ch: String = native_list_get(chars, i)
if ch == "\\" {
// escape: peek next char
let next_i = i + 1
if next_i < total {
let next_ch: String = native_list_get(chars, next_i)
if next_ch == "\"" {
let parts = native_list_append(parts, "\"")
let i = next_i + 1
} else {
if next_ch == "n" {
let parts = native_list_append(parts, "\n")
let i = next_i + 1
} else {
if next_ch == "t" {
let parts = native_list_append(parts, "\t")
let i = next_i + 1
} else {
if next_ch == "r" {
let parts = native_list_append(parts, "\r")
let i = next_i + 1
} else {
if next_ch == "\\" {
let parts = native_list_append(parts, "\\")
let i = next_i + 1
} else {
let parts = native_list_append(parts, next_ch)
let i = next_i + 1
}
}
}
}
}
} else {
let i = i + 1
}
} else {
if ch == "\"" {
let i = i + 1
let running = false
} else {
let parts = native_list_append(parts, ch)
let i = i + 1
}
}
}
}
{ "text": str_join(parts, ""), "pos": i }
}
// Main lexer
fn lex(source: String) -> [Map<String, Any>] {
let chars: [String] = native_string_chars(source)
let total: Int = native_list_len(chars)
let tokens: [Map<String, Any>] = native_list_empty()
let i: Int = 0
while i < total {
let ch: String = native_list_get(chars, i)
// Skip whitespace
if lex_is_whitespace(ch) {
let i = i + 1
} else {
// Line comments: //
if ch == "/" {
let next_i = i + 1
if next_i < total {
let next_ch: String = native_list_get(chars, next_i)
if next_ch == "/" {
// skip to end of line
let i = i + 2
let running2 = true
while running2 {
if i >= total {
let running2 = false
} else {
let lch: String = native_list_get(chars, i)
if lch == "\n" {
let running2 = false
} else {
let i = i + 1
}
}
}
} else {
let tokens = native_list_append(tokens, make_tok("Slash", "/"))
let i = i + 1
}
} else {
let tokens = native_list_append(tokens, make_tok("Slash", "/"))
let i = i + 1
}
} else {
// String literal
if ch == "\"" {
let result = scan_string(chars, i + 1, total)
let str_text: String = result["text"]
let new_pos: Int = result["pos"]
// Compile-time scrub: strings that embed JS or CSS get
// their // line comments and /* block comments stripped
// before the token reaches the parser. Plain prose passes
// through untouched.
let clean_text = str_text
if looks_like_code(str_text) {
let clean_text = strip_code_comments(str_text)
}
let tokens = native_list_append(tokens, make_tok("Str", clean_text))
let i = new_pos
} else {
// Number literal
if lex_is_digit(ch) {
let result = scan_digits(chars, i, total)
let num_text: String = result["text"]
let new_pos: Int = result["pos"]
// check for float (dot followed by digit)
if new_pos < total {
let dot_ch: String = native_list_get(chars, new_pos)
if dot_ch == "." {
let after_dot = new_pos + 1
if after_dot < total {
let after_dot_ch: String = native_list_get(chars, after_dot)
if lex_is_digit(after_dot_ch) {
let frac_result = scan_digits(chars, after_dot, total)
let frac_text: String = frac_result["text"]
let frac_pos: Int = frac_result["pos"]
let tokens = native_list_append(tokens, make_tok("Float", num_text + "." + frac_text))
let i = frac_pos
} else {
let tokens = native_list_append(tokens, make_tok("Int", num_text))
let i = new_pos
}
} else {
let tokens = native_list_append(tokens, make_tok("Int", num_text))
let i = new_pos
}
} else {
let tokens = native_list_append(tokens, make_tok("Int", num_text))
let i = new_pos
}
} else {
let tokens = native_list_append(tokens, make_tok("Int", num_text))
let i = new_pos
}
} else {
// Identifier or keyword
if lex_is_alpha(ch) || ch == "_" {
let result = scan_ident(chars, i, total)
let word: String = result["text"]
let new_pos: Int = result["pos"]
let kw = keyword_kind(word)
if kw == "" {
let tokens = native_list_append(tokens, make_tok("Ident", word))
} else {
let tokens = native_list_append(tokens, make_tok(kw, word))
}
let i = new_pos
} else {
// Multi-char and single-char operators/delimiters
let peek_i = i + 1
let peek_ch = ""
if peek_i < total {
let peek_ch: String = native_list_get(chars, peek_i)
}
if ch == "=" {
if peek_ch == "=" {
let tokens = native_list_append(tokens, make_tok("EqEq", "=="))
let i = i + 2
} else {
if peek_ch == ">" {
let tokens = native_list_append(tokens, make_tok("FatArrow", "=>"))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Eq", "="))
let i = i + 1
}
}
} else {
if ch == "!" {
if peek_ch == "=" {
let tokens = native_list_append(tokens, make_tok("NotEq", "!="))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Not", "!"))
let i = i + 1
}
} else {
if ch == "<" {
if peek_ch == "=" {
let tokens = native_list_append(tokens, make_tok("LtEq", "<="))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Lt", "<"))
let i = i + 1
}
} else {
if ch == ">" {
if peek_ch == "=" {
let tokens = native_list_append(tokens, make_tok("GtEq", ">="))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Gt", ">"))
let i = i + 1
}
} else {
if ch == "&" {
if peek_ch == "&" {
let tokens = native_list_append(tokens, make_tok("And", "&&"))
let i = i + 2
} else {
let i = i + 1
}
} else {
if ch == "|" {
if peek_ch == "|" {
let tokens = native_list_append(tokens, make_tok("Or", "||"))
let i = i + 2
} else {
if peek_ch == ">" {
let tokens = native_list_append(tokens, make_tok("PipeOp", "|>"))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Pipe", "|"))
let i = i + 1
}
}
} else {
if ch == "-" {
if peek_ch == ">" {
let tokens = native_list_append(tokens, make_tok("Arrow", "->"))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Minus", "-"))
let i = i + 1
}
} else {
if ch == ":" {
if peek_ch == ":" {
let tokens = native_list_append(tokens, make_tok("ColonColon", "::"))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Colon", ":"))
let i = i + 1
}
} else {
if ch == "+" {
let tokens = native_list_append(tokens, make_tok("Plus", "+"))
let i = i + 1
} else {
if ch == "*" {
let tokens = native_list_append(tokens, make_tok("Star", "*"))
let i = i + 1
} else {
if ch == "%" {
let tokens = native_list_append(tokens, make_tok("Percent", "%"))
let i = i + 1
} else {
if ch == "(" {
let tokens = native_list_append(tokens, make_tok("LParen", "("))
let i = i + 1
} else {
if ch == ")" {
let tokens = native_list_append(tokens, make_tok("RParen", ")"))
let i = i + 1
} else {
if ch == "{" {
let tokens = native_list_append(tokens, make_tok("LBrace", "{"))
let i = i + 1
} else {
if ch == "}" {
let tokens = native_list_append(tokens, make_tok("RBrace", "}"))
let i = i + 1
} else {
if ch == "[" {
let tokens = native_list_append(tokens, make_tok("LBracket", "["))
let i = i + 1
} else {
if ch == "]" {
let tokens = native_list_append(tokens, make_tok("RBracket", "]"))
let i = i + 1
} else {
if ch == "," {
let tokens = native_list_append(tokens, make_tok("Comma", ","))
let i = i + 1
} else {
if ch == "." {
let tokens = native_list_append(tokens, make_tok("Dot", "."))
let i = i + 1
} else {
if ch == ";" {
let tokens = native_list_append(tokens, make_tok("Semicolon", ";"))
let i = i + 1
} else {
if ch == "@" {
let tokens = native_list_append(tokens, make_tok("At", "@"))
let i = i + 1
} else {
if ch == "?" {
let tokens = native_list_append(tokens, make_tok("QuestionMark", "?"))
let i = i + 1
} else {
if ch == "#" {
let tokens = native_list_append(tokens, make_tok("Hash", "#"))
let i = i + 1
} else {
// unknown char skip
let i = i + 1
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
let tokens = native_list_append(tokens, make_tok("Eof", ""))
tokens
}
File diff suppressed because it is too large Load Diff
+6 -80
View File
@@ -77,33 +77,6 @@ fn parse_manifest_entry(src: String) -> String {
return "" return ""
} }
// parse_manifest_c_sources - collect all `c_source "path"` lines from the
// build block. Returns a flat list of path strings.
fn parse_manifest_c_sources(src: String) -> [String] {
let result: [String] = native_list_empty()
let lines: [String] = str_split(src, "\n")
let n: Int = native_list_len(lines)
let i = 0
while i < n {
let line: String = native_list_get(lines, i)
let t: String = str_trim(line)
if str_starts_with(t, "c_source ") {
let after: String = str_slice(t, 9, str_len(t))
let trimmed: String = str_trim(after)
if str_starts_with(trimmed, "\"") {
let inner: String = str_slice(trimmed, 1, str_len(trimmed))
let q: Int = str_index_of(inner, "\"")
if q >= 0 {
let path: String = str_slice(inner, 0, q)
let result = native_list_append(result, path)
}
}
}
let i = i + 1
}
return result
}
fn parse_manifest_name(src: String) -> String { fn parse_manifest_name(src: String) -> String {
let lines: [String] = str_split(src, "\n") let lines: [String] = str_split(src, "\n")
let n: Int = native_list_len(lines) let n: Int = native_list_len(lines)
@@ -252,7 +225,6 @@ fn compile_module(src_path: String, out_dir: String, elc_bin: String, dry_run: B
let bname: String = basename_noext(src_path) let bname: String = basename_noext(src_path)
let c_out: String = out_dir + "/" + bname + ".c" let c_out: String = out_dir + "/" + bname + ".c"
let elh_out: String = out_dir + "/" + bname + ".elh" let elh_out: String = out_dir + "/" + bname + ".elh"
let err_tmp: String = "/tmp/elb-err-" + bname + ".txt"
// Check if recompile needed // Check if recompile needed
if !file_is_newer(src_path, c_out) { if !file_is_newer(src_path, c_out) {
@@ -262,26 +234,18 @@ fn compile_module(src_path: String, out_dir: String, elc_bin: String, dry_run: B
return true return true
} }
// elc streams C to stdout; redirect stderr to a temp file so we can // elc streams C to stdout (collect mode not yet implemented); use
// surface the actual error message on failure instead of swallowing it. // shell redirection so the output lands in the file, not the terminal.
let cmd: String = elc_bin + " --emit-header " + src_path + " > " + c_out + " 2>" + err_tmp let cmd: String = elc_bin + " --emit-header " + src_path + " > " + c_out + " 2>&1"
println(" compile " + src_path) println(" compile " + src_path)
if dry_run { return true } if dry_run { return true }
let ret: Int = exec_command(cmd) let ret: Int = exec_command(cmd)
if ret != 0 { if ret != 0 {
// Surface the actual compiler error from stderr
let err_msg: String = str_trim(fs_read(err_tmp))
if !str_eq(err_msg, "") {
println(err_msg)
}
// Remove partial output so a retry starts clean
exec_command("rm -f " + c_out + " " + err_tmp)
println("elb: compile failed: " + src_path) println("elb: compile failed: " + src_path)
return false return false
} }
exec_command("rm -f " + err_tmp)
// Move the generated .elh (written next to the source by elc) into // Move the generated .elh (written next to the source by elc) into
// out_dir so that #include "module.elh" lines in the generated .c // out_dir so that #include "module.elh" lines in the generated .c
@@ -298,21 +262,7 @@ fn link_binary(c_files: [String], out_bin: String, runtime_path: String, out_dir
let parts: [String] = native_list_empty() let parts: [String] = native_list_empty()
// Include both the runtime dir (for el_runtime.h) and the output dir // Include both the runtime dir (for el_runtime.h) and the output dir
// (for module.elh cross-module forward declarations). // (for module.elh cross-module forward declarations).
// Detect clang vs gcc: -fbracket-depth is clang-only; silently ignored let parts = native_list_append(parts, "cc -O2 -I " + dirname_of(runtime_path) + " -I " + out_dir)
// if unsupported but gcc rejects it with an error.
let bracket_flag: String = "$(cc --version 2>&1 | grep -q clang && printf -- '-fbracket-depth=1024' || true)"
// On macOS, OpenSSL is not on the default linker path. Detect homebrew
// prefix and add it if present (no-op on Linux where libssl is in /usr/lib).
let ossl_lib_flag: String = "$(brew --prefix openssl 2>/dev/null | xargs -I{} printf -- '-L{}/lib' 2>/dev/null || true)"
let ossl_inc_flag: String = "$(brew --prefix openssl 2>/dev/null | xargs -I{} printf -- '-I{}/include' 2>/dev/null || true)"
// Force-include the C-level master declarations header so every translation
// unit sees all cross-module function signatures. Handles packages (like ELP)
// where modules call each other without explicit El import statements.
// The header is generated by elb --gen-decls or manually placed in out_dir.
let master_decls: String = out_dir + "/elp-c-decls.h"
let has_master: String = str_trim(exec_capture("test -f " + master_decls + " && echo yes || echo no"))
let include_flag: String = if str_eq(has_master, "yes") { "-include " + master_decls } else { "" }
let parts = native_list_append(parts, "cc -O2 " + bracket_flag + " " + ossl_inc_flag + " " + include_flag + " -I " + dirname_of(runtime_path) + " -I " + out_dir)
let i = 0 let i = 0
while i < n { while i < n {
let f: String = native_list_get(c_files, i) let f: String = native_list_get(c_files, i)
@@ -320,7 +270,7 @@ fn link_binary(c_files: [String], out_bin: String, runtime_path: String, out_dir
let i = i + 1 let i = i + 1
} }
let parts = native_list_append(parts, runtime_path) let parts = native_list_append(parts, runtime_path)
let parts = native_list_append(parts, ossl_lib_flag + " -lcurl -lssl -lcrypto -lpthread -lm") let parts = native_list_append(parts, "-lcurl -lpthread")
let parts = native_list_append(parts, "-o " + out_bin) let parts = native_list_append(parts, "-o " + out_bin)
let cmd: String = str_join(parts, " ") let cmd: String = str_join(parts, " ")
println(" link " + out_bin) println(" link " + out_bin)
@@ -353,7 +303,6 @@ fn main() -> Void {
let pkg_name: String = parse_manifest_name(manifest_src) let pkg_name: String = parse_manifest_name(manifest_src)
let entry: String = parse_manifest_entry(manifest_src) let entry: String = parse_manifest_entry(manifest_src)
let extra_c: [String] = parse_manifest_c_sources(manifest_src)
if str_eq(entry, "") { if str_eq(entry, "") {
println("elb: manifest.el has no 'entry' declaration") println("elb: manifest.el has no 'entry' declaration")
exit(1) exit(1)
@@ -368,21 +317,7 @@ fn main() -> Void {
let which_out: String = str_trim(exec_capture("which " + elc_bin + " 2>/dev/null")) let which_out: String = str_trim(exec_capture("which " + elc_bin + " 2>/dev/null"))
if !str_eq(which_out, "") { if !str_eq(which_out, "") {
let elc_dir: String = dirname_of(which_out) let elc_dir: String = dirname_of(which_out)
runtime_path = elc_dir + "/../runtime/el_runtime.c" runtime_path = elc_dir + "/../el-compiler/runtime/el_runtime.c"
}
}
// If --runtime points to a directory, auto-locate el_runtime.c inside it.
// This lets both forms work:
// --runtime=/opt/el/runtime (directory form)
// --runtime=/opt/el/runtime/el_runtime.c (file form)
if !str_eq(runtime_path, "") {
let is_dir: String = str_trim(exec_capture("test -d " + runtime_path + " && echo dir || echo file"))
if str_eq(is_dir, "dir") {
let candidate: String = runtime_path + "/el_runtime.c"
let has_file: String = str_trim(exec_capture("test -f " + candidate + " && echo yes || echo no"))
if str_eq(has_file, "yes") {
let runtime_path = candidate
}
} }
} }
if str_eq(runtime_path, "") { if str_eq(runtime_path, "") {
@@ -432,15 +367,6 @@ fn main() -> Void {
exit(1) exit(1)
} }
// Append any extra C sources declared in the manifest (e.g. platform stubs)
let ei = 0
let en: Int = native_list_len(extra_c)
while ei < en {
let ec: String = native_list_get(extra_c, ei)
let c_files = native_list_append(c_files, ec)
let ei = ei + 1
}
// Link // Link
let out_bin: String = out_dir + "/" + pkg_name let out_bin: String = out_dir + "/" + pkg_name
let linked: Bool = link_binary(c_files, out_bin, runtime_path, out_dir, dry_run) let linked: Bool = link_binary(c_files, out_bin, runtime_path, out_dir, dry_run)
View File
-3
View File
@@ -3797,9 +3797,6 @@ fn builtin_arity(name: String) -> Int {
if str_eq(name, "engram_activate") { return 2 } if str_eq(name, "engram_activate") { return 2 }
if str_eq(name, "engram_save") { return 1 } if str_eq(name, "engram_save") { return 1 }
if str_eq(name, "engram_load") { return 1 } if str_eq(name, "engram_load") { return 1 }
if str_eq(name, "engram_store_boot") { return 1 }
if str_eq(name, "engram_store_checkpoint") { return 0 }
if str_eq(name, "engram_store_close") { return 0 }
if str_eq(name, "engram_get_node_json") { return 1 } if str_eq(name, "engram_get_node_json") { return 1 }
if str_eq(name, "engram_search_json") { return 2 } if str_eq(name, "engram_search_json") { return 2 }
if str_eq(name, "engram_scan_nodes_json") { return 2 } if str_eq(name, "engram_scan_nodes_json") { return 2 }
-21
View File
@@ -1,21 +0,0 @@
# Compiled El bytecode
*.elc
# C codegen output
*.c
*.o
*.a
*.so
*.dylib
# Combined build artifacts
_combined.el
*-combined.el
# Distribution / build output
dist/
build/
out/
# OS
.DS_Store
-65
View File
@@ -1,65 +0,0 @@
# ELP language consolidation — full-lexicon backfill (stage)
Branch: `stage-elp-lang-consolidation` (stage-bound; NOT the live soul :8742).
Consolidates scattered Python language-realizer work (`~/Desktop/lang-realizers`,
`~/Desktop/lang-poetry-experiment`, `~/semitic_engine`) into the ELP `.el`
structure, generating **full lexicons** (complete UniMorph + kaikki.org
Wiktionary — real gender, real inflections) instead of the demo/curated subsets
the prototypes shipped.
## ELP before this branch
- 18 classical/ancient languages fully done (vocab + morphology + tests):
akk ang cop egy enm fro gez goh got grc non peo pi sa sga sux txb uga.
- 11 modern/classical languages had `morphology-<code>.el` in the build manifest
but **no vocabulary and no lang_profile**: es fr de ja ar he hi ru fi sw la.
- The ES port (`stage-elp-es-port`) had a *demo-scale* vocabulary-es.el (~350
entries, s-expr form).
## Landed on this branch (full-lexicon seed-fn format, matching the 18 ancients)
Vocabulary schema per row: `[lemma, pos, form0, form1, form2, en_gloss, hint]`.
Files are ELP runtime **seed data** (loaded via the Engram at runtime), so — like
all 18 classical `vocabulary-*.el` — they are intentionally NOT in the build
manifest. Syntax validated: the chunked `fn vocab_<code>_seed_pN` format
compiles cleanly to C via `elc` (correct UTF-8).
| code | in-ELP-morph? | vocab entries | verbs | nouns | adjs | profile |
|------|---------------|--------------:|------:|------:|-----:|---------|
| es | yes | 72,032 | 6,695 | 48,353 | 16,984 | yes |
| fr | yes | 130,517 | 7,534 | 77,344 | 45,639 | yes |
| de | yes | 144,692 | 6,661 | 133,162 | 4,869 | yes |
| la | yes | 22,590 | 82 | 13,436 | 9,072 | yes |
| it | no (bonus) | 193,675 | 10,008 | 109,459 | 74,208 | yes |
| pt | no (bonus) | 115,772 | 4,001 | 72,073 | 39,698 | yes |
| ro | no (bonus) | 86,504 | 1,216 | 65,915 | 19,373 | yes |
| ca | no (bonus) | 47,112 | 1,547 | 28,830 | 16,735 | yes |
|**total**| |**812,894** | | | | |
Generators (reproducible): `elp/tests/lang-gen/gen_elp_seed_full.py` (Romance),
`gen_elp_seed_de_la.py` (German declension + Latin case-paradigm mapping). They
read the pre-built morph caches in `~/Desktop/lang-realizers/data/` (UniMorph +
kaikki), which are too large to commit.
## Remaining (honest)
Of the 11 ELP backfill targets, 4 are done (es fr de la). The other 7 have **no
full-lexicon engine** yet — cannot be generated honestly without engine work:
- **ru**: only a 110-entry curated Slavic subset exists; full `rus.unimorph`
present but no `morphology_ru_full` productive loader. Needs a full Russian
morphology module (like the Romance ones) before vocab generation.
- **ja / ko / zh**: validated demo engines (~66-104 hardcoded words) in
`lang-poetry-experiment`, Python only. Agglutinative (ja/ko) + isolating (zh)
need `.el` engine ports + full-lexicon wiring (ja: jpn_unimorph; zh: CC-CEDICT).
- **ar / he (Semitic)**: template engines (16 AR / 8 HE patterns, ~6 roots) in
`~/semitic_engine`, Python only. Root-and-pattern; full UniMorph ara/heb
present but used only for validation. Needs productive root lexicon + `.el` port.
- **hi (Hindi), fi (Finnish), sw (Swahili)**: `morphology-<code>.el` exists in
ELP but there is NO scattered prototype and NO downloaded data for these —
full-lexicon collection (UniMorph/kaikki) + generator still to do.
De/nl/sv Germanic and it/ro/ca/pt Romance verb coverage note: German verbs here
are the ~6.6k caches carry; the it/ro/ca/pt bonus languages have full vocab but
**no `morphology-<code>.el` in ELP yet** (Python realizer exists; `.el` port is
the remaining engine work).
Construction coverage (separate from lexicon): French realizer was ~55%,
Semitic ~3% in the prototypes — full construction coverage remains its own task.
File diff suppressed because one or more lines are too long
-23
View File
@@ -1,23 +0,0 @@
{
"dataset": "british-rp-accent-transform",
"primitive_type": "accent_target",
"accent": "british-rp",
"grounding": "derived",
"provenance": "HONEST-DERIVED, COARSE FIRST PASS — NOT transcribed measured RP formants. The exact measured RP/GB tables (Deterding 1997 JIPA 27:47-55; Hawkins & Midgley 2005 JIPA 35:183-199) are the intended ground truth but were gated/figure-only at author time and were NOT transcribed. So these targets are DERIVED: each = the corresponding MEASURED Peterson&Barney(1952) base vowel transformed under the documented, citable RP-vs-GA structural rules of Wells (1982) 'Accents of English' — non-rhoticity (NURSE de-rhoticized: remove low F3), TRAP F2-lowering, LOT/THOUGHT back-rounding (F2 down), GOOSE-fronting (F2 up), GOAT centering. Shift MAGNITUDES are coarse/approximate (first pass), directions are cited. ground:derived (base measured + rule cited). Refine by transcribing Deterding/Hawkins&Midgley. No number is presented as a measured RP value it is not.",
"notes": "records with kind=vowel_override REPLACE the base phoneme's formant targets with the DERIVED RP realization. records with kind=rule encode non-formant transforms (non-rhoticity: drop post-vocalic coda /r/). The render composes: base geometry then accent override + rhoticity rule — voice + accent, separable.",
"records": [
{"key": "IY", "features": {"kind": "vowel_override", "set": "FLEECE"}, "attributes": {"f1": 280, "f2": 2249, "f3": 3000}},
{"key": "IH", "features": {"kind": "vowel_override", "set": "KIT"}, "attributes": {"f1": 360, "f2": 2100, "f3": 2550}},
{"key": "EH", "features": {"kind": "vowel_override", "set": "DRESS"}, "attributes": {"f1": 560, "f2": 1970, "f3": 2480}},
{"key": "AE", "features": {"kind": "vowel_override", "set": "TRAP"}, "attributes": {"f1": 730, "f2": 1590, "f3": 2410}},
{"key": "AA", "features": {"kind": "vowel_override", "set": "LOT"}, "attributes": {"f1": 560, "f2": 920, "f3": 2440}},
{"key": "AO", "features": {"kind": "vowel_override", "set": "THOUGHT"}, "attributes": {"f1": 415, "f2": 700, "f3": 2410}},
{"key": "UH", "features": {"kind": "vowel_override", "set": "FOOT"}, "attributes": {"f1": 380, "f2": 1100, "f3": 2240}},
{"key": "UW", "features": {"kind": "vowel_override", "set": "GOOSE"}, "attributes": {"f1": 310, "f2": 1650, "f3": 2240}},
{"key": "AH", "features": {"kind": "vowel_override", "set": "STRUT"}, "attributes": {"f1": 680, "f2": 1180, "f3": 2390}},
{"key": "ER", "features": {"kind": "vowel_override", "set": "NURSE", "rhotic": "no"}, "attributes": {"f1": 550, "f2": 1500, "f3": 2500}},
{"key": "AX", "features": {"kind": "vowel_override", "set": "commA"}, "attributes": {"f1": 500, "f2": 1500, "f3": 2500}},
{"key": "OW", "features": {"kind": "vowel_override", "set": "GOAT"}, "attributes": {"f1": 450, "f2": 1400, "f3": 2380}},
{"key": "R", "features": {"kind": "rule", "rule": "non_rhotic"}, "attributes": {"drop_coda_r": 1}}
]
}
-26
View File
@@ -1,26 +0,0 @@
# british-rp-accent TRANSFORM — INGESTIBLE DATA (a geometry/transform composed
# onto the base General-American phoneme targets; voice + accent, separable).
#
# PROVENANCE — HONEST, COARSE FIRST PASS. These are DERIVED targets, NOT
# transcribed measured RP formants. Measured RP tables (Deterding 1997 JIPA 27;
# Hawkins & Midgley 2005 JIPA 35) are the intended ground truth but were gated at
# author time and NOT transcribed. Each target = the MEASURED Peterson&Barney
# (1952) base vowel transformed under the documented, citable RP-vs-GA structural
# rules of Wells (1982): non-rhoticity, TRAP F2-lowering, LOT/THOUGHT back-
# rounding, GOOSE-fronting, GOAT centering, NURSE de-rhoticization. Shift
# magnitudes are coarse/approximate; directions are cited. ground=derived.
# Refine by transcribing the measured RP tables. No value is claimed as measured.
# Format: KEY|F1|F2|F3|KIND|SET
IY|280|2249|3000|vowel_override|FLEECE
IH|360|2100|2550|vowel_override|KIT
EH|560|1970|2480|vowel_override|DRESS
AE|730|1590|2410|vowel_override|TRAP
AA|560|920|2440|vowel_override|LOT
AO|415|700|2410|vowel_override|THOUGHT
UH|380|1100|2240|vowel_override|FOOT
UW|310|1650|2240|vowel_override|GOOSE
AH|680|1180|2390|vowel_override|STRUT
ER|550|1500|2500|vowel_override|NURSE-nonrhotic
AX|500|1500|2500|vowel_override|commA
OW|450|1400|2380|vowel_override|GOAT
R|0|0|0|rule|non_rhotic_drop_coda
-20
View File
@@ -1,20 +0,0 @@
# pronunciation lexicon SOURCE — word -> phoneme sequence, as INGESTIBLE DATA.
# Pronunciation is linguistic KNOWLEDGE (the language faculty's orthography->
# phonology map), ingested into the engram, not frozen in code. The render reads
# a word's phoneme sequence back from the engram. Covers the self-lexicon and the
# proof sentences; general G2P is the realizer/morphology faculty's remit.
# Diphthongs are written as two vowel targets (the render's transitions glide
# between them). Format: word|PH1 PH2 PH3 ...
i|AA IY
am|AE M
neuron|N UW R AA N
is|IH Z
memory|M EH M ER IY
hello|HH EH L OW
the|DH AH
a|AH
remember|R IH M EH M ER
i'm|AA IY M
you|Y UW
here|HH IY R
will|W IH L
File diff suppressed because one or more lines are too long
-528
View File
@@ -1,528 +0,0 @@
{
"dataset": "english-phoneme-formants",
"primitive_type": "phoneme",
"grounding": "extracted",
"provenance": "AUDITED per-field. The 10 monophthong-vowel F1/F2/F3 (IY,IH,EH,AE,AA,AO,UH,UW,AH,ER) are the MEASURED adult-male /hVd/ means of Peterson & Barney (1952) JASA 24:175-184, verified vs CRAN phonTools::pb52. AX=neutral uniform-tube resonances (Fant, physics). OW steady target = synthesis convention (diphthong). Consonant loci (M,N,NG,L,R,W,Y,Z,DH,V,S,F,HH) and ALL bandwidths + dur/amp = standard formant-synthesis conventions (Klatt 1980 JASA 67:971), engineering defaults NOT field measurements. No numbers invented/LLM-generated.",
"records": [
{
"key": "IY",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 270,
"f2": 2290,
"f3": 3010,
"bw1": 60,
"bw2": 90,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 130,
"amp": 100
}
},
{
"key": "IH",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 390,
"f2": 1990,
"f3": 2550,
"bw1": 70,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 110,
"amp": 100
}
},
{
"key": "EH",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 530,
"f2": 1840,
"f3": 2480,
"bw1": 80,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 130,
"amp": 100
}
},
{
"key": "AE",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 660,
"f2": 1720,
"f3": 2410,
"bw1": 90,
"bw2": 110,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 150,
"amp": 100
}
},
{
"key": "AA",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 730,
"f2": 1090,
"f3": 2440,
"bw1": 90,
"bw2": 110,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 150,
"amp": 100
}
},
{
"key": "AO",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 570,
"f2": 840,
"f3": 2410,
"bw1": 80,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 140,
"amp": 100
}
},
{
"key": "UH",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 440,
"f2": 1020,
"f3": 2240,
"bw1": 70,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 110,
"amp": 100
}
},
{
"key": "UW",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 300,
"f2": 870,
"f3": 2240,
"bw1": 70,
"bw2": 90,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 140,
"amp": 100
}
},
{
"key": "AH",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 640,
"f2": 1190,
"f3": 2390,
"bw1": 80,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 110,
"amp": 95
}
},
{
"key": "ER",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 490,
"f2": 1350,
"f3": 1690,
"bw1": 80,
"bw2": 100,
"bw3": 120,
"voiced": 1,
"nasal": 0,
"dur": 140,
"amp": 95
}
},
{
"key": "AX",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 500,
"f2": 1500,
"f3": 2500,
"bw1": 80,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 80,
"amp": 85
}
},
{
"key": "OW",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 490,
"f2": 910,
"f3": 2380,
"bw1": 80,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 140,
"amp": 100
}
},
{
"key": "M",
"features": {
"manner": "nasal",
"voiced": "yes",
"nasal": "yes"
},
"attributes": {
"f1": 250,
"f2": 900,
"f3": 2200,
"bw1": 90,
"bw2": 120,
"bw3": 180,
"voiced": 1,
"nasal": 1,
"dur": 80,
"amp": 60
}
},
{
"key": "N",
"features": {
"manner": "nasal",
"voiced": "yes",
"nasal": "yes"
},
"attributes": {
"f1": 250,
"f2": 1700,
"f3": 2600,
"bw1": 90,
"bw2": 120,
"bw3": 180,
"voiced": 1,
"nasal": 1,
"dur": 80,
"amp": 60
}
},
{
"key": "NG",
"features": {
"manner": "nasal",
"voiced": "yes",
"nasal": "yes"
},
"attributes": {
"f1": 250,
"f2": 2300,
"f3": 2700,
"bw1": 90,
"bw2": 120,
"bw3": 180,
"voiced": 1,
"nasal": 1,
"dur": 80,
"amp": 60
}
},
{
"key": "L",
"features": {
"manner": "approximant",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 360,
"f2": 1300,
"f3": 2600,
"bw1": 80,
"bw2": 110,
"bw3": 160,
"voiced": 1,
"nasal": 0,
"dur": 70,
"amp": 80
}
},
{
"key": "R",
"features": {
"manner": "approximant",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 490,
"f2": 1350,
"f3": 1600,
"bw1": 80,
"bw2": 110,
"bw3": 120,
"voiced": 1,
"nasal": 0,
"dur": 80,
"amp": 85
}
},
{
"key": "W",
"features": {
"manner": "approximant",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 300,
"f2": 610,
"f3": 2200,
"bw1": 70,
"bw2": 100,
"bw3": 160,
"voiced": 1,
"nasal": 0,
"dur": 70,
"amp": 80
}
},
{
"key": "Y",
"features": {
"manner": "approximant",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 270,
"f2": 2290,
"f3": 3010,
"bw1": 60,
"bw2": 90,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 60,
"amp": 80
}
},
{
"key": "Z",
"features": {
"manner": "fricative",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 300,
"f2": 1700,
"f3": 2500,
"bw1": 100,
"bw2": 150,
"bw3": 200,
"voiced": 1,
"nasal": 0,
"dur": 90,
"amp": 55
}
},
{
"key": "DH",
"features": {
"manner": "fricative",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 300,
"f2": 1400,
"f3": 2500,
"bw1": 100,
"bw2": 150,
"bw3": 200,
"voiced": 1,
"nasal": 0,
"dur": 70,
"amp": 55
}
},
{
"key": "V",
"features": {
"manner": "fricative",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 300,
"f2": 1000,
"f3": 2300,
"bw1": 100,
"bw2": 150,
"bw3": 200,
"voiced": 1,
"nasal": 0,
"dur": 70,
"amp": 55
}
},
{
"key": "S",
"features": {
"manner": "fricative",
"voiced": "no",
"nasal": "no"
},
"attributes": {
"f1": 320,
"f2": 1700,
"f3": 2500,
"bw1": 200,
"bw2": 200,
"bw3": 250,
"voiced": 0,
"nasal": 0,
"dur": 110,
"amp": 45
}
},
{
"key": "F",
"features": {
"manner": "fricative",
"voiced": "no",
"nasal": "no"
},
"attributes": {
"f1": 300,
"f2": 1200,
"f3": 2400,
"bw1": 200,
"bw2": 200,
"bw3": 250,
"voiced": 0,
"nasal": 0,
"dur": 100,
"amp": 40
}
},
{
"key": "HH",
"features": {
"manner": "fricative",
"voiced": "no",
"nasal": "no"
},
"attributes": {
"f1": 500,
"f2": 1500,
"f3": 2500,
"bw1": 200,
"bw2": 250,
"bw3": 300,
"voiced": 0,
"nasal": 0,
"dur": 70,
"amp": 40
}
},
{
"key": "SIL",
"features": {
"manner": "silence",
"voiced": "no",
"nasal": "no"
},
"attributes": {
"f1": 500,
"f2": 1500,
"f3": 2500,
"bw1": 100,
"bw2": 100,
"bw3": 100,
"voiced": 0,
"nasal": 0,
"dur": 55,
"amp": 0
}
}
]
}
-45
View File
@@ -1,45 +0,0 @@
# acoustic-phonetics SOURCE — the learned speech primitives, as INGESTIBLE DATA.
# NOT audio, NOT code: formant geometry of the phonemes, to be ingested via the
# ingest organ into the engram as a phoneme manifold. The render reads this
# geometry back from the engram; nothing is frozen in EL code.
#
# PROVENANCE (audited, per-field honesty — no invented numbers):
# * The 10 MONOPHTHONG VOWEL formants F1/F2/F3 (IY,IH,EH,AE,AA,AO,UH,UW,AH,ER)
# are the MEASURED adult-male means of Peterson & Barney (1952), JASA 24:175-184
# — the canonical /hVd/ table, verified digit-for-digit vs CRAN phonTools::pb52.
# These are real measured values.
# * AX (schwa) F1/F2/F3 = neutral uniform-tube resonances (2n-1)*500 — a PHYSICS
# value (Fant), not a P&B measurement.
# * OW is a diphthong; its listed steady target is a conventional synthesis value,
# not a P&B monophthong measurement.
# * CONSONANT loci (M,N,NG,L,R,W,Y,Z,DH,V,S,F,HH) and ALL BANDWIDTHS (B1,B2,B3)
# and dur/amp are STANDARD FORMANT-SYNTHESIS conventions (Klatt 1980, JASA 67:971
# "Software for a cascade/parallel formant synthesizer") — engineering defaults,
# NOT per-phoneme field measurements. Labeled as such, not attributed to P&B.
# Format: SYM|F1|F2|F3|B1|B2|B3|voiced|nasal|dur_ms|amp|class|example
IY|270|2290|3010|60|90|150|1|0|130|100|vowel|beet
IH|390|1990|2550|70|100|150|1|0|110|100|vowel|bit
EH|530|1840|2480|80|100|150|1|0|130|100|vowel|bet
AE|660|1720|2410|90|110|150|1|0|150|100|vowel|bat
AA|730|1090|2440|90|110|150|1|0|150|100|vowel|bot
AO|570|840|2410|80|100|150|1|0|140|100|vowel|bought
UH|440|1020|2240|70|100|150|1|0|110|100|vowel|book
UW|300|870|2240|70|90|150|1|0|140|100|vowel|boot
AH|640|1190|2390|80|100|150|1|0|110|95|vowel|but
ER|490|1350|1690|80|100|120|1|0|140|95|vowel|bird
AX|500|1500|2500|80|100|150|1|0|80|85|vowel|about
OW|490|910|2380|80|100|150|1|0|140|100|vowel|boat
M|250|900|2200|90|120|180|1|1|80|60|nasal|map
N|250|1700|2600|90|120|180|1|1|80|60|nasal|nap
NG|250|2300|2700|90|120|180|1|1|80|60|nasal|sing
L|360|1300|2600|80|110|160|1|0|70|80|approximant|lip
R|490|1350|1600|80|110|120|1|0|80|85|approximant|rip
W|300|610|2200|70|100|160|1|0|70|80|approximant|wet
Y|270|2290|3010|60|90|150|1|0|60|80|approximant|yet
Z|300|1700|2500|100|150|200|1|0|90|55|fricative|zoo
DH|300|1400|2500|100|150|200|1|0|70|55|fricative|the
V|300|1000|2300|100|150|200|1|0|70|55|fricative|van
S|320|1700|2500|200|200|250|0|0|110|45|fricative|see
F|300|1200|2400|200|200|250|0|0|100|40|fricative|fee
HH|500|1500|2500|200|250|300|0|0|70|40|fricative|hat
SIL|500|1500|2500|100|100|100|0|0|55|0|silence|_

Some files were not shown because too many files have changed in this diff Show More