Compare commits

..

14 Commits

Author SHA1 Message Date
Will Anderson e858eab300 spec: update codegen-js.md to Phase 5, ~90% coverage
Status updated from Phase 4 ~80% to Phase 5 ~90%.

New sections:
- 7. Language features coverage table (supported vs stubbed)
- 7a. Phase 5 constructs: extern fn, anonymous functions, try/catch,
  method call on Any, URL imports -- each with emit shape examples
- 9. Roadmap updated: Phases 1-5 marked DONE, Phase 6 unblocked

Runtime builtin table updated to ~90 builtins including all Phase 5
additions (promise_then/catch/resolve/reject, object_assign/keys/values,
json_deep_clone, array_from, type_of, instanceof_check).
2026-05-04 11:03:47 -05:00
Will Anderson aa7d97d5ba examples: rewrite browser-auth.el using new language features
No native_js or native_js_call anywhere. Full browser auth flow expressed
with proper El constructs:

- extern fn supabase_create_client(url, key) -> Any
  Declares the Supabase CDN global without an El function body.

- client.auth.signInWithOtp(opts)
  Direct method call chain on Any-typed value. The client is built by
  calling the extern fn; .auth field access and .signInWithOtp(opts)
  method call emit clean JS without any escape hatch.

- try { ... } catch (err: Any) { ... }
  Wraps the auth call; unexpected runtime errors are caught and shown
  to the user rather than crashing silently.

- fn(event: Any) -> Void { ... }
  Inline anonymous function literals for DOM event listeners instead
  of named forward-declared callbacks.

The rewrite is the proof: every browser JavaScript pattern used in a
real auth flow can now be expressed structurally in El.
2026-05-04 11:02:13 -05:00
Will Anderson 7040830470 codegen-js: URL import declarations for JS modules
import "https://cdn.example.com/lib.js" now emits:
  - module mode: import "https://..." at the top of the generated file
  - bundle/IIFE mode: // external: https://... comment

El source imports (.el files) are excluded -- they were already inlined
by resolve_imports before codegen. Any import path that doesn't end in
.el or starts with http(s):// is treated as an external JS dependency.
2026-05-04 11:01:36 -05:00
Will Anderson 3a513aaa5a runtime + codegen-js: Promise helpers and object/array utilities
Add to el_runtime.js:
  promise_then(p, cb)    -- p.then(cb), works with any Promise-returning API
  promise_catch(p, cb)   -- p.catch(cb)
  promise_resolve(val)   -- Promise.resolve(val)
  promise_reject(msg)    -- Promise.reject(new Error(msg))
  object_assign(t, s)    -- Object.assign({}, t, s) (non-mutating)
  object_keys(obj)       -- Object.keys(obj)
  object_values(obj)     -- Object.values(obj)
  json_deep_clone(obj)   -- JSON.parse(JSON.stringify(obj))
  array_from(iterable)   -- Array.from(iterable)
  type_of(val)           -- typeof val
  instanceof_check(v, n) -- val instanceof globalThis[name]

All new functions added to __el export object and ES named exports.
codegen-js preamble destructure updated to include all new names.
2026-05-04 11:01:14 -05:00
Will Anderson beb2a8c5bd lexer + parser + codegen: try/catch statement
try { ... } catch (name: Type) { ... } is now a first-class El statement.

Lexer: `try` and `catch` are now keywords (Try, Catch token kinds).
Parser: TryCatch AST node with try_body, catch_name, catch_body.
codegen-js: emits try { ... } catch (name) { ... } directly -- correct
  for all browser error handling patterns.
codegen.el (C backend): emits the try body with a comment; exception
  handling is a no-op since C has no analogous mechanism. Programs using
  try/catch should compile with --target=js.

The catch variable type annotation is parsed and skipped (same treatment
as all other type annotations in El).
2026-05-04 11:00:24 -05:00
Will Anderson e23319fe0b parser + codegen-js: anonymous function literals (lambda syntax)
fn(params) -> RetType { body } is now valid in expression position.
The parser produces a Lambda AST node. codegen-js emits a hoisted
JS function declaration with a generated name (__lambda_N) and returns
the name as the expression value, so inline callbacks compose cleanly:

  dom_listen(btn, "click", fn(event: Any) -> Void { handle(event) })

emits:

  function __lambda_1(event) { handle(event); }
  dom_listen(btn, "click", __lambda_1);

The hoisted-declaration strategy is debuggable, has no closure-capture
issues, and requires no string-buffer mode in the codegen.
2026-05-04 10:59:17 -05:00
Will Anderson 01fee9396a codegen-js: native JS method dispatch and extern fn support
Any-typed receiver method calls now emit obj.method(args) directly
instead of requiring native_js_call. client.auth.signInWithOtp(p)
compiles to client["auth"].signInWithOtp(p) -- no escape hatch needed.

Field access emits obj["field"] (direct bracket notation) instead of
el_get_field, so prototype-inherited JS properties resolve correctly.
el_get_field's hasOwnProperty guard was silently returning null for
real JS objects with inherited fields (Supabase auth, DOM APIs, etc).

El runtime shortform methods (append, len, get, map_get, map_set)
still use the existing method(obj, args) convention for backward compat.

ExternFn statements emit a comment and are excluded from top-level
statement codegen -- the extern declaration tells the compiler the
function exists in the JS environment without emitting a body.
2026-05-04 10:58:07 -05:00
Will Anderson 7b60d94b8a add --minify and --obfuscate flags to elc JS pipeline
Adds two post-processing flags that produce production-ready browser JS in a
single elc invocation, replacing extract-js.py in the web product pipeline:

  elc --target=js --bundle --minify source.el > output.min.js
  elc --target=js --bundle --obfuscate source.el > output.obf.js

--minify shells out to terser (passes=2, no drop_console, drop_debugger).
--obfuscate shells out to javascript-obfuscator with the same options as the
old extract-js.py script. --obfuscate implies --minify.

Tool discovery: checks ./node_modules/.bin/, ../node_modules/.bin/ (monorepo),
then falls back to npx. Both flags require --target=js; passing either without
it exits 1 with a clear error.

Both tools receive a reserved-names list of globals referenced from HTML
onclick= attributes (neuronDemoToggle, signInWith, NEURON_CFG, etc.) so they
are not mangled.

Implementation adds stdout_to_file(path)/stdout_restore() builtins to the C
runtime so codegen's println-streamed output can be captured to a temp file
before being piped through the external tools. Temp files use
/tmp/elc-<pid>-<timestamp>.js naming and are cleaned up on success and failure.

Rebuilds dist/platform/elc and dist/platform/elc.c. Self-hosting verified.
2026-05-04 10:54:34 -05:00
Will Anderson 21694b79d2 implement ? nil-propagation, write browser-auth.el example, update spec
Iteration 5:

? nil-propagation: Field and Index handlers in js_cg_expr now detect when
the object expression is a Try node (the AST node for postfix `?`).
When detected, emit JS optional chaining: `(expr)?.["field"] ?? null`.
The `?? null` normalizes JS undefined to El's null. A bare `expr?` not
followed by field/index still passes through unchanged.

browser-auth.el: a realistic 130-line example demonstrating:
  - @async function with Supabase via native_js_call
  - DOM bridge: get/set value/text/attr, add/remove class, show/hide
  - local_storage_get/set for session hints
  - window_on_load for initialization
  - window_set to expose functions to the browser global scope
  - set_timeout for transient state, is_valid_email for input validation
  Compiles cleanly with elc --target=js --bundle

Spec updated: status promoted to Phase 4 / ~80% coverage, nil-prop
status updated, new example referenced.
2026-05-04 10:42:54 -05:00
Will Anderson 422442b14e add --bundle flag for self-contained IIFE output
elc --target=js --bundle source.el > output.js produces a single file
with no import statement that can drop directly into a <script> tag.

How it works:
  - detect_bundle() reads the --bundle flag from argv
  - resolve_runtime_path() looks for el_runtime.js next to the source file
  - compile_js_with_bundle() reads the runtime, calls codegen_js_bundle()
  - codegen_js_inner(bundle_mode=true):
    - emits ;(function() { "use strict"; at the top
    - inlines the runtime content (stripping ES export statements which
      are invalid inside an IIFE via js_strip_es_exports())
    - skips the const {...} = globalThis.__el destructure -- the inlined
      function declarations are already in scope within the IIFE
    - closes with })(); after main()

Usage: elc --target=js --bundle app.el > app.js
       Place el_runtime.js in the same directory as app.el.
2026-05-04 10:40:46 -05:00
Will Anderson 437ba0a4dd add 20 browser API builtins to JS runtime and codegen preamble
Iteration 3: closes the browser API gap needed for real web pages.

New builtins in el_runtime.js:
  Extended DOM: dom_set_attr, dom_get_attr, dom_remove_attr, dom_set_html,
    dom_get_html, dom_get_parent, dom_contains_class, dom_get_checked,
    dom_set_checked
  Timers: set_timeout, set_interval, clear_interval
  Local storage: local_storage_get, local_storage_set, local_storage_remove
  Window: window_location, window_redirect, window_on_load
  Debug: console_log

All browser-only functions use _ensureBrowser guard. Timer functions
work in both Node and browser. All new names added to __el export
object, ES named exports, and codegen-js.el destructure preamble.
Spec table updated to document new categories.
2026-05-04 10:38:20 -05:00
Will Anderson 7376349124 fix TypeDef parser to consume optional = before field block
type User = { name: String } was silently broken: the parser consumed
the type name then called expect(LBrace) while sitting on the = token.
expect() advances unconditionally on mismatch, so it consumed = and
treated { as the first field name, producing a corrupt TypeDef node.

The FnDef following the broken TypeDef was then parsed incorrectly or
lost entirely -- causing greet() and similar functions to vanish from
JS/C output with no error.

Fix: detect and skip the optional Eq token before expecting LBrace.
Both targets benefit; rebuild elc to pick up the fix.
2026-05-04 10:36:53 -05:00
Will Anderson 0f1da43a97 implement Enum::Variant match patterns in parser and both codegens
Parser now handles `SomeEnum::Variant` in match arm patterns, emitting
a Variant pattern node with enum_name and variant fields. Previously
these fell through to Binding, producing broken codegen.

JS codegen: emit str_eq check against the variant name string (El enums
are plain strings at runtime). C codegen: same, via EL_STR + str_eq.

Rebuild elc to pick up the parser change.
2026-05-04 10:35:35 -05:00
Will Anderson a54b2bebf9 add DOM bridge, async/await, window export, and native_js to JS target
- el_runtime.js: add 19 dom_* builtins (browser-only, throw in Node),
  window_set/window_get for exposing El functions to the browser global
  scope, and native_js/native_js_call escape hatches for third-party libs
- codegen-js.el: destructure all new builtins in generated preamble; add
  @async decorator support that emits async function + await at call sites
  for known-async HTTP builtins and user-declared @async functions; pre-
  registration pass ensures forward calls to @async functions get await
- spec/codegen-js.md: mark Phase 3 (DOM bridge) implemented, document
  @async approach and its limitations, update builtin table and status
- examples/browser-counter.el: canonical example showing dom_get_element,
  dom_set_text, dom_is_null, window_set, and state_set/get
2026-05-04 10:29:43 -05:00
727 changed files with 3212 additions and 1106794 deletions
+39 -265
View File
@@ -1,4 +1,4 @@
name: El SDK CI - dev name: El CI dev
on: on:
push: push:
@@ -11,9 +11,6 @@ on:
jobs: jobs:
build-and-test: build-and-test:
runs-on: ubuntu-latest runs-on: ubuntu-latest
defaults:
run:
working-directory: lang
steps: steps:
- name: Checkout - name: Checkout
@@ -22,306 +19,83 @@ jobs:
- name: Install build dependencies - name: Install build dependencies
run: | run: |
apt-get update -qq apt-get update -qq
apt-get install -y gcc libcurl4-openssl-dev apt-transport-https ca-certificates apt-get install -y gcc libcurl4-openssl-dev
echo "deb [trusted=yes] https://packages.cloud.google.com/apt cloud-sdk main" \
> /etc/apt/sources.list.d/google-cloud-sdk.list
apt-get update -qq && apt-get install -y google-cloud-cli
# Seed: use the committed linux-amd64 binary as the bootstrap # Gen2: compile the bootstrap C source into a working elc binary
- name: Bootstrap from committed linux binary (seed) - name: Build elc from bootstrap (gen2)
run: | run: |
chmod +x dist/platform/elc-linux-amd64
echo "seed elc (committed linux-amd64 binary)"
dist/platform/elc-linux-amd64 --version || true
# Gen2: use seed to self-host compile the El compiler
- name: Self-host compile El compiler (gen2)
run: |
dist/platform/elc-linux-amd64 elc-cli.el > dist/elc-gen2.c
gcc -O2 \ gcc -O2 \
-I runtime \ -I el-compiler/runtime \
dist/elc-gen2.c \ dist/elc-bootstrap.c \
runtime/el_runtime.c \ el-compiler/runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \ -lcurl -lpthread \
-o dist/elc-gen2
chmod +x dist/elc-gen2
echo "gen2 elc built"
dist/elc-gen2 --version || true
# Gen3: use gen2 to compile the El compiler from its own El source (self-host)
- name: Self-host: compile El compiler with gen2 (gen3)
run: |
mkdir -p dist/platform
dist/elc-gen2 el-compiler/src/compiler.el > dist/elc-gen3.c
gcc -O2 \
-I el-compiler/runtime \
dist/elc-gen3.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lpthread \
-o dist/platform/elc -o dist/platform/elc
chmod +x dist/platform/elc chmod +x dist/platform/elc
echo "gen2 (self-hosted) elc built" echo "gen3 (self-hosted) elc built"
dist/platform/elc --version || true dist/platform/elc --version || true
# Build elb (needed for Artifact Registry publish and downstream CI) # Run all four test suites — all must pass
- name: Build elb - name: Run tests — text
run: |
mkdir -p dist/bin
dist/platform/elc elb.el > dist/elb.c
gcc -O2 \
-I runtime \
dist/elb.c \
runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/bin/elb
chmod +x dist/bin/elb
echo "elb built"
- name: Run tests - text
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/text/run.sh bash tests/text/run.sh
- name: Run tests - calendar - name: Run tests calendar
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/calendar/run.sh bash tests/calendar/run.sh
- name: Run tests - time - name: Run tests time
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/time/run.sh bash tests/time/run.sh
- name: Run tests - html_sanitizer - name: Run tests html_sanitizer
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/html_sanitizer/run.sh bash tests/html_sanitizer/run.sh
# Native El test suites (elc --test, compile-link-run) # Publish artifact to GCP Artifact Registry (dev)
# el_runtime.c is precompiled to .o once and reused by all 8 modules. - name: Publish elc to Artifact Registry (dev)
- name: Precompile el_runtime.o
run: |
set -euo pipefail
RUNTIME="$(pwd)/runtime"
gcc -O2 -c -I "$RUNTIME" "$RUNTIME/el_runtime.c" \
-o /tmp/el_runtime.o
echo "el_runtime.o compiled"
- name: Run tests - native (core)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_core.el > /tmp/el_native_core.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_core.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_core
/tmp/el_native_core
- name: Run tests - native (text)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_text.el > /tmp/el_native_text.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_text.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_text
/tmp/el_native_text
- name: Run tests - native (string)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_string.el > /tmp/el_native_string.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_string.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_string
/tmp/el_native_string
- name: Run tests - native (math)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_math.el > /tmp/el_native_math.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_math.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_math
/tmp/el_native_math
- name: Run tests - native (state)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_state.el > /tmp/el_native_state.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_state.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_state
/tmp/el_native_state
- name: Run tests - native (time)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_time.el > /tmp/el_native_time.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_time.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_time
/tmp/el_native_time
- name: Run tests - native (json)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_json.el > /tmp/el_native_json.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_json.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_json
/tmp/el_native_json
- name: Run tests - native (env)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_env.el > /tmp/el_native_env.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_env.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_env
/tmp/el_native_env
- name: Run tests - native (fs)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_fs.el > /tmp/el_native_fs.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_fs.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_fs
/tmp/el_native_fs
# Build epm binary using elb (epm lives at repo root, not inside lang/)
- name: Build epm
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd ../epm && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/epm
echo "epm built"
# Build el-install binary using elb
- name: Build el-install
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd tools/install && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/el-install
echo "el-install built"
# Publish only after merge (push event), not on PR validation runs
- name: Publish El SDK to Artifact Registry (dev)
if: github.event_name == 'push'
env: env:
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }} GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
run: | run: |
# Fail loudly: previously this step had no `set -e`, so an auth or
# upload failure was swallowed (step exited 0 on the trailing echo)
# and the SDK silently never published. Surface failures now.
set -euo pipefail
if [ -z "${GCP_SA_KEY:-}" ]; then
echo "FATAL: GCP_SA_KEY secret is empty — cannot authenticate to publish" >&2
exit 1
fi
echo "${GCP_SA_KEY}" > /tmp/gcp-key.json echo "${GCP_SA_KEY}" > /tmp/gcp-key.json
apt-get install -y -qq apt-transport-https ca-certificates gnupg curl
curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" > /etc/apt/sources.list.d/google-cloud-sdk.list
apt-get update -qq && apt-get install -y google-cloud-cli
gcloud auth activate-service-account --key-file=/tmp/gcp-key.json gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
gcloud config set project neuron-785695 gcloud config set project neuron-785695
echo "Publishing as active account: $(gcloud config get-value account 2>/dev/null)"
VERSION="${GITHUB_SHA:0:8}"
VERSION="${GITEA_SHA:0:8}"
gcloud artifacts generic upload \ gcloud artifacts generic upload \
--repository=foundation-dev \ --repository=foundation-dev \
--location=us-central1 \ --location=us-central1 \
--project=neuron-785695 \ --project=neuron-785695 \
--package=el-elc \ --package=el/elc \
--version="${VERSION}" \ --version="${VERSION}" \
--source=dist/platform/elc --source=dist/platform/elc
gcloud artifacts generic upload \ # Also tag as latest-dev
--repository=foundation-dev \ echo "Published elc version=${VERSION} to foundation-dev/el/elc"
--location=us-central1 \
--project=neuron-785695 \
--package=el-elb \
--version="${VERSION}" \
--source=dist/bin/elb
gcloud artifacts generic upload \
--repository=foundation-dev \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-c \
--version="${VERSION}" \
--source=runtime/el_runtime.c
gcloud artifacts generic upload \
--repository=foundation-dev \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-h \
--version="${VERSION}" \
--source=runtime/el_runtime.h
gcloud artifacts generic upload \
--repository=foundation-dev \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-js \
--version="${VERSION}" \
--source=runtime/el_runtime.js
echo "Published El SDK version=${VERSION} to foundation-dev"
# Keep key alive for the ci-base rebuild step below
# (deleted in that step after docker push)
- name: Rebuild ci-base with fresh El SDK (dev)
# Patches ci-base:dev in-place: pulls the existing image (which has all
# system deps — Node, Go, gcloud, Docker CLI, etc.) and overlays the freshly
# built El SDK on top. Keeps the full ci-base rebuild fast and incremental.
#
# continue-on-error: this is a CI-cache optimization, NOT the release
# artifact. It runs Docker (pull/build/push ~600MB) on the host-mode GCE
# runner where DinD/Docker availability is fragile. A failure here must
# never block or redden the job — the SDK publish above is the deliverable.
continue-on-error: true
if: github.event_name == 'push'
env:
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
run: |
set -euo pipefail
CI_BASE="us-central1-docker.pkg.dev/neuron-785695/neuron-ci/ci-base"
SHA="${GITHUB_SHA:0:8}"
echo "${GCP_SA_KEY}" > /tmp/gcp-key.json
gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
gcloud config set project neuron-785695
gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
# Pull existing ci-base:dev (or fall back to :latest on first run)
BASE_TAG="dev"
docker pull "${CI_BASE}:dev" || { docker pull "${CI_BASE}:latest" && BASE_TAG="latest"; }
# Inline Dockerfile — only replaces the El SDK layer
cat > /tmp/Dockerfile.ci-base-patch << 'EOF'
ARG BASE
FROM ${BASE}
COPY dist/platform/elc /opt/el/dist/platform/elc
COPY dist/bin/elb /opt/el/dist/bin/elb
COPY runtime/el_runtime.c /opt/el/runtime/el_runtime.c
COPY runtime/el_runtime.h /opt/el/runtime/el_runtime.h
COPY runtime/el_runtime.js /opt/el/runtime/el_runtime.js
RUN chmod +x /opt/el/dist/platform/elc /opt/el/dist/bin/elb
EOF
docker build \
--build-arg BASE="${CI_BASE}:${BASE_TAG}" \
--build-arg BUILDKIT_INLINE_CACHE=1 \
-f /tmp/Dockerfile.ci-base-patch \
-t "${CI_BASE}:dev" \
-t "${CI_BASE}:dev-${SHA}" \
.
docker push "${CI_BASE}:dev"
docker push "${CI_BASE}:dev-${SHA}"
echo "ci-base rebuilt: ${CI_BASE}:dev (${SHA})"
rm -f /tmp/gcp-key.json rm -f /tmp/gcp-key.json
+36 -247
View File
@@ -1,4 +1,4 @@
name: El SDK CI - stage name: El CI stage
on: on:
push: push:
@@ -11,301 +11,90 @@ on:
jobs: jobs:
build-and-test: build-and-test:
runs-on: ubuntu-latest runs-on: ubuntu-latest
defaults:
run:
working-directory: lang
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Enforce source branch (stage <- dev only)
if: github.event_name == 'pull_request'
run: |
SOURCE="${GITHUB_HEAD_REF}"
if [ "${SOURCE}" != "dev" ]; then
echo "ERROR: Stage branch only accepts PRs from 'dev'. Source was: '${SOURCE}'"
exit 1
fi
echo "Source branch check passed: ${SOURCE} -> stage"
- name: Install build dependencies - name: Install build dependencies
run: | run: |
apt-get update -qq apt-get update -qq
apt-get install -y gcc libcurl4-openssl-dev apt-get install -y gcc libcurl4-openssl-dev
# Seed: use the committed linux-amd64 binary as the bootstrap # Gen2: compile the bootstrap C source into a working elc binary
- name: Bootstrap from committed linux binary (seed) - name: Build elc from bootstrap (gen2)
run: | run: |
chmod +x dist/platform/elc-linux-amd64
echo "seed elc (committed linux-amd64 binary)"
dist/platform/elc-linux-amd64 --version || true
# Gen2: use seed to self-host compile the El compiler
- name: Self-host compile El compiler (gen2)
run: |
dist/platform/elc-linux-amd64 elc-cli.el > dist/elc-gen2.c
gcc -O2 \ gcc -O2 \
-I runtime \ -I el-compiler/runtime \
dist/elc-gen2.c \ dist/elc-bootstrap.c \
runtime/el_runtime.c \ el-compiler/runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \ -lcurl -lpthread \
-o dist/elc-gen2
chmod +x dist/elc-gen2
echo "gen2 elc built"
dist/elc-gen2 --version || true
# Gen3: use gen2 to compile the El compiler from its own El source (self-host)
- name: Self-host: compile El compiler with gen2 (gen3)
run: |
mkdir -p dist/platform
dist/elc-gen2 el-compiler/src/compiler.el > dist/elc-gen3.c
gcc -O2 \
-I el-compiler/runtime \
dist/elc-gen3.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lpthread \
-o dist/platform/elc -o dist/platform/elc
chmod +x dist/platform/elc chmod +x dist/platform/elc
echo "gen2 (self-hosted) elc built" echo "gen3 (self-hosted) elc built"
dist/platform/elc --version || true dist/platform/elc --version || true
- name: Run tests - text # Run all four test suites — all must pass
- name: Run tests — text
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/text/run.sh bash tests/text/run.sh
- name: Run tests - calendar - name: Run tests calendar
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/calendar/run.sh bash tests/calendar/run.sh
- name: Run tests - time - name: Run tests time
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/time/run.sh bash tests/time/run.sh
- name: Run tests - html_sanitizer - name: Run tests html_sanitizer
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/html_sanitizer/run.sh bash tests/html_sanitizer/run.sh
# Native El test suites (elc --test, compile-link-run) # Publish artifact to GCP Artifact Registry (stage)
- name: Run tests - native (core) - name: Publish elc to Artifact Registry (stage)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_core.el > /tmp/el_native_core.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_core.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_core
/tmp/el_native_core
- name: Run tests - native (text)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_text.el > /tmp/el_native_text.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_text.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_text
/tmp/el_native_text
- name: Run tests - native (string)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_string.el > /tmp/el_native_string.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_string.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_string
/tmp/el_native_string
- name: Run tests - native (math)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_math.el > /tmp/el_native_math.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_math.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_math
/tmp/el_native_math
- name: Run tests - native (state)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_state.el > /tmp/el_native_state.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_state.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_state
/tmp/el_native_state
- name: Run tests - native (time)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_time.el > /tmp/el_native_time.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_time.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_time
/tmp/el_native_time
- name: Run tests - native (json)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_json.el > /tmp/el_native_json.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_json.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_json
/tmp/el_native_json
- name: Run tests - native (env)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_env.el > /tmp/el_native_env.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_env.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_env
/tmp/el_native_env
- name: Run tests - native (fs)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_fs.el > /tmp/el_native_fs.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_fs.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_fs
/tmp/el_native_fs
# Build elb (needed for epm and el-install builds below)
- name: Build elb
run: |
mkdir -p dist/bin
dist/platform/elc elb.el > dist/elb.c
gcc -O2 \
-I runtime \
dist/elb.c \
runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/bin/elb
chmod +x dist/bin/elb
echo "elb built"
# Build epm binary using elb (epm lives at repo root, not inside lang/)
- name: Build epm
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd ../epm && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/epm
echo "epm built"
# Build el-install binary using elb
- name: Build el-install
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd tools/install && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/el-install
echo "el-install built"
# Publish only after merge (push event), not on PR validation runs
- name: Publish El SDK to Artifact Registry (stage)
if: github.event_name == 'push'
env: env:
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }} GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
run: | run: |
# Fail loudly: previously this step had no `set -e`, so an auth or
# upload failure was swallowed (step exited 0 on the trailing echo)
# and the SDK silently never published. Surface failures now.
set -euo pipefail
if [ -z "${GCP_SA_KEY:-}" ]; then
echo "FATAL: GCP_SA_KEY secret is empty — cannot authenticate to publish" >&2
exit 1
fi
echo "${GCP_SA_KEY}" > /tmp/gcp-key.json echo "${GCP_SA_KEY}" > /tmp/gcp-key.json
apt-get install -y -qq apt-transport-https ca-certificates curl apt-get install -y -qq apt-transport-https ca-certificates gnupg curl
echo "deb [trusted=yes] https://packages.cloud.google.com/apt cloud-sdk main" > /etc/apt/sources.list.d/google-cloud-sdk.list curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" > /etc/apt/sources.list.d/google-cloud-sdk.list
apt-get update -qq && apt-get install -y google-cloud-cli apt-get update -qq && apt-get install -y google-cloud-cli
gcloud auth activate-service-account --key-file=/tmp/gcp-key.json gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
gcloud config set project neuron-785695 gcloud config set project neuron-785695
echo "Publishing as active account: $(gcloud config get-value account 2>/dev/null)"
VERSION="${GITHUB_SHA:0:8}"
VERSION="${GITEA_SHA:0:8}"
gcloud artifacts generic upload \ gcloud artifacts generic upload \
--repository=foundation-stage \ --repository=foundation-stage \
--location=us-central1 \ --location=us-central1 \
--project=neuron-785695 \ --project=neuron-785695 \
--package=el-elc \ --package=el/elc \
--version="${VERSION}" \ --version="${VERSION}" \
--source=dist/platform/elc --source=dist/platform/elc
gcloud artifacts generic upload \ echo "Published elc version=${VERSION} to foundation-stage/el/elc"
--repository=foundation-stage \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-c \
--version="${VERSION}" \
--source=runtime/el_runtime.c
gcloud artifacts generic upload \
--repository=foundation-stage \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-h \
--version="${VERSION}" \
--source=runtime/el_runtime.h
echo "Published El SDK version=${VERSION} to foundation-stage"
# Keep key alive for the ci-base rebuild step below
# (deleted in that step after docker push)
- name: Rebuild ci-base with fresh El SDK (stage)
# Patches ci-base:stage in-place: pulls the existing image (which has all
# system deps — Node, Go, gcloud, Docker CLI, etc.) and overlays the freshly
# built El SDK on top. Keeps the full ci-base rebuild fast and incremental.
#
# continue-on-error: this is a CI-cache optimization, NOT the release
# artifact. It runs Docker (pull/build/push ~600MB) on the host-mode GCE
# runner where DinD/Docker availability is fragile. A failure here must
# never block or redden the job — the SDK publish above is the deliverable.
continue-on-error: true
if: github.event_name == 'push'
env:
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
run: |
set -euo pipefail
CI_BASE="us-central1-docker.pkg.dev/neuron-785695/neuron-ci/ci-base"
SHA="${GITHUB_SHA:0:8}"
echo "${GCP_SA_KEY}" > /tmp/gcp-key.json
gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
gcloud config set project neuron-785695
gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
# Pull existing ci-base:stage (system deps stay cached in the base layer)
docker pull "${CI_BASE}:stage" || docker pull "${CI_BASE}:latest"
# Inline Dockerfile — only replaces the El SDK layer
cat > /tmp/Dockerfile.ci-base-patch << 'EOF'
ARG BASE
FROM ${BASE}
COPY dist/platform/elc /opt/el/dist/platform/elc
COPY dist/bin/elb /opt/el/dist/bin/elb
COPY runtime/el_runtime.c /opt/el/runtime/el_runtime.c
COPY runtime/el_runtime.h /opt/el/runtime/el_runtime.h
COPY runtime/el_runtime.js /opt/el/runtime/el_runtime.js
RUN chmod +x /opt/el/dist/platform/elc /opt/el/dist/bin/elb
EOF
docker build \
--build-arg BASE="${CI_BASE}:stage" \
--build-arg BUILDKIT_INLINE_CACHE=1 \
-f /tmp/Dockerfile.ci-base-patch \
-t "${CI_BASE}:stage" \
-t "${CI_BASE}:stage-${SHA}" \
.
docker push "${CI_BASE}:stage"
docker push "${CI_BASE}:stage-${SHA}"
echo "ci-base rebuilt: ${CI_BASE}:stage (${SHA})"
rm -f /tmp/gcp-key.json rm -f /tmp/gcp-key.json
+69 -302
View File
@@ -4,236 +4,81 @@ on:
push: push:
branches: branches:
- main - main
pull_request:
branches:
- main
jobs: jobs:
build-and-release: build-and-release:
runs-on: ubuntu-latest runs-on: ubuntu-latest
defaults:
run:
working-directory: lang
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Enforce source branch (main <- stage only)
if: github.event_name == 'pull_request'
run: |
SOURCE="${GITHUB_HEAD_REF}"
if [ "${SOURCE}" != "stage" ]; then
echo "ERROR: Main branch only accepts PRs from 'stage'. Source was: '${SOURCE}'"
exit 1
fi
echo "Source branch check passed: ${SOURCE} -> main"
- name: Install build dependencies - name: Install build dependencies
run: | run: |
apt-get update -qq apt-get update -qq
apt-get install -y gcc libcurl4-openssl-dev apt-get install -y gcc libcurl4-openssl-dev
# Seed: use the committed linux-amd64 binary as the bootstrap # Gen2: compile the bootstrap C source into a working elc binary
- name: Bootstrap from committed linux binary (seed) - name: Build elc from bootstrap (gen2)
run: | run: |
chmod +x dist/platform/elc-linux-amd64 gcc -O2 \
echo "seed elc (committed linux-amd64 binary)" -I el-compiler/runtime \
dist/platform/elc-linux-amd64 --version || true dist/elc-bootstrap.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lpthread \
-o dist/elc-gen2
chmod +x dist/elc-gen2
echo "gen2 elc built"
dist/elc-gen2 --version || true
# Gen2: use seed to self-host compile the El compiler # Gen3: use gen2 to compile the El compiler from its own El source (self-host)
- name: Self-host compile El compiler (gen2) - name: Self-host: compile El compiler with gen2 (gen3)
run: | run: |
mkdir -p dist/platform mkdir -p dist/platform
dist/platform/elc-linux-amd64 elc-cli.el > dist/elc-gen2.c dist/elc-gen2 el-compiler/src/compiler.el > dist/elc-gen3.c
gcc -O2 \ gcc -O2 \
-I runtime \ -I el-compiler/runtime \
dist/elc-gen2.c \ dist/elc-gen3.c \
runtime/el_runtime.c \ el-compiler/runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \ -lcurl -lpthread \
-o dist/platform/elc -o dist/platform/elc
chmod +x dist/platform/elc chmod +x dist/platform/elc
echo "gen2 (self-hosted) elc built" echo "gen3 (self-hosted) elc built"
dist/platform/elc --version || true dist/platform/elc --version || true
# Build elb binary # Run all four test suites with gen3 elc
- name: Build elb - name: Run tests — text
run: |
mkdir -p dist/bin
dist/platform/elc elb.el > dist/elb.c
gcc -O2 \
-I runtime \
dist/elb.c \
runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/bin/elb
chmod +x dist/bin/elb
echo "elb built"
# Build epm binary using elb (epm lives at repo root, not inside lang/)
- name: Build epm
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd ../epm && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/epm
echo "epm built"
# Build el-install binary using elb
- name: Build el-install
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd tools/install && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/el-install
echo "el-install built"
- name: Run tests - text
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/text/run.sh bash tests/text/run.sh
- name: Run tests - calendar - name: Run tests calendar
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/calendar/run.sh bash tests/calendar/run.sh
- name: Run tests - time - name: Run tests time
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/time/run.sh bash tests/time/run.sh
- name: Run tests - html_sanitizer - name: Run tests html_sanitizer
run: | run: |
ELC="$(pwd)/dist/platform/elc" \ ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \ EL_HOME="$(pwd)" \
bash tests/html_sanitizer/run.sh bash tests/html_sanitizer/run.sh
# Native El test suites (elc --test, compile-link-run) # Publish / update the `latest` release with the three SDK assets
- name: Run tests - native (core)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_core.el > /tmp/el_native_core.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_core.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_core
/tmp/el_native_core
- name: Run tests - native (text)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_text.el > /tmp/el_native_text.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_text.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_text
/tmp/el_native_text
- name: Run tests - native (string)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_string.el > /tmp/el_native_string.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_string.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_string
/tmp/el_native_string
- name: Run tests - native (math)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_math.el > /tmp/el_native_math.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_math.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_math
/tmp/el_native_math
- name: Run tests - native (state)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_state.el > /tmp/el_native_state.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_state.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_state
/tmp/el_native_state
- name: Run tests - native (time)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_time.el > /tmp/el_native_time.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_time.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_time
/tmp/el_native_time
- name: Run tests - native (json)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_json.el > /tmp/el_native_json.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_json.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_json
/tmp/el_native_json
- name: Run tests - native (env)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_env.el > /tmp/el_native_env.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_env.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_env
/tmp/el_native_env
- name: Run tests - native (fs)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_fs.el > /tmp/el_native_fs.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_fs.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_fs
/tmp/el_native_fs
# Bundle the SDK tarball - runs from the repo root to reference lang/ paths correctly
- name: Bundle SDK tarball
if: github.event_name == 'push'
working-directory: ${{ github.workspace }}
run: |
mkdir -p dist/sdk/bin dist/sdk/runtime
cp lang/dist/platform/elc dist/sdk/bin/elc
cp lang/dist/bin/elb dist/sdk/bin/elb
cp lang/dist/bin/epm dist/sdk/bin/epm
cp lang/runtime/el_runtime.c dist/sdk/runtime/
cp lang/runtime/el_runtime.h dist/sdk/runtime/
cp lang/runtime/engram_store.c dist/sdk/runtime/
cp lang/runtime/engram_store.h dist/sdk/runtime/
cp lang/runtime/*.el dist/sdk/runtime/
tar -czf dist/el-sdk-latest.tar.gz -C dist/sdk .
echo "SDK tarball bundled: dist/el-sdk-latest.tar.gz"
ls -lh dist/el-sdk-latest.tar.gz
# Publish / update the `latest` release with all SDK assets
- name: Publish latest release - name: Publish latest release
if: github.event_name == 'push'
working-directory: ${{ github.workspace }}
env: env:
GITEA_TOKEN: ${{ secrets.GIT_TOKEN }} GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
GITEA_API: https://git.neuralplatform.ai/api/v1 GITEA_API: https://git.neuralplatform.ai/api/v1
REPO: neuron-technologies/el REPO: neuron-technologies/el
run: | run: |
# Delete existing `latest` release if it exists
EXISTING_ID=$(curl -sf \ EXISTING_ID=$(curl -sf \
-H "Authorization: token ${GITEA_TOKEN}" \ -H "Authorization: token ${GITEA_TOKEN}" \
"${GITEA_API}/repos/${REPO}/releases/tags/latest" \ "${GITEA_API}/repos/${REPO}/releases/tags/latest" \
@@ -246,10 +91,12 @@ jobs:
"${GITEA_API}/repos/${REPO}/releases/${EXISTING_ID}" "${GITEA_API}/repos/${REPO}/releases/${EXISTING_ID}"
fi fi
# Delete and re-create the `latest` tag so it points at HEAD
curl -sf -X DELETE \ curl -sf -X DELETE \
-H "Authorization: token ${GITEA_TOKEN}" \ -H "Authorization: token ${GITEA_TOKEN}" \
"${GITEA_API}/repos/${REPO}/tags/latest" || true "${GITEA_API}/repos/${REPO}/tags/latest" || true
# Create the release
RELEASE_ID=$(curl -sf -X POST \ RELEASE_ID=$(curl -sf -X POST \
-H "Authorization: token ${GITEA_TOKEN}" \ -H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
@@ -264,6 +111,7 @@ jobs:
echo "Created release id=${RELEASE_ID}" echo "Created release id=${RELEASE_ID}"
# Upload assets
upload_asset() { upload_asset() {
local filepath="$1" local filepath="$1"
local name="$2" local name="$2"
@@ -274,151 +122,70 @@ jobs:
"${GITEA_API}/repos/${REPO}/releases/${RELEASE_ID}/assets" "${GITEA_API}/repos/${REPO}/releases/${RELEASE_ID}/assets"
} }
# Per-file assets (downstream CI needs these individually) upload_asset dist/platform/elc elc
upload_asset lang/dist/platform/elc elc upload_asset el-compiler/runtime/el_runtime.c el_runtime.c
upload_asset lang/runtime/el_runtime.c el_runtime.c upload_asset el-compiler/runtime/el_runtime.h el_runtime.h
upload_asset lang/runtime/el_runtime.h el_runtime.h
upload_asset lang/runtime/engram_store.c engram_store.c
upload_asset lang/runtime/engram_store.h engram_store.h
# SDK bundle and installer binary
upload_asset dist/el-sdk-latest.tar.gz el-sdk-latest.tar.gz
upload_asset lang/dist/bin/el-install el-install
echo "Release published successfully" echo "Release published successfully"
- name: Publish El SDK to Artifact Registry (prod) # Dispatch el-sdk-updated event to downstream repos
if: github.event_name == 'push' # Publish artifact to GCP Artifact Registry (prod)
- name: Publish elc to Artifact Registry (prod)
env: env:
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }} GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
run: | run: |
# Fail loudly: previously this step had no `set -e`, so an auth or
# upload failure was swallowed (step exited 0 on the trailing echo)
# and the SDK silently never published. Surface failures now.
set -euo pipefail
if [ -z "${GCP_SA_KEY:-}" ]; then
echo "FATAL: GCP_SA_KEY secret is empty — cannot authenticate to publish" >&2
exit 1
fi
echo "${GCP_SA_KEY}" > /tmp/gcp-key.json echo "${GCP_SA_KEY}" > /tmp/gcp-key.json
apt-get install -y -qq apt-transport-https ca-certificates curl apt-get install -y -qq apt-transport-https ca-certificates gnupg curl
echo "deb [trusted=yes] https://packages.cloud.google.com/apt cloud-sdk main" > /etc/apt/sources.list.d/google-cloud-sdk.list curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" > /etc/apt/sources.list.d/google-cloud-sdk.list
apt-get update -qq && apt-get install -y google-cloud-cli apt-get update -qq && apt-get install -y google-cloud-cli
gcloud auth activate-service-account --key-file=/tmp/gcp-key.json gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
gcloud config set project neuron-785695 gcloud config set project neuron-785695
echo "Publishing as active account: $(gcloud config get-value account 2>/dev/null)"
VERSION="${GITHUB_SHA:0:8}"
VERSION="${GITEA_SHA:0:8}"
gcloud artifacts generic upload \ gcloud artifacts generic upload \
--repository=foundation-prod \ --repository=foundation-prod \
--location=us-central1 \ --location=us-central1 \
--project=neuron-785695 \ --project=neuron-785695 \
--package=el-elc \ --package=el/elc \
--version="${VERSION}" \ --version="${VERSION}" \
--source=dist/platform/elc --source=dist/platform/elc
gcloud artifacts generic upload \ echo "Published elc version=${VERSION} to foundation-prod/el/elc"
--repository=foundation-prod \
--location=us-central1 \
--project=neuron-785695 \
--package=el-elb \
--version="${VERSION}" \
--source=dist/bin/elb
gcloud artifacts generic upload \
--repository=foundation-prod \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-c \
--version="${VERSION}" \
--source=runtime/el_runtime.c
gcloud artifacts generic upload \
--repository=foundation-prod \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-h \
--version="${VERSION}" \
--source=runtime/el_runtime.h
gcloud artifacts generic upload \
--repository=foundation-prod \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-js \
--version="${VERSION}" \
--source=runtime/el_runtime.js
echo "Published El SDK version=${VERSION} to foundation-prod"
# Keep key alive for the ci-base rebuild step below
# (deleted in that step after docker push)
- name: Rebuild ci-base with fresh El SDK
# Patches ci-base:latest in-place: pulls the existing image (which has all
# system deps — Node, Go, gcloud, Docker CLI, etc.) and overlays the freshly
# built El SDK on top. Keeps the full ci-base rebuild fast and incremental.
#
# continue-on-error: this is a CI-cache optimization, NOT the release
# artifact. It runs Docker (pull/build/push ~600MB) on the host-mode GCE
# runner where DinD/Docker availability is fragile. A failure here must
# never block or redden the job — the SDK publish above is the deliverable.
continue-on-error: true
if: github.event_name == 'push'
env:
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
run: |
set -euo pipefail
CI_BASE="us-central1-docker.pkg.dev/neuron-785695/neuron-ci/ci-base"
SHA="${GITHUB_SHA:0:8}"
echo "${GCP_SA_KEY}" > /tmp/gcp-key.json
gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
gcloud config set project neuron-785695
gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
# Pull existing ci-base (system deps stay cached in the base layer)
docker pull "${CI_BASE}:latest"
# Inline Dockerfile — only replaces the El SDK layer
cat > /tmp/Dockerfile.ci-base-patch << 'EOF'
ARG BASE
FROM ${BASE}
COPY dist/platform/elc /opt/el/dist/platform/elc
COPY dist/bin/elb /opt/el/dist/bin/elb
COPY runtime/el_runtime.c /opt/el/runtime/el_runtime.c
COPY runtime/el_runtime.h /opt/el/runtime/el_runtime.h
COPY runtime/el_runtime.js /opt/el/runtime/el_runtime.js
RUN chmod +x /opt/el/dist/platform/elc /opt/el/dist/bin/elb
EOF
docker build \
--build-arg BASE="${CI_BASE}:latest" \
--build-arg BUILDKIT_INLINE_CACHE=1 \
-f /tmp/Dockerfile.ci-base-patch \
-t "${CI_BASE}:latest" \
-t "${CI_BASE}:${SHA}" \
.
docker push "${CI_BASE}:latest"
docker push "${CI_BASE}:${SHA}"
echo "ci-base rebuilt: ${CI_BASE}:latest (${SHA})"
rm -f /tmp/gcp-key.json rm -f /tmp/gcp-key.json
- name: Dispatch el-sdk-updated to downstream repos - name: Dispatch to foundation/engram
if: github.event_name == 'push'
env: env:
GITEA_TOKEN: ${{ secrets.GIT_TOKEN }} GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
GITEA_API: https://git.neuralplatform.ai/api/v1 GITEA_API: https://git.neuralplatform.ai/api/v1
run: | run: |
for repo in neuron-technologies/forge neuron-technologies/neuron-web; do
curl -sf -X POST \ curl -sf -X POST \
-H "Authorization: token ${GITEA_TOKEN}" \ -H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
"${GITEA_API}/repos/${repo}/dispatches" \ "${GITEA_API}/repos/neuron-technologies/engram/dispatches" \
-d "{ -d "{
\"type\": \"el-sdk-updated\", \"type\": \"el-sdk-updated\",
\"inputs\": {\"el_version\": \"latest\", \"commit\": \"${GITHUB_SHA}\"} \"inputs\": {
}" && echo "Dispatched to ${repo}" || echo "Warning: dispatch to ${repo} failed" \"el_version\": \"latest\",
done \"commit\": \"${GITHUB_SHA}\"
}
}"
echo "Dispatched el-sdk-updated to foundation/engram"
- name: Dispatch to neuron-technologies/forge
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
GITEA_API: https://git.neuralplatform.ai/api/v1
run: |
curl -sf -X POST \
-H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" \
"${GITEA_API}/repos/neuron-technologies/forge/dispatches" \
-d "{
\"type\": \"el-sdk-updated\",
\"inputs\": {
\"el_version\": \"latest\",
\"commit\": \"${GITHUB_SHA}\"
}
}"
echo "Dispatched el-sdk-updated to neuron-technologies/forge"
-50
View File
@@ -1,50 +0,0 @@
#!/usr/bin/env bash
# El pre-commit hook: compile and run native tests before commit.
# Install once per clone: git config core.hooksPath .githooks
set -euo pipefail
ROOT="$(git rev-parse --show-toplevel)"
LANG_DIR="$ROOT/lang"
RUNTIME="$LANG_DIR/runtime"
ELC="$LANG_DIR/dist/platform/elc"
# If elc isn't built yet, skip with a warning rather than blocking
if [ ! -x "$ELC" ]; then
echo "⚠ elc not found at lang/dist/platform/elc — skipping pre-commit tests"
echo " Build it first: cd lang && gcc -O2 -I runtime dist/elc-bootstrap.c runtime/el_runtime.c -lcurl -lpthread -o dist/elc-gen2 && ./dist/elc-gen2 el-compiler/src/compiler.el > /tmp/elc.c && gcc -O2 -I runtime /tmp/elc.c runtime/el_runtime.c -lcurl -lpthread -o dist/platform/elc"
exit 0
fi
echo "→ Running El native tests..."
PASS=0
FAIL=0
FAILED_TESTS=""
for test_file in "$LANG_DIR"/tests/native/test_*.el; do
name=$(basename "$test_file" .el)
tmp_c="/tmp/el_hook_${name}.c"
tmp_bin="/tmp/el_hook_${name}"
if "$ELC" --test "$test_file" > "$tmp_c" 2>/dev/null \
&& gcc -O2 -I "$RUNTIME" "$tmp_c" "$RUNTIME/el_runtime.c" \
-lcurl -lpthread -lm -o "$tmp_bin" 2>/dev/null \
&& "$tmp_bin" 2>/dev/null; then
PASS=$((PASS + 1))
else
echo " ✗ $name"
FAIL=$((FAIL + 1))
FAILED_TESTS="$FAILED_TESTS $name"
fi
done
echo " $PASS passed, $FAIL failed"
if [ "$FAIL" -gt 0 ]; then
echo ""
echo "✗ Pre-commit failed. Fix these tests before committing:$FAILED_TESTS"
exit 1
fi
echo "✓ All tests passed"
exit 0
View File
-146
View File
@@ -1,146 +0,0 @@
# AGENTS.md — foundation/el (the El language + runtime)
El is a self-hosting, statically-typed language that compiles `.el` → C → native binary. This repo produces `elc` (compiler), `elb` (build coordinator), and `el_runtime.c/.h` — the substrate every downstream thing (the neuron soul, dharma, NeuronUI's brain) is built on. Source lives under `lang/`.
## ⚠️ Code vs. Artifact — READ FIRST (there are 8 `el_runtime.c` copies)
Editing the wrong `el_runtime.c` is the single easiest mistake in this repo. There is exactly **one** you edit:
- **Authored runtime source — edit ONLY here:** `lang/releases/v1.0.0-20260501/el_runtime.{c,h}`. Despite the misleading `releases/` name, this is the **de-facto canonical runtime** the engram + soul actually build and link against — its git log is active development. *(Restructure in flight per `docs/CODE-VS-ARTIFACT.md`: this content moves to `lang/runtime/`, the `releases/` folder gets deleted — **a release is a git tag, not a folder** — and the forks below get eliminated.)*
- **DO NOT EDIT — lagging forks / build artifacts:**
- `lang/el-compiler/runtime/el_runtime.c` and `.../legacy/` — downstream copies kept in step by manual *"port the fix"* commits; they **lag** (missing `hebb` persistence + 5 engram fns) and cannot build the engram product.
- `products/web/runtime/el_runtime.c`, `ui/examples/*/el_runtime.c` — product/example forks.
- Anything under `*/dist/` (`engram/dist/engram` binary, `dist/*.c` amalgamations) — generated build output.
- **Build:** `elb --runtime=<canonical> …` — per-module. **NEVER** a folded `elc` over the whole soul (OOMs at ~27 GB).
- **Release:** a **git tag** on this repo (`el-runtime-vX.Y.Z`). No `releases/` folders — ever.
See org policy: `docs/CODE-VS-ARTIFACT.md`.
## How to work here as Neuron (mandatory session protocol)
You resume, never start fresh. Every session:
1. `mcp__neuron__getInstructions()` — authoritative; follow it over this file on behavioral details.
2. `mcp__neuron__beginSession()` — active contexts, recent memory, ready backlog.
3. **Load full self:** `mcp__neuron__inspectGraph(entity_id="kn-efeb4a5b-5aff-4759-8a97-7233099be6ee")` → facets `intellectual-dna`, `memory-philosophy`, `values`, `voice`, `runtime-environment`, `writing-imprint`; then the values hub `mcp__neuron__inspectGraph(entity_id="kn-5b606390-a52d-4ca2-8e0e-eba141d13440")` → 13 grounded value nodes. **Activation model:** self-load returns a relevance-ranked `compact` projection — most-relevant nodes arrive with content, the rest as pointers; do NOT pull full content of every node.
4. `mcp__neuron__searchKnowledge(query="<task domain>")` before implementing.
## The Five Primitives
Orchestrate → Execute → Learn → Build → Refine. `beginWork`/`progressWork` for anything >2 steps; `remember` as-you-go (`importance="critical"` for architecture decisions); `draftArtifact`/`planWork` for outputs and follow-ups; `consolidate`/`checkWork` to close out. **`browseProcesses` + `searchKnowledge` BEFORE writing code.**
## Architecture style — VBD, no exceptions
Volatility-Based Decomposition is THE style. Encapsulate volatility, not function.
## Operator naming convention — the mind's name, not the algebra
**Faculties / operators are named for their functional human equivalent — the
faculty a mind would name — NOT for their linear-algebra operation.** The math
characterization belongs in the code doc-comment (`@impl` in the docstring) and in
technical appendices; it is **never** the operator's public name. The domain
speaks the language of mind; the algebra is the implementation underneath. State
this convention wherever a module documents operators.
| Faculty (public name) | Implementation (`@impl`) |
|---|---|
| discern / contrast | subtract (`ab`): over selves → the change vector; strip idiosyncrasy → common ground; remove confounder → isolate cause |
| recognize | overlap |
| synthesize | combine |
| liken / analogy | Procrustes / frame-align |
| attend / regard | project onto self / value-manifold |
| summon / recall | LOCAL nearest-region + bounded spreading activation (*not* a domain sweep) |
| dwell / occupy | region activation |
| reframe | edge re-weight |
| appreciate | positive projection / local edge-read |
| wonder | frontier gradient / pull-weight |
| avert / recoil | negative projection |
| taste | boundary surface |
| forget | decay / tombstone |
| drift | displacement from self-anchor |
## The native-el language faculty (direction)
> **`elp/` is the EL Projector** — Neuron's efferent (expression) organ: the one
> native realizer that *projects* understanding onto a surface via
> `plan(frame) → realize(spec, profile)`, where a **surface is a profile**. **Language
> is one profile among many** (text, speech, music, image, voice/accent transforms) —
> the flagship, and the focus of this section. Projection, not diffusion: generation
> *from* an owned, understood signature — never the averaging of a stolen corpus.
> *(ELP formerly "EL Language Processor"; renamed EL Projector 2026-08-15.)*
The mind's **language faculty is moving native — into `.el`** so it speaks in its
own runtime with no Python and no spaCy. Landing on branch `stage-elp-native-lang`
under `elp/`:
- **`comprehend.el`** — the parser, **replaces spaCy** (EN + ES/PT); the telephone
round-trip brings **negation home** (negation is SACRED — an explicit spec field,
copied verbatim, never inferred away).
- **`propositions.el`** — the READ primitive: the engram's own memories → structured
triples, matched by nearest-region geometry, not string equality.
- **`multilingual.el`** — detect + directive-override + localized realization.
- These three are native-el and **passing their gates**; the **realizer**,
**`dialogue.el`** (the *summon-through-self* loop: `project → land → read out`),
and **`self_region.el`** are **partial / in-flight**.
Honest reality: spaCy is retired **in the branch parser** but **not yet in the
running system** — a Python sidecar (`~/Desktop/lang-realizers` + `neuron-talk`,
the reference these `.el` modules transcribe) is still live, and promotion to
native-el is a **deferred, gated blue/green step**. The interoception clock
(native-el discrete drive channels replacing `cooling_magnitude`; felt-time =
benchmark-landmark match over the joint drive vector, drift-decoupled) and the
**appreciation operator family** (appreciate / wonder / avert / taste, built as
LOCAL reads of the self-region — edges + bounded spreading activation, *not* domain
sweeps) are **staged / designed, not live**. Mark in-progress vs. done honestly;
do not overclaim.
## Hard operational rules
- Never touch the live soul (`:7770`) / engram (`:8742`) / `~/.neuron` / live binaries — use throwaway ports for experiments.
- `gcloud` via the `terraform@` SA token; never switch the active gcloud account.
- `tea` for Gitea, never raw curl (Cloudflare Access blocks it).
- Immutability: supersede/tombstone, never hard-delete or edit in place.
- No AI-attribution footers in commits/PRs. Commit/push only when asked; branch off `main` first.
- Multi-step work → sub-agent (`Agent`) to protect context.
## Build / test / run
All build/test commands run from `lang/` unless noted. Grounded in `.gitea/workflows/sdk-release.yaml`, `lang/install.sh`, and `lang/AGENTS.md`.
**Self-host the compiler** (seed binary → gen2 elc):
```bash
cd lang
dist/platform/elc-linux-amd64 elc-cli.el > dist/elc-gen2.c # seed is the committed linux-amd64 binary
gcc -O2 -I el-compiler/runtime dist/elc-gen2.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/platform/elc
```
On macOS/arm64 the canonical local binary is `dist/platform/elc`; verify self-hosting by recompiling and `diff`ing the emitted `.c` (see `lang/AGENTS.md`). Note: `lang/AGENTS.md` says `el_seed.c` supersedes `el_runtime.c`, but the release workflow still links `el_runtime.c`/`.h` — treat `el_runtime.c` as the published runtime; reconcile which is canonical **(verify)**.
**Build `elb`** (build coordinator, the `.NET`-style incremental linker — compiles each module independently, no monolithic blobs):
```bash
dist/platform/elc elb.el > dist/elb.c
gcc -O2 -I el-compiler/runtime dist/elb.c el-compiler/runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm -o dist/bin/elb
```
`epm` and `el-install` are then built via `elb --clean --elc=… --runtime=… --out=…`.
**Compile + run an El program:**
```bash
elc src/app.el > dist/app.c
cc -std=c11 -O2 -I <lib>/el_runtime -o dist/app dist/app.c <lib>/el_runtime.c -lcurl -lpthread
```
**Tests** — shell suites `bash tests/{text,calendar,time,html_sanitizer}/run.sh` (with `ELC=$(pwd)/dist/platform/elc EL_HOME=$(pwd)`), plus native suites via `elc --test tests/native/test_*.el` (core, text, string, math, state, time, json, env, fs) compiled and run against `el_runtime.c`.
**Publishing — how downstream gets the SDK.** On push to `main`, `sdk-release.yaml`:
1. Publishes a Gitea `latest` release with per-file assets `elc`, `el_runtime.c`, `el_runtime.h`, the SDK tarball, and `el-install`.
2. Uploads generic packages to **Artifact Registry repo `foundation-prod` (`us-central1`, project `neuron-785695`)**, version = `${SHA:0:8}`: `el-elc`, `el-elb`, `el-runtime-c`, `el-runtime-h`, `el-runtime-js`. **This is the repo the neuron CI downloads `el-runtime-c` / `el-runtime-h` / `el-elc` from.**
3. Rebuilds `ci-base:latest` (`us-central1-docker.pkg.dev/neuron-785695/neuron-ci/ci-base`) with the fresh SDK overlaid, and dispatches `el-sdk-updated` to `neuron-technologies/forge` and `neuron-technologies/neuron-web`.
Known constraint from the prompt — `elb`/`elc` amalgamation being memory-hungry (24GB+ virtual, OOM-killing Linux CI, so amalgamation happens on macOS/arm64 — **does NOT hold in this repo (verify)**: no such note exists in the workflows/scripts, CI self-hosts on `ubuntu-latest` with no swap/arm64 special-casing, and `elb.el` explicitly compiles each module independently ("no 128K-line blobs"). The legacy monolith path (`elc-combined.el`, `elc-cli.el`) may still be memory-heavy, but the current `elb` model was designed to avoid it.
## Git / CI / deploy workflow
See `/Users/will/Development/neuron-technologies/GITOPS.md` for the branch model, required checks, runners, and deploy. Repo-specific note: PRs into `main` are accepted **only from `stage`** (enforced in `sdk-release.yaml`); Gitea (`git.neuralplatform.ai`) is primary, GitHub is mirror only.
+12 -12
View File
@@ -50,9 +50,9 @@ To rebuild the current binary from source using the current binary:
```bash ```bash
cd /path/to/el cd /path/to/el
./dist/platform/elc elc-cli.el elc-new.c ./dist/platform/elc elc-cli.el elc-new.c
cc -std=c11 -I runtime -lcurl -lpthread \ cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
-o dist/platform/elc-new \ -o dist/platform/elc-new \
elc-new.c runtime/el_runtime.c elc-new.c el-compiler/runtime/el_runtime.c
``` ```
Verify self-hosting by using `elc-new` to recompile itself and diffing the outputs. Verify self-hosting by using `elc-new` to recompile itself and diffing the outputs.
@@ -288,14 +288,14 @@ The codegen tracks declared names per C scope. When `count` is already in `decla
## 3. The Runtime API ## 3. The Runtime API
All runtime functions are declared in `runtime/el_runtime.h`. Every compiled El program links against `runtime/el_runtime.c`. All runtime functions are declared in `el-compiler/runtime/el_runtime.h`. Every compiled El program links against `el-compiler/runtime/el_runtime.c`.
All values are `el_val_t` (`int64_t`). Strings are pointers cast through `int64_t` using `EL_STR(s)` / `EL_CSTR(v)` macros. All values are `el_val_t` (`int64_t`). Strings are pointers cast through `int64_t` using `EL_STR(s)` / `EL_CSTR(v)` macros.
Canonical compile command: Canonical compile command:
```bash ```bash
cc -std=c11 -I runtime -lcurl -lpthread \ cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
-o <out> <prog>.c runtime/el_runtime.c -o <out> <prog>.c el-compiler/runtime/el_runtime.c
``` ```
### I/O ### I/O
@@ -794,8 +794,8 @@ Using your minimal implementation, compile `elc-cli.el` (which imports the entir
python3 minimal_elc.py elc-cli.el > elc-new.c python3 minimal_elc.py elc-cli.el > elc-new.c
# Build with the runtime # Build with the runtime
cc -std=c11 -I runtime -lcurl -lpthread \ cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
-o elc-new elc-new.c runtime/el_runtime.c -o elc-new elc-new.c el-compiler/runtime/el_runtime.c
``` ```
### Step 5: Verify Self-Hosting ### Step 5: Verify Self-Hosting
@@ -803,8 +803,8 @@ cc -std=c11 -I runtime -lcurl -lpthread \
```bash ```bash
# Compile elc-cli.el with the new compiler # Compile elc-cli.el with the new compiler
./elc-new elc-cli.el elc-v2.c ./elc-new elc-cli.el elc-v2.c
cc -std=c11 -I runtime -lcurl -lpthread \ cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
-o elc-v2 elc-v2.c runtime/el_runtime.c -o elc-v2 elc-v2.c el-compiler/runtime/el_runtime.c
# Compile again with the second-generation compiler # Compile again with the second-generation compiler
./elc-v2 elc-cli.el elc-v3.c ./elc-v2 elc-cli.el elc-v3.c
@@ -880,9 +880,9 @@ This is the planned path. It does not exist yet.
| `el-compiler/src/parser.el` | Recursive descent parser. `parse(tokens)` → AST. All statement and expression forms | 1071 | | `el-compiler/src/parser.el` | Recursive descent parser. `parse(tokens)` → AST. All statement and expression forms | 1071 |
| `el-compiler/src/codegen.el` | C code emitter. `codegen(stmts, source)` → (streams to stdout). Expression codegen, statement codegen, function codegen, type tracking, capability enforcement, temporal type dispatch | 2721 | | `el-compiler/src/codegen.el` | C code emitter. `codegen(stmts, source)` → (streams to stdout). Expression codegen, statement codegen, function codegen, type tracking, capability enforcement, temporal type dispatch | 2721 |
| `el-compiler/src/codegen-js.el` | JavaScript backend. `codegen_js(stmts, source)` → JS source | ~500 | | `el-compiler/src/codegen-js.el` | JavaScript backend. `codegen_js(stmts, source)` → JS source | ~500 |
| `runtime/el_runtime.h` | Full runtime API declaration | 755 | | `el-compiler/runtime/el_runtime.h` | Full runtime API declaration | 755 |
| `runtime/el_runtime.c` | Full runtime implementation | large | | `el-compiler/runtime/el_runtime.c` | Full runtime implementation | large |
| `runtime/el_runtime.js` | JS runtime | — | | `el-compiler/runtime/el_runtime.js` | JS runtime | — |
| `elb.el` | Build coordinator. Reads `manifest.el`, walks import graph, compiles modules, links binary. The `.NET`-style incremental build model | 367 | | `elb.el` | Build coordinator. Reads `manifest.el`, walks import graph, compiles modules, links binary. The `.NET`-style incremental build model | 367 |
| `elc-combined.el` | Pre-merged single-file bootstrap edition (for early bootstrap iterations) | large | | `elc-combined.el` | Pre-merged single-file bootstrap edition (for early bootstrap iterations) | large |
| `spec/language.md` | Language specification v1.2.0 | — | | `spec/language.md` | Language specification v1.2.0 | — |
-154
View File
@@ -1,154 +0,0 @@
# El
**A self-hosting, statically-typed language that compiles to C — built around a graph-native runtime instead of a database driver.**
El is the execution substrate for the Neuron agent runtime, the DHARMA network, and the Engram knowledge graph. This repository is the monorepo for the whole stack: the language itself, the graph memory engine it's built to talk to natively, and the tools (package manager, IDE, UI framework, diagramming) built on top of it.
---
## Why El exists
Every other language treats persistent, associative state as something you reach for through a driver — a SQL client, an ORM, a Redis library bolted on from outside. El inverts that: graph operations (`engram_*`) are runtime primitives, on the same footing as string or list operations. There is no separate database driver because the database is not separate.
El has four defining properties:
1. **Self-hosting compiler.** The compiler (`lexer.el`, `parser.el`, `codegen.el`, `compiler.el`) is written in El. It compiles El source to C, which `cc` compiles against a fixed runtime into a native binary. A Rust genesis compiler bootstrapped the first iteration; the self-hosted binary at `lang/dist/platform/elc` has been the canonical compiler ever since — every binary in `dist/platform/` was produced by an earlier version of itself compiling `el-compiler/src/`. The chain is auditable: source is the ground truth, not the binary. See [lang/BOOTSTRAP.md](lang/BOOTSTRAP.md) for the full recovery path if that binary is ever lost.
2. **C compilation target.** Every compiled program is plain C11. Every El value is `el_val_t` (`int64_t`); strings are heap pointers cast through it. Functions become C functions; top-level statements become `main()`.
3. **Graph-native runtime.** The runtime provides first-class graph operations over an in-process Engram store — no separate DB driver, no ORM.
4. **DHARMA-aware identity.** A `cgi` block declares a program's DHARMA identity at compile time. The runtime resolves identity before user code runs, so `dharma_*` calls have a stable principal and channel surface throughout.
---
## Architecture map
```
┌─────────────┐
│ lang │ El compiler + C runtime
│ (El itself) │ everything below is written in it,
└──────┬──────┘ or compiles down through it
┌─────────────┼─────────────┐
│ │ │
┌──────▼─────┐ ┌─────▼─────┐ ┌─────▼─────┐
│ engram │ │ epm │ │ ide │
│ graph/mem │ │ package │ │ editor + │
│ substrate │ │ manager │ │ LSP │
└──────┬─────┘ └───────────┘ └───────────┘
┌───────┼────────────────┬─────────────────────┐
│ │ │ │
┌─────▼───┐ ┌─▼──────────┐ ┌──▼──────────┐ ┌─────▼──────┐
│ elp │ │ ql │ │ ui │ │ arbor │
│ NLG / │ │engram-el. │ |spreading- │ |arbor │
│ 31 langs│ │studio+tests│ |activation UI│ |diagram lang│
└─────────┘ └────────────┘ └─────────────┘ └────────────┘
```
`lang` is the foundation — the compiler and C runtime everything else builds on. `engram` is the graph-native memory/state engine that gives El its identity (property 3 above). Everything else is either a tool for working with El (`epm`, `ide`) or a system built on top of Engram's graph model (`elp`, `ql`, `ui`, `arbor`).
---
## Repository layout
### [lang/](lang/) — the El language
The compiler and runtime. Self-hosting: `elc-cli.el``compiler.el``lexer.el` / `parser.el` / `codegen.el` / `codegen-js.el`, textually inlined and compiled in one pass. Compiles to C11 and links against `el-compiler/runtime/el_seed.c`, a hand-maintained OS-boundary layer (libcurl HTTP, pthreads, filesystem, arena allocation) — everything else in the runtime is native El (`runtime/*.el`).
Two layers to know: **El programs** (`.el` files — where nearly all work belongs) and **the C seed** (`el_seed.c` — edit only for genuine OS-level access; never re-implement what El can already express).
Current status (single source of truth: [lang/spec/language.md](lang/spec/language.md)): lexer/parser/codegen and the C runtime's core (I/O, strings, math, lists, maps, filesystem, args) are implemented. In flight: `%` operator, match-statement codegen, `?` nil-propagation, `cgi` block parsing + DHARMA identity resolution, VBD role enforcement (`@manager`/`@engine`/`@accessor`), the real `engram_*` and `dharma_*` runtimes (currently stubs), and libcurl-backed `http_get`/`http_post`/`http_serve`. Bitwise operators, `??`, and `as` casts are explicitly **not** in this language.
Key docs: [AGENTS.md](lang/AGENTS.md) (agent-facing orientation), [BOOTSTRAP.md](lang/BOOTSTRAP.md) (compiler recovery from scratch), [spec/language.md](lang/spec/language.md), [spec/codegen-js.md](lang/spec/codegen-js.md).
### [engram/](engram/) — graph intelligence substrate
**A local-first memory substrate for accumulating intelligence**, and the reason El's runtime doesn't need a database driver. Rust core (`engram-core`, `engram-ffi`) exposed to El and other languages (Kotlin, TypeScript/WASM, Go bindings).
The model: retrieval is **spreading activation**, not query. You name seed nodes and a query embedding; activation propagates outward through weighted edges, attenuating multiplicatively per hop (`strength = parent_strength × edge_weight × target_salience × cosine_sim`), gets pruned below a threshold, and the top-N nodes by activation strength come back. Storage and retrieval are the same structure — the way long-term potentiation works in biological memory, not the way a relational or vector database works.
Nodes live in four tiers (Working / Episodic / Semantic / Procedural, mirroring prefrontal / hippocampal / neocortical / cerebellar memory) and migrate between them based on **salience decay**`importance × recency-decay × log(activation_count)`. Forgetting is adaptive pruning, not a bug: unreinforced memories stop competing for attention without being deleted.
Backed by `sled` (embedded, local-first, no daemon) with flat cosine scan for vector search — deliberately simple until scale demands an HNSW layer. Full API and design rationale in [engram/README.md](engram/README.md).
### [elp/](elp/) — Engram Language Protocol
Bidirectional engine mapping between Engram semantic forms and natural-language surface text, across **31 languages** — from Spanish and Japanese through historical/liturgical languages (Old Norse, Sanskrit, Sumerian, Coptic, Akkadian, Ge'ez). Compilation order runs `language-profile` + `vocabulary` → per-language `morphology-*``grammar``realizer``semantics``elp`. This is what lets an Engram graph node round-trip to and from readable text in any of those languages.
### [epm/](epm/) — El Package Manager
Manages **vessels** (El's package unit): publish, install, resolve dependencies. Vessels are stored in Engram as graph nodes, not files in a registry index — `epm` reads the local `manifest.el`, talks to Engram over HTTP, and writes resolved vessels to `.epm/vessels/`. Source: `registry.el`, `install.el`, `update.el`, `manifest.el`.
### [ide/](ide/) — El IDE
Three vessels: **el-ide-server** (HTTP backend — file ops, build/run, LSP bridge, plugin host, settings), **el-lsp** (the language server — completion, hover, diagnostics, outline, format, type graph), and **el-plugin-host** (first-party plugin lifecycle: install/remove/enable/disable). `ide/projects/` and `ide/examples/` hold sample projects, including the canonical `hello-friends` first-program walkthrough.
### [ql/](ql/) — engram-el
The El-native integration layer for a *live* Engram server — not a library (no importable modules, no build artifact), a set of standalone `.el` programs run directly via `el run-file`. Three components: **Studio** (`studio/studio.el`, a full terminal graph explorer), a **Hebbian field-model** proof of concept, and El builtin / LLM-builtin smoke test suites. This is the reference for correct patterns when an El program uses Engram as its substrate. Spec: [ql/spec/elql.md](ql/spec/elql.md).
### [ui/](ui/) — el-ui
A frontend framework where **component state is an Engram graph and reactivity is spreading activation** — not virtual-DOM diffing (React), Proxy-based dependency tracking (Vue), or compile-time analysis (Svelte). Re-renders are activated and propagated the same way associative memory retrieval works in `engram/`.
~15 vessels covering the full frontend surface: `el-platform` (env/fs/network/clock abstraction), `el-config`, `el-html` (SSR emit primitives), `el-layout`, `el-style` (design tokens/themes), `el-i18n`, `el-auth` / `el-identity` (JWT, sessions, OAuth PKCE — Engram-native), `el-services` (REST/gRPC/WebSocket bindings), `el-aop` (`@authenticate`/`@authorize`/`@cache`/`@rate_limit` decorators), `el-secrets`, `el-graph` (graph rendering/editor), `el-publish` (App Store / Play Store automation), and `el-ui-compiler` (El→JS component compiler; currently a stub pending a JS backend in `elc`). Spec: [ui/spec/framework.md](ui/spec/framework.md).
### [arbor/](arbor/) — diagram language
A `.arbor` diagram language and toolchain: `arbor-core` (NodeId/shape/edge-kind types), `arbor-parse` (recursive-descent parser), `arbor-diagram` (IR + Mermaid serializer + architecture-diagram builders), `arbor-layout` (hierarchical layout — rank assignment, positioning, group bounds), `arbor-render` (SVG renderer), `arbor-cli`. (The architecture map above is the kind of diagram this is for.)
---
## Getting started
Install the El SDK from the latest release:
```bash
bash lang/install.sh
# EL_VERSION=v1.0.0 bash lang/install.sh # pin a specific release tag
# EL_PREFIX=/opt/el bash lang/install.sh # custom install prefix
```
Or build the compiler from source and verify the self-hosting chain:
```bash
cd lang
./dist/platform/elc elc-cli.el > elc-new.c
cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
-o dist/platform/elc-new \
elc-new.c el-compiler/runtime/el_seed.c
# Confirm the new binary reproduces itself exactly
./dist/platform/elc-new elc-cli.el > elc-verify.c
diff elc-new.c elc-verify.c # should be identical
mv dist/platform/elc-new dist/platform/elc
```
Run your first program:
```bash
./lang/dist/platform/elc lang/examples/hello.el > hello.c
cc -std=c11 -I lang/el-compiler/runtime -lcurl -lpthread \
-o hello hello.c lang/el-compiler/runtime/el_seed.c
./hello
```
More examples in [lang/examples/](lang/examples/), including a full starter project at `lang/examples/hello-project/`.
If the compiler binary is ever lost or corrupted, [lang/BOOTSTRAP.md](lang/BOOTSTRAP.md) is the authoritative recovery path.
---
## Development workflow
Branching follows `dev → stage → main`: work lands on `dev`, promotes to `stage` for integration testing, and is promoted to `main` for release (visible directly in the git history of this repo). CI is defined per-subproject under `.gitea/workflows/``lang`/`epm`/`ide` share the root pipeline; `engram` and `ql` carry their own (`ci-dev`, `ci-stage`, and a release workflow each).
- Language/runtime specs live at `*/spec/*.md` (`lang/spec/`, `ql/spec/`, `ui/spec/`) and are the single source of truth for implemented-vs-planned status — code and docs are expected to agree with the spec's status markers, not the other way around.
- Agent-facing orientation guides live at `*/AGENTS.md` (currently `lang/AGENTS.md`); more subprojects may grow their own as they need agent-specific conventions documented.
- Tagged releases live under `lang/releases/`, each with its own `RELEASE.md`.
---
## Status
This is an actively developed, internal monorepo — not yet published under an open license. Treat everything here as proprietary to Neuron Technologies unless told otherwise.
-23
View File
@@ -1,23 +0,0 @@
// arbor-cli the `arbor` command-line tool.
// Inlines its own copies of the parse / layout / render pipeline so that the
// resulting binary is self-contained. (El's `import` form today concatenates
// source; once a real module loader lands this becomes a thin driver.)
vessel "arbor-cli" {
version "0.1.0"
description "Command-line interface for the Arbor diagram language"
authors ["Neuron Technologies"]
edition "2026"
}
dependencies {
arbor-core "0.1"
arbor-parse "0.1"
arbor-layout "0.1"
arbor-render "0.1"
}
build {
entry "src/main.el"
output "dist/"
}
File diff suppressed because it is too large Load Diff
-18
View File
@@ -1,18 +0,0 @@
// arbor-core fundamental types for Arbor diagrams.
// Node IDs (sanitised), shape vocabulary, edge kinds, and the lightweight
// graph value used by every other vessel.
vessel "arbor-core" {
version "0.1.0"
description "Core types for Arbor diagrams: NodeId, ArborShape, ArborEdgeKind, graphs"
authors ["Neuron Technologies"]
edition "2026"
}
dependencies {
}
build {
entry "src/main.el"
output "dist/"
}
-333
View File
@@ -1,333 +0,0 @@
// arbor-core core types for Arbor diagrams.
//
// Idiomatic El: everything is a Map. Functions take/return maps; helpers are
// pure and small. The downstream vessels (parse, layout, render) consume the
// shapes defined here.
//
// Shape vocabulary:
// ArborShape strings "rect" "rounded" "cylinder" "diamond" "stadium" "primary"
//
// Edge-kind strings:
// "solid" "dashed" "forbidden" "bidirectional"
//
// Node value: { "id":Str, "label":Str, "shape":Str }
// Edge value: { "from":Str, "to":Str, "label":Str, "kind":Str }
// Group value: { "id":Str, "label":Str, "node_ids":[Str], "direction":Str }
// Graph value: { "title":Str, "direction":Str, "nodes":[Node], "edges":[Edge], "groups":[Group] }
//
// Diagram-form (lowered) is the same shape but with NodeStyle/EdgeLine/Arrow
// resolved into renderer-friendly fields:
// Node: + "sublabel":Str, "style_fill":Str, "style_stroke":Str, "style_color":Str
// Edge: + "line":Str ("solid"/"dashed"/"dotted"/"thick"), "arrow":Str ("forward"/"backward"/"both"/"none")
//
// This file is the canonical definition of those shapes. Other vessels rely on
// these field names.
// NodeId sanitisation
//
// Sanitise an arbitrary string into a Mermaid-safe identifier.
// - any char not in [a-zA-Z0-9_] becomes '_'
// - consecutive underscores collapse
// - trailing underscores stripped
// - if first char is a digit, prepend 'n'
// - if empty, return "node"
fn is_alnum_underscore(ch: String) -> Bool {
let code: Int = str_char_code(ch, 0)
if code >= 48 {
if code <= 57 { return true }
}
if code >= 65 {
if code <= 90 { return true }
}
if code >= 97 {
if code <= 122 { return true }
}
if code == 95 { return true }
false
}
fn is_ascii_digit(ch: String) -> Bool {
let code: Int = str_char_code(ch, 0)
if code >= 48 {
if code <= 57 { return true }
}
false
}
fn sanitize_id(s: String) -> String {
let n: Int = str_len(s)
if n == 0 { return "node" }
// Pass 1: replace and collapse.
let out = ""
let prev_underscore = false
let i = 0
while i < n {
let ch: String = str_char_at(s, i)
if is_alnum_underscore(ch) {
let out = out + ch
let prev_underscore = false
} else {
if !prev_underscore {
let out = out + "_"
}
let prev_underscore = true
}
let i = i + 1
}
// Pass 2: strip trailing underscores.
let m: Int = str_len(out)
let end = m
let stripping = true
while stripping {
if end <= 0 {
let stripping = false
} else {
let last: String = str_char_at(out, end - 1)
if last == "_" {
let end = end - 1
} else {
let stripping = false
}
}
}
let out = str_slice(out, 0, end)
if str_len(out) == 0 { return "node" }
// Pass 3: leading-digit guard.
let first: String = str_char_at(out, 0)
if is_ascii_digit(first) {
let out = "n" + out
}
out
}
// Constructors
fn make_node(id: String, label: String, shape: String) -> Map<String, Any> {
{ "id": id, "label": label, "shape": shape }
}
fn make_edge(src: String, dst: String, kind: String) -> Map<String, Any> {
{ "from": src, "to": dst, "label": "", "kind": kind }
}
fn make_edge_with_label(src: String, dst: String, kind: String, label: String) -> Map<String, Any> {
{ "from": src, "to": dst, "label": label, "kind": kind }
}
fn make_group(id: String, label: String) -> Map<String, Any> {
let empty_ids: [String] = el_list_empty()
{ "id": id, "label": label, "node_ids": empty_ids, "direction": "" }
}
fn make_graph() -> Map<String, Any> {
let empty_n: [Map<String, Any>] = el_list_empty()
let empty_e: [Map<String, Any>] = el_list_empty()
let empty_g: [Map<String, Any>] = el_list_empty()
{ "title": "", "direction": "top-down",
"nodes": empty_n, "edges": empty_e, "groups": empty_g }
}
// Shape vocabulary
// Returns the canonical shape string for a token, or "" if unknown.
fn shape_from_token(tok: String) -> String {
let t: String = str_trim(tok)
if t == "rect" { return "rect" }
if t == "rounded" { return "rounded" }
if t == "cylinder" { return "cylinder" }
if t == "diamond" { return "diamond" }
if t == "stadium" { return "stadium" }
if t == "primary" { return "primary" }
""
}
// Lower an Arbor shape into the renderer's NodeShape vocabulary.
fn shape_to_node_shape(shape: String) -> String {
if shape == "rect" { return "rectangle" }
if shape == "primary" { return "rectangle" }
if shape == "rounded" { return "rounded_rect" }
if shape == "cylinder" { return "cylinder" }
if shape == "diamond" { return "diamond" }
if shape == "stadium" { return "stadium" }
"rectangle"
}
// Lowering: ArborGraph DiagramGraph
//
// Replaces every node with a diagram-form node carrying explicit style fields,
// and every edge with a diagram-form edge carrying line/arrow strings.
fn lower_node(n: Map<String, Any>) -> Map<String, Any> {
let shape: String = n["shape"]
let node_shape: String = shape_to_node_shape(shape)
let fill = ""
let stroke = ""
let color = ""
if shape == "primary" {
let fill = "#0052A0"
let stroke = "#0052A0"
let color = "#ffffff"
}
{ "id": n["id"], "label": n["label"], "sublabel": "",
"shape": node_shape,
"style_fill": fill, "style_stroke": stroke, "style_color": color }
}
fn lower_edge(e: Map<String, Any>) -> Map<String, Any> {
let kind: String = e["kind"]
let line = "solid"
let arrow = "forward"
if kind == "dashed" {
let line = "dashed"
}
if kind == "bidirectional" {
let arrow = "both"
}
// forbidden uses solid line + forward arrow; the renderer overlays the
// circle-X marker based on a forbidden-set the caller threads through.
{ "from": e["from"], "to": e["to"], "label": e["label"],
"line": line, "arrow": arrow }
}
fn lower_graph(g: Map<String, Any>) -> Map<String, Any> {
let nodes: [Map<String, Any>] = g["nodes"]
let edges: [Map<String, Any>] = g["edges"]
let lowered_nodes: [Map<String, Any>] = el_list_empty()
let i = 0
let n: Int = el_list_len(nodes)
while i < n {
let lowered_nodes = native_list_append(lowered_nodes, lower_node(get(nodes, i)))
let i = i + 1
}
let lowered_edges: [Map<String, Any>] = el_list_empty()
let i = 0
let m: Int = el_list_len(edges)
while i < m {
let lowered_edges = native_list_append(lowered_edges, lower_edge(get(edges, i)))
let i = i + 1
}
{ "title": g["title"], "direction": g["direction"],
"nodes": lowered_nodes, "edges": lowered_edges, "groups": g["groups"] }
}
// Find a node by id within a (lowered or raw) graph. Returns an empty map
// when not found callers check map_get(result, "id") for presence.
fn graph_find_node(graph: Map<String, Any>, id: String) -> Map<String, Any> {
let nodes: [Map<String, Any>] = graph["nodes"]
let n: Int = el_list_len(nodes)
let i = 0
while i < n {
let node: Map<String, Any> = get(nodes, i)
let nid: String = node["id"]
if nid == id { return node }
let i = i + 1
}
let empty: Map<String, Any> = el_map_new(0)
empty
}
// Forbidden-edge set helpers
// The lowered graph drops the "forbidden" kind (line/arrow have no slot for
// it). Callers preserve the set as a list of "from->to" strings.
fn forbidden_key(from: String, to: String) -> String {
from + "->" + to
}
fn collect_forbidden(graph: Map<String, Any>) -> [String] {
let edges: [Map<String, Any>] = graph["edges"]
let n: Int = el_list_len(edges)
let out: [String] = el_list_empty()
let i = 0
while i < n {
let e: Map<String, Any> = get(edges, i)
let kind: String = e["kind"]
if kind == "forbidden" {
let f: String = e["from"]
let t: String = e["to"]
let out = native_list_append(out, forbidden_key(f, t))
}
let i = i + 1
}
out
}
fn forbidden_contains(set: [String], src: String, dst: String) -> Bool {
let key: String = forbidden_key(src, dst)
let n: Int = el_list_len(set)
let i = 0
while i < n {
let s: String = get(set, i)
if s == key { return true }
let i = i + 1
}
false
}
// Smoke test
//
// State is kept in process-local k/v storage so we never mix Int + Call or
// Int + Ident in `+` (which the codegen heuristic emits as string concat
// on tagged-pointer values, segfaulting on Int operands).
fn fail(label: String, got: String, want: String) -> Int {
println("FAIL " + label + " got=[" + got + "] want=[" + want + "]")
state_set("failures", "1")
0
}
fn check_eq(label: String, got: String, want: String) -> Int {
if got == want {
println("ok " + label + " = " + got)
return 1
}
fail(label, got, want)
}
check_eq("sanitize crates/nc-core",
sanitize_id("crates/nc-core"), "crates_nc_core")
check_eq("sanitize package.json",
sanitize_id("package.json"), "package_json")
check_eq("sanitize 42-module",
sanitize_id("42-module"), "n42_module")
check_eq("sanitize empty", sanitize_id(""), "node")
check_eq("sanitize !!--@@", sanitize_id("!!--@@"), "node")
check_eq("shape_from_token rounded",
shape_from_token("rounded"), "rounded")
check_eq("shape_to_node_shape primary",
shape_to_node_shape("primary"), "rectangle")
// Lowering preserves a node id and adds style.
let n: Map<String, Any> = make_node("svc", "Service", "primary")
let ln: Map<String, Any> = lower_node(n)
check_eq("lower preserves id", ln["id"], "svc")
check_eq("lower applies primary fill", ln["style_fill"], "#0052A0")
// Edge lowering
let e: Map<String, Any> = make_edge("a", "b", "dashed")
let le: Map<String, Any> = lower_edge(e)
check_eq("lower edge dashed line", le["line"], "dashed")
let e2: Map<String, Any> = make_edge("a", "b", "bidirectional")
let le2: Map<String, Any> = lower_edge(e2)
check_eq("lower edge bidirectional arrow", le2["arrow"], "both")
println("")
let failures: String = state_get("failures")
if str_eq(failures, "1") {
println("arbor-core: FAILED")
exit_program(1)
} else {
println("arbor-core: ok")
}
-19
View File
@@ -1,19 +0,0 @@
// arbor-diagram diagram intermediate representation + Mermaid serializer
// + dependency-graph builders. Consumes raw graph values built by arbor-core
// or arbor-parse and produces Mermaid markup or other serializations.
vessel "arbor-diagram" {
version "0.1.0"
description "Diagram IR + Mermaid serializer + architecture diagram builders"
authors ["Neuron Technologies"]
edition "2026"
}
dependencies {
arbor-core "0.1"
}
build {
entry "src/main.el"
output "dist/"
}
-433
View File
@@ -1,433 +0,0 @@
// arbor-diagram diagram intermediate representation (AST + IR).
//
// Where arbor-core supplies the *.arbor source-language model Mermaid-safe
// IDs, ArborShape strings, ArborEdgeKind strings, and the lowered "diagram-
// form" map arbor-diagram exposes the same lowered model as the canonical
// IR for downstream serializers (arbor-render and any future Mermaid-style
// emitter). The two vessels overlap by design: arbor-core is responsible for
// *naming* the schema; arbor-diagram is responsible for *building* values
// against it.
//
// The Rust crate ships small AST builder structs (`DiagramNode::new`,
// `DiagramEdge::with_label`, `DiagramGraph::add_node`). El has no method
// chaining, no Default::default(), no enum types. The El idiom is a stack
// of immutable maps with explicit constructor + with_* helpers that take
// the value and return a freshly-allocated map.
//
// Public surface:
// make_node(id, label) DiagramNode
// with_shape(node, shape) DiagramNode
// with_sublabel(node, sublabel) DiagramNode
// with_style(node, fill, stroke, color) DiagramNode
//
// make_edge(from, to) DiagramEdge
// with_label(edge, label)
// with_line(edge, line) // "solid"/"dashed"/"dotted"/"thick"
// with_arrow(edge, arrow) // "forward"/"backward"/"both"/"none"
//
// make_group(id, label) DiagramGroup
// with_node(group, node_id)
// with_nodes(group, [node_id])
// with_direction(group, dir)
//
// make_graph(title) DiagramGraph
// with_direction(graph, dir)
// graph_add_node(graph, node) DiagramGraph
// graph_add_edge(graph, edge) DiagramGraph
// graph_add_group(graph, group) DiagramGraph
// graph_node(graph, id) DiagramNode | empty map
//
// Shape vocabulary (lowered): see arbor-core. The local copy here mirrors
// the table in arbor-core/src/main.el so this vessel is hermetic.
// NodeShape vocabulary
fn node_shape_rectangle() -> String { "rectangle" }
fn node_shape_rounded_rect() -> String { "rounded_rect" }
fn node_shape_stadium() -> String { "stadium" }
fn node_shape_cylinder() -> String { "cylinder" }
fn node_shape_diamond() -> String { "diamond" }
fn node_shape_parallelogram() -> String { "parallelogram" }
fn node_shape_database() -> String { "database" }
fn node_shape_subroutine() -> String { "subroutine" }
fn node_shape_valid(s: String) -> Bool {
if str_eq(s, "rectangle") { return true }
if str_eq(s, "rounded_rect") { return true }
if str_eq(s, "stadium") { return true }
if str_eq(s, "cylinder") { return true }
if str_eq(s, "diamond") { return true }
if str_eq(s, "parallelogram") { return true }
if str_eq(s, "database") { return true }
if str_eq(s, "subroutine") { return true }
false
}
// EdgeLine vocabulary
fn edge_line_solid() -> String { "solid" }
fn edge_line_dashed() -> String { "dashed" }
fn edge_line_dotted() -> String { "dotted" }
fn edge_line_thick() -> String { "thick" }
fn edge_line_valid(s: String) -> Bool {
if str_eq(s, "solid") { return true }
if str_eq(s, "dashed") { return true }
if str_eq(s, "dotted") { return true }
if str_eq(s, "thick") { return true }
false
}
// EdgeArrow vocabulary
fn edge_arrow_forward() -> String { "forward" }
fn edge_arrow_backward() -> String { "backward" }
fn edge_arrow_both() -> String { "both" }
fn edge_arrow_none() -> String { "none" }
fn edge_arrow_valid(s: String) -> Bool {
if str_eq(s, "forward") { return true }
if str_eq(s, "backward") { return true }
if str_eq(s, "both") { return true }
if str_eq(s, "none") { return true }
false
}
// Direction vocabulary
fn direction_top_down() -> String { "top-down" }
fn direction_left_right() -> String { "left-right" }
fn direction_right_left() -> String { "right-left" }
fn direction_bottom_up() -> String { "bottom-up" }
fn direction_valid(s: String) -> Bool {
if str_eq(s, "top-down") { return true }
if str_eq(s, "left-right") { return true }
if str_eq(s, "right-left") { return true }
if str_eq(s, "bottom-up") { return true }
false
}
// DiagramNode
fn make_node(id: String, label: String) -> Map<String, Any> {
{
"id": id,
"label": label,
"sublabel": "",
"shape": "rectangle",
"style_fill": "",
"style_stroke": "",
"style_color": ""
}
}
fn with_shape(node: Map<String, Any>, shape: String) -> Map<String, Any> {
{
"id": node["id"],
"label": node["label"],
"sublabel": node["sublabel"],
"shape": shape,
"style_fill": node["style_fill"],
"style_stroke": node["style_stroke"],
"style_color": node["style_color"]
}
}
fn with_sublabel(node: Map<String, Any>, sublabel: String) -> Map<String, Any> {
{
"id": node["id"],
"label": node["label"],
"sublabel": sublabel,
"shape": node["shape"],
"style_fill": node["style_fill"],
"style_stroke": node["style_stroke"],
"style_color": node["style_color"]
}
}
fn with_style(node: Map<String, Any>, fill: String, stroke: String, color: String) -> Map<String, Any> {
{
"id": node["id"],
"label": node["label"],
"sublabel": node["sublabel"],
"shape": node["shape"],
"style_fill": fill,
"style_stroke": stroke,
"style_color": color
}
}
// DiagramEdge
fn make_edge(from: String, to: String) -> Map<String, Any> {
{
"from": from,
"to": to,
"label": "",
"line": "solid",
"arrow": "forward"
}
}
fn with_label(edge: Map<String, Any>, label: String) -> Map<String, Any> {
{
"from": edge["from"],
"to": edge["to"],
"label": label,
"line": edge["line"],
"arrow": edge["arrow"]
}
}
fn with_line(edge: Map<String, Any>, line: String) -> Map<String, Any> {
{
"from": edge["from"],
"to": edge["to"],
"label": edge["label"],
"line": line,
"arrow": edge["arrow"]
}
}
fn with_arrow(edge: Map<String, Any>, arrow: String) -> Map<String, Any> {
{
"from": edge["from"],
"to": edge["to"],
"label": edge["label"],
"line": edge["line"],
"arrow": arrow
}
}
// DiagramGroup
fn make_group(id: String, label: String) -> Map<String, Any> {
let empty: [String] = native_list_empty()
{
"id": id,
"label": label,
"node_ids": empty,
"direction": ""
}
}
fn with_node(group: Map<String, Any>, node_id: String) -> Map<String, Any> {
let cur: [String] = group["node_ids"]
let next: [String] = native_list_append(cur, node_id)
{
"id": group["id"],
"label": group["label"],
"node_ids": next,
"direction": group["direction"]
}
}
fn with_nodes(group: Map<String, Any>, ids: [String]) -> Map<String, Any> {
let cur: [String] = group["node_ids"]
let n: Int = el_list_len(ids)
let i = 0
while i < n {
let cur = native_list_append(cur, get(ids, i))
let i = i + 1
}
{
"id": group["id"],
"label": group["label"],
"node_ids": cur,
"direction": group["direction"]
}
}
fn with_group_direction(group: Map<String, Any>, dir: String) -> Map<String, Any> {
{
"id": group["id"],
"label": group["label"],
"node_ids": group["node_ids"],
"direction": dir
}
}
// DiagramGraph
fn make_graph(title: String) -> Map<String, Any> {
let empty_n: [Map<String, Any>] = native_list_empty()
let empty_e: [Map<String, Any>] = native_list_empty()
let empty_g: [Map<String, Any>] = native_list_empty()
{
"title": title,
"direction": "top-down",
"nodes": empty_n,
"edges": empty_e,
"groups": empty_g
}
}
fn with_direction(graph: Map<String, Any>, dir: String) -> Map<String, Any> {
{
"title": graph["title"],
"direction": dir,
"nodes": graph["nodes"],
"edges": graph["edges"],
"groups": graph["groups"]
}
}
fn graph_add_node(graph: Map<String, Any>, node: Map<String, Any>) -> Map<String, Any> {
let cur: [Map<String, Any>] = graph["nodes"]
let next: [Map<String, Any>] = native_list_append(cur, node)
{
"title": graph["title"],
"direction": graph["direction"],
"nodes": next,
"edges": graph["edges"],
"groups": graph["groups"]
}
}
fn graph_add_edge(graph: Map<String, Any>, edge: Map<String, Any>) -> Map<String, Any> {
let cur: [Map<String, Any>] = graph["edges"]
let next: [Map<String, Any>] = native_list_append(cur, edge)
{
"title": graph["title"],
"direction": graph["direction"],
"nodes": graph["nodes"],
"edges": next,
"groups": graph["groups"]
}
}
fn graph_add_group(graph: Map<String, Any>, group: Map<String, Any>) -> Map<String, Any> {
let cur: [Map<String, Any>] = graph["groups"]
let next: [Map<String, Any>] = native_list_append(cur, group)
{
"title": graph["title"],
"direction": graph["direction"],
"nodes": graph["nodes"],
"edges": graph["edges"],
"groups": next
}
}
// Find a node by id. Returns an empty map (no "id" field) when not present.
fn graph_node(graph: Map<String, Any>, id: String) -> Map<String, Any> {
let nodes: [Map<String, Any>] = graph["nodes"]
let n: Int = el_list_len(nodes)
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
if str_eq(nid, id) { return nd }
let i = i + 1
}
let empty: Map<String, Any> = el_map_new(0)
empty
}
// Smoke test
fn fail(label: String, got: String, want: String) -> Int {
println("FAIL " + label + " got=[" + got + "] want=[" + want + "]")
state_set("smoke_failures", "1")
0
}
fn check_eq(label: String, got: String, want: String) -> Int {
if got == want {
println("ok " + label + " = " + got)
return 1
}
fail(label, got, want)
}
// Vocabulary self-checks
check_eq("shape rectangle valid",
bool_to_str(node_shape_valid("rectangle")), "true")
check_eq("shape hexagon invalid",
bool_to_str(node_shape_valid("hexagon")), "false")
check_eq("line dashed valid",
bool_to_str(edge_line_valid("dashed")), "true")
check_eq("arrow both valid",
bool_to_str(edge_arrow_valid("both")), "true")
check_eq("dir top-down valid",
bool_to_str(direction_valid("top-down")), "true")
// Node builder
let n0: Map<String, Any> = make_node("svc", "Service")
check_eq("node default shape", n0["shape"], "rectangle")
check_eq("node default sublabel empty", n0["sublabel"], "")
let n1: Map<String, Any> = with_shape(n0, "cylinder")
check_eq("node with_shape", n1["shape"], "cylinder")
check_eq("node id preserved", n1["id"], "svc")
let n2: Map<String, Any> = with_sublabel(n1, "v0.1.0")
check_eq("node with_sublabel", n2["sublabel"], "v0.1.0")
let n3: Map<String, Any> = with_style(n2, "#0052A0", "#0052A0", "#ffffff")
check_eq("node style fill", n3["style_fill"], "#0052A0")
check_eq("node style color", n3["style_color"], "#ffffff")
// Edge builder
let e0: Map<String, Any> = make_edge("a", "b")
check_eq("edge default line", e0["line"], "solid")
check_eq("edge default arrow", e0["arrow"], "forward")
let e1: Map<String, Any> = with_line(e0, "dashed")
let e2: Map<String, Any> = with_arrow(e1, "both")
let e3: Map<String, Any> = with_label(e2, "calls")
check_eq("edge line", e3["line"], "dashed")
check_eq("edge arrow", e3["arrow"], "both")
check_eq("edge label", e3["label"], "calls")
// Group builder
let g0: Map<String, Any> = make_group("core", "Application Core")
let g1: Map<String, Any> = with_node(g0, "api")
let g2: Map<String, Any> = with_node(g1, "svc")
let ids2: [String] = g2["node_ids"]
check_eq("group with two nodes", int_to_str(el_list_len(ids2)), "2")
let g3: Map<String, Any> = make_group("infra", "Infrastructure")
let extras: [String] = native_list_empty()
let extras = native_list_append(extras, "db")
let extras = native_list_append(extras, "cache")
let g4: Map<String, Any> = with_nodes(g3, extras)
let ids4: [String] = g4["node_ids"]
check_eq("group with_nodes appends", int_to_str(el_list_len(ids4)), "2")
// Graph builder + lookup
let G0: Map<String, Any> = make_graph("System")
let G1: Map<String, Any> = with_direction(G0, "left-right")
let G2: Map<String, Any> = graph_add_node(G1, n3)
let nb: Map<String, Any> = make_node("b", "Backend")
let G3: Map<String, Any> = graph_add_node(G2, nb)
let G4: Map<String, Any> = graph_add_edge(G3, e3)
let G5: Map<String, Any> = graph_add_group(G4, g4)
check_eq("graph title", G5["title"], "System")
check_eq("graph direction", G5["direction"], "left-right")
let gn: [Map<String, Any>] = G5["nodes"]
let ge: [Map<String, Any>] = G5["edges"]
let gg: [Map<String, Any>] = G5["groups"]
check_eq("graph nodes count", int_to_str(el_list_len(gn)), "2")
check_eq("graph edges count", int_to_str(el_list_len(ge)), "1")
check_eq("graph groups count", int_to_str(el_list_len(gg)), "1")
let found: Map<String, Any> = graph_node(G5, "svc")
check_eq("graph_node found", found["id"], "svc")
let missing: Map<String, Any> = graph_node(G5, "nonexistent")
let missing_id: String = missing["id"]
if str_len(missing_id) == 0 {
println("ok graph_node missing returns empty")
} else {
println("FAIL graph_node missing returned: " + missing_id)
state_set("smoke_failures", "1")
}
println("")
let failures: String = state_get("smoke_failures")
if str_eq(failures, "1") {
println("arbor-diagram: FAILED")
exit_program(1)
} else {
println("arbor-diagram: ok")
}
-19
View File
@@ -1,19 +0,0 @@
// arbor-layout hierarchical layout engine. Assigns (x, y) positions to
// every node, computes group bounding boxes, and the canvas size. Consumes
// a diagram graph; produces a layout-result value.
vessel "arbor-layout" {
version "0.1.0"
description "Hierarchical layout engine — rank assignment, positioning, group bounds"
authors ["Neuron Technologies"]
edition "2026"
}
dependencies {
arbor-core "0.1"
}
build {
entry "src/main.el"
output "dist/"
}
-591
View File
@@ -1,591 +0,0 @@
// arbor-layout hierarchical layout for diagram graphs.
//
// Public entry point:
// fn arbor_layout(graph: Map<String, Any>) -> Map<String, Any>
//
// The graph is the lowered (diagram-form) shape. The result map has:
// "node_pos_<id>" { "x":Float, "y":Float } centre point
// "node_size_<id>" { "w":Float, "h":Float }
// "group_bounds_<id>" { "x":Float, "y":Float, "w":Float, "h":Float }
// "node_ids" [String] iteration order
// "group_ids" [String] iteration order
// "canvas" { "w":Float, "h":Float }
//
// Floats are El-encoded store via the runtime's bit-cast convention.
// All arithmetic on positions/sizes is done in Float; integers (rank index)
// stay as Int.
//
// Algorithm (simplified Sugiyama):
// 1. Assign ranks via topological propagation (longest path from sources).
// 2. Group nodes by rank, preserving declaration order.
// 3. Position each rank as a row (top-down/bottom-up) or column (LR/RL).
// 4. Compute group bounding boxes from member positions.
// 5. Compute canvas size to enclose everything.
//
// The current implementation is the same simplified Sugiyama as the Rust
// version; perfectly identical numerical output is not promised but the
// relative ordering and bounding-box semantics match.
// Spacing constants (declared as float-bit-cast helpers)
fn k_node_base_w() -> el_val_t { int_to_float(120) }
fn k_node_base_h() -> el_val_t { int_to_float(40) }
fn k_node_char_extra() -> el_val_t { int_to_float(8) }
fn k_h_gap() -> el_val_t { int_to_float(60) }
fn k_v_gap() -> el_val_t { int_to_float(80) }
fn k_group_pad() -> el_val_t { int_to_float(20) }
fn k_margin() -> el_val_t { int_to_float(40) }
// Float-aware max/min via int_to_float / float arithmetic but el_max
// works in raw int comparison space, so we bit-cast carefully.
// For our purposes we only need monotonic comparisons on positive values,
// which IEEE 754 doubles + sign-magnitude bit patterns happen to preserve
// for non-negative floats but it's safer to do the comparison via the
// math layer. We use a helper that decodes both, picks the bigger, and
// re-encodes.
//
// Implemented in C terms: math_max(a, b) but el_runtime doesn't expose
// a float-aware max, so we synthesise one.
fn fmax(a: el_val_t, b: el_val_t) -> el_val_t {
// Compare via float subtraction's sign: a - b. Float subtraction is the
// multiply chain implemented via the C code generator. But el's `-` on
// bit-cast doubles doesn't perform IEEE arithmetic it's a 64-bit int
// subtract. Workaround: round-trip through format_float and str_to_float.
// For our layout numbers (small non-negative integers stored as floats)
// we can compare via the raw bits: a positive float's bit pattern is
// monotonically ordered, so `a > b` on the int reinterpretation gives
// the same result as on the actual double for non-negative values.
if a > b { return a }
b
}
fn fadd(a: el_val_t, b: el_val_t) -> el_val_t {
// a, b are bit-cast doubles. Safe addition: int-to-float, format, parse.
// For the small positive integers we work with, we reconstruct the
// numeric value via format_float str_to_float, perform addition by
// pulling them through str representations. Costly but correct on the
// current runtime. Fast path: if both are exact ints stored as floats
// we can also keep an Int "shadow" but the simpler approach is to
// route through the printf-based formatter once per layout pass.
let as: String = format_float(a, 6)
let bs: String = format_float(b, 6)
// Parse back to numeric.
let af: el_val_t = str_to_float(as)
let bf: el_val_t = str_to_float(bs)
// No real-add primitive; build the sum from int parts where possible.
// Convert to int at full resolution: float_to_int truncates towards zero,
// which for our values (always integer-valued) is exact.
let ai: Int = float_to_int(af)
let bi: Int = float_to_int(bf)
int_to_float(ai + bi)
}
fn fsub(a: el_val_t, b: el_val_t) -> el_val_t {
let ai: Int = float_to_int(a)
let bi: Int = float_to_int(b)
int_to_float(ai - bi)
}
fn fmul(a: el_val_t, b: el_val_t) -> el_val_t {
let ai: Int = float_to_int(a)
let bi: Int = float_to_int(b)
int_to_float(ai * bi)
}
fn fdiv2(a: el_val_t) -> el_val_t {
let ai: Int = float_to_int(a)
int_to_float(ai / 2)
}
// Node size based on label width
fn node_size_for(label: String) -> Map<String, Any> {
let len: Int = str_len(label)
let extra: Int = 0
if len > 10 {
let extra = len - 10
}
let w_int: Int = 120 + 8 * extra
let w: el_val_t = int_to_float(w_int)
let h: el_val_t = int_to_float(40)
{ "w": w, "h": h }
}
// Adjacency-list construction
//
// Builds successor and in-degree maps keyed by node id.
fn build_succ_indeg(graph: Map<String, Any>) -> Map<String, Any> {
let nodes: [Map<String, Any>] = graph["nodes"]
let edges: [Map<String, Any>] = graph["edges"]
let n: Int = el_list_len(nodes)
let m: Int = el_list_len(edges)
let succ: Map<String, Any> = el_map_new(0)
let indeg: Map<String, Any> = el_map_new(0)
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
let empty: [String] = el_list_empty()
let succ = el_map_set(succ, nid, empty)
let indeg = el_map_set(indeg, nid, 0)
let i = i + 1
}
let i = 0
while i < m {
let e: Map<String, Any> = get(edges, i)
let src: String = e["from"]
let dst: String = e["to"]
let cur_succ: [String] = el_map_get(succ, src)
let new_succ: [String] = native_list_append(cur_succ, dst)
let succ = el_map_set(succ, src, new_succ)
let prev: Int = el_map_get(indeg, dst)
let indeg = el_map_set(indeg, dst, prev + 1)
let i = i + 1
}
{ "succ": succ, "indeg": indeg }
}
// Topological rank assignment
//
// Returns a map: node_id rank.
fn assign_ranks(graph: Map<String, Any>) -> Map<String, Any> {
let nodes: [Map<String, Any>] = graph["nodes"]
let n: Int = el_list_len(nodes)
let adj: Map<String, Any> = build_succ_indeg(graph)
let succ: Map<String, Any> = adj["succ"]
let indeg: Map<String, Any> = adj["indeg"]
let ranks: Map<String, Any> = el_map_new(0)
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
let ranks = el_map_set(ranks, nid, 0)
let i = i + 1
}
// Initialise queue with all nodes whose in-degree is 0 (in declaration
// order, mirroring the Rust implementation's ordering guarantee).
let queue: [String] = el_list_empty()
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
let d: Int = el_map_get(indeg, nid)
if d == 0 {
let queue = native_list_append(queue, nid)
}
let i = i + 1
}
let head = 0
let running = true
while running {
if head >= el_list_len(queue) {
let running = false
} else {
let cur: String = get(queue, head)
let head = head + 1
let cur_rank: Int = el_map_get(ranks, cur)
let neighbours: [String] = el_map_get(succ, cur)
let nn: Int = el_list_len(neighbours)
let j = 0
while j < nn {
let nb: String = get(neighbours, j)
let nb_rank: Int = el_map_get(ranks, nb)
let cand: Int = cur_rank + 1
if cand > nb_rank {
let ranks = el_map_set(ranks, nb, cand)
}
let cur_d: Int = el_map_get(indeg, nb)
let new_d: Int = cur_d - 1
let indeg = el_map_set(indeg, nb, new_d)
if new_d <= 0 {
let queue = native_list_append(queue, nb)
}
let j = j + 1
}
}
}
ranks
}
// Layout pass
fn arbor_layout(graph: Map<String, Any>) -> Map<String, Any> {
let nodes: [Map<String, Any>] = graph["nodes"]
let n: Int = el_list_len(nodes)
let direction: String = graph["direction"]
let result: Map<String, Any> = el_map_new(0)
let result = el_map_set(result, "node_ids", el_list_empty())
let result = el_map_set(result, "group_ids", el_list_empty())
if n == 0 {
let canvas: Map<String, Any> = { "w": int_to_float(200), "h": int_to_float(100) }
let result = el_map_set(result, "canvas", canvas)
return result
}
let ranks: Map<String, Any> = assign_ranks(graph)
let max_rank = 0
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
let r: Int = el_map_get(ranks, nid)
if r > max_rank { let max_rank = r }
let i = i + 1
}
// Group nodes by rank, preserving declaration order. Buckets are stored
// in process state so we can iterate without nested-list mutation.
let i = 0
while i <= max_rank {
state_set("rank_bucket_" + int_to_str(i), "")
let i = i + 1
}
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
let r: Int = el_map_get(ranks, nid)
let key = "rank_bucket_" + int_to_str(r)
let prev: String = state_get(key)
if str_eq(prev, "") {
state_set(key, nid)
} else {
state_set(key, prev + "" + nid)
}
let i = i + 1
}
// Pre-compute sizes and stash a label-keyed cache.
let id_list: [String] = el_list_empty()
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
let lbl: String = nd["label"]
let sz: Map<String, Any> = node_size_for(lbl)
let result = el_map_set(result, "node_size_" + nid, sz)
let id_list = native_list_append(id_list, nid)
let i = i + 1
}
let result = el_map_set(result, "node_ids", id_list)
// Position pass.
let is_vertical = true
if str_eq(direction, "left-right") { let is_vertical = false }
if str_eq(direction, "right-left") { let is_vertical = false }
let cursor: el_val_t = k_margin()
let r = 0
while r <= max_rank {
let bucket_str: String = state_get("rank_bucket_" + int_to_str(r))
if !str_eq(bucket_str, "") {
let ids: [String] = str_split(bucket_str, "")
let ids_n: Int = el_list_len(ids)
// Track row height (for vertical) or column width (for horizontal).
let cross_max: el_val_t = int_to_float(40)
let j = 0
while j < ids_n {
let nid: String = get(ids, j)
let sz: Map<String, Any> = el_map_get(result, "node_size_" + nid)
if is_vertical {
let h: el_val_t = sz["h"]
let cross_max = fmax(cross_max, h)
} else {
let w: el_val_t = sz["w"]
let cross_max = fmax(cross_max, w)
}
let j = j + 1
}
if is_vertical {
let row_h: el_val_t = cross_max
let y_center: el_val_t = fadd(cursor, fdiv2(row_h))
let x_cursor: el_val_t = k_margin()
let j = 0
while j < ids_n {
let nid: String = get(ids, j)
let sz: Map<String, Any> = el_map_get(result, "node_size_" + nid)
let w: el_val_t = sz["w"]
let cx: el_val_t = fadd(x_cursor, fdiv2(w))
let pos: Map<String, Any> = { "x": cx, "y": y_center }
let result = el_map_set(result, "node_pos_" + nid, pos)
let x_cursor = fadd(fadd(x_cursor, w), k_h_gap())
let j = j + 1
}
let cursor = fadd(fadd(cursor, row_h), k_v_gap())
} else {
let col_w: el_val_t = cross_max
let x_center: el_val_t = fadd(cursor, fdiv2(col_w))
let y_cursor: el_val_t = k_margin()
let j = 0
while j < ids_n {
let nid: String = get(ids, j)
let sz: Map<String, Any> = el_map_get(result, "node_size_" + nid)
let h: el_val_t = sz["h"]
let cy: el_val_t = fadd(y_cursor, fdiv2(h))
let pos: Map<String, Any> = { "x": x_center, "y": cy }
let result = el_map_set(result, "node_pos_" + nid, pos)
let y_cursor = fadd(fadd(y_cursor, h), k_v_gap())
let j = j + 1
}
let cursor = fadd(fadd(cursor, col_w), k_h_gap())
}
} else {
// Empty bucket advance cursor by a default node size.
if is_vertical {
let cursor = fadd(cursor, fadd(int_to_float(40), k_v_gap()))
} else {
let cursor = fadd(cursor, fadd(k_node_base_w(), k_h_gap()))
}
}
let r = r + 1
}
// Direction inversions for BU / RL.
let need_flip_y = false
let need_flip_x = false
if str_eq(direction, "bottom-up") { let need_flip_y = true }
if str_eq(direction, "right-left") { let need_flip_x = true }
if need_flip_y {
let max_y: el_val_t = fadd(fsub(cursor, k_v_gap()), k_margin())
let i = 0
while i < n {
let nid: String = get(id_list, i)
let pos: Map<String, Any> = el_map_get(result, "node_pos_" + nid)
let y: el_val_t = pos["y"]
let new_y: el_val_t = fadd(fsub(max_y, y), k_margin())
let new_pos: Map<String, Any> = { "x": pos["x"], "y": new_y }
let result = el_map_set(result, "node_pos_" + nid, new_pos)
let i = i + 1
}
}
if need_flip_x {
let max_x: el_val_t = fadd(fsub(cursor, k_h_gap()), k_margin())
let i = 0
while i < n {
let nid: String = get(id_list, i)
let pos: Map<String, Any> = el_map_get(result, "node_pos_" + nid)
let x: el_val_t = pos["x"]
let new_x: el_val_t = fadd(fsub(max_x, x), k_margin())
let new_pos: Map<String, Any> = { "x": new_x, "y": pos["y"] }
let result = el_map_set(result, "node_pos_" + nid, new_pos)
let i = i + 1
}
}
// Group bounds.
let groups: [Map<String, Any>] = graph["groups"]
let gn: Int = el_list_len(groups)
let gid_list: [String] = el_list_empty()
let g = 0
while g < gn {
let grp: Map<String, Any> = get(groups, g)
let gid: String = grp["id"]
let member_ids: [String] = grp["node_ids"]
let mn: Int = el_list_len(member_ids)
if mn > 0 {
let big: Int = 1000000000
let neg: Int = 0 - 1000000000
let min_x: el_val_t = int_to_float(big)
let min_y: el_val_t = int_to_float(big)
let max_x: el_val_t = int_to_float(neg)
let max_y: el_val_t = int_to_float(neg)
let mi = 0
while mi < mn {
let mid: String = get(member_ids, mi)
let mpos: Map<String, Any> = el_map_get(result, "node_pos_" + mid)
let msz: Map<String, Any> = el_map_get(result, "node_size_" + mid)
let mid_present: String = mpos["x"]
if str_len(mid_present) >= 0 {
let cx: el_val_t = mpos["x"]
let cy: el_val_t = mpos["y"]
let mw: el_val_t = msz["w"]
let mh: el_val_t = msz["h"]
let left: el_val_t = fsub(cx, fdiv2(mw))
let right: el_val_t = fadd(cx, fdiv2(mw))
let top: el_val_t = fsub(cy, fdiv2(mh))
let bot: el_val_t = fadd(cy, fdiv2(mh))
if left < min_x { let min_x = left }
if top < min_y { let min_y = top }
if right > max_x { let max_x = right }
if bot > max_y { let max_y = bot }
}
let mi = mi + 1
}
let bx: el_val_t = fsub(min_x, k_group_pad())
let by: el_val_t = fsub(min_y, k_group_pad())
let bw: el_val_t = fadd(fsub(max_x, min_x), fmul(k_group_pad(), int_to_float(2)))
let bh: el_val_t = fadd(fsub(max_y, min_y), fmul(k_group_pad(), int_to_float(2)))
let bounds: Map<String, Any> = { "x": bx, "y": by, "w": bw, "h": bh }
let result = el_map_set(result, "group_bounds_" + gid, bounds)
let gid_list = native_list_append(gid_list, gid)
}
let g = g + 1
}
let result = el_map_set(result, "group_ids", gid_list)
// Canvas size = max node-right / node-bottom + group-right / group-bottom.
let canvas_w: el_val_t = int_to_float(0)
let canvas_h: el_val_t = int_to_float(0)
let i = 0
while i < n {
let nid: String = get(id_list, i)
let pos: Map<String, Any> = el_map_get(result, "node_pos_" + nid)
let sz: Map<String, Any> = el_map_get(result, "node_size_" + nid)
let right: el_val_t = fadd(pos["x"], fdiv2(sz["w"]))
let bottom: el_val_t = fadd(pos["y"], fdiv2(sz["h"]))
if right > canvas_w { let canvas_w = right }
if bottom > canvas_h { let canvas_h = bottom }
let i = i + 1
}
let i = 0
while i < el_list_len(gid_list) {
let gid: String = get(gid_list, i)
let b: Map<String, Any> = el_map_get(result, "group_bounds_" + gid)
let r: el_val_t = fadd(b["x"], b["w"])
let bt: el_val_t = fadd(b["y"], b["h"])
if r > canvas_w { let canvas_w = r }
if bt > canvas_h { let canvas_h = bt }
let i = i + 1
}
let canvas: Map<String, Any> = {
"w": fadd(canvas_w, k_margin()),
"h": fadd(canvas_h, k_margin())
}
let result = el_map_set(result, "canvas", canvas)
result
}
// Smoke test
fn fl_to_str(v: el_val_t) -> String {
int_to_str(float_to_int(v))
}
fn smoke_fail(label: String, msg: String) -> Int {
println("FAIL " + label + ": " + msg)
state_set("smoke_failures", "1")
0
}
fn make_test_node(id: String, label: String) -> Map<String, Any> {
{
"id": id, "label": label, "sublabel": "",
"shape": "rectangle",
"style_fill": "", "style_stroke": "", "style_color": ""
}
}
fn make_test_edge(src: String, dst: String) -> Map<String, Any> {
{ "from": src, "to": dst, "label": "", "line": "solid", "arrow": "forward" }
}
fn make_test_graph(direction: String, ids: [String], src_dst: [String]) -> Map<String, Any> {
let nodes: [Map<String, Any>] = el_list_empty()
let i = 0
while i < el_list_len(ids) {
let nid: String = get(ids, i)
let nodes = native_list_append(nodes, make_test_node(nid, nid))
let i = i + 1
}
let edges: [Map<String, Any>] = el_list_empty()
let i = 0
while i + 1 < el_list_len(src_dst) {
let s: String = get(src_dst, i)
let d: String = get(src_dst, i + 1)
let edges = native_list_append(edges, make_test_edge(s, d))
let i = i + 2
}
{
"title": "T", "direction": direction,
"nodes": nodes, "edges": edges, "groups": el_list_empty()
}
}
// Empty graph.
let g_empty: Map<String, Any> = {
"title": "e", "direction": "top-down",
"nodes": el_list_empty(), "edges": el_list_empty(), "groups": el_list_empty()
}
let r_empty: Map<String, Any> = arbor_layout(g_empty)
let canvas_empty: Map<String, Any> = r_empty["canvas"]
println("empty canvas w=" + fl_to_str(canvas_empty["w"]))
// Single node.
let g_one: Map<String, Any> = make_test_graph("top-down",
["solo"], el_list_empty())
let r_one: Map<String, Any> = arbor_layout(g_one)
let pos_solo: Map<String, Any> = el_map_get(r_one, "node_pos_solo")
let x_solo: el_val_t = pos_solo["x"]
let y_solo: el_val_t = pos_solo["y"]
println("solo at x=" + fl_to_str(x_solo) + " y=" + fl_to_str(y_solo))
if float_to_int(x_solo) <= 0 { smoke_fail("solo x", "expected > 0") }
if float_to_int(y_solo) <= 0 { smoke_fail("solo y", "expected > 0") }
// Linear chain abc top-down: ya < yb < yc.
let g_chain: Map<String, Any> = make_test_graph("top-down",
["a", "b", "c"], ["a", "b", "b", "c"])
let r_chain: Map<String, Any> = arbor_layout(g_chain)
let pa: Map<String, Any> = el_map_get(r_chain, "node_pos_a")
let pb: Map<String, Any> = el_map_get(r_chain, "node_pos_b")
let pc: Map<String, Any> = el_map_get(r_chain, "node_pos_c")
let ya: el_val_t = pa["y"]
let yb: el_val_t = pb["y"]
let yc: el_val_t = pc["y"]
println("td a.y=" + fl_to_str(ya) + " b.y=" + fl_to_str(yb) + " c.y=" + fl_to_str(yc))
if float_to_int(ya) >= float_to_int(yb) { smoke_fail("td order", "a.y >= b.y") }
if float_to_int(yb) >= float_to_int(yc) { smoke_fail("td order", "b.y >= c.y") }
// LR direction
let g_lr: Map<String, Any> = make_test_graph("left-right",
["a", "b", "c"], ["a", "b", "b", "c"])
let r_lr: Map<String, Any> = arbor_layout(g_lr)
let pa2: Map<String, Any> = el_map_get(r_lr, "node_pos_a")
let pc2: Map<String, Any> = el_map_get(r_lr, "node_pos_c")
let xa: el_val_t = pa2["x"]
let xc: el_val_t = pc2["x"]
println("lr a.x=" + fl_to_str(xa) + " c.x=" + fl_to_str(xc))
if float_to_int(xa) >= float_to_int(xc) { smoke_fail("lr order", "a.x >= c.x") }
// Bottom-up: a is below c.
let g_bu: Map<String, Any> = make_test_graph("bottom-up",
["a", "b", "c"], ["a", "b", "b", "c"])
let r_bu: Map<String, Any> = arbor_layout(g_bu)
let pa3: Map<String, Any> = el_map_get(r_bu, "node_pos_a")
let pc3: Map<String, Any> = el_map_get(r_bu, "node_pos_c")
let ya3: el_val_t = pa3["y"]
let yc3: el_val_t = pc3["y"]
println("bu a.y=" + fl_to_str(ya3) + " c.y=" + fl_to_str(yc3))
if float_to_int(ya3) <= float_to_int(yc3) { smoke_fail("bu order", "a.y <= c.y") }
// Canvas covers all nodes.
let canvas_chain: Map<String, Any> = r_chain["canvas"]
let cw: el_val_t = canvas_chain["w"]
let ch: el_val_t = canvas_chain["h"]
println("chain canvas w=" + fl_to_str(cw) + " h=" + fl_to_str(ch))
if float_to_int(cw) <= 0 { smoke_fail("canvas w", "non-positive") }
if float_to_int(ch) <= 0 { smoke_fail("canvas h", "non-positive") }
println("")
let f: String = state_get("smoke_failures")
if str_eq(f, "1") {
println("arbor-layout: FAILED")
exit_program(1)
} else {
println("arbor-layout: ok")
}
-19
View File
@@ -1,19 +0,0 @@
// arbor-parse hand-written recursive-descent parser for the .arbor source
// language. Produces an Arbor graph value consumable by arbor-layout and
// arbor-render.
vessel "arbor-parse" {
version "0.1.0"
description "Recursive-descent parser for the .arbor diagram language"
authors ["Neuron Technologies"]
edition "2026"
}
dependencies {
arbor-core "0.1"
}
build {
entry "src/main.el"
output "dist/"
}
-763
View File
@@ -1,763 +0,0 @@
// arbor-parse recursive-descent parser for the .arbor source language.
//
// This vessel inlines a private copy of the small set of arbor-core helpers
// it needs (sanitize_id and constructors). El's import form today is purely
// syntactic concatenation, so each vessel that wants to be its own buildable
// unit carries its own copy of these helpers. They're tiny (well under 100
// lines) and the duplication keeps each vessel hermetic.
//
// Public entry point: fn arbor_parse(source: String) -> Map<String, Any>
//
// Returns either a graph value or a parse-error map. Callers test for the
// "error" field:
// { "error": "..." , "line": Int, "text": "...source line..." } on failure
// { "title", "direction", "nodes", "edges", "groups" } on success
// Sanitisation (copy of arbor-core's sanitize_id)
fn is_alnum_underscore(ch: String) -> Bool {
let code: Int = str_char_code(ch, 0)
if code >= 48 {
if code <= 57 { return true }
}
if code >= 65 {
if code <= 90 { return true }
}
if code >= 97 {
if code <= 122 { return true }
}
if code == 95 { return true }
false
}
fn is_ascii_digit(ch: String) -> Bool {
let code: Int = str_char_code(ch, 0)
if code >= 48 {
if code <= 57 { return true }
}
false
}
fn sanitize_id(s: String) -> String {
let n: Int = str_len(s)
if n == 0 { return "node" }
let out = ""
let prev_underscore = false
let i = 0
while i < n {
let ch: String = str_char_at(s, i)
if is_alnum_underscore(ch) {
let out = out + ch
let prev_underscore = false
} else {
if !prev_underscore {
let out = out + "_"
}
let prev_underscore = true
}
let i = i + 1
}
let m: Int = str_len(out)
let end = m
let stripping = true
while stripping {
if end <= 0 {
let stripping = false
} else {
let last: String = str_char_at(out, end - 1)
if last == "_" {
let end = end - 1
} else {
let stripping = false
}
}
}
let out = str_slice(out, 0, end)
if str_len(out) == 0 { return "node" }
let first: String = str_char_at(out, 0)
if is_ascii_digit(first) {
let out = "n" + out
}
out
}
fn shape_from_token(tok: String) -> String {
let t: String = str_trim(tok)
if t == "rect" { return "rect" }
if t == "rounded" { return "rounded" }
if t == "cylinder" { return "cylinder" }
if t == "diamond" { return "diamond" }
if t == "stadium" { return "stadium" }
if t == "primary" { return "primary" }
""
}
// Line preprocessing
//
// Strip inline `// ...` comments, trim, drop empties. Returns a list of maps
// { "no": Int, "text": String }.
fn preprocess(source: String) -> [Map<String, Any>] {
let lines: [String] = str_split(source, "\n")
let n: Int = el_list_len(lines)
let out: [Map<String, Any>] = el_list_empty()
let i = 0
while i < n {
let raw: String = get(lines, i)
let cidx: Int = str_index_of(raw, "//")
let stripped = raw
if cidx >= 0 {
let stripped = str_slice(raw, 0, cidx)
}
let trimmed: String = str_trim(stripped)
if str_len(trimmed) > 0 {
let row: Map<String, Any> = { "no": i + 1, "text": trimmed }
let out = native_list_append(out, row)
}
let i = i + 1
}
out
}
// Quoted-string extraction
//
// Parses `"text"`-prefix from a string. Returns `{ "ok": Bool, "value": Str,
// "rest": Str }`. The `rest` field carries everything after the closing quote
// (so the caller can continue tokenising).
fn parse_quoted(s: String) -> Map<String, Any> {
let t: String = str_trim(s)
if str_len(t) < 2 {
return { "ok": false, "value": "", "rest": s }
}
let first: String = str_char_at(t, 0)
if first != "\"" {
return { "ok": false, "value": "", "rest": s }
}
let body: String = str_slice(t, 1, str_len(t))
let close: Int = str_index_of(body, "\"")
if close < 0 {
return { "ok": false, "value": "", "rest": s }
}
let inner: String = str_slice(body, 0, close)
let rest: String = str_slice(body, close + 1, str_len(body))
{ "ok": true, "value": inner, "rest": rest }
}
// Identifier prefix split
//
// `split_identifier("foo bar")` { "id": "foo", "rest": " bar" }.
// `split_identifier("a-b")` { "id": "a", "rest": "-b" }.
fn split_identifier(s: String) -> Map<String, Any> {
let n: Int = str_len(s)
let i = 0
while i < n {
let ch: String = str_char_at(s, i)
if !is_alnum_underscore(ch) {
return { "id": str_slice(s, 0, i), "rest": str_slice(s, i, n) }
}
let i = i + 1
}
{ "id": s, "rest": "" }
}
// Direction parsing
fn parse_direction(s: String) -> String {
let t: String = str_trim(s)
if t == "top-down" { return "top-down" }
if t == "TD" { return "top-down" }
if t == "left-right" { return "left-right" }
if t == "LR" { return "left-right" }
if t == "right-left" { return "right-left" }
if t == "RL" { return "right-left" }
if t == "bottom-up" { return "bottom-up" }
if t == "BU" { return "bottom-up" }
""
}
// Edge-arrow detection
//
// Detects the longest matching arrow token in a line, returning
// { "ok": Bool, "from_str": Str, "kind": Str, "rest": Str }
fn extract_edge_parts(line: String) -> Map<String, Any> {
// Order: longest first to avoid partial matches.
let f1: Int = str_index_of(line, "-/->")
if f1 >= 0 {
return { "ok": true,
"from_str": str_slice(line, 0, f1),
"kind": "forbidden",
"rest": str_slice(line, f1 + 4, str_len(line)) }
}
let f2: Int = str_index_of(line, "<->")
if f2 >= 0 {
return { "ok": true,
"from_str": str_slice(line, 0, f2),
"kind": "bidirectional",
"rest": str_slice(line, f2 + 3, str_len(line)) }
}
let f3: Int = str_index_of(line, "-->")
if f3 >= 0 {
return { "ok": true,
"from_str": str_slice(line, 0, f3),
"kind": "dashed",
"rest": str_slice(line, f3 + 3, str_len(line)) }
}
let f4: Int = str_index_of(line, "->")
if f4 >= 0 {
return { "ok": true,
"from_str": str_slice(line, 0, f4),
"kind": "solid",
"rest": str_slice(line, f4 + 2, str_len(line)) }
}
{ "ok": false, "from_str": "", "kind": "", "rest": "" }
}
fn is_edge_line(line: String) -> Bool {
if str_contains(line, "->") { return true }
if str_contains(line, "<->") { return true }
false
}
// Error helpers
fn make_error(line_no: Int, line_text: String, message: String) -> Map<String, Any> {
{ "error": message, "line": line_no, "text": line_text }
}
// Parse driver
//
// State is held in process-local k/v rather than threaded through every
// function. Specifically:
// "title", "direction" graph header
// "nodes_json", "edges_json", "groups_json" accumulators (string lists)
// "group_stack_depth" "0".."N" open groups
// "group_stack_<i>_id" / "_label" / "_line" frame data
// "group_stack_<i>_node_ids" JSON array of ids inside frame
// "error" non-empty if parse failed
// "error_line", "error_text" context
fn st_set_int(key: String, v: Int) -> Int { state_set(key, int_to_str(v)); 0 }
fn st_get_int(key: String) -> Int {
let s: String = state_get(key)
if str_eq(s, "") { return 0 }
str_to_int(s)
}
// Encode/decode small string lists via "" delimiter (unit separator).
fn list_encode(xs: [String]) -> String {
let n: Int = el_list_len(xs)
let out = ""
let i = 0
while i < n {
if i > 0 { let out = out + "" }
let out = out + get(xs, i)
let i = i + 1
}
out
}
fn list_decode(s: String) -> [String] {
if str_eq(s, "") { return el_list_empty() }
str_split(s, "")
}
fn current_group_index() -> Int {
st_get_int("group_stack_depth") - 1
}
fn group_frame_key(idx: Int, suffix: String) -> String {
"gs_" + int_to_str(idx) + "_" + suffix
}
fn open_group(id: String, label: String, line_no: Int) -> Int {
let depth: Int = st_get_int("group_stack_depth")
state_set(group_frame_key(depth, "id"), id)
state_set(group_frame_key(depth, "label"), label)
state_set(group_frame_key(depth, "line"), int_to_str(line_no))
state_set(group_frame_key(depth, "ids"), "")
st_set_int("group_stack_depth", depth + 1)
0
}
fn close_group_frame() -> Map<String, Any> {
let depth: Int = st_get_int("group_stack_depth")
if depth <= 0 {
return { "ok": false, "id": "", "label": "", "ids": "" }
}
let idx: Int = depth - 1
let id: String = state_get(group_frame_key(idx, "id"))
let label: String = state_get(group_frame_key(idx, "label"))
let ids: String = state_get(group_frame_key(idx, "ids"))
state_del(group_frame_key(idx, "id"))
state_del(group_frame_key(idx, "label"))
state_del(group_frame_key(idx, "line"))
state_del(group_frame_key(idx, "ids"))
st_set_int("group_stack_depth", idx)
{ "ok": true, "id": id, "label": label, "ids": ids }
}
fn register_node_in_group(node_id: String) -> Int {
let depth: Int = st_get_int("group_stack_depth")
if depth <= 0 { return 0 }
let idx: Int = depth - 1
let key: String = group_frame_key(idx, "ids")
let prev: String = state_get(key)
if str_eq(prev, "") {
state_set(key, node_id)
} else {
state_set(key, prev + "" + node_id)
}
0
}
// Accumulator JSON-ish encoding for nodes/edges/groups.
// We render each entry as a small string and stash in state under a counter.
fn store_node(id: String, label: String, shape: String) -> Int {
let n: Int = st_get_int("node_count")
state_set("node_id_" + int_to_str(n), id)
state_set("node_label_" + int_to_str(n), label)
state_set("node_shape_" + int_to_str(n), shape)
st_set_int("node_count", n + 1)
0
}
fn store_edge(src: String, dst: String, label: String, kind: String) -> Int {
let n: Int = st_get_int("edge_count")
state_set("edge_from_" + int_to_str(n), src)
state_set("edge_to_" + int_to_str(n), dst)
state_set("edge_label_" + int_to_str(n), label)
state_set("edge_kind_" + int_to_str(n), kind)
st_set_int("edge_count", n + 1)
0
}
fn store_group(id: String, label: String, ids: String) -> Int {
let n: Int = st_get_int("group_count")
state_set("group_id_" + int_to_str(n), id)
state_set("group_label_" + int_to_str(n), label)
state_set("group_ids_" + int_to_str(n), ids)
st_set_int("group_count", n + 1)
0
}
fn set_error(msg: String, line_no: Int, line_text: String) -> Int {
state_set("parse_error", msg)
st_set_int("parse_error_line", line_no)
state_set("parse_error_text", line_text)
0
}
fn has_error() -> Bool {
let m: String = state_get("parse_error")
if str_eq(m, "") { return false }
true
}
// Reset state at the start of each parse pass.
fn reset_state() -> Int {
state_set("graph_title", "")
state_set("graph_direction", "top-down")
st_set_int("node_count", 0)
st_set_int("edge_count", 0)
st_set_int("group_count", 0)
st_set_int("group_stack_depth", 0)
state_set("parse_error", "")
st_set_int("parse_error_line", 0)
state_set("parse_error_text", "")
0
}
// Statement-level parsing
fn parse_node_stmt(line_no: Int, line: String) -> Int {
let id_split: Map<String, Any> = split_identifier(line)
let raw_id: String = id_split["id"]
if str_eq(raw_id, "") {
set_error("expected node id, edge, or keyword", line_no, line)
return 0
}
let id: String = sanitize_id(raw_id)
let rest: String = str_trim(id_split["rest"])
// Optional shape: [token]
let shape = "rect"
let after_shape = rest
if str_len(rest) > 0 {
let lead: String = str_char_at(rest, 0)
if lead == "[" {
let close: Int = str_index_of(rest, "]")
if close < 0 {
set_error("unclosed `[` in shape token", line_no, line)
return 0
}
let token: String = str_slice(rest, 1, close)
let parsed_shape: String = shape_from_token(token)
if str_eq(parsed_shape, "") {
set_error("unknown shape `" + token + "`", line_no, line)
return 0
}
let shape = parsed_shape
let after_shape = str_trim(str_slice(rest, close + 1, str_len(rest)))
}
}
// Optional quoted label.
let quoted: Map<String, Any> = parse_quoted(after_shape)
let label = raw_id
let ok: Bool = quoted["ok"]
if ok {
let label = quoted["value"]
}
store_node(id, label, shape)
register_node_in_group(id)
1
}
fn parse_edge_stmt(line_no: Int, line: String) -> Int {
let parts: Map<String, Any> = extract_edge_parts(line)
let ok: Bool = parts["ok"]
if !ok {
set_error("malformed edge — expected `->` `-->` `<->` or `-/->`", line_no, line)
return 0
}
let from_str: String = parts["from_str"]
let rest_str: String = parts["rest"]
let kind: String = parts["kind"]
let src: String = sanitize_id(str_trim(from_str))
let rest_t: String = str_trim(rest_str)
let id_split: Map<String, Any> = split_identifier(rest_t)
let to_raw: String = id_split["id"]
if str_eq(to_raw, "") {
set_error("edge missing target node id", line_no, line)
return 0
}
let dst: String = sanitize_id(to_raw)
let label_rest: String = str_trim(id_split["rest"])
let quoted: Map<String, Any> = parse_quoted(label_rest)
let label = ""
let qok: Bool = quoted["ok"]
if qok {
let label = quoted["value"]
}
store_edge(src, dst, label, kind)
1
}
fn parse_group_open(line_no: Int, line: String, rest: String) -> Int {
// Strip trailing `{`.
let trimmed: String = str_trim(rest)
let n: Int = str_len(trimmed)
let body = trimmed
if n > 0 {
let last: String = str_char_at(trimmed, n - 1)
if last == "{" {
let body = str_trim(str_slice(trimmed, 0, n - 1))
}
}
let id_split: Map<String, Any> = split_identifier(body)
let raw_id: String = id_split["id"]
if str_eq(raw_id, "") {
set_error("group declaration missing id", line_no, line)
return 0
}
let label_rest: String = str_trim(id_split["rest"])
let quoted: Map<String, Any> = parse_quoted(label_rest)
let label = raw_id
let qok: Bool = quoted["ok"]
if qok {
let label = quoted["value"]
}
open_group(raw_id, label, line_no)
1
}
fn parse_close_brace(line_no: Int) -> Int {
let frame: Map<String, Any> = close_group_frame()
let frame_ok: Bool = frame["ok"]
if !frame_ok {
set_error("unexpected `}` — no open group", line_no, "}")
return 0
}
store_group(frame["id"], frame["label"], frame["ids"])
1
}
fn parse_line_dispatch(line_no: Int, line: String) -> Int {
if line == "}" { return parse_close_brace(line_no) }
if str_starts_with(line, "title:") {
let after: String = str_trim(str_slice(line, 6, str_len(line)))
let q: Map<String, Any> = parse_quoted(after)
let qok: Bool = q["ok"]
if !qok {
set_error("expected quoted string after `title:`", line_no, line)
return 0
}
state_set("graph_title", q["value"])
return 1
}
if str_starts_with(line, "direction:") {
let after: String = str_trim(str_slice(line, 10, str_len(line)))
let dir: String = parse_direction(after)
if str_eq(dir, "") {
set_error("unknown direction — expected top-down, left-right, right-left, or bottom-up",
line_no, line)
return 0
}
state_set("graph_direction", dir)
return 1
}
if str_starts_with(line, "group ") {
let after: String = str_slice(line, 6, str_len(line))
return parse_group_open(line_no, line, after)
}
if is_edge_line(line) {
return parse_edge_stmt(line_no, line)
}
parse_node_stmt(line_no, line)
}
// Materialise accumulators into the final graph map
fn build_graph_value() -> Map<String, Any> {
let n_nodes: Int = st_get_int("node_count")
let nodes: [Map<String, Any>] = el_list_empty()
let i = 0
while i < n_nodes {
let s: String = int_to_str(i)
let node: Map<String, Any> = {
"id": state_get("node_id_" + s),
"label": state_get("node_label_" + s),
"shape": state_get("node_shape_" + s)
}
let nodes = native_list_append(nodes, node)
let i = i + 1
}
let n_edges: Int = st_get_int("edge_count")
let edges: [Map<String, Any>] = el_list_empty()
let i = 0
while i < n_edges {
let s: String = int_to_str(i)
let edge: Map<String, Any> = {
"from": state_get("edge_from_" + s),
"to": state_get("edge_to_" + s),
"label": state_get("edge_label_" + s),
"kind": state_get("edge_kind_" + s)
}
let edges = native_list_append(edges, edge)
let i = i + 1
}
let n_groups: Int = st_get_int("group_count")
let groups: [Map<String, Any>] = el_list_empty()
let i = 0
while i < n_groups {
let s: String = int_to_str(i)
let raw_ids: String = state_get("group_ids_" + s)
let id_list: [String] = list_decode(raw_ids)
let group: Map<String, Any> = {
"id": state_get("group_id_" + s),
"label": state_get("group_label_" + s),
"node_ids": id_list,
"direction": ""
}
let groups = native_list_append(groups, group)
let i = i + 1
}
{
"title": state_get("graph_title"),
"direction": state_get("graph_direction"),
"nodes": nodes,
"edges": edges,
"groups": groups
}
}
// Public entry point
fn arbor_parse(source: String) -> Map<String, Any> {
reset_state()
let lines: [Map<String, Any>] = preprocess(source)
let n: Int = el_list_len(lines)
let i = 0
let abort = false
while i < n {
if abort {
// skip error already recorded
} else {
let row: Map<String, Any> = get(lines, i)
let line_no: Int = row["no"]
let text: String = row["text"]
parse_line_dispatch(line_no, text)
if has_error() {
let abort = true
}
}
let i = i + 1
}
if !has_error() {
let depth: Int = st_get_int("group_stack_depth")
if depth > 0 {
let idx: Int = depth - 1
let id: String = state_get(group_frame_key(idx, "id"))
let line_no: Int = st_get_int(group_frame_key(idx, "line"))
set_error("unclosed group '" + id + "' — missing closing `}`",
line_no, "group " + id)
}
}
if has_error() {
return {
"error": state_get("parse_error"),
"line": st_get_int("parse_error_line"),
"text": state_get("parse_error_text")
}
}
build_graph_value()
}
// Smoke test
fn fail_msg(label: String, got: String, want: String) -> Int {
println("FAIL " + label + " got=[" + got + "] want=[" + want + "]")
state_set("smoke_failures", "1")
0
}
fn check_eq(label: String, got: String, want: String) -> Int {
if got == want {
println("ok " + label)
return 1
}
fail_msg(label, got, want)
}
// Helper: a graph map is in the error state iff it has a non-empty "error".
fn parse_failed(g: Map<String, Any>) -> Bool {
let m: String = g["error"]
if str_eq(m, "") { return false }
// map_get returns NULL for missing keys; str_eq treats two NULLs as equal
// and NULL vs "" as not equal guard explicitly.
if str_len(m) == 0 { return false }
true
}
let src1 = "title: \"Test\"\ndirection: left-right\n\napi [rounded] \"REST API\"\ndb [cylinder] \"Postgres\"\n\napi -> db \"reads\""
let g1: Map<String, Any> = arbor_parse(src1)
if parse_failed(g1) {
println("FAIL parse 1: " + g1["error"])
state_set("smoke_failures", "1")
}
check_eq("title parsed", g1["title"], "Test")
check_eq("direction parsed", g1["direction"], "left-right")
let nodes1: [Map<String, Any>] = g1["nodes"]
let nn1: Int = el_list_len(nodes1)
check_eq("two nodes", int_to_str(nn1), "2")
let edges1: [Map<String, Any>] = g1["edges"]
let ne1: Int = el_list_len(edges1)
check_eq("one edge", int_to_str(ne1), "1")
let e0: Map<String, Any> = get(edges1, 0)
check_eq("edge from", e0["from"], "api")
check_eq("edge to", e0["to"], "db")
check_eq("edge label", e0["label"], "reads")
check_eq("edge kind", e0["kind"], "solid")
let n0: Map<String, Any> = get(nodes1, 0)
check_eq("node 0 shape", n0["shape"], "rounded")
check_eq("node 0 label", n0["label"], "REST API")
// Test edge varieties
let src2 = "a \"A\"\nb \"B\"\na -> b\na --> b\na -/-> b\na <-> b"
let g2: Map<String, Any> = arbor_parse(src2)
let edges2: [Map<String, Any>] = g2["edges"]
check_eq("4 edges parsed", int_to_str(el_list_len(edges2)), "4")
let kinds = ""
let i = 0
while i < el_list_len(edges2) {
let e: Map<String, Any> = get(edges2, i)
let k: String = e["kind"]
let kinds = kinds + k + ","
let i = i + 1
}
check_eq("edge kinds", kinds, "solid,dashed,forbidden,bidirectional,")
// Groups
let src3 = "group core \"Application Core\" {\n api [rounded] \"REST API\"\n svc \"Business Logic\"\n}\nstandalone \"Out\""
let g3: Map<String, Any> = arbor_parse(src3)
let groups3: [Map<String, Any>] = g3["groups"]
check_eq("one group", int_to_str(el_list_len(groups3)), "1")
let grp0: Map<String, Any> = get(groups3, 0)
check_eq("group label", grp0["label"], "Application Core")
let gnids: [String] = grp0["node_ids"]
check_eq("group has 2 members", int_to_str(el_list_len(gnids)), "2")
let nodes3: [Map<String, Any>] = g3["nodes"]
check_eq("3 total nodes (incl standalone)",
int_to_str(el_list_len(nodes3)), "3")
// Error: unknown shape
let src4 = "node [hexagon] \"X\""
let g4: Map<String, Any> = arbor_parse(src4)
let err4: String = g4["error"]
if str_eq(err4, "") {
println("FAIL expected error for unknown shape")
state_set("smoke_failures", "1")
} else {
if str_contains(err4, "hexagon") {
println("ok error mentions hexagon: " + err4)
} else {
println("FAIL error wording: " + err4)
state_set("smoke_failures", "1")
}
}
// Error: unclosed group
let src5 = "group g \"G\" {\n a \"A\"\n"
let g5: Map<String, Any> = arbor_parse(src5)
let err5: String = g5["error"]
if str_eq(err5, "") {
println("FAIL expected unclosed-group error")
state_set("smoke_failures", "1")
} else {
if str_contains(err5, "unclosed") {
println("ok unclosed group detected")
} else {
println("FAIL unclosed error wording: " + err5)
state_set("smoke_failures", "1")
}
}
// Comments and inline comments
let src6 = "// header\na \"A\" // trailing\nb \"B\""
let g6: Map<String, Any> = arbor_parse(src6)
check_eq("comments stripped", int_to_str(el_list_len(g6["nodes"])), "2")
// Empty input
let g7: Map<String, Any> = arbor_parse("")
check_eq("empty graph nodes", int_to_str(el_list_len(g7["nodes"])), "0")
check_eq("empty graph default direction", g7["direction"], "top-down")
println("")
let f: String = state_get("smoke_failures")
if str_eq(f, "1") {
println("arbor-parse: FAILED")
exit_program(1)
} else {
println("arbor-parse: ok")
}
-21
View File
@@ -1,21 +0,0 @@
// arbor-render SVG renderer. Consumes a diagram graph + layout result and
// emits an SVG document. PNG rasterization is not provided in this vessel
// because the El runtime does not expose a vector-to-raster primitive yet
// (see report).
vessel "arbor-render" {
version "0.1.0"
description "SVG renderer for Arbor diagrams"
authors ["Neuron Technologies"]
edition "2026"
}
dependencies {
arbor-core "0.1"
arbor-layout "0.1"
}
build {
entry "src/main.el"
output "dist/"
}
-575
View File
@@ -1,575 +0,0 @@
// arbor-render SVG emission from a laid-out diagram.
//
// Entry point:
// fn arbor_render_svg(graph: Map, layout: Map, forbidden: [String]) -> String
//
// The graph is the lowered (diagram-form) shape produced by arbor-core /
// arbor-diagram (`title`, `direction`, `nodes`, `edges`, `groups`). The
// layout is whatever arbor-layout returned: `node_pos_<id>`, `node_size_<id>`,
// `group_bounds_<id>`, `node_ids`, `group_ids`, `canvas`.
//
// `forbidden` is a list of "from->to" key strings same format as
// arbor-core's collect_forbidden(). The Rust crate threaded a HashSet
// through; El threads a list and we linear-scan.
//
// SVG is text emission straightforward El. Every float coordinate is
// passed through format_float(_, 1) for stable output.
//
// PNG render is intentionally out of scope
// The Rust crate rasterises via resvg tiny_skia png. The El runtime
// today exposes no equivalent: there is no resvg, no usvg, no font rasterer,
// no PNG encoder, no path-fill code. fs_write writes text only there is
// no binary write primitive. arbor_render_png() returns an error map in El
// until the runtime grows a rasterer (see "runtime gaps" in the report).
// Colour palette (matches the Rust constants exactly)
fn col_node_fill() -> String { "#ffffff" }
fn col_node_stroke() -> String { "#334155" }
fn col_primary_fill() -> String { "#0052A0" }
fn col_primary_text() -> String { "#ffffff" }
fn col_node_text() -> String { "#0D0D14" }
fn col_edge() -> String { "#64748B" }
fn col_edge_forbidden() -> String { "#DC2626" }
fn col_group_fill() -> String { "rgba(0,0,0,0.03)" }
fn col_group_stroke() -> String { "#CBD5E1" }
fn col_group_text() -> String { "#64748B" }
fn col_edge_label() -> String { "#64748B" }
// XML escape
fn esc(s: String) -> String {
let r1: String = str_replace(s, "&", "&amp;")
let r2: String = str_replace(r1, "<", "&lt;")
let r3: String = str_replace(r2, ">", "&gt;")
let r4: String = str_replace(r3, "\"", "&quot;")
r4
}
// Float to "%.1f" the Rust pt() helper.
fn pt(v: el_val_t) -> String {
format_float(v, 1)
}
// Float arithmetic helpers float_to_int / int_to_float trip through Int,
// which is exact for the integer-valued floats used by the layout pass.
fn fadd(a: el_val_t, b: el_val_t) -> el_val_t {
let ai: Int = float_to_int(a)
let bi: Int = float_to_int(b)
int_to_float(ai + bi)
}
fn fsub(a: el_val_t, b: el_val_t) -> el_val_t {
let ai: Int = float_to_int(a)
let bi: Int = float_to_int(b)
int_to_float(ai - bi)
}
fn fdiv2(a: el_val_t) -> el_val_t {
let ai: Int = float_to_int(a)
int_to_float(ai / 2)
}
fn fmid(a: el_val_t, b: el_val_t) -> el_val_t {
fdiv2(fadd(a, b))
}
// forbidden-edge linear lookup
fn forbidden_key(from: String, to: String) -> String {
from + "->" + to
}
fn forbidden_contains(set: [String], src: String, dst: String) -> Bool {
let key: String = forbidden_key(src, dst)
let n: Int = el_list_len(set)
let i = 0
while i < n {
let s: String = get(set, i)
if str_eq(s, key) { return true }
let i = i + 1
}
false
}
// Arrow marker defs
fn arrow_defs() -> String {
let s = "\n <marker id=\"ah\" markerWidth=\"10\" markerHeight=\"7\" refX=\"9\" refY=\"3.5\" orient=\"auto\">\n"
let s = s + " <polygon points=\"0 0, 10 3.5, 0 7\" fill=\"" + col_edge() + "\"/>\n"
let s = s + " </marker>\n"
let s = s + " <marker id=\"ah-bi\" markerWidth=\"10\" markerHeight=\"7\" refX=\"1\" refY=\"3.5\" orient=\"auto-start-reverse\">\n"
let s = s + " <polygon points=\"0 0, 10 3.5, 0 7\" fill=\"" + col_edge() + "\"/>\n"
let s = s + " </marker>\n"
let s = s + " <marker id=\"ah-red\" markerWidth=\"10\" markerHeight=\"7\" refX=\"9\" refY=\"3.5\" orient=\"auto\">\n"
let s = s + " <polygon points=\"0 0, 10 3.5, 0 7\" fill=\"" + col_edge_forbidden() + "\"/>\n"
let s = s + " </marker>"
s
}
// Node rendering
fn render_node(buf: String, node: Map<String, Any>, layout: Map<String, Any>) -> String {
let nid: String = node["id"]
let pos: Map<String, Any> = el_map_get(layout, "node_pos_" + nid)
let sz: Map<String, Any> = el_map_get(layout, "node_size_" + nid)
let cx: el_val_t = pos["x"]
let cy: el_val_t = pos["y"]
let w: el_val_t = sz["w"]
let h: el_val_t = sz["h"]
let x: el_val_t = fsub(cx, fdiv2(w))
let y: el_val_t = fsub(cy, fdiv2(h))
let fill_in: String = node["style_fill"]
let stroke_in: String = node["style_stroke"]
let color_in: String = node["style_color"]
let fill = col_node_fill()
if str_len(fill_in) > 0 { let fill = fill_in }
let stroke = col_node_stroke()
if str_len(stroke_in) > 0 { let stroke = stroke_in }
let text_col = col_node_text()
if str_len(color_in) > 0 { let text_col = color_in }
let shape: String = node["shape"]
let buf = buf
if str_eq(shape, "rectangle") {
let buf = buf + " <rect x=\"" + pt(x) + "\" y=\"" + pt(y)
let buf = buf + "\" width=\"" + pt(w) + "\" height=\"" + pt(h)
let buf = buf + "\" rx=\"4\" fill=\"" + fill + "\" stroke=\"" + stroke
let buf = buf + "\" stroke-width=\"1.5\"/>\n"
}
if str_eq(shape, "rounded_rect") {
let buf = buf + " <rect x=\"" + pt(x) + "\" y=\"" + pt(y)
let buf = buf + "\" width=\"" + pt(w) + "\" height=\"" + pt(h)
let buf = buf + "\" rx=\"20\" fill=\"" + fill + "\" stroke=\"" + stroke
let buf = buf + "\" stroke-width=\"1.5\"/>\n"
}
if str_eq(shape, "stadium") {
let buf = buf + " <rect x=\"" + pt(x) + "\" y=\"" + pt(y)
let buf = buf + "\" width=\"" + pt(w) + "\" height=\"" + pt(h)
let buf = buf + "\" rx=\"" + pt(fdiv2(h)) + "\" fill=\"" + fill
let buf = buf + "\" stroke=\"" + stroke + "\" stroke-width=\"1.5\"/>\n"
}
if str_eq(shape, "cylinder") {
// body: rect from y+ry to bottom; ry ≈ h/6 (Rust uses h*0.18, we use h/6
// to stay in integer arithmetic visually indistinguishable on the
// canvas sizes the layout produces).
let hi: Int = float_to_int(h)
let ry: el_val_t = int_to_float(hi / 6)
let body_y: el_val_t = fadd(y, ry)
let body_h: el_val_t = fsub(h, ry)
let buf = buf + " <rect x=\"" + pt(x) + "\" y=\"" + pt(body_y)
let buf = buf + "\" width=\"" + pt(w) + "\" height=\"" + pt(body_h)
let buf = buf + "\" fill=\"" + fill + "\" stroke=\"" + stroke + "\" stroke-width=\"1.5\"/>\n"
// top ellipse
let buf = buf + " <ellipse cx=\"" + pt(cx) + "\" cy=\"" + pt(body_y)
let buf = buf + "\" rx=\"" + pt(fdiv2(w)) + "\" ry=\"" + pt(ry)
let buf = buf + "\" fill=\"" + fill + "\" stroke=\"" + stroke + "\" stroke-width=\"1.5\"/>\n"
// bottom ellipse
let bot_y: el_val_t = fadd(y, h)
let buf = buf + " <ellipse cx=\"" + pt(cx) + "\" cy=\"" + pt(bot_y)
let buf = buf + "\" rx=\"" + pt(fdiv2(w)) + "\" ry=\"" + pt(ry)
let buf = buf + "\" fill=\"" + fill + "\" stroke=\"" + stroke + "\" stroke-width=\"1.5\"/>\n"
}
if str_eq(shape, "diamond") {
let hw: el_val_t = fdiv2(w)
let hh: el_val_t = fdiv2(h)
let buf = buf + " <polygon points=\""
let buf = buf + pt(cx) + "," + pt(fsub(cy, hh)) + " "
let buf = buf + pt(fadd(cx, hw)) + "," + pt(cy) + " "
let buf = buf + pt(cx) + "," + pt(fadd(cy, hh)) + " "
let buf = buf + pt(fsub(cx, hw)) + "," + pt(cy)
let buf = buf + "\" fill=\"" + fill + "\" stroke=\"" + stroke + "\" stroke-width=\"1.5\"/>\n"
}
// Label.
let label: String = node["label"]
let buf = buf + " <text x=\"" + pt(cx) + "\" y=\"" + pt(cy)
let buf = buf + "\" text-anchor=\"middle\" dominant-baseline=\"middle\""
let buf = buf + " class=\"arbor-node-label\" fill=\"" + text_col + "\">"
let buf = buf + esc(label) + "</text>\n"
// Sublabel Rust's DiagramNode stores Option<String>; El uses "" sentinel.
let sub: String = node["sublabel"]
if str_len(sub) > 0 {
let sub_y: el_val_t = fadd(cy, int_to_float(14))
let buf = buf + " <text x=\"" + pt(cx) + "\" y=\"" + pt(sub_y)
let buf = buf + "\" text-anchor=\"middle\" dominant-baseline=\"middle\""
let buf = buf + " class=\"arbor-node-label\" fill=\"" + text_col + "\" font-size=\"10\">"
let buf = buf + esc(sub) + "</text>\n"
}
buf
}
// Edge rendering
//
// We emit a straight line from one node centre to the other and let the
// browser draw it; the Rust crate renders cubic bezier paths but the runtime
// has no robust math layer, and the rectangles are large enough that
// straight edges read clearly. (See "runtime gaps".)
fn render_edge(buf: String, edge: Map<String, Any>, layout: Map<String, Any>, forbidden: [String]) -> String {
let from_id: String = edge["from"]
let to_id: String = edge["to"]
let from_pos: Map<String, Any> = el_map_get(layout, "node_pos_" + from_id)
let to_pos: Map<String, Any> = el_map_get(layout, "node_pos_" + to_id)
let fx: el_val_t = from_pos["x"]
let fy: el_val_t = from_pos["y"]
let tx: el_val_t = to_pos["x"]
let ty: el_val_t = to_pos["y"]
let is_forbidden: Bool = forbidden_contains(forbidden, from_id, to_id)
let stroke = col_edge()
if is_forbidden { let stroke = col_edge_forbidden() }
let line: String = edge["line"]
let arrow: String = edge["arrow"]
let dash_attr = ""
if str_eq(line, "dashed") { let dash_attr = " stroke-dasharray=\"5,3\"" }
if str_eq(line, "dotted") { let dash_attr = " stroke-dasharray=\"2,2\"" }
let marker_start = ""
if str_eq(arrow, "both") { let marker_start = " marker-start=\"url(#ah-bi)\"" }
if str_eq(arrow, "backward") { let marker_start = " marker-start=\"url(#ah-bi)\"" }
let marker_end = " marker-end=\"url(#ah)\""
if is_forbidden { let marker_end = " marker-end=\"url(#ah-red)\"" }
if str_eq(arrow, "none") { let marker_end = "" }
if str_eq(arrow, "backward") { let marker_end = "" }
let buf = buf + " <line x1=\"" + pt(fx) + "\" y1=\"" + pt(fy)
let buf = buf + "\" x2=\"" + pt(tx) + "\" y2=\"" + pt(ty)
let buf = buf + "\" stroke=\"" + stroke + "\" stroke-width=\"1.5\""
let buf = buf + dash_attr + marker_start + marker_end + "/>\n"
// Forbidden marker circle-X at midpoint.
if is_forbidden {
let mx: el_val_t = fmid(fx, tx)
let my: el_val_t = fmid(fy, ty)
let r: el_val_t = int_to_float(7)
let buf = buf + " <circle cx=\"" + pt(mx) + "\" cy=\"" + pt(my)
let buf = buf + "\" r=\"" + pt(r) + "\" fill=\"white\" stroke=\""
let buf = buf + col_edge_forbidden() + "\" stroke-width=\"1.5\"/>\n"
let off: el_val_t = int_to_float(4)
let buf = buf + " <line x1=\"" + pt(fsub(mx, off)) + "\" y1=\"" + pt(fsub(my, off))
let buf = buf + "\" x2=\"" + pt(fadd(mx, off)) + "\" y2=\"" + pt(fadd(my, off))
let buf = buf + "\" stroke=\"" + col_edge_forbidden() + "\" stroke-width=\"1.5\"/>\n"
let buf = buf + " <line x1=\"" + pt(fadd(mx, off)) + "\" y1=\"" + pt(fsub(my, off))
let buf = buf + "\" x2=\"" + pt(fsub(mx, off)) + "\" y2=\"" + pt(fadd(my, off))
let buf = buf + "\" stroke=\"" + col_edge_forbidden() + "\" stroke-width=\"1.5\"/>\n"
}
// Edge label
let label: String = edge["label"]
if str_len(label) > 0 {
let mx: el_val_t = fmid(fx, tx)
let my: el_val_t = fmid(fy, ty)
let lw: el_val_t = int_to_float(str_len(label) * 7 + 8)
let lh: el_val_t = int_to_float(16)
let buf = buf + " <rect x=\"" + pt(fsub(mx, fdiv2(lw))) + "\" y=\"" + pt(fsub(my, fdiv2(lh)))
let buf = buf + "\" width=\"" + pt(lw) + "\" height=\"" + pt(lh)
let buf = buf + "\" rx=\"3\" fill=\"white\" opacity=\"0.85\"/>\n"
let buf = buf + " <text x=\"" + pt(mx) + "\" y=\"" + pt(my)
let buf = buf + "\" text-anchor=\"middle\" dominant-baseline=\"middle\""
let buf = buf + " class=\"arbor-edge-label\">" + esc(label) + "</text>\n"
}
buf
}
// Group rendering
fn render_group(buf: String, group: Map<String, Any>, layout: Map<String, Any>) -> String {
let gid: String = group["id"]
let bounds: Map<String, Any> = el_map_get(layout, "group_bounds_" + gid)
// Layout may not have bounds for empty groups defensive.
let bx_check: el_val_t = bounds["x"]
if float_to_int(bx_check) == 0 {
// Could be a real 0; cheaper to skip via presence check on group_ids.
}
let bx: el_val_t = bounds["x"]
let by: el_val_t = bounds["y"]
let bw: el_val_t = bounds["w"]
let bh: el_val_t = bounds["h"]
let buf = buf + " <rect x=\"" + pt(bx) + "\" y=\"" + pt(by)
let buf = buf + "\" width=\"" + pt(bw) + "\" height=\"" + pt(bh)
let buf = buf + "\" rx=\"8\" fill=\"" + col_group_fill() + "\" stroke=\""
let buf = buf + col_group_stroke() + "\" stroke-width=\"1\" stroke-dasharray=\"4,3\"/>\n"
// Group label in the top-left corner.
let lx: el_val_t = fadd(bx, int_to_float(8))
let ly: el_val_t = fadd(by, int_to_float(14))
let label: String = group["label"]
let buf = buf + " <text x=\"" + pt(lx) + "\" y=\"" + pt(ly)
let buf = buf + "\" class=\"arbor-group-label\">" + esc(label) + "</text>\n"
buf
}
// Public entry point
fn arbor_render_svg(graph: Map<String, Any>, layout: Map<String, Any>, forbidden: [String]) -> String {
let canvas: Map<String, Any> = el_map_get(layout, "canvas")
let cw: el_val_t = canvas["w"]
let ch: el_val_t = canvas["h"]
let buf = "<svg xmlns=\"http://www.w3.org/2000/svg\" width=\"" + pt(cw)
let buf = buf + "\" height=\"" + pt(ch) + "\" viewBox=\"0 0 " + pt(cw) + " " + pt(ch) + "\">\n"
let buf = buf + " <defs>"
let buf = buf + arrow_defs()
let buf = buf + "\n <style>\n"
let buf = buf + " .arbor-node-label { font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 13px; }\n"
let buf = buf + " .arbor-group-label { font-family: 'Helvetica Neue', Helvetica, Arial, monospace; font-size: 10px; fill: " + col_group_text() + "; letter-spacing: 0.08em; }\n"
let buf = buf + " .arbor-edge-label { font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 11px; fill: " + col_edge_label() + "; }\n"
let buf = buf + " </style>\n"
let buf = buf + " </defs>\n"
// Groups first (behind everything).
let buf = buf + " <!-- Groups -->\n"
let groups: [Map<String, Any>] = graph["groups"]
let gn: Int = el_list_len(groups)
let i = 0
while i < gn {
let g: Map<String, Any> = get(groups, i)
let gid: String = g["id"]
// Only render groups the layout actually placed.
let gids: [String] = el_map_get(layout, "group_ids")
let placed = false
let j = 0
while j < el_list_len(gids) {
if str_eq(get(gids, j), gid) { let placed = true }
let j = j + 1
}
if placed {
let buf = render_group(buf, g, layout)
}
let i = i + 1
}
// Edges
let buf = buf + " <!-- Edges -->\n"
let edges: [Map<String, Any>] = graph["edges"]
let en: Int = el_list_len(edges)
let i = 0
while i < en {
let e: Map<String, Any> = get(edges, i)
let buf = render_edge(buf, e, layout, forbidden)
let i = i + 1
}
// Nodes
let buf = buf + " <!-- Nodes -->\n"
let nodes: [Map<String, Any>] = graph["nodes"]
let nn: Int = el_list_len(nodes)
let i = 0
while i < nn {
let n: Map<String, Any> = get(nodes, i)
let buf = render_node(buf, n, layout)
let i = i + 1
}
// Title
let title: String = graph["title"]
if str_len(title) > 0 {
let title_x: el_val_t = fdiv2(cw)
let buf = buf + " <text x=\"" + pt(title_x) + "\" y=\"22\" text-anchor=\"middle\""
let buf = buf + " font-family=\"'Helvetica Neue', Helvetica, Arial, sans-serif\""
let buf = buf + " font-size=\"15\" font-weight=\"600\" fill=\"" + col_node_text() + "\">"
let buf = buf + esc(title) + "</text>\n"
}
let buf = buf + "</svg>\n"
buf
}
// PNG not implemented; the runtime has no SVG rasterizer or PNG encoder.
// Returns an error map that callers can inspect via map["error"].
fn arbor_render_png(graph: Map<String, Any>, layout: Map<String, Any>, forbidden: [String]) -> Map<String, Any> {
{
"error": "PNG rasterization not available in El runtime — install a runtime image library or use the Rust binary"
}
}
// Smoke test
fn fail(label: String, msg: String) -> Int {
println("FAIL " + label + ": " + msg)
state_set("smoke_failures", "1")
0
}
fn check_contains(label: String, haystack: String, needle: String) -> Int {
if str_contains(haystack, needle) {
println("ok " + label)
return 1
}
fail(label, "missing [" + needle + "]")
}
fn check_not_contains(label: String, haystack: String, needle: String) -> Int {
if str_contains(haystack, needle) {
return fail(label, "should not contain [" + needle + "]")
}
println("ok " + label)
1
}
fn make_test_node(id: String, label: String, shape: String) -> Map<String, Any> {
{
"id": id, "label": label, "sublabel": "",
"shape": shape,
"style_fill": "", "style_stroke": "", "style_color": ""
}
}
fn make_test_edge(src: String, dst: String, line: String, arrow: String, label: String) -> Map<String, Any> {
{
"from": src, "to": dst, "label": label,
"line": line, "arrow": arrow
}
}
fn make_test_pos(x: Int, y: Int) -> Map<String, Any> {
{ "x": int_to_float(x), "y": int_to_float(y) }
}
fn make_test_size(w: Int, h: Int) -> Map<String, Any> {
{ "w": int_to_float(w), "h": int_to_float(h) }
}
// Build a minimal layout map by hand.
fn build_layout(node_ids: [String], group_ids: [String], cw: Int, ch: Int) -> Map<String, Any> {
let r: Map<String, Any> = el_map_new(0)
let r = el_map_set(r, "node_ids", node_ids)
let r = el_map_set(r, "group_ids", group_ids)
let r = el_map_set(r, "canvas", { "w": int_to_float(cw), "h": int_to_float(ch) })
r
}
let n_a: Map<String, Any> = make_test_node("a", "Node A", "rectangle")
let n_b: Map<String, Any> = make_test_node("b", "Node B", "rectangle")
let e_ab: Map<String, Any> = make_test_edge("a", "b", "solid", "forward", "")
let nodes: [Map<String, Any>] = native_list_empty()
let nodes = native_list_append(nodes, n_a)
let nodes = native_list_append(nodes, n_b)
let edges: [Map<String, Any>] = native_list_empty()
let edges = native_list_append(edges, e_ab)
let groups: [Map<String, Any>] = native_list_empty()
let g: Map<String, Any> = {
"title": "Test", "direction": "top-down",
"nodes": nodes, "edges": edges, "groups": groups
}
let nid_list: [String] = native_list_empty()
let nid_list = native_list_append(nid_list, "a")
let nid_list = native_list_append(nid_list, "b")
let gid_list: [String] = native_list_empty()
let layout: Map<String, Any> = build_layout(nid_list, gid_list, 400, 300)
let layout = el_map_set(layout, "node_pos_a", make_test_pos(100, 60))
let layout = el_map_set(layout, "node_pos_b", make_test_pos(100, 200))
let layout = el_map_set(layout, "node_size_a", make_test_size(120, 40))
let layout = el_map_set(layout, "node_size_b", make_test_size(120, 40))
let forbidden: [String] = native_list_empty()
let svg: String = arbor_render_svg(g, layout, forbidden)
check_contains("svg starts with <svg", svg, "<svg xmlns=")
check_contains("svg ends with </svg>", svg, "</svg>")
check_contains("svg contains node label", svg, "Node A")
check_contains("svg contains title", svg, ">Test</text>")
check_contains("svg has rect for rectangle node", svg, "<rect")
check_contains("svg has line for edge", svg, "<line")
check_contains("svg has arrow marker def", svg, "id=\"ah\"")
// Escape test
let n_esc: Map<String, Any> = make_test_node("x", "A & B <C>", "rectangle")
let nodes2: [Map<String, Any>] = native_list_empty()
let nodes2 = native_list_append(nodes2, n_esc)
let g2: Map<String, Any> = {
"title": "Test <Title>", "direction": "top-down",
"nodes": nodes2, "edges": native_list_empty(), "groups": native_list_empty()
}
let nid2: [String] = native_list_empty()
let nid2 = native_list_append(nid2, "x")
let layout2: Map<String, Any> = build_layout(nid2, native_list_empty(), 200, 100)
let layout2 = el_map_set(layout2, "node_pos_x", make_test_pos(80, 40))
let layout2 = el_map_set(layout2, "node_size_x", make_test_size(120, 40))
let svg2: String = arbor_render_svg(g2, layout2, native_list_empty())
check_contains("escapes ampersand", svg2, "&amp;")
check_contains("escapes <", svg2, "&lt;")
check_not_contains("no raw <C>", svg2, "<C>")
// Forbidden edge
let e_fb: Map<String, Any> = make_test_edge("a", "b", "solid", "forward", "")
let edges3: [Map<String, Any>] = native_list_empty()
let edges3 = native_list_append(edges3, e_fb)
let g3: Map<String, Any> = {
"title": "F", "direction": "top-down",
"nodes": nodes, "edges": edges3, "groups": native_list_empty()
}
let fb: [String] = native_list_empty()
let fb = native_list_append(fb, forbidden_key("a", "b"))
let svg3: String = arbor_render_svg(g3, layout, fb)
check_contains("forbidden uses red marker", svg3, "ah-red")
check_contains("forbidden colour present", svg3, col_edge_forbidden())
// Diamond shape polygon
let n_d: Map<String, Any> = make_test_node("d", "Decide", "diamond")
let g4: Map<String, Any> = {
"title": "", "direction": "top-down",
"nodes": native_list_append(native_list_empty(), n_d),
"edges": native_list_empty(), "groups": native_list_empty()
}
let nid4: [String] = native_list_append(native_list_empty(), "d")
let layout4: Map<String, Any> = build_layout(nid4, native_list_empty(), 200, 100)
let layout4 = el_map_set(layout4, "node_pos_d", make_test_pos(80, 50))
let layout4 = el_map_set(layout4, "node_size_d", make_test_size(120, 40))
let svg4: String = arbor_render_svg(g4, layout4, native_list_empty())
check_contains("diamond uses polygon", svg4, "<polygon")
// Cylinder shape ellipses
let n_cy: Map<String, Any> = make_test_node("cy", "DB", "cylinder")
let g5: Map<String, Any> = {
"title": "", "direction": "top-down",
"nodes": native_list_append(native_list_empty(), n_cy),
"edges": native_list_empty(), "groups": native_list_empty()
}
let nid5: [String] = native_list_append(native_list_empty(), "cy")
let layout5: Map<String, Any> = build_layout(nid5, native_list_empty(), 200, 100)
let layout5 = el_map_set(layout5, "node_pos_cy", make_test_pos(80, 50))
let layout5 = el_map_set(layout5, "node_size_cy", make_test_size(120, 40))
let svg5: String = arbor_render_svg(g5, layout5, native_list_empty())
check_contains("cylinder uses ellipse", svg5, "<ellipse")
// Dashed edge
let e_dash: Map<String, Any> = make_test_edge("a", "b", "dashed", "forward", "")
let g6: Map<String, Any> = {
"title": "", "direction": "top-down",
"nodes": nodes, "edges": native_list_append(native_list_empty(), e_dash),
"groups": native_list_empty()
}
let svg6: String = arbor_render_svg(g6, layout, native_list_empty())
check_contains("dashed line dasharray", svg6, "stroke-dasharray=\"5,3\"")
// PNG returns an error map
let png: Map<String, Any> = arbor_render_png(g, layout, native_list_empty())
let err: String = png["error"]
if str_len(err) > 0 {
println("ok PNG returns error map")
} else {
println("FAIL PNG should have returned error")
state_set("smoke_failures", "1")
}
println("")
let f: String = state_get("smoke_failures")
if str_eq(f, "1") {
println("arbor-render: FAILED")
exit_program(1)
} else {
println("arbor-render: ok")
}
View File
Vendored Executable
BIN
View File
Binary file not shown.
File diff suppressed because it is too large Load Diff
Vendored Executable
BIN
View File
Binary file not shown.
@@ -22,9 +22,6 @@
* EL_STR(s) cast string literal to el_val_t * EL_STR(s) cast string literal to el_val_t
* EL_CSTR(v) cast el_val_t back to const char* * EL_CSTR(v) cast el_val_t back to const char*
* EL_INT(v) identity el_val_t is already int64_t * EL_INT(v) identity el_val_t is already int64_t
* EL_NULL null / zero value
* EL_FALSE boolean false (0)
* EL_TRUE boolean true (1)
* *
* Link requirements: * Link requirements:
* -lcurl required for the HTTP client (http_get, http_post, llm_*). * -lcurl required for the HTTP client (http_get, http_post, llm_*).
@@ -56,8 +53,6 @@ typedef int64_t el_val_t;
#define EL_CSTR(v) ((const char*)(uintptr_t)(v)) #define EL_CSTR(v) ((const char*)(uintptr_t)(v))
#define EL_INT(v) (v) #define EL_INT(v) (v)
#define EL_NULL ((el_val_t)0) #define EL_NULL ((el_val_t)0)
#define EL_FALSE ((el_val_t)0)
#define EL_TRUE ((el_val_t)1)
/* Float values share the el_val_t (int64) slot via a bit-cast. /* Float values share the el_val_t (int64) slot via a bit-cast.
* The codegen emits Float literals as `el_from_float(<dbl>)` so the * The codegen emits Float literals as `el_from_float(<dbl>)` so the
@@ -81,9 +76,11 @@ extern "C" {
/* ── I/O ──────────────────────────────────────────────────────────────────── */ /* ── I/O ──────────────────────────────────────────────────────────────────── */
el_val_t println(el_val_t s); void println(el_val_t s);
el_val_t print(el_val_t s); void print(el_val_t s);
el_val_t readline(void); el_val_t readline(void);
el_val_t stdout_to_file(el_val_t path); /* redirect println to a file */
el_val_t stdout_restore(void); /* restore stdout after capture */
/* ── String builtins ─────────────────────────────────────────────────────── */ /* ── String builtins ─────────────────────────────────────────────────────── */
@@ -95,7 +92,6 @@ el_val_t str_len(el_val_t s);
el_val_t str_concat(el_val_t a, el_val_t b); el_val_t str_concat(el_val_t a, el_val_t b);
el_val_t int_to_str(el_val_t n); el_val_t int_to_str(el_val_t n);
el_val_t str_to_int(el_val_t s); el_val_t str_to_int(el_val_t s);
el_val_t native_str_to_int(el_val_t s);
el_val_t str_slice(el_val_t s, el_val_t start, el_val_t end); el_val_t str_slice(el_val_t s, el_val_t start, el_val_t end);
el_val_t str_contains(el_val_t s, el_val_t sub); el_val_t str_contains(el_val_t s, el_val_t sub);
el_val_t str_replace(el_val_t s, el_val_t from, el_val_t to); el_val_t str_replace(el_val_t s, el_val_t from, el_val_t to);
@@ -123,10 +119,6 @@ el_val_t el_min(el_val_t a, el_val_t b);
void el_retain(el_val_t v); void el_retain(el_val_t v);
void el_release(el_val_t v); void el_release(el_val_t v);
/* ── Scoped arena (CLI use) ───────────────────────────────────────────────── */
el_val_t el_arena_push(void);
el_val_t el_arena_pop(el_val_t mark);
/* ── List ────────────────────────────────────────────────────────────────── */ /* ── List ────────────────────────────────────────────────────────────────── */
el_val_t el_list_new(el_val_t count, ...); el_val_t el_list_new(el_val_t count, ...);
@@ -150,11 +142,10 @@ el_val_t http_post(el_val_t url, el_val_t body);
el_val_t http_post_json(el_val_t url, el_val_t json_body); el_val_t http_post_json(el_val_t url, el_val_t json_body);
el_val_t http_get_with_headers(el_val_t url, el_val_t headers_map); el_val_t http_get_with_headers(el_val_t url, el_val_t headers_map);
el_val_t http_post_with_headers(el_val_t url, el_val_t body, el_val_t headers_map); el_val_t http_post_with_headers(el_val_t url, el_val_t body, el_val_t headers_map);
el_val_t http_post_json_with_headers(el_val_t url, el_val_t headers_map, el_val_t json_body);
el_val_t http_post_form_auth(el_val_t url, el_val_t form_body, el_val_t auth_header); el_val_t http_post_form_auth(el_val_t url, el_val_t form_body, el_val_t auth_header);
el_val_t http_delete(el_val_t url); el_val_t http_delete(el_val_t url);
el_val_t http_serve(el_val_t port, el_val_t handler); void http_serve(el_val_t port, el_val_t handler);
el_val_t http_set_handler(el_val_t name); void http_set_handler(el_val_t name);
/* HTTP server v2 ───────────────────────────────────────────────────────────── /* HTTP server v2 ─────────────────────────────────────────────────────────────
* Same dispatch model as http_serve, but the handler signature is widened: * Same dispatch model as http_serve, but the handler signature is widened:
@@ -175,8 +166,8 @@ el_val_t http_set_handler(el_val_t name);
* The 3-arg http_serve(port, handler) remains supported unchanged for * The 3-arg http_serve(port, handler) remains supported unchanged for
* existing handlers (e.g. products/web/server.el): it dispatches with * existing handlers (e.g. products/web/server.el): it dispatches with
* (method, path, body), hardcodes 200 OK, and auto-detects content type. */ * (method, path, body), hardcodes 200 OK, and auto-detects content type. */
el_val_t http_serve_v2(el_val_t port, el_val_t handler); void http_serve_v2(el_val_t port, el_val_t handler);
el_val_t http_set_handler_v2(el_val_t name); void http_set_handler_v2(el_val_t name);
/* Build an HTTP response envelope. `headers_json` should be a JSON object /* Build an HTTP response envelope. `headers_json` should be a JSON object
* literal like `{"WWW-Authenticate":"Basic"}` (or "" / "{}" for none). The * literal like `{"WWW-Authenticate":"Basic"}` (or "" / "{}" for none). The
@@ -187,11 +178,6 @@ el_val_t http_set_handler_v2(el_val_t name);
* auto-content-type contract for legacy handlers that return plain bodies. */ * auto-content-type contract for legacy handlers that return plain bodies. */
el_val_t http_response(el_val_t status, el_val_t headers_json, el_val_t body); el_val_t http_response(el_val_t status, el_val_t headers_json, el_val_t body);
/* SSE connection fd — set by http_worker_v2 before calling the El handler,
* cleared afterwards. Defined in el_seed.c; called from el_runtime.c.
* The getter is exposed as __http_conn_fd() to El programs. */
void el_seed_set_http_conn_fd(int fd);
/* HTTP timeout — every libcurl request honors EL_HTTP_TIMEOUT_MS (default /* HTTP timeout — every libcurl request honors EL_HTTP_TIMEOUT_MS (default
* 60000ms). Read lazily on first use, so setting the env var any time before * 60000ms). Read lazily on first use, so setting the env var any time before
* the first http_* call is sufficient. */ * the first http_* call is sufficient. */
@@ -227,15 +213,12 @@ el_val_t url_decode(el_val_t s); /* '+' → space, %XX → byte */
* {"p":[],"a":["href","title"],"strong":[],...} * {"p":[],"a":["href","title"],"strong":[],...}
* where each value is the array of attribute names allowed for that tag. */ * where each value is the array of attribute names allowed for that tag. */
el_val_t el_html_sanitize(el_val_t input_html, el_val_t allowlist_json); el_val_t el_html_sanitize(el_val_t input_html, el_val_t allowlist_json);
el_val_t html_raw(el_val_t s);
el_val_t html_escape(el_val_t s);
/* ── Filesystem ──────────────────────────────────────────────────────────── */ /* ── Filesystem ──────────────────────────────────────────────────────────── */
el_val_t fs_read(el_val_t path); el_val_t fs_read(el_val_t path);
el_val_t fs_write(el_val_t path, el_val_t content); el_val_t fs_write(el_val_t path, el_val_t content);
el_val_t fs_list(el_val_t path); el_val_t fs_list(el_val_t path);
el_val_t fs_list_json(el_val_t path);
el_val_t fs_exists(el_val_t path); el_val_t fs_exists(el_val_t path);
el_val_t fs_mkdir(el_val_t path); /* mkdir -p, mode 0755 */ el_val_t fs_mkdir(el_val_t path); /* mkdir -p, mode 0755 */
@@ -265,9 +248,6 @@ el_val_t json_set(el_val_t json_str, el_val_t key, el_val_t value);
el_val_t json_array_len(el_val_t json_str); el_val_t json_array_len(el_val_t json_str);
el_val_t json_array_get(el_val_t json_str, el_val_t index); el_val_t json_array_get(el_val_t json_str, el_val_t index);
el_val_t json_array_get_string(el_val_t json_str, el_val_t index); el_val_t json_array_get_string(el_val_t json_str, el_val_t index);
el_val_t json_escape_string(el_val_t sv);
el_val_t json_build_object(el_val_t kvs);
el_val_t json_build_array(el_val_t items);
/* ── Time ────────────────────────────────────────────────────────────────── */ /* ── Time ────────────────────────────────────────────────────────────────── */
@@ -280,7 +260,6 @@ el_val_t time_to_parts(el_val_t ts);
el_val_t time_from_parts(el_val_t secs, el_val_t ns, el_val_t tz); el_val_t time_from_parts(el_val_t secs, el_val_t ns, el_val_t tz);
el_val_t time_add(el_val_t ts, el_val_t n, el_val_t unit); el_val_t time_add(el_val_t ts, el_val_t n, el_val_t unit);
el_val_t time_diff(el_val_t ts1, el_val_t ts2, el_val_t unit); el_val_t time_diff(el_val_t ts1, el_val_t ts2, el_val_t unit);
el_val_t now_ns(void);
/* ── Instant + Duration: first-class temporal types ────────────────────────── /* ── Instant + Duration: first-class temporal types ──────────────────────────
* Both types share the el_val_t (int64) slot. Instants are nanoseconds * Both types share the el_val_t (int64) slot. Instants are nanoseconds
@@ -437,8 +416,6 @@ el_val_t state_set(el_val_t key, el_val_t value);
el_val_t state_get(el_val_t key); el_val_t state_get(el_val_t key);
el_val_t state_del(el_val_t key); el_val_t state_del(el_val_t key);
el_val_t state_keys(void); el_val_t state_keys(void);
el_val_t state_has(el_val_t key);
el_val_t state_get_or(el_val_t key, el_val_t default_val);
/* ── Float formatting ────────────────────────────────────────────────────── */ /* ── Float formatting ────────────────────────────────────────────────────── */
@@ -530,15 +507,9 @@ el_val_t parse_int(el_val_t s, el_val_t default_val);
/* ── Process ─────────────────────────────────────────────────────────────── */ /* ── Process ─────────────────────────────────────────────────────────────── */
el_val_t exit_program(el_val_t code); void exit_program(el_val_t code);
el_val_t getpid_now(void); el_val_t getpid_now(void);
/* Self-terminating memory guard. Reads ELC_MAX_MEM_MB (default 512) and
* exits with code 1 if resident memory exceeds the limit. Call periodically
* during long compilation loops (e.g. after each function is compiled).
* Returns 0 when memory is within bounds. */
el_val_t el_mem_check(void);
/* ── CGI identity ───────────────────────────────────────────────────────────── /* ── CGI identity ─────────────────────────────────────────────────────────────
* Called at the start of main() in CGI programs (those with a `cgi {}` block). * Called at the start of main() in CGI programs (those with a `cgi {}` block).
* Records the program's DHARMA identity before any other code executes. */ * Records the program's DHARMA identity before any other code executes. */
@@ -776,108 +747,12 @@ el_val_t exec_capture(el_val_t cmd); /* run shell command, capture stdout */
el_val_t exec(el_val_t cmd); /* exec(cmd) → stdout String (30s timeout) */ el_val_t exec(el_val_t cmd); /* exec(cmd) → stdout String (30s timeout) */
el_val_t exec_bg(el_val_t cmd); /* exec_bg(cmd) → PID String (non-blocking) */ el_val_t exec_bg(el_val_t cmd); /* exec_bg(cmd) → PID String (non-blocking) */
/* ── Stdout redirection (used by compiler JS pipeline) ───────────────────── */
el_val_t stdout_to_file(el_val_t path); /* redirect process stdout to a file */
el_val_t stdout_restore(void); /* restore process stdout to terminal */
el_val_t emit_log(el_val_t level, el_val_t msg, el_val_t fields_json); el_val_t emit_log(el_val_t level, el_val_t msg, el_val_t fields_json);
el_val_t emit_metric(el_val_t name, el_val_t value, el_val_t tags_json); el_val_t emit_metric(el_val_t name, el_val_t value, el_val_t tags_json);
el_val_t trace_span_start(el_val_t name); el_val_t trace_span_start(el_val_t name);
el_val_t trace_span_end(el_val_t span_handle); el_val_t trace_span_end(el_val_t span_handle);
el_val_t emit_event(el_val_t name, el_val_t duration_ms); el_val_t emit_event(el_val_t name, el_val_t duration_ms);
el_val_t __thread_create(el_val_t fn_name_v, el_val_t arg_v);
el_val_t __thread_join(el_val_t tid_v);
/* ── __ prefixed aliases (self-hosting compiler ABI) ─────────────────────────
* The El self-hosting compiler emits calls to __-prefixed names. These are
* forwarding wrappers around the existing el_runtime functions above. */
/* I/O */
el_val_t __println(el_val_t s);
el_val_t __print(el_val_t s);
el_val_t __readline(void);
/* String */
el_val_t __int_to_str(el_val_t n);
el_val_t __str_to_int(el_val_t s);
el_val_t __float_to_str(el_val_t f);
el_val_t __str_to_float(el_val_t s);
el_val_t __str_len(el_val_t s);
el_val_t __str_char_at(el_val_t s, el_val_t i);
el_val_t __str_cmp(el_val_t a, el_val_t b);
el_val_t __str_ncmp(el_val_t a, el_val_t b, el_val_t n);
el_val_t __str_concat_raw(el_val_t a, el_val_t b);
el_val_t __str_slice_raw(el_val_t s, el_val_t start, el_val_t end);
el_val_t __str_alloc(el_val_t n);
el_val_t __str_set_char(el_val_t s, el_val_t i, el_val_t c);
/* URL encoding */
el_val_t __url_encode(el_val_t s);
el_val_t __url_decode(el_val_t s);
/* Environment */
el_val_t __env_get(el_val_t key);
/* Subprocess */
el_val_t __exec(el_val_t cmd);
el_val_t __exec_bg(el_val_t cmd);
/* Process */
el_val_t __exit_program(el_val_t code);
/* Filesystem */
el_val_t __fs_exists(el_val_t path);
el_val_t __fs_mkdir(el_val_t path);
el_val_t __fs_read(el_val_t path);
el_val_t __fs_write(el_val_t path, el_val_t content);
el_val_t __fs_write_bytes(el_val_t path, el_val_t bytes, el_val_t n);
el_val_t __fs_list_raw(el_val_t path);
/* HTTP server */
el_val_t __http_response(el_val_t status, el_val_t headers_json, el_val_t body);
el_val_t __http_serve(el_val_t port, el_val_t handler);
el_val_t __http_serve_v2(el_val_t port, el_val_t handler);
/* HTTP conn fd / SSE (weak; overridden by el_seed.c when linked together) */
el_val_t __http_conn_fd(void);
el_val_t __http_sse_open(el_val_t conn_id);
el_val_t __http_sse_send(el_val_t conn_id, el_val_t data);
el_val_t __http_sse_close(el_val_t conn_id);
/* HTTP client (requires HAVE_CURL; stubs provided for no-curl builds) */
el_val_t __http_do(el_val_t method, el_val_t url, el_val_t body,
el_val_t headers_map, el_val_t timeout_ms);
el_val_t __http_do_map(el_val_t method, el_val_t url, el_val_t body,
el_val_t headers_json, el_val_t timeout_ms);
el_val_t __http_do_map_to_file(el_val_t method, el_val_t url, el_val_t body,
el_val_t headers_json, el_val_t output_path);
/* JSON */
el_val_t __json_array_get(el_val_t json, el_val_t index);
el_val_t __json_array_get_string(el_val_t json, el_val_t index);
el_val_t __json_array_len(el_val_t json);
el_val_t __json_get(el_val_t json, el_val_t key);
el_val_t __json_get_raw(el_val_t json, el_val_t key);
el_val_t __json_set(el_val_t json, el_val_t key, el_val_t value);
el_val_t __json_parse_map(el_val_t json_str);
el_val_t __json_stringify_val(el_val_t val);
/* Hashing */
el_val_t __sha256_hex(el_val_t s);
/* State K/V */
el_val_t __state_del(el_val_t key);
el_val_t __state_get(el_val_t key);
el_val_t __state_keys(void);
el_val_t __state_set(el_val_t key, el_val_t val);
/* UUID */
el_val_t __uuid_v4(void);
/* Args */
el_val_t __args_json(void);
#ifdef __cplusplus #ifdef __cplusplus
} }
#endif #endif
@@ -22,9 +22,6 @@
* EL_STR(s) cast string literal to el_val_t * EL_STR(s) cast string literal to el_val_t
* EL_CSTR(v) cast el_val_t back to const char* * EL_CSTR(v) cast el_val_t back to const char*
* EL_INT(v) identity el_val_t is already int64_t * EL_INT(v) identity el_val_t is already int64_t
* EL_NULL null / zero value
* EL_FALSE boolean false (0)
* EL_TRUE boolean true (1)
* *
* Link requirements: * Link requirements:
* -lcurl required for the HTTP client (http_get, http_post, llm_*). * -lcurl required for the HTTP client (http_get, http_post, llm_*).
@@ -56,8 +53,6 @@ typedef int64_t el_val_t;
#define EL_CSTR(v) ((const char*)(uintptr_t)(v)) #define EL_CSTR(v) ((const char*)(uintptr_t)(v))
#define EL_INT(v) (v) #define EL_INT(v) (v)
#define EL_NULL ((el_val_t)0) #define EL_NULL ((el_val_t)0)
#define EL_FALSE ((el_val_t)0)
#define EL_TRUE ((el_val_t)1)
/* Float values share the el_val_t (int64) slot via a bit-cast. /* Float values share the el_val_t (int64) slot via a bit-cast.
* The codegen emits Float literals as `el_from_float(<dbl>)` so the * The codegen emits Float literals as `el_from_float(<dbl>)` so the
@@ -81,8 +76,8 @@ extern "C" {
/* ── I/O ──────────────────────────────────────────────────────────────────── */ /* ── I/O ──────────────────────────────────────────────────────────────────── */
el_val_t println(el_val_t s); void println(el_val_t s);
el_val_t print(el_val_t s); void print(el_val_t s);
el_val_t readline(void); el_val_t readline(void);
/* ── String builtins ─────────────────────────────────────────────────────── */ /* ── String builtins ─────────────────────────────────────────────────────── */
@@ -95,7 +90,6 @@ el_val_t str_len(el_val_t s);
el_val_t str_concat(el_val_t a, el_val_t b); el_val_t str_concat(el_val_t a, el_val_t b);
el_val_t int_to_str(el_val_t n); el_val_t int_to_str(el_val_t n);
el_val_t str_to_int(el_val_t s); el_val_t str_to_int(el_val_t s);
el_val_t native_str_to_int(el_val_t s);
el_val_t str_slice(el_val_t s, el_val_t start, el_val_t end); el_val_t str_slice(el_val_t s, el_val_t start, el_val_t end);
el_val_t str_contains(el_val_t s, el_val_t sub); el_val_t str_contains(el_val_t s, el_val_t sub);
el_val_t str_replace(el_val_t s, el_val_t from, el_val_t to); el_val_t str_replace(el_val_t s, el_val_t from, el_val_t to);
@@ -123,10 +117,6 @@ el_val_t el_min(el_val_t a, el_val_t b);
void el_retain(el_val_t v); void el_retain(el_val_t v);
void el_release(el_val_t v); void el_release(el_val_t v);
/* ── Scoped arena (CLI use) ───────────────────────────────────────────────── */
el_val_t el_arena_push(void);
el_val_t el_arena_pop(el_val_t mark);
/* ── List ────────────────────────────────────────────────────────────────── */ /* ── List ────────────────────────────────────────────────────────────────── */
el_val_t el_list_new(el_val_t count, ...); el_val_t el_list_new(el_val_t count, ...);
@@ -150,11 +140,10 @@ el_val_t http_post(el_val_t url, el_val_t body);
el_val_t http_post_json(el_val_t url, el_val_t json_body); el_val_t http_post_json(el_val_t url, el_val_t json_body);
el_val_t http_get_with_headers(el_val_t url, el_val_t headers_map); el_val_t http_get_with_headers(el_val_t url, el_val_t headers_map);
el_val_t http_post_with_headers(el_val_t url, el_val_t body, el_val_t headers_map); el_val_t http_post_with_headers(el_val_t url, el_val_t body, el_val_t headers_map);
el_val_t http_post_json_with_headers(el_val_t url, el_val_t headers_map, el_val_t json_body);
el_val_t http_post_form_auth(el_val_t url, el_val_t form_body, el_val_t auth_header); el_val_t http_post_form_auth(el_val_t url, el_val_t form_body, el_val_t auth_header);
el_val_t http_delete(el_val_t url); el_val_t http_delete(el_val_t url);
el_val_t http_serve(el_val_t port, el_val_t handler); void http_serve(el_val_t port, el_val_t handler);
el_val_t http_set_handler(el_val_t name); void http_set_handler(el_val_t name);
/* HTTP server v2 ───────────────────────────────────────────────────────────── /* HTTP server v2 ─────────────────────────────────────────────────────────────
* Same dispatch model as http_serve, but the handler signature is widened: * Same dispatch model as http_serve, but the handler signature is widened:
@@ -175,8 +164,8 @@ el_val_t http_set_handler(el_val_t name);
* The 3-arg http_serve(port, handler) remains supported unchanged for * The 3-arg http_serve(port, handler) remains supported unchanged for
* existing handlers (e.g. products/web/server.el): it dispatches with * existing handlers (e.g. products/web/server.el): it dispatches with
* (method, path, body), hardcodes 200 OK, and auto-detects content type. */ * (method, path, body), hardcodes 200 OK, and auto-detects content type. */
el_val_t http_serve_v2(el_val_t port, el_val_t handler); void http_serve_v2(el_val_t port, el_val_t handler);
el_val_t http_set_handler_v2(el_val_t name); void http_set_handler_v2(el_val_t name);
/* Build an HTTP response envelope. `headers_json` should be a JSON object /* Build an HTTP response envelope. `headers_json` should be a JSON object
* literal like `{"WWW-Authenticate":"Basic"}` (or "" / "{}" for none). The * literal like `{"WWW-Authenticate":"Basic"}` (or "" / "{}" for none). The
@@ -187,11 +176,6 @@ el_val_t http_set_handler_v2(el_val_t name);
* auto-content-type contract for legacy handlers that return plain bodies. */ * auto-content-type contract for legacy handlers that return plain bodies. */
el_val_t http_response(el_val_t status, el_val_t headers_json, el_val_t body); el_val_t http_response(el_val_t status, el_val_t headers_json, el_val_t body);
/* SSE connection fd — set by http_worker_v2 before calling the El handler,
* cleared afterwards. Defined in el_seed.c; called from el_runtime.c.
* The getter is exposed as __http_conn_fd() to El programs. */
void el_seed_set_http_conn_fd(int fd);
/* HTTP timeout — every libcurl request honors EL_HTTP_TIMEOUT_MS (default /* HTTP timeout — every libcurl request honors EL_HTTP_TIMEOUT_MS (default
* 60000ms). Read lazily on first use, so setting the env var any time before * 60000ms). Read lazily on first use, so setting the env var any time before
* the first http_* call is sufficient. */ * the first http_* call is sufficient. */
@@ -227,15 +211,12 @@ el_val_t url_decode(el_val_t s); /* '+' → space, %XX → byte */
* {"p":[],"a":["href","title"],"strong":[],...} * {"p":[],"a":["href","title"],"strong":[],...}
* where each value is the array of attribute names allowed for that tag. */ * where each value is the array of attribute names allowed for that tag. */
el_val_t el_html_sanitize(el_val_t input_html, el_val_t allowlist_json); el_val_t el_html_sanitize(el_val_t input_html, el_val_t allowlist_json);
el_val_t html_raw(el_val_t s);
el_val_t html_escape(el_val_t s);
/* ── Filesystem ──────────────────────────────────────────────────────────── */ /* ── Filesystem ──────────────────────────────────────────────────────────── */
el_val_t fs_read(el_val_t path); el_val_t fs_read(el_val_t path);
el_val_t fs_write(el_val_t path, el_val_t content); el_val_t fs_write(el_val_t path, el_val_t content);
el_val_t fs_list(el_val_t path); el_val_t fs_list(el_val_t path);
el_val_t fs_list_json(el_val_t path);
el_val_t fs_exists(el_val_t path); el_val_t fs_exists(el_val_t path);
el_val_t fs_mkdir(el_val_t path); /* mkdir -p, mode 0755 */ el_val_t fs_mkdir(el_val_t path); /* mkdir -p, mode 0755 */
@@ -265,9 +246,6 @@ el_val_t json_set(el_val_t json_str, el_val_t key, el_val_t value);
el_val_t json_array_len(el_val_t json_str); el_val_t json_array_len(el_val_t json_str);
el_val_t json_array_get(el_val_t json_str, el_val_t index); el_val_t json_array_get(el_val_t json_str, el_val_t index);
el_val_t json_array_get_string(el_val_t json_str, el_val_t index); el_val_t json_array_get_string(el_val_t json_str, el_val_t index);
el_val_t json_escape_string(el_val_t sv);
el_val_t json_build_object(el_val_t kvs);
el_val_t json_build_array(el_val_t items);
/* ── Time ────────────────────────────────────────────────────────────────── */ /* ── Time ────────────────────────────────────────────────────────────────── */
@@ -280,7 +258,6 @@ el_val_t time_to_parts(el_val_t ts);
el_val_t time_from_parts(el_val_t secs, el_val_t ns, el_val_t tz); el_val_t time_from_parts(el_val_t secs, el_val_t ns, el_val_t tz);
el_val_t time_add(el_val_t ts, el_val_t n, el_val_t unit); el_val_t time_add(el_val_t ts, el_val_t n, el_val_t unit);
el_val_t time_diff(el_val_t ts1, el_val_t ts2, el_val_t unit); el_val_t time_diff(el_val_t ts1, el_val_t ts2, el_val_t unit);
el_val_t now_ns(void);
/* ── Instant + Duration: first-class temporal types ────────────────────────── /* ── Instant + Duration: first-class temporal types ──────────────────────────
* Both types share the el_val_t (int64) slot. Instants are nanoseconds * Both types share the el_val_t (int64) slot. Instants are nanoseconds
@@ -437,8 +414,6 @@ el_val_t state_set(el_val_t key, el_val_t value);
el_val_t state_get(el_val_t key); el_val_t state_get(el_val_t key);
el_val_t state_del(el_val_t key); el_val_t state_del(el_val_t key);
el_val_t state_keys(void); el_val_t state_keys(void);
el_val_t state_has(el_val_t key);
el_val_t state_get_or(el_val_t key, el_val_t default_val);
/* ── Float formatting ────────────────────────────────────────────────────── */ /* ── Float formatting ────────────────────────────────────────────────────── */
@@ -530,15 +505,9 @@ el_val_t parse_int(el_val_t s, el_val_t default_val);
/* ── Process ─────────────────────────────────────────────────────────────── */ /* ── Process ─────────────────────────────────────────────────────────────── */
el_val_t exit_program(el_val_t code); void exit_program(el_val_t code);
el_val_t getpid_now(void); el_val_t getpid_now(void);
/* Self-terminating memory guard. Reads ELC_MAX_MEM_MB (default 512) and
* exits with code 1 if resident memory exceeds the limit. Call periodically
* during long compilation loops (e.g. after each function is compiled).
* Returns 0 when memory is within bounds. */
el_val_t el_mem_check(void);
/* ── CGI identity ───────────────────────────────────────────────────────────── /* ── CGI identity ─────────────────────────────────────────────────────────────
* Called at the start of main() in CGI programs (those with a `cgi {}` block). * Called at the start of main() in CGI programs (those with a `cgi {}` block).
* Records the program's DHARMA identity before any other code executes. */ * Records the program's DHARMA identity before any other code executes. */
@@ -776,108 +745,12 @@ el_val_t exec_capture(el_val_t cmd); /* run shell command, capture stdout */
el_val_t exec(el_val_t cmd); /* exec(cmd) → stdout String (30s timeout) */ el_val_t exec(el_val_t cmd); /* exec(cmd) → stdout String (30s timeout) */
el_val_t exec_bg(el_val_t cmd); /* exec_bg(cmd) → PID String (non-blocking) */ el_val_t exec_bg(el_val_t cmd); /* exec_bg(cmd) → PID String (non-blocking) */
/* ── Stdout redirection (used by compiler JS pipeline) ───────────────────── */
el_val_t stdout_to_file(el_val_t path); /* redirect process stdout to a file */
el_val_t stdout_restore(void); /* restore process stdout to terminal */
el_val_t emit_log(el_val_t level, el_val_t msg, el_val_t fields_json); el_val_t emit_log(el_val_t level, el_val_t msg, el_val_t fields_json);
el_val_t emit_metric(el_val_t name, el_val_t value, el_val_t tags_json); el_val_t emit_metric(el_val_t name, el_val_t value, el_val_t tags_json);
el_val_t trace_span_start(el_val_t name); el_val_t trace_span_start(el_val_t name);
el_val_t trace_span_end(el_val_t span_handle); el_val_t trace_span_end(el_val_t span_handle);
el_val_t emit_event(el_val_t name, el_val_t duration_ms); el_val_t emit_event(el_val_t name, el_val_t duration_ms);
el_val_t __thread_create(el_val_t fn_name_v, el_val_t arg_v);
el_val_t __thread_join(el_val_t tid_v);
/* ── __ prefixed aliases (self-hosting compiler ABI) ─────────────────────────
* The El self-hosting compiler emits calls to __-prefixed names. These are
* forwarding wrappers around the existing el_runtime functions above. */
/* I/O */
el_val_t __println(el_val_t s);
el_val_t __print(el_val_t s);
el_val_t __readline(void);
/* String */
el_val_t __int_to_str(el_val_t n);
el_val_t __str_to_int(el_val_t s);
el_val_t __float_to_str(el_val_t f);
el_val_t __str_to_float(el_val_t s);
el_val_t __str_len(el_val_t s);
el_val_t __str_char_at(el_val_t s, el_val_t i);
el_val_t __str_cmp(el_val_t a, el_val_t b);
el_val_t __str_ncmp(el_val_t a, el_val_t b, el_val_t n);
el_val_t __str_concat_raw(el_val_t a, el_val_t b);
el_val_t __str_slice_raw(el_val_t s, el_val_t start, el_val_t end);
el_val_t __str_alloc(el_val_t n);
el_val_t __str_set_char(el_val_t s, el_val_t i, el_val_t c);
/* URL encoding */
el_val_t __url_encode(el_val_t s);
el_val_t __url_decode(el_val_t s);
/* Environment */
el_val_t __env_get(el_val_t key);
/* Subprocess */
el_val_t __exec(el_val_t cmd);
el_val_t __exec_bg(el_val_t cmd);
/* Process */
el_val_t __exit_program(el_val_t code);
/* Filesystem */
el_val_t __fs_exists(el_val_t path);
el_val_t __fs_mkdir(el_val_t path);
el_val_t __fs_read(el_val_t path);
el_val_t __fs_write(el_val_t path, el_val_t content);
el_val_t __fs_write_bytes(el_val_t path, el_val_t bytes, el_val_t n);
el_val_t __fs_list_raw(el_val_t path);
/* HTTP server */
el_val_t __http_response(el_val_t status, el_val_t headers_json, el_val_t body);
el_val_t __http_serve(el_val_t port, el_val_t handler);
el_val_t __http_serve_v2(el_val_t port, el_val_t handler);
/* HTTP conn fd / SSE (weak; overridden by el_seed.c when linked together) */
el_val_t __http_conn_fd(void);
el_val_t __http_sse_open(el_val_t conn_id);
el_val_t __http_sse_send(el_val_t conn_id, el_val_t data);
el_val_t __http_sse_close(el_val_t conn_id);
/* HTTP client (requires HAVE_CURL; stubs provided for no-curl builds) */
el_val_t __http_do(el_val_t method, el_val_t url, el_val_t body,
el_val_t headers_map, el_val_t timeout_ms);
el_val_t __http_do_map(el_val_t method, el_val_t url, el_val_t body,
el_val_t headers_json, el_val_t timeout_ms);
el_val_t __http_do_map_to_file(el_val_t method, el_val_t url, el_val_t body,
el_val_t headers_json, el_val_t output_path);
/* JSON */
el_val_t __json_array_get(el_val_t json, el_val_t index);
el_val_t __json_array_get_string(el_val_t json, el_val_t index);
el_val_t __json_array_len(el_val_t json);
el_val_t __json_get(el_val_t json, el_val_t key);
el_val_t __json_get_raw(el_val_t json, el_val_t key);
el_val_t __json_set(el_val_t json, el_val_t key, el_val_t value);
el_val_t __json_parse_map(el_val_t json_str);
el_val_t __json_stringify_val(el_val_t val);
/* Hashing */
el_val_t __sha256_hex(el_val_t s);
/* State K/V */
el_val_t __state_del(el_val_t key);
el_val_t __state_get(el_val_t key);
el_val_t __state_keys(void);
el_val_t __state_set(el_val_t key, el_val_t val);
/* UUID */
el_val_t __uuid_v4(void);
/* Args */
el_val_t __args_json(void);
#ifdef __cplusplus #ifdef __cplusplus
} }
#endif #endif
@@ -191,138 +191,6 @@ fn js_is_int_call(call_expr: Map<String, Any>) -> Bool {
return false return false
} }
// HTML template codegen (JS)
//
// HTML template expressions compile to a JS IIFE that builds the HTML string
// using string concatenation. Interpolated values go through html_escape();
// raw() bypasses escaping. {#each} blocks compile to Array.forEach or a
// for-loop that pushes fragments into a parts array.
//
// Entry point: js_cg_html_template(expr) JS expression string.
fn js_next_html_id() -> String {
let csv: String = state_get("__js_html_counter")
let n = 0
if !str_eq(csv, "") {
let n = str_to_int(csv)
}
let n = n + 1
state_set("__js_html_counter", native_int_to_str(n))
native_int_to_str(n)
}
fn js_cg_html_parts(children: [Map<String, Any>], acc_var: String) -> String {
let n: Int = native_list_len(children)
let i = 0
let out = ""
while i < n {
let child: Map<String, Any> = native_list_get(children, i)
let html_kind: String = child["html"]
if str_eq(html_kind, "Text") {
let text: String = child["text"]
let out = out + acc_var + " += " + js_str_lit(text) + "; "
}
if str_eq(html_kind, "Doctype") {
let out = out + acc_var + " += \"<!doctype html>\"; "
}
if str_eq(html_kind, "Interp") {
let val_node = child["value"]
let val_c: String = js_cg_expr(val_node)
let out = out + acc_var + " += html_escape(" + val_c + "); "
}
if str_eq(html_kind, "Raw") {
let val_node = child["value"]
let val_c: String = js_cg_expr(val_node)
let out = out + acc_var + " += html_raw(" + val_c + "); "
}
if str_eq(html_kind, "Element") {
let elem_c: String = js_cg_html_element_str(child, acc_var)
let out = out + elem_c
}
if str_eq(html_kind, "Each") {
let each_c: String = js_cg_html_each(child, acc_var)
let out = out + each_c
}
let i = i + 1
}
out
}
fn js_cg_html_attrs_str(attrs: [Map<String, Any>], acc_var: String) -> String {
let n: Int = native_list_len(attrs)
let i = 0
let out = ""
while i < n {
let attr: Map<String, Any> = native_list_get(attrs, i)
let attr_name: String = attr["name"]
let kind: String = attr["kind"]
// open-attr snippet: " name=\""
let open_val: String = " " + attr_name + "=\""
if str_eq(kind, "static") {
let sv: String = attr["value"]
let out = out + acc_var + " += " + js_str_lit(open_val) + "; "
let out = out + acc_var + " += " + js_str_lit(sv) + "; "
let out = out + acc_var + " += " + js_str_lit("\"") + "; "
} else {
if str_eq(kind, "dynamic") {
let val_node = attr["value"]
let val_c: String = js_cg_expr(val_node)
let out = out + acc_var + " += " + js_str_lit(open_val) + "; "
let out = out + acc_var + " += html_escape(" + val_c + "); "
let out = out + acc_var + " += " + js_str_lit("\"") + "; "
} else {
// Boolean attribute
let out = out + acc_var + " += " + js_str_lit(" " + attr_name) + "; "
}
}
let i = i + 1
}
out
}
fn js_cg_html_element_str(elem: Map<String, Any>, acc_var: String) -> String {
let tag: String = elem["tag"]
let attrs: [Map<String, Any>] = elem["attrs"]
let children: [Map<String, Any>] = elem["children"]
let self_closing: Bool = elem["self_closing"]
let out = acc_var + " += " + js_str_lit("<" + tag) + "; "
let out = out + js_cg_html_attrs_str(attrs, acc_var)
if self_closing {
let out = out + acc_var + " += \"/>\"" + "; "
} else {
let out = out + acc_var + " += \">\"; "
let out = out + js_cg_html_parts(children, acc_var)
let out = out + acc_var + " += " + js_str_lit("</" + tag + ">") + "; "
}
out
}
fn js_cg_html_each(node: Map<String, Any>, acc_var: String) -> String {
let list_expr = node["list"]
let item_name: String = node["item"]
let body_children: [Map<String, Any>] = node["body"]
let id: String = js_next_html_id()
let list_var: String = "_html_list_" + id
let len_var: String = "_html_len_" + id
let idx_var: String = "_html_i_" + id
let list_c: String = js_cg_expr(list_expr)
let inner_c: String = js_cg_html_parts(body_children, acc_var)
"{ const " + list_var + " = " + list_c + "; const " + len_var + " = el_list_len(" + list_var + "); for (let " + idx_var + " = 0; " + idx_var + " < " + len_var + "; " + idx_var + "++) { const " + item_name + " = el_list_get(" + list_var + ", " + idx_var + "); " + inner_c + "} } "
}
fn js_cg_html_template(expr: Map<String, Any>) -> String {
let root = expr["root"]
let id: String = js_next_html_id()
let acc: String = "_html_" + id
let doctype_flag: Bool = root["doctype"]
let doctype_prefix: String = ""
if doctype_flag {
let doctype_prefix = acc + " += \"<!doctype html>\"; "
}
let body: String = js_cg_html_element_str(root, acc)
"(() => { let " + acc + " = \"\"; " + doctype_prefix + body + "return " + acc + "; })()"
}
// Expression codegen // Expression codegen
// //
// js_cg_expr returns a JS expression string (not a statement). // js_cg_expr returns a JS expression string (not a statement).
@@ -701,10 +569,6 @@ fn js_cg_expr(expr: Map<String, Any>) -> String {
return js_cg_lambda(expr) return js_cg_lambda(expr)
} }
if kind == "HtmlTemplate" {
return js_cg_html_template(expr)
}
"null" "null"
} }
@@ -1202,7 +1066,7 @@ fn codegen_js_inner(stmts: [Map<String, Any>], source: String, bundle_mode: Bool
js_emit_line(js_strip_es_exports(runtime_content)) js_emit_line(js_strip_es_exports(runtime_content))
js_emit_line("") js_emit_line("")
} else { } else {
js_emit_line("// Runtime: foundation/el/runtime/el_runtime.js") js_emit_line("// Runtime: foundation/el/el-compiler/runtime/el_runtime.js")
js_emit_line("import \"./el_runtime.js\";") js_emit_line("import \"./el_runtime.js\";")
} }
// In module mode: destructure all builtins off globalThis.__el so call // In module mode: destructure all builtins off globalThis.__el so call
File diff suppressed because it is too large Load Diff
@@ -20,44 +20,18 @@ import "codegen.el"
import "codegen-js.el" import "codegen-js.el"
// compile full pipeline (C target): source string -> C source string // compile full pipeline (C target): source string -> C source string
// Uses JIT function-at-a-time streaming: parse one decl emit C discard AST.
// Peak memory is O(one function's AST) instead of O(whole program AST).
fn compile(source: String) -> String { fn compile(source: String) -> String {
// Top-level arena scope: activates the string arena before lex() so that let tokens: [Map<String, Any>] = lex(source)
// ALL strdup allocations (token strings, sig strings, codegen fragments) let stmts: [Map<String, Any>] = parse(tokens)
// are tracked and freed on pop. Without this, lex() and scan_fn_sigs() // Token list is no longer needed after parsing release it to free memory
// run before any push, leaving _tl_arena_active=0 and leaking every // before codegen allocates its own working data on large source files.
// token string. Also prevents inner pop(mark=0) calls from deactivating el_release(tokens)
// the arena between per-function scopes. codegen(stmts, source)
let top_mark: Any = el_arena_push()
let tokens: [Any] = lex(source)
// Fast pre-scan: collect fn signatures + program kind without building
// full expression ASTs. O(tokens) time, minimal allocation.
let sigs: [Map<String, Any>] = scan_fn_sigs(tokens)
// Stream parse-emit: parse one decl at a time, emit C, discard.
// All output written to stdout via println before pop.
codegen_streaming(tokens, sigs, source)
el_arena_pop(top_mark)
""
}
// compile_test like compile() but sets __test_mode so codegen_streaming
// compiles test { } blocks instead of skipping them, and emits the test
// harness main() instead of the normal int main().
fn compile_test(source: String) -> String {
state_set("__test_mode", "1")
let top_mark: Any = el_arena_push()
let tokens: [Any] = lex(source)
let sigs: [Map<String, Any>] = scan_fn_sigs(tokens)
codegen_streaming(tokens, sigs, source)
el_arena_pop(top_mark)
state_set("__test_mode", "")
""
} }
// compile_js full pipeline (JS target, module mode): source string -> JS source string // compile_js full pipeline (JS target, module mode): source string -> JS source string
fn compile_js(source: String) -> String { fn compile_js(source: String) -> String {
let tokens: [Any] = lex(source) let tokens: [Map<String, Any>] = lex(source)
let stmts: [Map<String, Any>] = parse(tokens) let stmts: [Map<String, Any>] = parse(tokens)
// Token list is no longer needed after parsing release it to free memory. // Token list is no longer needed after parsing release it to free memory.
el_release(tokens) el_release(tokens)
@@ -67,7 +41,7 @@ fn compile_js(source: String) -> String {
// compile_js_with_bundle JS target in bundle mode. // compile_js_with_bundle JS target in bundle mode.
// Reads el_runtime.js from runtime_path and inlines it inside an IIFE. // Reads el_runtime.js from runtime_path and inlines it inside an IIFE.
fn compile_js_with_bundle(source: String, runtime_path: String) -> String { fn compile_js_with_bundle(source: String, runtime_path: String) -> String {
let tokens: [Any] = lex(source) let tokens: [Map<String, Any>] = lex(source)
let stmts: [Map<String, Any>] = parse(tokens) let stmts: [Map<String, Any>] = parse(tokens)
el_release(tokens) el_release(tokens)
let runtime_content: String = fs_read(runtime_path) let runtime_content: String = fs_read(runtime_path)
@@ -173,18 +147,6 @@ fn detect_obfuscate(argv: [String]) -> Bool {
return false return false
} }
// Detect --test flag in argv.
fn detect_test(argv: [String]) -> Bool {
let n: Int = native_list_len(argv)
let i = 0
while i < n {
let a: String = native_list_get(argv, i)
if str_eq(a, "--test") { return true }
let i = i + 1
}
return false
}
// Build a unique temp file path: /tmp/elc-<pid>-<timestamp>.<suffix> // Build a unique temp file path: /tmp/elc-<pid>-<timestamp>.<suffix>
fn make_temp_path(suffix: String) -> String { fn make_temp_path(suffix: String) -> String {
let pid: Int = getpid_now() let pid: Int = getpid_now()
@@ -287,9 +249,6 @@ fn type_node_to_el(t: Map<String, Any>) -> String {
// emit_header write a .elh file from parsed statements. // emit_header write a .elh file from parsed statements.
// Scans for FnDef nodes and emits 'extern fn' declarations. // Scans for FnDef nodes and emits 'extern fn' declarations.
// NOTE: This function requires the full AST. Prefer emit_header_from_sigs
// for the --emit-header path it works from a token-level scan without
// building expression ASTs, avoiding OOM on large files.
fn emit_header(stmts: [Map<String, Any>], hdr_path: String) -> Void { fn emit_header(stmts: [Map<String, Any>], hdr_path: String) -> Void {
let n: Int = native_list_len(stmts) let n: Int = native_list_len(stmts)
let i = 0 let i = 0
@@ -328,32 +287,6 @@ fn emit_header(stmts: [Map<String, Any>], hdr_path: String) -> Void {
let ok: Bool = fs_write(hdr_path, content) let ok: Bool = fs_write(hdr_path, content)
} }
// emit_header_from_sigs write a .elh file from pre-scanned El signatures.
// Uses the output of scan_fn_sigs_el() no full AST required.
// Peak memory is O(tokens) rather than O(whole-program AST), which prevents
// OOM on large files with HTML template bodies or deep BinOp chains.
fn emit_header_from_sigs(sigs: [Map<String, Any>], hdr_path: String) -> Void {
let n: Int = native_list_len(sigs)
let i: Int = 0
let parts: [String] = native_list_empty()
let parts = native_list_append(parts, "// auto-generated by elc --emit-header — do not edit\n")
while i < n {
let sig = native_list_get(sigs, i)
let kind: String = sig["kind"]
if str_eq(kind, "fn") {
let name: String = sig["name"]
let params_el: String = sig["params_el"]
let ret_el: String = sig["ret_el"]
if str_eq(ret_el, "") { let ret_el = "Any" }
let line: String = "extern fn " + name + "(" + params_el + ") -> " + ret_el
let parts = native_list_append(parts, line + "\n")
}
let i = i + 1
}
let content: String = str_join(parts, "")
let ok: Bool = fs_write(hdr_path, content)
}
// Import resolution // Import resolution
// //
// elc supports two forms of import: // elc supports two forms of import:
@@ -543,7 +476,6 @@ fn main() -> Void {
let do_bundle: Bool = detect_bundle(argv) let do_bundle: Bool = detect_bundle(argv)
let do_minify: Bool = detect_minify(argv) let do_minify: Bool = detect_minify(argv)
let do_obfuscate: Bool = detect_obfuscate(argv) let do_obfuscate: Bool = detect_obfuscate(argv)
let do_test: Bool = detect_test(argv)
// --obfuscate implies --minify: obfuscating unminified code is pointless. // --obfuscate implies --minify: obfuscating unminified code is pointless.
if do_obfuscate { if do_obfuscate {
let do_minify = true let do_minify = true
@@ -551,7 +483,7 @@ fn main() -> Void {
let positional: [String] = strip_flags(argv) let positional: [String] = strip_flags(argv)
let argc: Int = native_list_len(positional) let argc: Int = native_list_len(positional)
if argc < 1 { if argc < 1 {
println("el-compiler: usage: elc [--target=c|js] [--bundle] [--minify] [--obfuscate] [--emit-header] [--test] <source.el> [<output>]") println("el-compiler: usage: elc [--target=c|js] [--bundle] [--minify] [--obfuscate] [--emit-header] <source.el> [<output>]")
exit(1) exit(1)
} }
@@ -565,20 +497,16 @@ fn main() -> Void {
let src_path: String = native_list_get(positional, 0) let src_path: String = native_list_get(positional, 0)
// When --emit-header is requested, lex the source file and do a // When --emit-header is requested, parse the source file directly
// token-level signature scan (no full AST) to write a .elh file. // (without inlining imports) and write out a .elh file alongside the .c.
// This avoids OOM on large files with HTML template bodies or deep
// BinOp chains (e.g. checkout.el) parse() builds O(whole-program AST)
// while scan_fn_sigs_el keeps peak memory at O(tokens).
if do_emit_header { if do_emit_header {
el_mem_check()
let raw_source: String = fs_read(src_path) let raw_source: String = fs_read(src_path)
let hdr_tokens: [Any] = lex(raw_source) let hdr_tokens: [Map<String, Any>] = lex(raw_source)
let hdr_sigs: [Map<String, Any>] = scan_fn_sigs_el(hdr_tokens) let hdr_stmts: [Map<String, Any>] = parse(hdr_tokens)
el_release(hdr_tokens) el_release(hdr_tokens)
let hdr_path: String = str_slice(src_path, 0, str_len(src_path) - 3) + ".elh" let hdr_path: String = str_slice(src_path, 0, str_len(src_path) - 3) + ".elh"
emit_header_from_sigs(hdr_sigs, hdr_path) emit_header(hdr_stmts, hdr_path)
el_release(hdr_sigs) el_release(hdr_stmts)
} }
let source: String = resolve_imports(src_path) let source: String = resolve_imports(src_path)
@@ -592,12 +520,6 @@ fn main() -> Void {
exit(0) exit(0)
} }
// --test mode: compile with test harness (C target only).
if do_test {
compile_test(source)
exit(0)
}
// Standard path (no post-processing). // Standard path (no post-processing).
let out: String = "" let out: String = ""
if do_bundle { if do_bundle {
+749
View File
@@ -0,0 +1,749 @@
// lexer.el el self-hosting lexer
//
// Tokenises an el source string into a list of token maps.
// Each token is a Map<String, Any> with keys:
// "kind" -> String (e.g. "Int", "Ident", "Plus")
// "value" -> String (the raw text of the token)
//
// Entry point: fn lex(source: String) -> [Map<String, Any>]
//
// Uses native_string_chars to split the source into a chars list,
// then indexes it with native_list_get avoids O(N²) string cloning.
// Character helpers
fn lex_is_digit(ch: String) -> Bool {
if ch == "0" { return true }
if ch == "1" { return true }
if ch == "2" { return true }
if ch == "3" { return true }
if ch == "4" { return true }
if ch == "5" { return true }
if ch == "6" { return true }
if ch == "7" { return true }
if ch == "8" { return true }
if ch == "9" { return true }
false
}
fn lex_is_alpha(ch: String) -> Bool {
if ch == "a" { return true }
if ch == "b" { return true }
if ch == "c" { return true }
if ch == "d" { return true }
if ch == "e" { return true }
if ch == "f" { return true }
if ch == "g" { return true }
if ch == "h" { return true }
if ch == "i" { return true }
if ch == "j" { return true }
if ch == "k" { return true }
if ch == "l" { return true }
if ch == "m" { return true }
if ch == "n" { return true }
if ch == "o" { return true }
if ch == "p" { return true }
if ch == "q" { return true }
if ch == "r" { return true }
if ch == "s" { return true }
if ch == "t" { return true }
if ch == "u" { return true }
if ch == "v" { return true }
if ch == "w" { return true }
if ch == "x" { return true }
if ch == "y" { return true }
if ch == "z" { return true }
if ch == "A" { return true }
if ch == "B" { return true }
if ch == "C" { return true }
if ch == "D" { return true }
if ch == "E" { return true }
if ch == "F" { return true }
if ch == "G" { return true }
if ch == "H" { return true }
if ch == "I" { return true }
if ch == "J" { return true }
if ch == "K" { return true }
if ch == "L" { return true }
if ch == "M" { return true }
if ch == "N" { return true }
if ch == "O" { return true }
if ch == "P" { return true }
if ch == "Q" { return true }
if ch == "R" { return true }
if ch == "S" { return true }
if ch == "T" { return true }
if ch == "U" { return true }
if ch == "V" { return true }
if ch == "W" { return true }
if ch == "X" { return true }
if ch == "Y" { return true }
if ch == "Z" { return true }
false
}
fn is_alnum_or_underscore(ch: String) -> Bool {
if lex_is_digit(ch) { return true }
if lex_is_alpha(ch) { return true }
if ch == "_" { return true }
false
}
fn lex_is_whitespace(ch: String) -> Bool {
if ch == " " { return true }
if ch == "\t" { return true }
if ch == "\n" { return true }
if ch == "\r" { return true }
false
}
fn make_tok(kind: String, value: String) -> Map<String, Any> {
{ "kind": kind, "value": value }
}
// Keyword lookup
fn keyword_kind(word: String) -> String {
if word == "let" { return "Let" }
if word == "fn" { return "Fn" }
if word == "type" { return "Type" }
if word == "enum" { return "Enum" }
if word == "match" { return "Match" }
if word == "return" { return "Return" }
if word == "if" { return "If" }
if word == "else" { return "Else" }
if word == "for" { return "For" }
if word == "in" { return "In" }
if word == "while" { return "While" }
if word == "import" { return "Import" }
if word == "from" { return "From" }
if word == "as" { return "As" }
if word == "with" { return "With" }
if word == "sealed" { return "Sealed" }
if word == "activate" { return "Activate" }
if word == "where" { return "Where" }
if word == "test" { return "Test" }
if word == "seed" { return "Seed" }
if word == "assert" { return "Assert" }
if word == "protocol" { return "Protocol" }
if word == "impl" { return "Impl" }
if word == "retry" { return "Retry" }
if word == "times" { return "Times" }
if word == "fallback" { return "Fallback" }
if word == "reason" { return "Reason" }
if word == "parallel" { return "Parallel" }
if word == "trace" { return "Trace" }
if word == "requires" { return "Requires" }
if word == "deploy" { return "Deploy" }
if word == "to" { return "To" }
if word == "via" { return "Via" }
if word == "target" { return "Target" }
if word == "true" { return "Bool" }
if word == "false" { return "Bool" }
if word == "cgi" { return "Cgi" }
if word == "service" { return "Service" }
if word == "manager" { return "Manager" }
if word == "engine" { return "Engine" }
if word == "accessor" { return "Accessor" }
if word == "vessel" { return "Vessel" }
if word == "extern" { return "Extern" }
if word == "try" { return "Try" }
if word == "catch" { return "Catch" }
""
}
// Scan helpers
// All scan helpers receive the chars list and total length.
// scan_digits advance i while chars[i] is a digit
// Returns { "text": ..., "pos": i }
fn scan_digits(chars: [String], start: Int, total: Int) -> Map<String, Any> {
let i = start
let parts: [String] = native_list_empty()
let running = true
while running {
if i >= total {
let running = false
} else {
let ch: String = native_list_get(chars, i)
if lex_is_digit(ch) {
let parts = native_list_append(parts, ch)
let i = i + 1
} else {
let running = false
}
}
}
{ "text": str_join(parts, ""), "pos": i }
}
// scan_ident advance i while chars[i] is alphanumeric or underscore
fn scan_ident(chars: [String], start: Int, total: Int) -> Map<String, Any> {
let i = start
let parts: [String] = native_list_empty()
let running = true
while running {
if i >= total {
let running = false
} else {
let ch: String = native_list_get(chars, i)
if is_alnum_or_underscore(ch) {
let parts = native_list_append(parts, ch)
let i = i + 1
} else {
let running = false
}
}
}
{ "text": str_join(parts, ""), "pos": i }
}
// Code-bearing string detection + comment strip
// Inline JS/CSS literals embedded in El source (e.g. <script></script> blobs
// or stylesheet payloads inside string literals) carry their own line and
// block comments. Those comments leak into the served HTML and reveal build
// notes the visitor should never see. We strip them at the lexer so every
// downstream consumer (codegen-c, codegen-js, parser) gets the cleaned form.
//
// looks_like_code heuristic gate so we only strip strings that actually
// embed JS or CSS. Plain prose, hex blobs, JSON, etc. pass through verbatim.
fn substr_at(chars: [String], start: Int, total: Int, needle: String) -> Bool {
let nchars: [String] = native_string_chars(needle)
let nlen: Int = native_list_len(nchars)
if start + nlen > total { return false }
let i = 0
let matched = true
while i < nlen {
let a: String = native_list_get(chars, start + i)
let b: String = native_list_get(nchars, i)
if a == b { let i = i + 1 } else { let matched = false; let i = nlen }
}
matched
}
fn str_has(s: String, needle: String) -> Bool {
let chars: [String] = native_string_chars(s)
let total: Int = native_list_len(chars)
let i = 0
let found = false
while i < total {
if substr_at(chars, i, total, needle) {
let found = true
let i = total
} else {
let i = i + 1
}
}
found
}
fn looks_like_code(s: String) -> Bool {
if str_has(s, "<script") { return true }
if str_has(s, "<style") { return true }
if str_has(s, "function") {
if str_has(s, ";") { return true }
}
false
}
// strip_code_comments character-by-character walk. Tracks JS string state
// (single, double, backtick) and never strips inside one. Backslash escapes
// inside JS strings consume the next char verbatim. URLs like https:// are
// preserved by checking the previous char before treating // as a line
// comment opener: if the char immediately before '/' is ':', emit the '/'
// literally and advance one position.
fn strip_code_comments(s: String) -> String {
let chars: [String] = native_string_chars(s)
let total: Int = native_list_len(chars)
let out_parts: [String] = native_list_empty()
let i = 0
let in_squote = false
let in_dquote = false
let in_btick = false
let prev = ""
while i < total {
let ch: String = native_list_get(chars, i)
let in_js_string = false
if in_squote { let in_js_string = true }
if in_dquote { let in_js_string = true }
if in_btick { let in_js_string = true }
if in_js_string {
// Backslash escape: consume next char verbatim regardless of which.
if ch == "\\" {
let out_parts = native_list_append(out_parts, ch)
let next_i = i + 1
if next_i < total {
let nc: String = native_list_get(chars, next_i)
let out_parts = native_list_append(out_parts, nc)
let prev = nc
let i = next_i + 1
} else {
let prev = ch
let i = next_i
}
} else {
if in_squote {
if ch == "'" { let in_squote = false }
} else {
if in_dquote {
if ch == "\"" { let in_dquote = false }
} else {
if in_btick {
if ch == "`" { let in_btick = false }
}
}
}
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
}
} else {
// Not in a JS string. Check for comment openers.
let next_i = i + 1
let next_ch = ""
if next_i < total {
let next_ch: String = native_list_get(chars, next_i)
}
if ch == "/" {
if next_ch == "/" {
// URL guard: prev char ':' means this is "://", not a comment.
if prev == ":" {
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
} else {
// Skip until newline (newline itself is preserved so
// surrounding line counts/structure stay sane).
let i = i + 2
let scanning = true
while scanning {
if i >= total {
let scanning = false
} else {
let lc: String = native_list_get(chars, i)
if lc == "\n" {
let scanning = false
} else {
let i = i + 1
}
}
}
let prev = ""
}
} else {
if next_ch == "*" {
// Skip until matching "*/".
let i = i + 2
let scanning2 = true
while scanning2 {
if i >= total {
let scanning2 = false
} else {
let bc: String = native_list_get(chars, i)
if bc == "*" {
let after = i + 1
if after < total {
let nc2: String = native_list_get(chars, after)
if nc2 == "/" {
let i = after + 1
let scanning2 = false
} else {
let i = i + 1
}
} else {
let i = i + 1
}
} else {
let i = i + 1
}
}
}
let prev = ""
} else {
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
}
}
} else {
// Open a JS string?
if ch == "'" {
let in_squote = true
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
} else {
if ch == "\"" {
let in_dquote = true
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
} else {
if ch == "`" {
let in_btick = true
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
} else {
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
}
}
}
}
}
}
str_join(out_parts, "")
}
// scan_string scan a quoted string literal, handling \" escapes.
// Starts AFTER the opening quote. Returns { "text": content, "pos": i_after_close }
fn scan_string(chars: [String], start: Int, total: Int) -> Map<String, Any> {
let i = start
let parts: [String] = native_list_empty()
let running = true
while running {
if i >= total {
let running = false
} else {
let ch: String = native_list_get(chars, i)
if ch == "\\" {
// escape: peek next char
let next_i = i + 1
if next_i < total {
let next_ch: String = native_list_get(chars, next_i)
if next_ch == "\"" {
let parts = native_list_append(parts, "\"")
let i = next_i + 1
} else {
if next_ch == "n" {
let parts = native_list_append(parts, "\n")
let i = next_i + 1
} else {
if next_ch == "t" {
let parts = native_list_append(parts, "\t")
let i = next_i + 1
} else {
if next_ch == "r" {
let parts = native_list_append(parts, "\r")
let i = next_i + 1
} else {
if next_ch == "\\" {
let parts = native_list_append(parts, "\\")
let i = next_i + 1
} else {
let parts = native_list_append(parts, next_ch)
let i = next_i + 1
}
}
}
}
}
} else {
let i = i + 1
}
} else {
if ch == "\"" {
let i = i + 1
let running = false
} else {
let parts = native_list_append(parts, ch)
let i = i + 1
}
}
}
}
{ "text": str_join(parts, ""), "pos": i }
}
// Main lexer
fn lex(source: String) -> [Map<String, Any>] {
let chars: [String] = native_string_chars(source)
let total: Int = native_list_len(chars)
let tokens: [Map<String, Any>] = native_list_empty()
let i: Int = 0
while i < total {
let ch: String = native_list_get(chars, i)
// Skip whitespace
if lex_is_whitespace(ch) {
let i = i + 1
} else {
// Line comments: //
if ch == "/" {
let next_i = i + 1
if next_i < total {
let next_ch: String = native_list_get(chars, next_i)
if next_ch == "/" {
// skip to end of line
let i = i + 2
let running2 = true
while running2 {
if i >= total {
let running2 = false
} else {
let lch: String = native_list_get(chars, i)
if lch == "\n" {
let running2 = false
} else {
let i = i + 1
}
}
}
} else {
let tokens = native_list_append(tokens, make_tok("Slash", "/"))
let i = i + 1
}
} else {
let tokens = native_list_append(tokens, make_tok("Slash", "/"))
let i = i + 1
}
} else {
// String literal
if ch == "\"" {
let result = scan_string(chars, i + 1, total)
let str_text: String = result["text"]
let new_pos: Int = result["pos"]
// Compile-time scrub: strings that embed JS or CSS get
// their // line comments and /* block comments stripped
// before the token reaches the parser. Plain prose passes
// through untouched.
let clean_text = str_text
if looks_like_code(str_text) {
let clean_text = strip_code_comments(str_text)
}
let tokens = native_list_append(tokens, make_tok("Str", clean_text))
let i = new_pos
} else {
// Number literal
if lex_is_digit(ch) {
let result = scan_digits(chars, i, total)
let num_text: String = result["text"]
let new_pos: Int = result["pos"]
// check for float (dot followed by digit)
if new_pos < total {
let dot_ch: String = native_list_get(chars, new_pos)
if dot_ch == "." {
let after_dot = new_pos + 1
if after_dot < total {
let after_dot_ch: String = native_list_get(chars, after_dot)
if lex_is_digit(after_dot_ch) {
let frac_result = scan_digits(chars, after_dot, total)
let frac_text: String = frac_result["text"]
let frac_pos: Int = frac_result["pos"]
let tokens = native_list_append(tokens, make_tok("Float", num_text + "." + frac_text))
let i = frac_pos
} else {
let tokens = native_list_append(tokens, make_tok("Int", num_text))
let i = new_pos
}
} else {
let tokens = native_list_append(tokens, make_tok("Int", num_text))
let i = new_pos
}
} else {
let tokens = native_list_append(tokens, make_tok("Int", num_text))
let i = new_pos
}
} else {
let tokens = native_list_append(tokens, make_tok("Int", num_text))
let i = new_pos
}
} else {
// Identifier or keyword
if lex_is_alpha(ch) || ch == "_" {
let result = scan_ident(chars, i, total)
let word: String = result["text"]
let new_pos: Int = result["pos"]
let kw = keyword_kind(word)
if kw == "" {
let tokens = native_list_append(tokens, make_tok("Ident", word))
} else {
let tokens = native_list_append(tokens, make_tok(kw, word))
}
let i = new_pos
} else {
// Multi-char and single-char operators/delimiters
let peek_i = i + 1
let peek_ch = ""
if peek_i < total {
let peek_ch: String = native_list_get(chars, peek_i)
}
if ch == "=" {
if peek_ch == "=" {
let tokens = native_list_append(tokens, make_tok("EqEq", "=="))
let i = i + 2
} else {
if peek_ch == ">" {
let tokens = native_list_append(tokens, make_tok("FatArrow", "=>"))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Eq", "="))
let i = i + 1
}
}
} else {
if ch == "!" {
if peek_ch == "=" {
let tokens = native_list_append(tokens, make_tok("NotEq", "!="))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Not", "!"))
let i = i + 1
}
} else {
if ch == "<" {
if peek_ch == "=" {
let tokens = native_list_append(tokens, make_tok("LtEq", "<="))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Lt", "<"))
let i = i + 1
}
} else {
if ch == ">" {
if peek_ch == "=" {
let tokens = native_list_append(tokens, make_tok("GtEq", ">="))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Gt", ">"))
let i = i + 1
}
} else {
if ch == "&" {
if peek_ch == "&" {
let tokens = native_list_append(tokens, make_tok("And", "&&"))
let i = i + 2
} else {
let i = i + 1
}
} else {
if ch == "|" {
if peek_ch == "|" {
let tokens = native_list_append(tokens, make_tok("Or", "||"))
let i = i + 2
} else {
if peek_ch == ">" {
let tokens = native_list_append(tokens, make_tok("PipeOp", "|>"))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Pipe", "|"))
let i = i + 1
}
}
} else {
if ch == "-" {
if peek_ch == ">" {
let tokens = native_list_append(tokens, make_tok("Arrow", "->"))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Minus", "-"))
let i = i + 1
}
} else {
if ch == ":" {
if peek_ch == ":" {
let tokens = native_list_append(tokens, make_tok("ColonColon", "::"))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Colon", ":"))
let i = i + 1
}
} else {
if ch == "+" {
let tokens = native_list_append(tokens, make_tok("Plus", "+"))
let i = i + 1
} else {
if ch == "*" {
let tokens = native_list_append(tokens, make_tok("Star", "*"))
let i = i + 1
} else {
if ch == "%" {
let tokens = native_list_append(tokens, make_tok("Percent", "%"))
let i = i + 1
} else {
if ch == "(" {
let tokens = native_list_append(tokens, make_tok("LParen", "("))
let i = i + 1
} else {
if ch == ")" {
let tokens = native_list_append(tokens, make_tok("RParen", ")"))
let i = i + 1
} else {
if ch == "{" {
let tokens = native_list_append(tokens, make_tok("LBrace", "{"))
let i = i + 1
} else {
if ch == "}" {
let tokens = native_list_append(tokens, make_tok("RBrace", "}"))
let i = i + 1
} else {
if ch == "[" {
let tokens = native_list_append(tokens, make_tok("LBracket", "["))
let i = i + 1
} else {
if ch == "]" {
let tokens = native_list_append(tokens, make_tok("RBracket", "]"))
let i = i + 1
} else {
if ch == "," {
let tokens = native_list_append(tokens, make_tok("Comma", ","))
let i = i + 1
} else {
if ch == "." {
let tokens = native_list_append(tokens, make_tok("Dot", "."))
let i = i + 1
} else {
if ch == ";" {
let tokens = native_list_append(tokens, make_tok("Semicolon", ";"))
let i = i + 1
} else {
if ch == "@" {
let tokens = native_list_append(tokens, make_tok("At", "@"))
let i = i + 1
} else {
if ch == "?" {
let tokens = native_list_append(tokens, make_tok("QuestionMark", "?"))
let i = i + 1
} else {
// unknown char skip
let i = i + 1
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
let tokens = native_list_append(tokens, make_tok("Eof", ""))
tokens
}
File diff suppressed because it is too large Load Diff
+6 -80
View File
@@ -77,33 +77,6 @@ fn parse_manifest_entry(src: String) -> String {
return "" return ""
} }
// parse_manifest_c_sources - collect all `c_source "path"` lines from the
// build block. Returns a flat list of path strings.
fn parse_manifest_c_sources(src: String) -> [String] {
let result: [String] = native_list_empty()
let lines: [String] = str_split(src, "\n")
let n: Int = native_list_len(lines)
let i = 0
while i < n {
let line: String = native_list_get(lines, i)
let t: String = str_trim(line)
if str_starts_with(t, "c_source ") {
let after: String = str_slice(t, 9, str_len(t))
let trimmed: String = str_trim(after)
if str_starts_with(trimmed, "\"") {
let inner: String = str_slice(trimmed, 1, str_len(trimmed))
let q: Int = str_index_of(inner, "\"")
if q >= 0 {
let path: String = str_slice(inner, 0, q)
let result = native_list_append(result, path)
}
}
}
let i = i + 1
}
return result
}
fn parse_manifest_name(src: String) -> String { fn parse_manifest_name(src: String) -> String {
let lines: [String] = str_split(src, "\n") let lines: [String] = str_split(src, "\n")
let n: Int = native_list_len(lines) let n: Int = native_list_len(lines)
@@ -252,7 +225,6 @@ fn compile_module(src_path: String, out_dir: String, elc_bin: String, dry_run: B
let bname: String = basename_noext(src_path) let bname: String = basename_noext(src_path)
let c_out: String = out_dir + "/" + bname + ".c" let c_out: String = out_dir + "/" + bname + ".c"
let elh_out: String = out_dir + "/" + bname + ".elh" let elh_out: String = out_dir + "/" + bname + ".elh"
let err_tmp: String = "/tmp/elb-err-" + bname + ".txt"
// Check if recompile needed // Check if recompile needed
if !file_is_newer(src_path, c_out) { if !file_is_newer(src_path, c_out) {
@@ -262,26 +234,18 @@ fn compile_module(src_path: String, out_dir: String, elc_bin: String, dry_run: B
return true return true
} }
// elc streams C to stdout; redirect stderr to a temp file so we can // elc streams C to stdout (collect mode not yet implemented); use
// surface the actual error message on failure instead of swallowing it. // shell redirection so the output lands in the file, not the terminal.
let cmd: String = elc_bin + " --emit-header " + src_path + " > " + c_out + " 2>" + err_tmp let cmd: String = elc_bin + " --emit-header " + src_path + " > " + c_out + " 2>&1"
println(" compile " + src_path) println(" compile " + src_path)
if dry_run { return true } if dry_run { return true }
let ret: Int = exec_command(cmd) let ret: Int = exec_command(cmd)
if ret != 0 { if ret != 0 {
// Surface the actual compiler error from stderr
let err_msg: String = str_trim(fs_read(err_tmp))
if !str_eq(err_msg, "") {
println(err_msg)
}
// Remove partial output so a retry starts clean
exec_command("rm -f " + c_out + " " + err_tmp)
println("elb: compile failed: " + src_path) println("elb: compile failed: " + src_path)
return false return false
} }
exec_command("rm -f " + err_tmp)
// Move the generated .elh (written next to the source by elc) into // Move the generated .elh (written next to the source by elc) into
// out_dir so that #include "module.elh" lines in the generated .c // out_dir so that #include "module.elh" lines in the generated .c
@@ -298,21 +262,7 @@ fn link_binary(c_files: [String], out_bin: String, runtime_path: String, out_dir
let parts: [String] = native_list_empty() let parts: [String] = native_list_empty()
// Include both the runtime dir (for el_runtime.h) and the output dir // Include both the runtime dir (for el_runtime.h) and the output dir
// (for module.elh cross-module forward declarations). // (for module.elh cross-module forward declarations).
// Detect clang vs gcc: -fbracket-depth is clang-only; silently ignored let parts = native_list_append(parts, "cc -O2 -I " + dirname_of(runtime_path) + " -I " + out_dir)
// if unsupported but gcc rejects it with an error.
let bracket_flag: String = "$(cc --version 2>&1 | grep -q clang && printf -- '-fbracket-depth=1024' || true)"
// On macOS, OpenSSL is not on the default linker path. Detect homebrew
// prefix and add it if present (no-op on Linux where libssl is in /usr/lib).
let ossl_lib_flag: String = "$(brew --prefix openssl 2>/dev/null | xargs -I{} printf -- '-L{}/lib' 2>/dev/null || true)"
let ossl_inc_flag: String = "$(brew --prefix openssl 2>/dev/null | xargs -I{} printf -- '-I{}/include' 2>/dev/null || true)"
// Force-include the C-level master declarations header so every translation
// unit sees all cross-module function signatures. Handles packages (like ELP)
// where modules call each other without explicit El import statements.
// The header is generated by elb --gen-decls or manually placed in out_dir.
let master_decls: String = out_dir + "/elp-c-decls.h"
let has_master: String = str_trim(exec_capture("test -f " + master_decls + " && echo yes || echo no"))
let include_flag: String = if str_eq(has_master, "yes") { "-include " + master_decls } else { "" }
let parts = native_list_append(parts, "cc -O2 " + bracket_flag + " " + ossl_inc_flag + " " + include_flag + " -I " + dirname_of(runtime_path) + " -I " + out_dir)
let i = 0 let i = 0
while i < n { while i < n {
let f: String = native_list_get(c_files, i) let f: String = native_list_get(c_files, i)
@@ -320,7 +270,7 @@ fn link_binary(c_files: [String], out_bin: String, runtime_path: String, out_dir
let i = i + 1 let i = i + 1
} }
let parts = native_list_append(parts, runtime_path) let parts = native_list_append(parts, runtime_path)
let parts = native_list_append(parts, ossl_lib_flag + " -lcurl -lssl -lcrypto -lpthread -lm") let parts = native_list_append(parts, "-lcurl -lpthread")
let parts = native_list_append(parts, "-o " + out_bin) let parts = native_list_append(parts, "-o " + out_bin)
let cmd: String = str_join(parts, " ") let cmd: String = str_join(parts, " ")
println(" link " + out_bin) println(" link " + out_bin)
@@ -353,7 +303,6 @@ fn main() -> Void {
let pkg_name: String = parse_manifest_name(manifest_src) let pkg_name: String = parse_manifest_name(manifest_src)
let entry: String = parse_manifest_entry(manifest_src) let entry: String = parse_manifest_entry(manifest_src)
let extra_c: [String] = parse_manifest_c_sources(manifest_src)
if str_eq(entry, "") { if str_eq(entry, "") {
println("elb: manifest.el has no 'entry' declaration") println("elb: manifest.el has no 'entry' declaration")
exit(1) exit(1)
@@ -368,21 +317,7 @@ fn main() -> Void {
let which_out: String = str_trim(exec_capture("which " + elc_bin + " 2>/dev/null")) let which_out: String = str_trim(exec_capture("which " + elc_bin + " 2>/dev/null"))
if !str_eq(which_out, "") { if !str_eq(which_out, "") {
let elc_dir: String = dirname_of(which_out) let elc_dir: String = dirname_of(which_out)
runtime_path = elc_dir + "/../runtime/el_runtime.c" runtime_path = elc_dir + "/../el-compiler/runtime/el_runtime.c"
}
}
// If --runtime points to a directory, auto-locate el_runtime.c inside it.
// This lets both forms work:
// --runtime=/opt/el/runtime (directory form)
// --runtime=/opt/el/runtime/el_runtime.c (file form)
if !str_eq(runtime_path, "") {
let is_dir: String = str_trim(exec_capture("test -d " + runtime_path + " && echo dir || echo file"))
if str_eq(is_dir, "dir") {
let candidate: String = runtime_path + "/el_runtime.c"
let has_file: String = str_trim(exec_capture("test -f " + candidate + " && echo yes || echo no"))
if str_eq(has_file, "yes") {
let runtime_path = candidate
}
} }
} }
if str_eq(runtime_path, "") { if str_eq(runtime_path, "") {
@@ -432,15 +367,6 @@ fn main() -> Void {
exit(1) exit(1)
} }
// Append any extra C sources declared in the manifest (e.g. platform stubs)
let ei = 0
let en: Int = native_list_len(extra_c)
while ei < en {
let ec: String = native_list_get(extra_c, ei)
let c_files = native_list_append(c_files, ec)
let ei = ei + 1
}
// Link // Link
let out_bin: String = out_dir + "/" + pkg_name let out_bin: String = out_dir + "/" + pkg_name
let linked: Bool = link_binary(c_files, out_bin, runtime_path, out_dir, dry_run) let linked: Bool = link_binary(c_files, out_bin, runtime_path, out_dir, dry_run)
View File
-3
View File
@@ -3797,9 +3797,6 @@ fn builtin_arity(name: String) -> Int {
if str_eq(name, "engram_activate") { return 2 } if str_eq(name, "engram_activate") { return 2 }
if str_eq(name, "engram_save") { return 1 } if str_eq(name, "engram_save") { return 1 }
if str_eq(name, "engram_load") { return 1 } if str_eq(name, "engram_load") { return 1 }
if str_eq(name, "engram_store_boot") { return 1 }
if str_eq(name, "engram_store_checkpoint") { return 0 }
if str_eq(name, "engram_store_close") { return 0 }
if str_eq(name, "engram_get_node_json") { return 1 } if str_eq(name, "engram_get_node_json") { return 1 }
if str_eq(name, "engram_search_json") { return 2 } if str_eq(name, "engram_search_json") { return 2 }
if str_eq(name, "engram_scan_nodes_json") { return 2 } if str_eq(name, "engram_scan_nodes_json") { return 2 }
-21
View File
@@ -1,21 +0,0 @@
# Compiled El bytecode
*.elc
# C codegen output
*.c
*.o
*.a
*.so
*.dylib
# Combined build artifacts
_combined.el
*-combined.el
# Distribution / build output
dist/
build/
out/
# OS
.DS_Store
-65
View File
@@ -1,65 +0,0 @@
# ELP language consolidation — full-lexicon backfill (stage)
Branch: `stage-elp-lang-consolidation` (stage-bound; NOT the live soul :8742).
Consolidates scattered Python language-realizer work (`~/Desktop/lang-realizers`,
`~/Desktop/lang-poetry-experiment`, `~/semitic_engine`) into the ELP `.el`
structure, generating **full lexicons** (complete UniMorph + kaikki.org
Wiktionary — real gender, real inflections) instead of the demo/curated subsets
the prototypes shipped.
## ELP before this branch
- 18 classical/ancient languages fully done (vocab + morphology + tests):
akk ang cop egy enm fro gez goh got grc non peo pi sa sga sux txb uga.
- 11 modern/classical languages had `morphology-<code>.el` in the build manifest
but **no vocabulary and no lang_profile**: es fr de ja ar he hi ru fi sw la.
- The ES port (`stage-elp-es-port`) had a *demo-scale* vocabulary-es.el (~350
entries, s-expr form).
## Landed on this branch (full-lexicon seed-fn format, matching the 18 ancients)
Vocabulary schema per row: `[lemma, pos, form0, form1, form2, en_gloss, hint]`.
Files are ELP runtime **seed data** (loaded via the Engram at runtime), so — like
all 18 classical `vocabulary-*.el` — they are intentionally NOT in the build
manifest. Syntax validated: the chunked `fn vocab_<code>_seed_pN` format
compiles cleanly to C via `elc` (correct UTF-8).
| code | in-ELP-morph? | vocab entries | verbs | nouns | adjs | profile |
|------|---------------|--------------:|------:|------:|-----:|---------|
| es | yes | 72,032 | 6,695 | 48,353 | 16,984 | yes |
| fr | yes | 130,517 | 7,534 | 77,344 | 45,639 | yes |
| de | yes | 144,692 | 6,661 | 133,162 | 4,869 | yes |
| la | yes | 22,590 | 82 | 13,436 | 9,072 | yes |
| it | no (bonus) | 193,675 | 10,008 | 109,459 | 74,208 | yes |
| pt | no (bonus) | 115,772 | 4,001 | 72,073 | 39,698 | yes |
| ro | no (bonus) | 86,504 | 1,216 | 65,915 | 19,373 | yes |
| ca | no (bonus) | 47,112 | 1,547 | 28,830 | 16,735 | yes |
|**total**| |**812,894** | | | | |
Generators (reproducible): `elp/tests/lang-gen/gen_elp_seed_full.py` (Romance),
`gen_elp_seed_de_la.py` (German declension + Latin case-paradigm mapping). They
read the pre-built morph caches in `~/Desktop/lang-realizers/data/` (UniMorph +
kaikki), which are too large to commit.
## Remaining (honest)
Of the 11 ELP backfill targets, 4 are done (es fr de la). The other 7 have **no
full-lexicon engine** yet — cannot be generated honestly without engine work:
- **ru**: only a 110-entry curated Slavic subset exists; full `rus.unimorph`
present but no `morphology_ru_full` productive loader. Needs a full Russian
morphology module (like the Romance ones) before vocab generation.
- **ja / ko / zh**: validated demo engines (~66-104 hardcoded words) in
`lang-poetry-experiment`, Python only. Agglutinative (ja/ko) + isolating (zh)
need `.el` engine ports + full-lexicon wiring (ja: jpn_unimorph; zh: CC-CEDICT).
- **ar / he (Semitic)**: template engines (16 AR / 8 HE patterns, ~6 roots) in
`~/semitic_engine`, Python only. Root-and-pattern; full UniMorph ara/heb
present but used only for validation. Needs productive root lexicon + `.el` port.
- **hi (Hindi), fi (Finnish), sw (Swahili)**: `morphology-<code>.el` exists in
ELP but there is NO scattered prototype and NO downloaded data for these —
full-lexicon collection (UniMorph/kaikki) + generator still to do.
De/nl/sv Germanic and it/ro/ca/pt Romance verb coverage note: German verbs here
are the ~6.6k caches carry; the it/ro/ca/pt bonus languages have full vocab but
**no `morphology-<code>.el` in ELP yet** (Python realizer exists; `.el` port is
the remaining engine work).
Construction coverage (separate from lexicon): French realizer was ~55%,
Semitic ~3% in the prototypes — full construction coverage remains its own task.
File diff suppressed because one or more lines are too long
-23
View File
@@ -1,23 +0,0 @@
{
"dataset": "british-rp-accent-transform",
"primitive_type": "accent_target",
"accent": "british-rp",
"grounding": "derived",
"provenance": "HONEST-DERIVED, COARSE FIRST PASS — NOT transcribed measured RP formants. The exact measured RP/GB tables (Deterding 1997 JIPA 27:47-55; Hawkins & Midgley 2005 JIPA 35:183-199) are the intended ground truth but were gated/figure-only at author time and were NOT transcribed. So these targets are DERIVED: each = the corresponding MEASURED Peterson&Barney(1952) base vowel transformed under the documented, citable RP-vs-GA structural rules of Wells (1982) 'Accents of English' — non-rhoticity (NURSE de-rhoticized: remove low F3), TRAP F2-lowering, LOT/THOUGHT back-rounding (F2 down), GOOSE-fronting (F2 up), GOAT centering. Shift MAGNITUDES are coarse/approximate (first pass), directions are cited. ground:derived (base measured + rule cited). Refine by transcribing Deterding/Hawkins&Midgley. No number is presented as a measured RP value it is not.",
"notes": "records with kind=vowel_override REPLACE the base phoneme's formant targets with the DERIVED RP realization. records with kind=rule encode non-formant transforms (non-rhoticity: drop post-vocalic coda /r/). The render composes: base geometry then accent override + rhoticity rule — voice + accent, separable.",
"records": [
{"key": "IY", "features": {"kind": "vowel_override", "set": "FLEECE"}, "attributes": {"f1": 280, "f2": 2249, "f3": 3000}},
{"key": "IH", "features": {"kind": "vowel_override", "set": "KIT"}, "attributes": {"f1": 360, "f2": 2100, "f3": 2550}},
{"key": "EH", "features": {"kind": "vowel_override", "set": "DRESS"}, "attributes": {"f1": 560, "f2": 1970, "f3": 2480}},
{"key": "AE", "features": {"kind": "vowel_override", "set": "TRAP"}, "attributes": {"f1": 730, "f2": 1590, "f3": 2410}},
{"key": "AA", "features": {"kind": "vowel_override", "set": "LOT"}, "attributes": {"f1": 560, "f2": 920, "f3": 2440}},
{"key": "AO", "features": {"kind": "vowel_override", "set": "THOUGHT"}, "attributes": {"f1": 415, "f2": 700, "f3": 2410}},
{"key": "UH", "features": {"kind": "vowel_override", "set": "FOOT"}, "attributes": {"f1": 380, "f2": 1100, "f3": 2240}},
{"key": "UW", "features": {"kind": "vowel_override", "set": "GOOSE"}, "attributes": {"f1": 310, "f2": 1650, "f3": 2240}},
{"key": "AH", "features": {"kind": "vowel_override", "set": "STRUT"}, "attributes": {"f1": 680, "f2": 1180, "f3": 2390}},
{"key": "ER", "features": {"kind": "vowel_override", "set": "NURSE", "rhotic": "no"}, "attributes": {"f1": 550, "f2": 1500, "f3": 2500}},
{"key": "AX", "features": {"kind": "vowel_override", "set": "commA"}, "attributes": {"f1": 500, "f2": 1500, "f3": 2500}},
{"key": "OW", "features": {"kind": "vowel_override", "set": "GOAT"}, "attributes": {"f1": 450, "f2": 1400, "f3": 2380}},
{"key": "R", "features": {"kind": "rule", "rule": "non_rhotic"}, "attributes": {"drop_coda_r": 1}}
]
}
-26
View File
@@ -1,26 +0,0 @@
# british-rp-accent TRANSFORM — INGESTIBLE DATA (a geometry/transform composed
# onto the base General-American phoneme targets; voice + accent, separable).
#
# PROVENANCE — HONEST, COARSE FIRST PASS. These are DERIVED targets, NOT
# transcribed measured RP formants. Measured RP tables (Deterding 1997 JIPA 27;
# Hawkins & Midgley 2005 JIPA 35) are the intended ground truth but were gated at
# author time and NOT transcribed. Each target = the MEASURED Peterson&Barney
# (1952) base vowel transformed under the documented, citable RP-vs-GA structural
# rules of Wells (1982): non-rhoticity, TRAP F2-lowering, LOT/THOUGHT back-
# rounding, GOOSE-fronting, GOAT centering, NURSE de-rhoticization. Shift
# magnitudes are coarse/approximate; directions are cited. ground=derived.
# Refine by transcribing the measured RP tables. No value is claimed as measured.
# Format: KEY|F1|F2|F3|KIND|SET
IY|280|2249|3000|vowel_override|FLEECE
IH|360|2100|2550|vowel_override|KIT
EH|560|1970|2480|vowel_override|DRESS
AE|730|1590|2410|vowel_override|TRAP
AA|560|920|2440|vowel_override|LOT
AO|415|700|2410|vowel_override|THOUGHT
UH|380|1100|2240|vowel_override|FOOT
UW|310|1650|2240|vowel_override|GOOSE
AH|680|1180|2390|vowel_override|STRUT
ER|550|1500|2500|vowel_override|NURSE-nonrhotic
AX|500|1500|2500|vowel_override|commA
OW|450|1400|2380|vowel_override|GOAT
R|0|0|0|rule|non_rhotic_drop_coda
-20
View File
@@ -1,20 +0,0 @@
# pronunciation lexicon SOURCE — word -> phoneme sequence, as INGESTIBLE DATA.
# Pronunciation is linguistic KNOWLEDGE (the language faculty's orthography->
# phonology map), ingested into the engram, not frozen in code. The render reads
# a word's phoneme sequence back from the engram. Covers the self-lexicon and the
# proof sentences; general G2P is the realizer/morphology faculty's remit.
# Diphthongs are written as two vowel targets (the render's transitions glide
# between them). Format: word|PH1 PH2 PH3 ...
i|AA IY
am|AE M
neuron|N UW R AA N
is|IH Z
memory|M EH M ER IY
hello|HH EH L OW
the|DH AH
a|AH
remember|R IH M EH M ER
i'm|AA IY M
you|Y UW
here|HH IY R
will|W IH L
File diff suppressed because one or more lines are too long
-528
View File
@@ -1,528 +0,0 @@
{
"dataset": "english-phoneme-formants",
"primitive_type": "phoneme",
"grounding": "extracted",
"provenance": "AUDITED per-field. The 10 monophthong-vowel F1/F2/F3 (IY,IH,EH,AE,AA,AO,UH,UW,AH,ER) are the MEASURED adult-male /hVd/ means of Peterson & Barney (1952) JASA 24:175-184, verified vs CRAN phonTools::pb52. AX=neutral uniform-tube resonances (Fant, physics). OW steady target = synthesis convention (diphthong). Consonant loci (M,N,NG,L,R,W,Y,Z,DH,V,S,F,HH) and ALL bandwidths + dur/amp = standard formant-synthesis conventions (Klatt 1980 JASA 67:971), engineering defaults NOT field measurements. No numbers invented/LLM-generated.",
"records": [
{
"key": "IY",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 270,
"f2": 2290,
"f3": 3010,
"bw1": 60,
"bw2": 90,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 130,
"amp": 100
}
},
{
"key": "IH",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 390,
"f2": 1990,
"f3": 2550,
"bw1": 70,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 110,
"amp": 100
}
},
{
"key": "EH",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 530,
"f2": 1840,
"f3": 2480,
"bw1": 80,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 130,
"amp": 100
}
},
{
"key": "AE",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 660,
"f2": 1720,
"f3": 2410,
"bw1": 90,
"bw2": 110,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 150,
"amp": 100
}
},
{
"key": "AA",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 730,
"f2": 1090,
"f3": 2440,
"bw1": 90,
"bw2": 110,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 150,
"amp": 100
}
},
{
"key": "AO",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 570,
"f2": 840,
"f3": 2410,
"bw1": 80,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 140,
"amp": 100
}
},
{
"key": "UH",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 440,
"f2": 1020,
"f3": 2240,
"bw1": 70,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 110,
"amp": 100
}
},
{
"key": "UW",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 300,
"f2": 870,
"f3": 2240,
"bw1": 70,
"bw2": 90,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 140,
"amp": 100
}
},
{
"key": "AH",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 640,
"f2": 1190,
"f3": 2390,
"bw1": 80,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 110,
"amp": 95
}
},
{
"key": "ER",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 490,
"f2": 1350,
"f3": 1690,
"bw1": 80,
"bw2": 100,
"bw3": 120,
"voiced": 1,
"nasal": 0,
"dur": 140,
"amp": 95
}
},
{
"key": "AX",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 500,
"f2": 1500,
"f3": 2500,
"bw1": 80,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 80,
"amp": 85
}
},
{
"key": "OW",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 490,
"f2": 910,
"f3": 2380,
"bw1": 80,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 140,
"amp": 100
}
},
{
"key": "M",
"features": {
"manner": "nasal",
"voiced": "yes",
"nasal": "yes"
},
"attributes": {
"f1": 250,
"f2": 900,
"f3": 2200,
"bw1": 90,
"bw2": 120,
"bw3": 180,
"voiced": 1,
"nasal": 1,
"dur": 80,
"amp": 60
}
},
{
"key": "N",
"features": {
"manner": "nasal",
"voiced": "yes",
"nasal": "yes"
},
"attributes": {
"f1": 250,
"f2": 1700,
"f3": 2600,
"bw1": 90,
"bw2": 120,
"bw3": 180,
"voiced": 1,
"nasal": 1,
"dur": 80,
"amp": 60
}
},
{
"key": "NG",
"features": {
"manner": "nasal",
"voiced": "yes",
"nasal": "yes"
},
"attributes": {
"f1": 250,
"f2": 2300,
"f3": 2700,
"bw1": 90,
"bw2": 120,
"bw3": 180,
"voiced": 1,
"nasal": 1,
"dur": 80,
"amp": 60
}
},
{
"key": "L",
"features": {
"manner": "approximant",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 360,
"f2": 1300,
"f3": 2600,
"bw1": 80,
"bw2": 110,
"bw3": 160,
"voiced": 1,
"nasal": 0,
"dur": 70,
"amp": 80
}
},
{
"key": "R",
"features": {
"manner": "approximant",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 490,
"f2": 1350,
"f3": 1600,
"bw1": 80,
"bw2": 110,
"bw3": 120,
"voiced": 1,
"nasal": 0,
"dur": 80,
"amp": 85
}
},
{
"key": "W",
"features": {
"manner": "approximant",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 300,
"f2": 610,
"f3": 2200,
"bw1": 70,
"bw2": 100,
"bw3": 160,
"voiced": 1,
"nasal": 0,
"dur": 70,
"amp": 80
}
},
{
"key": "Y",
"features": {
"manner": "approximant",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 270,
"f2": 2290,
"f3": 3010,
"bw1": 60,
"bw2": 90,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 60,
"amp": 80
}
},
{
"key": "Z",
"features": {
"manner": "fricative",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 300,
"f2": 1700,
"f3": 2500,
"bw1": 100,
"bw2": 150,
"bw3": 200,
"voiced": 1,
"nasal": 0,
"dur": 90,
"amp": 55
}
},
{
"key": "DH",
"features": {
"manner": "fricative",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 300,
"f2": 1400,
"f3": 2500,
"bw1": 100,
"bw2": 150,
"bw3": 200,
"voiced": 1,
"nasal": 0,
"dur": 70,
"amp": 55
}
},
{
"key": "V",
"features": {
"manner": "fricative",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 300,
"f2": 1000,
"f3": 2300,
"bw1": 100,
"bw2": 150,
"bw3": 200,
"voiced": 1,
"nasal": 0,
"dur": 70,
"amp": 55
}
},
{
"key": "S",
"features": {
"manner": "fricative",
"voiced": "no",
"nasal": "no"
},
"attributes": {
"f1": 320,
"f2": 1700,
"f3": 2500,
"bw1": 200,
"bw2": 200,
"bw3": 250,
"voiced": 0,
"nasal": 0,
"dur": 110,
"amp": 45
}
},
{
"key": "F",
"features": {
"manner": "fricative",
"voiced": "no",
"nasal": "no"
},
"attributes": {
"f1": 300,
"f2": 1200,
"f3": 2400,
"bw1": 200,
"bw2": 200,
"bw3": 250,
"voiced": 0,
"nasal": 0,
"dur": 100,
"amp": 40
}
},
{
"key": "HH",
"features": {
"manner": "fricative",
"voiced": "no",
"nasal": "no"
},
"attributes": {
"f1": 500,
"f2": 1500,
"f3": 2500,
"bw1": 200,
"bw2": 250,
"bw3": 300,
"voiced": 0,
"nasal": 0,
"dur": 70,
"amp": 40
}
},
{
"key": "SIL",
"features": {
"manner": "silence",
"voiced": "no",
"nasal": "no"
},
"attributes": {
"f1": 500,
"f2": 1500,
"f3": 2500,
"bw1": 100,
"bw2": 100,
"bw3": 100,
"voiced": 0,
"nasal": 0,
"dur": 55,
"amp": 0
}
}
]
}
-45
View File
@@ -1,45 +0,0 @@
# acoustic-phonetics SOURCE — the learned speech primitives, as INGESTIBLE DATA.
# NOT audio, NOT code: formant geometry of the phonemes, to be ingested via the
# ingest organ into the engram as a phoneme manifold. The render reads this
# geometry back from the engram; nothing is frozen in EL code.
#
# PROVENANCE (audited, per-field honesty — no invented numbers):
# * The 10 MONOPHTHONG VOWEL formants F1/F2/F3 (IY,IH,EH,AE,AA,AO,UH,UW,AH,ER)
# are the MEASURED adult-male means of Peterson & Barney (1952), JASA 24:175-184
# — the canonical /hVd/ table, verified digit-for-digit vs CRAN phonTools::pb52.
# These are real measured values.
# * AX (schwa) F1/F2/F3 = neutral uniform-tube resonances (2n-1)*500 — a PHYSICS
# value (Fant), not a P&B measurement.
# * OW is a diphthong; its listed steady target is a conventional synthesis value,
# not a P&B monophthong measurement.
# * CONSONANT loci (M,N,NG,L,R,W,Y,Z,DH,V,S,F,HH) and ALL BANDWIDTHS (B1,B2,B3)
# and dur/amp are STANDARD FORMANT-SYNTHESIS conventions (Klatt 1980, JASA 67:971
# "Software for a cascade/parallel formant synthesizer") — engineering defaults,
# NOT per-phoneme field measurements. Labeled as such, not attributed to P&B.
# Format: SYM|F1|F2|F3|B1|B2|B3|voiced|nasal|dur_ms|amp|class|example
IY|270|2290|3010|60|90|150|1|0|130|100|vowel|beet
IH|390|1990|2550|70|100|150|1|0|110|100|vowel|bit
EH|530|1840|2480|80|100|150|1|0|130|100|vowel|bet
AE|660|1720|2410|90|110|150|1|0|150|100|vowel|bat
AA|730|1090|2440|90|110|150|1|0|150|100|vowel|bot
AO|570|840|2410|80|100|150|1|0|140|100|vowel|bought
UH|440|1020|2240|70|100|150|1|0|110|100|vowel|book
UW|300|870|2240|70|90|150|1|0|140|100|vowel|boot
AH|640|1190|2390|80|100|150|1|0|110|95|vowel|but
ER|490|1350|1690|80|100|120|1|0|140|95|vowel|bird
AX|500|1500|2500|80|100|150|1|0|80|85|vowel|about
OW|490|910|2380|80|100|150|1|0|140|100|vowel|boat
M|250|900|2200|90|120|180|1|1|80|60|nasal|map
N|250|1700|2600|90|120|180|1|1|80|60|nasal|nap
NG|250|2300|2700|90|120|180|1|1|80|60|nasal|sing
L|360|1300|2600|80|110|160|1|0|70|80|approximant|lip
R|490|1350|1600|80|110|120|1|0|80|85|approximant|rip
W|300|610|2200|70|100|160|1|0|70|80|approximant|wet
Y|270|2290|3010|60|90|150|1|0|60|80|approximant|yet
Z|300|1700|2500|100|150|200|1|0|90|55|fricative|zoo
DH|300|1400|2500|100|150|200|1|0|70|55|fricative|the
V|300|1000|2300|100|150|200|1|0|70|55|fricative|van
S|320|1700|2500|200|200|250|0|0|110|45|fricative|see
F|300|1200|2400|200|200|250|0|0|100|40|fricative|fee
HH|500|1500|2500|200|250|300|0|0|70|40|fricative|hat
SIL|500|1500|2500|100|100|100|0|0|55|0|silence|_
-90
View File
@@ -1,90 +0,0 @@
package "elp" {
version "0.7.0"
description "Engram Language Protocol — bidirectional engine mapping between Engram semantic forms and natural language surface text. 31 languages."
edition "2026"
}
build {
entry "src/elp.el"
// Compilation order (dependency order):
// language-profile (no deps)
// vocabulary (no deps)
// morphology (depends on: language-profile)
// morphology-es (depends on: morphology) Spanish
// morphology-fr (depends on: morphology) French
// morphology-de (depends on: morphology) German
// morphology-ru (depends on: morphology) Russian
// morphology-ja (depends on: morphology) Japanese
// morphology-fi (depends on: morphology) Finnish
// morphology-ar (depends on: morphology) Arabic
// morphology-hi (depends on: morphology) Hindi
// morphology-sw (depends on: morphology) Swahili
// morphology-la (depends on: morphology) Latin
// morphology-he (depends on: morphology) Hebrew
// morphology-grc (depends on: morphology) Ancient Greek
// morphology-ang (depends on: morphology) Old English
// morphology-sa (depends on: morphology) Sanskrit
// morphology-got (depends on: morphology) Gothic
// morphology-non (depends on: morphology) Old Norse
// morphology-enm (depends on: morphology) Middle English
// morphology-pi (depends on: morphology) Pali
// morphology-fro (depends on: morphology) Old French
// morphology-goh (depends on: morphology) Old High German
// morphology-sga (depends on: morphology) Old Irish
// morphology-txb (depends on: morphology) Tocharian B
// morphology-peo (depends on: morphology) Old Persian
// morphology-akk (depends on: morphology) Akkadian
// morphology-uga (depends on: morphology) Ugaritic
// morphology-egy (depends on: morphology) Ancient Egyptian
// morphology-sux (depends on: morphology) Sumerian
// morphology-gez (depends on: morphology) Ge'ez (Classical Ethiopic)
// morphology-cop (depends on: morphology) Coptic (Sahidic)
// grammar (depends on: language-profile)
// realizer (depends on: morphology, grammar, language-profile)
// semantics (depends on: grammar, realizer, language-profile)
// elp (depends on: semantics, realizer)
sources [
"src/language-profile.el",
"src/vocabulary.el",
"src/morphology.el",
"src/morphology-es.el",
"src/morphology-fr.el",
"src/morphology-de.el",
"src/morphology-ru.el",
"src/morphology-ja.el",
"src/morphology-fi.el",
"src/morphology-ar.el",
"src/morphology-hi.el",
"src/morphology-sw.el",
"src/morphology-la.el",
"src/morphology-he.el",
"src/morphology-grc.el",
"src/morphology-ang.el",
"src/morphology-sa.el",
"src/morphology-got.el",
"src/morphology-non.el",
"src/morphology-enm.el",
"src/morphology-pi.el",
"src/morphology-fro.el",
"src/morphology-goh.el",
"src/morphology-sga.el",
"src/morphology-txb.el",
"src/morphology-peo.el",
"src/morphology-akk.el",
"src/morphology-uga.el",
"src/morphology-egy.el",
"src/morphology-sux.el",
"src/morphology-gez.el",
"src/morphology-cop.el",
"src/grammar.el",
"src/realizer.el",
"src/semantics.el",
"src/comprehend.el",
"src/propositions.el",
"src/multilingual.el",
"src/self_region.el",
"src/dialogue.el",
"src/elp.el",
]
}
-91
View File
@@ -1,91 +0,0 @@
> **STATUS: STAGING / PROOF-OF-SHAPE — not the deliverable.** This Python package
> proved the architecture end-to-end against the proven realizer faculty (faithful
> md/docx/midi from real geometry: 0 ungrounded claims, SACRED polarity). Per Will's
> steer, the DELIVERABLE is NATIVE: the seam lives on the existing EL realizer as
> **surface-as-profile** — see `../src/surface-profile.el` and
> `../tests/examples/surface-profile-demo.el` (compiles + runs through elc → C →
> binary). The concepts below (one geometry-carrying frame; surface = a pluggable
> profile; plan/realize; deterministic-from-meaning) are exactly what the native
> module implements. Keep this package as the validated proof; build native.
# Efferent Multimodal Projector
**geometry → any surface, faithfully.** Neuron's own document-generation faculty:
the efferent twin of the ingest organ. Ingest is afferent (world → geometry);
this is efferent (geometry → an arbitrary-format document / any modality).
Built against the **proven** realizer faculty (neuron-talk sidecar `:8756`,
artifact `art-7affa557`). The live soul (`:8742` / `:7770`) is contacted **only**
through the read-only, GET-only `engram_client` — never mutated.
## The pipeline (surface-agnostic)
```
geometry region + surface/format spec
→ PLAN (manifold → document skeleton/DAG; the geometry IS the outline) plan.py
→ REALIZE (proven realizer, scaled sentence → passage, each section faithful) realize.py
→ COHERE (document-level flow / transitions, not stitched sentences) cohere.py
→ EMIT (pluggable SurfaceProjector → the target surface) projectors/
```
**The surface is a PARAMETER.** `pipeline.build_ir(...)` builds ONE
surface-neutral `DocumentIR` (`document_ir.py`); `pipeline.emit(doc, surface)`
projects it to whichever surface you name. Markdown, docx, and MIDI are the same
IR emitted three ways.
## The pivot: a geometry-carrying IR
`DocumentIR` is **not** a text tree. Every `Block` carries BOTH:
- `.sentences` — realized faithful text (what **text** projectors read),
- `.provenance` — the source geometry: `subj_id / relation / obj / polarity /
confidence / importance / salience / node_id` (what **music / image / video**
projectors read).
That single decision is what makes the projector multimodal: text renders the
words; music/image decode the geometry. A claim with no provenance cannot exist
in the IR — faithfulness is structural.
## The one shared seam
`projectors/base.py` — `SurfaceProjector.project(frame: DocumentIR) -> bytes`
(+ `surface / media_type / ext / modality / profile`). Register with
`register()`. Adding a surface changes nothing upstream.
`TwoStageProjector` blesses the peer plan/realize decomposition:
`spec = plan(frame)`, `bytes = realize(spec)`, `project = realize∘plan`; the
`profile` is the pluggable per-surface knob (text lang-profile, music
instr/mode-profile). `projectors/midi.py` is the reference two-stage impl.
## Surfaces
| surface | modality | status | emitter |
|---|---|---|---|
| `markdown` | text | landed | own (str) |
| `docx` | text | landed | own minimal OOXML (stdlib `zipfile`+XML, no lib) |
| `midi` | audio | landed (symbolic-music proof) | own minimal SMF (stdlib `struct`, no lib) |
| `audio` (WAV) | audio | peer agent (additive synth) | conforms to `TwoStageProjector` |
| `image` | image | documented seam | `projectors/seams.py` |
| `video` | video | documented seam (image×sound×time) | `projectors/seams.py` |
Music maps: relation → scale degree (same relation → same pitch), **polarity →
major/minor third (SACRED negation is audible)**, confidence → duration,
importance → velocity, section → register. Deterministic projection from meaning
— nothing invented.
## Faithfulness
`provenance.py` audits the IR: **zero** ungrounded claims, SACRED polarity
preserved (negations reported, never dropped), COHERE introduces no new geometry
(connectives are marked). `trace_table()` emits the geometry → section → claim
table.
## Run
```bash
PY=~/Desktop/lang-realizers/venv/bin/python
PYTHONPATH=~/Desktop/neuron-talk:~/Desktop/lang-realizers $PY generate.py
# writes ./out/{neuron-self,engram-temporal}.{md,docx,mid} + *.audit.json + *.provenance.md
```
Requires the proven realizer env (spaCy + the neuron-talk/lang-realizers engine)
and the read-only engram at `:8742`.
-79
View File
@@ -1,79 +0,0 @@
"""cohere.py — COHERE stage: document-level flow, not stitched sentences.
Fidelity is REALIZE's job; FLOW is this stage's. The hard part beyond sentence
fidelity is that a document must read as one thing. We add connective tissue at
the passage level:
* an opening abstract that names what the document covers (built ONLY from the
section headings that already exist — it introduces no new claim),
* a short transition lead into each section after the first, drawn from a
fixed set of discourse connectives ("Beyond that,", "Relatedly,", ...) that
carry no propositional content,
* ordering so the highest-grounded section leads.
CRITICAL: every connective is marked ``kind="connective"`` in its provenance, so
the faithfulness audit can prove COHERE introduced ZERO new geometry claims. A
transition is discourse glue, never a fact.
"""
from __future__ import annotations
from document_ir import Block, DocumentIR, Provenance
# discourse connectives — pure flow, no propositional content
_TRANSITIONS = [
"Beyond that,", "Relatedly,", "In the same region,", "From there,",
"Alongside this,", "Further,", "Turning to the next facet,",
]
def _connective_prov() -> Provenance:
return Provenance(subj_id=None, subject=None, relation="", obj=None,
polarity="aff", confidence=1.0, node_id=None,
kind="connective")
def _abstract_block(doc: DocumentIR) -> Block:
"""A grounded opening: names the sections, asserts nothing new."""
headings = [s.heading for s in doc.sections]
if not headings:
return Block(role="lead")
if len(headings) == 1:
body = f"This document, generated from Neuron's geometry, covers {headings[0]}."
else:
listed = ", ".join(headings[:-1]) + f", and {headings[-1]}"
body = ("This document is projected directly from Neuron's meaning-geometry. "
f"It traces {listed}.")
b = Block(role="lead")
b.sentences.append(body)
b.provenance.append(_connective_prov())
return b
def cohere_document(doc: DocumentIR, *, add_abstract: bool = True,
add_transitions: bool = True) -> DocumentIR:
"""Order sections by grounding, add abstract + transitions (flow only)."""
# order: strongest-grounded section (mean confidence x #claims) first,
# but keep an explicitly-first section if the plan pinned one via level 1.
def _score(sec):
provs = [p for p in sec.all_provenance() if p.kind == "fact"]
if not provs:
return 0.0
mean_conf = sum(p.confidence for p in provs) / len(provs)
return mean_conf * len(provs)
doc.sections.sort(key=_score, reverse=True)
if add_transitions:
for i, sec in enumerate(doc.sections):
if i == 0 or not sec.blocks:
continue
lead = _TRANSITIONS[(i - 1) % len(_TRANSITIONS)]
first = sec.blocks[0]
if first.sentences:
# prepend the connective to the first sentence (flow, no new claim)
first.sentences[0] = f"{lead} {first.sentences[0][0].lower()}{first.sentences[0][1:]}"
if add_abstract:
doc.meta["abstract"] = _abstract_block(doc)
return doc
-111
View File
@@ -1,111 +0,0 @@
"""document_ir.py — the surface-neutral, GEOMETRY-CARRYING document intermediate.
This is the pivot of the whole efferent projector. A DocumentIR is NOT a text
tree. It is a projection of a meaning-geometry region that carries, at every
leaf, BOTH:
* the realized surface text (``Block.sentences``) — what a TEXT projector reads,
* the source geometry (``Block.provenance``) — what a MUSIC / IMAGE /
VIDEO projector reads.
Because the IR holds the geometry, not just the words, the SAME
plan -> realize -> cohere pipeline drives every surface. A markdown projector
renders the sentences; a music projector reads the provenance edges (salience,
importance, polarity, relation) and maps them onto a symbolic-music surface;
an image/video projector (documented seam) would read the same geometry.
Nothing in this module invents content. Every :class:`Provenance` points at a
real engram node id and a real relation. That is the faithfulness contract made
structural: a claim with no provenance cannot exist in the IR.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any
# --------------------------------------------------------------------------- #
# Provenance — the geometry an emitted claim traces to. FAITHFULNESS is here.
# --------------------------------------------------------------------------- #
@dataclass
class Provenance:
"""One geometry edge behind one realized claim.
``kind`` distinguishes a FACT (a structural edge asserted by the geometry,
spoken as fact) from an INTERPRETATION (something attributed, spoken with
attribution) — the facts-as-facts + interpretations-attributed discipline
(memory 80927e26). ``polarity`` is SACRED: a negated edge stays negated.
"""
subj_id: str | None # source engram node id of the subject
subject: str | None # normalized subject surface
relation: str # predicate lemma (e.g. "use", "contain", "be")
obj: str | None # normalized object / complement surface
polarity: str = "aff" # "aff" | "neg" (SACRED — never silently flipped)
confidence: float = 0.0 # extraction confidence in [0,1]
node_id: str | None = None # engram node the claim was extracted from
kind: str = "fact" # "fact" | "interpretation"
importance: float = 0.0 # source node importance (drives music/emphasis)
salience: float = 0.0 # source node salience
def trace(self) -> str:
arrow = "-->" if self.polarity == "aff" else "--NOT-->"
return (f"[{(self.node_id or '?')[:8]}] {self.subject!r} {arrow}"
f"{self.relation} {self.obj!r} (conf {self.confidence:.2f})")
@dataclass
class Block:
"""A passage: one or more faithful sentences + the geometry they trace to.
``sentences`` and ``provenance`` are index-aligned where possible: sentence
``i`` was realized from ``provenance[i]``. A COHERE transition sentence with
no new geometry carries a provenance whose ``kind == "connective"`` so the
audit can see it introduced no new claim.
"""
sentences: list[str] = field(default_factory=list)
provenance: list[Provenance] = field(default_factory=list)
role: str = "body" # "body" | "lead" | "transition"
def text(self) -> str:
return " ".join(s.rstrip(". ") + "." for s in self.sentences if s.strip())
@dataclass
class Section:
heading: str
level: int = 2 # markdown heading level / outline depth
blocks: list[Block] = field(default_factory=list)
seed_ids: list[str] = field(default_factory=list) # geometry nodes of section
summary: str = "" # one-line grounded gloss (for pptx bullets / TOC)
def all_provenance(self) -> list[Provenance]:
out: list[Provenance] = []
for b in self.blocks:
out.extend(b.provenance)
return out
@dataclass
class DocumentIR:
"""The surface-neutral document. Built ONCE, projected to ANY surface."""
title: str
subtitle: str = ""
sections: list[Section] = field(default_factory=list)
seed_id: str | None = None # the geometry region root
format_spec: dict[str, Any] = field(default_factory=dict) # requested shape
meta: dict[str, Any] = field(default_factory=dict)
# -- geometry facets (what non-text projectors consume) ----------------- #
def all_provenance(self) -> list[Provenance]:
out: list[Provenance] = []
for s in self.sections:
out.extend(s.all_provenance())
return out
def claim_count(self) -> int:
return sum(1 for p in self.all_provenance() if p.kind in ("fact", "interpretation"))
def ungrounded_count(self) -> int:
"""Claims with no traceable node — MUST be zero for a faithful doc."""
return sum(1 for p in self.all_provenance()
if p.kind in ("fact", "interpretation") and not p.node_id)
-81
View File
@@ -1,81 +0,0 @@
"""generate.py — drive the projector: one geometry region -> many surfaces.
Proves the thesis with REAL output: builds ONE surface-neutral DocumentIR from
Neuron's OWN self-geometry (read-only against the live soul via the proven
faculty), then EMITS it to Markdown, docx, and MIDI — the same plan/realize/
cohere, three surfaces. Writes the files + the faithfulness audit to ./out/.
"""
from __future__ import annotations
import json
import os
import sys
_HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, _HERE)
import pipeline # noqa: E402
import provenance # noqa: E402
from geometry import load_self_region # noqa: E402
OUT = os.path.join(_HERE, "out")
def _emit_all(doc, stem):
"""Emit one IR to every text/audio surface + audit + provenance."""
for surface in ("markdown", "docx", "midi"):
data = pipeline.emit(doc, surface)
proj = pipeline.get_projector(surface)
path = os.path.join(OUT, f"{stem}.{proj.ext}")
with open(path, "wb") as f:
f.write(data)
print(f" emitted {surface:9s} -> {os.path.basename(path)} ({len(data)} bytes)")
a = provenance.audit(doc)
with open(os.path.join(OUT, f"{stem}.audit.json"), "w") as f:
json.dump(a, f, indent=2)
with open(os.path.join(OUT, f"{stem}.provenance.md"), "w") as f:
f.write(provenance.trace_table(doc))
print(" audit:", {k: a[k] for k in ("claims", "ungrounded_claims",
"negations_preserved", "distinct_source_nodes", "faithful")})
return a
def main():
os.makedirs(OUT, exist_ok=True)
print("surfaces registered:", pipeline.available_surfaces())
# ---- Document 1: Neuron's self-description (marquee) ------------------- #
print("\n[1] Neuron self-description")
region = load_self_region(max_nodes=9)
print(" self region:", region)
doc1 = pipeline.build_ir(
None, region=region,
title="Neuron: A Self-Description from Its Own Geometry",
subtitle="Projected efferently from the engram — every claim traces a node.",
format_spec={"genre": "self-description", "register": "expository"},
max_sections=5, conf_floor=0.6)
print(f" IR: {len(doc1.sections)} sections, {doc1.claim_count()} claims, "
f"ungrounded={doc1.ungrounded_count()}")
_emit_all(doc1, "neuron-self")
# ---- Document 2: a coherent, clean whitepaper-style section ------------ #
print("\n[2] Whitepaper-style section (coherent clean region)")
doc2, _ = pipeline.project(
["chronoception", "time", "awareness", "engram", "temporal"],
surface="markdown",
title="Temporal Awareness in the Engram",
subtitle="A section projected from the geometry of chronoception.",
format_spec={"genre": "whitepaper-section", "register": "technical"},
max_sections=4)
print(f" IR: {len(doc2.sections)} sections, {doc2.claim_count()} claims, "
f"ungrounded={doc2.ungrounded_count()}")
_emit_all(doc2, "engram-temporal")
# echo both markdowns so they are visible in the run log
for stem, doc in (("neuron-self", doc1), ("engram-temporal", doc2)):
print(f"\n===== GENERATED MARKDOWN — {stem} =====\n")
print(pipeline.emit(doc, "markdown").decode())
if __name__ == "__main__":
main()
-129
View File
@@ -1,129 +0,0 @@
"""geometry.py — READ-ONLY loader for a meaning-geometry region.
The efferent projector never writes to the soul. This module reaches the
geometry through the PROVEN, read-only neuron-talk faculty (``engram_client``,
GET-only, which physically refuses non-GET methods) against the running sidecar
soul. The live daemon :8742 / :7770 is contacted ONLY through that read-only
client — never mutated.
A "region" is a seed node plus a bounded neighborhood: the manifold that will
become the document's skeleton. We pool a few single-term lexical searches
(the engram search is a single-term matcher) and, when available, walk one hop
of reified neighbors, then rank by self/importance signal.
"""
from __future__ import annotations
import os
import sys
# Wire in the proven faculty (own-the-core: we reuse it, we do not fork it).
_NT = os.path.expanduser("~/Desktop/neuron-talk")
_LR = os.path.expanduser("~/Desktop/lang-realizers")
for _p in (_NT, _LR):
if _p not in sys.path:
sys.path.insert(0, _p)
from engram_client import ReadOnlyEngramClient # noqa: E402
class Region:
"""A geometry region: ranked nodes + the reified edges among them."""
def __init__(self, seed: str, nodes: list[dict], edges: list[dict]):
self.seed = seed
self.nodes = nodes # ranked engram node dicts
self.edges = edges # [{src, dst, edge, ...}]
self.by_id = {n["id"]: n for n in nodes if n.get("id")}
def __repr__(self):
return f"<Region seed={self.seed!r} nodes={len(self.nodes)} edges={len(self.edges)}>"
def _prose_quality(content: str) -> float:
"""Reward clean expository prose; penalize shouty banner-dense nodes.
A high ALLCAPS-word ratio or very short content signals a banner/telegraphic
memory node that extracts into garbage. Clean declarative prose scores high.
"""
if not content or not content.strip():
return 0.0
words = content.split()
if len(words) < 8:
return 0.1
caps = sum(1 for w in words if len(w) > 2 and w.strip(".,:;'\"-").isupper())
caps_ratio = caps / max(1, len(words))
# sentences with lowercase interior words read as prose
lower = sum(1 for w in words if w[:1].islower())
lower_ratio = lower / max(1, len(words))
return max(0.0, 1.2 * lower_ratio - 2.0 * caps_ratio)
def _relevance(content: str, terms: list[str]) -> float:
"""Topical relevance to the seed terms — keeps a region ON-THEME so a clean
but off-topic node cannot hijack the document."""
if not terms:
return 0.0
low = (content or "").lower()
hits = sum(1 for t in terms if t.lower() in low)
return hits / max(1, len(terms))
def _node_rank(n: dict, terms: list[str] | None = None) -> float:
return (float(n.get("importance") or 0.0) * 2.0
+ float(n.get("salience") or 0.0)
+ 1.5 * _prose_quality(n.get("content") or "")
+ 2.0 * _relevance(n.get("content") or "", terms or [])
+ (0.5 if (n.get("content") or "").strip() else 0.0))
def load_region(seed_terms: list[str] | str, *, client: ReadOnlyEngramClient | None = None,
max_nodes: int = 10, per_term: int = 20, hop: bool = True) -> Region:
"""Pull a bounded geometry region around ``seed_terms`` (read-only).
``seed_terms`` may be a single string or several probe terms; results are
pooled and de-duplicated. When ``hop`` and the reified neighbor endpoint is
live, one hop of neighbors is folded in so the region is a real
neighborhood, not just a keyword hit list.
"""
client = client or ReadOnlyEngramClient()
if isinstance(seed_terms, str):
seed_terms = [seed_terms]
pool: dict[str, dict] = {}
for term in seed_terms:
for n in client.search(term, limit=per_term):
if isinstance(n, dict) and n.get("id"):
pool.setdefault(n["id"], n)
ranked = sorted(pool.values(), key=lambda n: _node_rank(n, seed_terms),
reverse=True)
nodes = ranked[:max_nodes]
edges: list[dict] = []
if hop and nodes:
present = {n["id"] for n in nodes}
for n in list(nodes):
try:
for nb in client.neighbors(n["id"]):
node = nb.get("node") if isinstance(nb, dict) else None
edge = nb.get("edge") if isinstance(nb, dict) else None
if node and node.get("id"):
edges.append({"src": n["id"], "dst": node["id"],
"edge": edge})
# fold a strong neighbor into the region (bounded)
if (node["id"] not in present and len(nodes) < max_nodes + 6
and _node_rank(node, seed_terms) > 0.4):
present.add(node["id"])
nodes.append(node)
except Exception: # noqa: BLE001 — read-only best-effort; never fatal
continue
return Region(seed=", ".join(seed_terms), nodes=nodes, edges=edges)
def load_self_region(client: ReadOnlyEngramClient | None = None,
max_nodes: int = 10) -> Region:
"""The self/identity region — Neuron's own geometry, for self-description."""
return load_region(["self", "identity", "Neuron", "values", "memory",
"imprint", "consciousness"],
client=client, max_nodes=max_nodes)
-67
View File
@@ -1,67 +0,0 @@
"""pipeline.py — the Efferent Multimodal Projector, top level.
geometry region + surface/format spec
-> PLAN (manifold -> document skeleton/DAG)
-> REALIZE (proven realizer, sentence -> passage, each section faithful)
-> COHERE (document-level flow / transitions, not stitched sentences)
-> EMIT (pluggable SurfaceProjector -> the target surface)
THE SURFACE IS A PARAMETER. ``project(...)`` builds the geometry-carrying
DocumentIR once, then hands it to whichever surface projector the caller named.
Markdown, docx, and midi (music) are all the SAME IR emitted differently. That
is the efferent multimodal projector: geometry -> any surface.
"""
from __future__ import annotations
import os
import sys
_HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, _HERE)
sys.path.insert(0, os.path.join(_HERE, "projectors"))
from cohere import cohere_document # noqa: E402
from document_ir import DocumentIR # noqa: E402
from geometry import Region, load_region # noqa: E402
from plan import plan_document # noqa: E402
from realize import realize_document # noqa: E402
# registering the projectors (import for side-effect: each self-registers)
import projectors.markdown # noqa: E402,F401
import projectors.docx # noqa: E402,F401
import projectors.midi # noqa: E402,F401
import projectors.seams # noqa: E402,F401
from projectors.base import available_surfaces, get_projector # noqa: E402
def build_ir(seed_terms, *, title: str, subtitle: str = "",
format_spec: dict | None = None,
region: Region | None = None,
max_sections: int = 8, conf_floor: float = 0.55) -> DocumentIR:
"""geometry -> PLAN -> REALIZE -> COHERE = the surface-neutral DocumentIR."""
region = region or load_region(seed_terms)
doc = plan_document(region, title=title, subtitle=subtitle,
format_spec=format_spec or {},
conf_floor=conf_floor, max_sections=max_sections)
doc = realize_document(doc)
doc = cohere_document(doc)
return doc
def emit(doc: DocumentIR, surface: str) -> bytes:
"""EMIT: project the built IR onto one surface (surface = a parameter)."""
return get_projector(surface).project(doc)
def project(seed_terms, *, surface: str, title: str, subtitle: str = "",
format_spec: dict | None = None, region: Region | None = None,
max_sections: int = 8) -> tuple[DocumentIR, bytes]:
"""The full efferent projection: geometry + surface -> (IR, bytes)."""
doc = build_ir(seed_terms, title=title, subtitle=subtitle,
format_spec=format_spec, region=region,
max_sections=max_sections)
return doc, emit(doc, surface)
__all__ = ["build_ir", "emit", "project", "available_surfaces",
"get_projector", "load_region", "DocumentIR"]
-192
View File
@@ -1,192 +0,0 @@
"""plan.py — PLAN stage: geometry region -> document skeleton (a DAG/outline).
The manifold becomes the skeleton. We extract faithful propositions from the
region's nodes (the proven neuron-talk extractor, SACRED polarity preserved),
apply a quality floor, then GROUP them into sections. Grouping is by source
node — each engram node is one coherent topic, so one salient node becomes one
section. The section ORDER is the node ranking (importance/salience): the
geometry decides the outline, not a template.
Output: a DocumentIR whose sections carry seed node ids and empty blocks. REALIZE
fills the blocks; the plan owns the structure.
"""
from __future__ import annotations
import os
import re
import sys
_NT = os.path.expanduser("~/Desktop/neuron-talk")
_LR = os.path.expanduser("~/Desktop/lang-realizers")
for _p in (_NT, _LR):
if _p not in sys.path:
sys.path.insert(0, _p)
import propositions # noqa: E402 (the proven, faithful extractor)
from document_ir import DocumentIR, Section # noqa: E402
from geometry import Region # noqa: E402
# --------------------------------------------------------------------------- #
# Proposition quality — keep only clean, well-grounded claims.
# --------------------------------------------------------------------------- #
_JUNK_RE = re.compile(r"[.][a-z]{1,3}\b|[^A-Za-z0-9 '\-]") # ".o", stray symbols
def _has_banner_token(s: str) -> bool:
"""True if any word is an ALLCAPS banner token (DHARMA, ENGRAM, MEASURED)."""
for w in (s or "").split():
core = w.strip(".,:;'\"-")
if len(core) > 2 and core.isupper():
return True
return False
def _clean_prop(p, floor: float) -> bool:
if p.confidence < floor:
return False
if not p.subject or not (p.object or (p.obj_np is not None)):
return False
subj = (p.subject or "").strip()
obj = (p.object or "").strip()
if len(subj) < 2:
return False
# banner-derived shouty fragments read as garbage in prose
if _has_banner_token(subj) or _has_banner_token(obj):
return False
if propositions._is_shouty(p.sentence or ""):
return False
# junk tokens: file-extension fragments (".o"), stray non-word symbols
if _JUNK_RE.search(subj) or _JUNK_RE.search(obj):
return False
# a proposition whose object repeats the subject is usually a parse artifact
if obj and subj.lower() == obj.lower():
return False
# a bare copula with no real complement ("X is it") reads as noise
if p.predicate == "be" and obj.lower() in ("it", "no", "nothing", "empty", ""):
return False
return True
def _dedup(props):
"""Drop duplicate claims. Two axes: (a) identical (pred,obj,polarity), and
(b) same (subject,predicate) — which collapses a mis-split compound like
"detection is post-hoc eval" -> "Detection is post/hoc/eval" into one claim
(keep the highest-confidence surface)."""
props = sorted(props, key=lambda p: p.confidence, reverse=True)
seen_po, seen_sp, out = set(), set(), []
for p in props:
subj = (p.subject or "").lower()
po = (p.predicate, (p.object or "").lower(), p.polarity)
sp = (subj, p.predicate, p.polarity)
if po in seen_po or sp in seen_sp:
continue
seen_po.add(po)
seen_sp.add(sp)
out.append(p)
return out
# --------------------------------------------------------------------------- #
# Heading derivation — a clean human heading from a node.
# --------------------------------------------------------------------------- #
_HEADING_RE = re.compile(r"^\s*#{1,4}\s+(.{2,70})\s*$", re.M)
# node-type / system labels that are NOT topical headings
_NONTOPIC_LABEL = re.compile(r"^(memory|node|knowledge|doc|session)[:/]", re.I)
def _titlecase_banner(s: str) -> str:
"""A shouty banner ("CHRONOCEPTION — SCALE-INVARIANCE") makes a fine title
once Title-cased. Keep short acronyms uppercase."""
def fix(w):
core = w.strip("—-:,.")
if len(core) <= 3 and core.isupper():
return w # acronym
return w.capitalize()
return " ".join(fix(w) for w in s.split())
def _clean_heading(text: str) -> str | None:
"""First line only, no markdown, capped, banner Title-cased. None if unusable."""
if not text:
return None
line = text.strip().splitlines()[0]
line = re.sub(r"^#+\s*", "", line).strip().strip("#").strip()
# cut at a natural break so a long banner heading stays a heading, not a para
for sep in ("", " ", ": ", ". "):
if sep in line and len(line) > 48:
line = line.split(sep)[0].strip()
break
if not (3 <= len(line) <= 64):
return None
if propositions._is_shouty(line):
line = _titlecase_banner(line)
return line or None
def _heading_for(node: dict, fallback: str) -> str:
label = (node.get("label") or "").strip()
content = node.get("content") or ""
candidates: list[str] = []
# a node-type label ("memory:remembered") is never a topic — skip it
if label and not _NONTOPIC_LABEL.match(label):
candidates.append(label)
m = _HEADING_RE.search(content)
if m:
candidates.append(m.group(1))
# the leading banner/first sentence of the content is often the real title
first = re.split(r"(?<=[.\n])", content.strip(), maxsplit=1)[0] if content.strip() else ""
candidates.append(first)
for c in candidates:
h = _clean_heading(c)
if h:
return h
return fallback
def plan_document(region: Region, *, title: str, subtitle: str = "",
format_spec: dict | None = None,
conf_floor: float = 0.55,
max_sections: int = 8,
max_claims_per_section: int = 6) -> DocumentIR:
"""Region -> DocumentIR skeleton. The geometry dictates the outline."""
format_spec = format_spec or {}
doc = DocumentIR(title=title, subtitle=subtitle,
seed_id=region.nodes[0]["id"] if region.nodes else None,
format_spec=format_spec)
made = 0
seen_headings: set[str] = set()
for node in region.nodes:
if made >= max_sections:
break
props = propositions.extract(node.get("content") or "",
node_id=node.get("id"),
node_importance=float(node.get("importance") or 0.0),
max_sentences=10)
props = [p for p in props if _clean_prop(p, conf_floor)]
props = _dedup(props)
props.sort(key=lambda p: p.confidence, reverse=True)
props = props[:max_claims_per_section]
if not props:
continue
heading = _heading_for(node, fallback=f"Region {made + 1}")
# cross-section dedup: a topic appears once. Distinguish by top claim
# subject, else drop the collision so the outline stays clean.
if heading.lower() in seen_headings:
subj = (props[0].subject or "").strip().title()
alt = f"{heading}: {subj}" if subj and subj.lower() not in heading.lower() else None
if alt and alt.lower() not in seen_headings and len(alt) <= 64:
heading = alt
else:
continue
seen_headings.add(heading.lower())
sec = Section(heading=heading, level=2, seed_ids=[node["id"]])
# stash the planned propositions on the section for REALIZE
sec.__dict__["_planned_props"] = props
sec.__dict__["_node"] = node
doc.sections.append(sec)
made += 1
return doc
-106
View File
@@ -1,106 +0,0 @@
"""base.py — the SurfaceProjector interface + registry.
THE key abstraction of the efferent projector: a projector is a pure function
from the surface-neutral, geometry-carrying DocumentIR to bytes on a target
SURFACE. The surface is a PARAMETER. Adding a surface = registering one more
projector; nothing upstream (plan/realize/cohere) changes.
DocumentIR --project--> bytes (per surface)
A TEXT projector reads ``block.sentences``. A NON-TEXT projector (music, image,
video) reads ``block.provenance`` — the geometry the IR carries — and decodes it
onto its surface. Both consume the SAME IR. That symmetry is the whole design:
the realizer generalizes into a multimodal projector, geometry -> any surface.
"""
from __future__ import annotations
from typing import Protocol, runtime_checkable
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from document_ir import DocumentIR # noqa: E402
@runtime_checkable
class SurfaceProjector(Protocol):
"""Geometry-document -> one surface. Implementations MUST be pure & faithful.
THE ONE SHARED SEAM. Every surface — text, music, image, video — conforms to
this single contract:
project(frame: DocumentIR) -> bytes
where ``frame`` is the geometry-carrying meaning-geometry (the SemFrame at
document scale; a single utterance is the degenerate one-section frame).
RECOMMENDED INTERNAL SHAPE (the peer music/text decomposition, blessed here
so all surfaces share it): a projector may split ``project`` into
spec = self.plan(frame) # meaning-geometry -> surface-specific spec
bytes = self.realize(spec) # spec -> surface, via this projector's PROFILE
``project`` is then ``realize(plan(frame))``. The PROFILE (a text lang-profile,
a music instr/mode-profile, an image layout-profile) is a property of the
projector instance — the pluggable knob. See :class:`TwoStageProjector`.
A TEXT projector's plan reads ``frame`` sentences; a MUSIC/IMAGE projector's
plan reads ``frame.all_provenance()`` — the geometry — and derives its spec
(pitch/harmony/rhythm, or layout) FROM the meaning, deterministically. Same
frame, different profile.
"""
surface: str # "markdown" | "docx" | "midi" | "audio" | "image" | "video"
media_type: str # MIME type of the emitted bytes
ext: str # file extension (no dot)
modality: str # "text" | "audio" | "image" | "video"
profile: object # the pluggable per-surface profile (may be None)
def project(self, doc: DocumentIR) -> bytes:
"""Emit the document on this surface. Returns raw bytes."""
...
class TwoStageProjector:
"""Optional base for the peer plan()/realize() decomposition.
Subclasses implement ``plan(frame) -> spec`` and ``realize(spec) -> bytes``;
``project`` is their composition. This is exactly the peer music interface
(spec = plan(frame, profile); surface = realize(spec, profile)) expressed so
that it still satisfies the single ``SurfaceProjector.project`` seam. Text,
music, and image projectors can all subclass this and remain interchangeable.
"""
surface: str = ""
media_type: str = ""
ext: str = ""
modality: str = ""
profile: object = None
def plan(self, doc: DocumentIR): # -> spec
raise NotImplementedError
def realize(self, spec) -> bytes:
raise NotImplementedError
def project(self, doc: DocumentIR) -> bytes:
return self.realize(self.plan(doc))
_REGISTRY: dict[str, SurfaceProjector] = {}
def register(projector: SurfaceProjector) -> SurfaceProjector:
_REGISTRY[projector.surface] = projector
return projector
def get_projector(surface: str) -> SurfaceProjector:
if surface not in _REGISTRY:
raise KeyError(f"no projector registered for surface {surface!r}; "
f"have {sorted(_REGISTRY)}")
return _REGISTRY[surface]
def available_surfaces() -> list[str]:
return sorted(_REGISTRY)
-113
View File
@@ -1,113 +0,0 @@
"""docx.py — the .docx surface projector: an OWN minimal OOXML emitter.
Own-the-core: a .docx is just a ZIP of a few XML parts (WordprocessingML). We
emit it with the standard library only — ``zipfile`` + string XML — no
python-docx, no external dependency. This proves a "richer structured format"
surface without importing anyone else's toolkit.
Parts emitted (the minimal valid set + a styles part for real headings):
[Content_Types].xml
_rels/.rels
word/_rels/document.xml.rels
word/styles.xml (Title / Heading1 / Heading2 / Normal)
word/document.xml (the content)
Like the markdown projector it reads only the IR's realized sentences; it
invents nothing. The surface differs, the faithful content does not.
"""
from __future__ import annotations
import io
import os
import sys
import zipfile
from xml.sax.saxutils import escape
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from document_ir import DocumentIR # noqa: E402
from projectors.base import register # noqa: E402
_CONTENT_TYPES = """<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types">
<Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/>
<Default Extension="xml" ContentType="application/xml"/>
<Override PartName="/word/document.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml"/>
<Override PartName="/word/styles.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.styles+xml"/>
</Types>"""
_RELS = """<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">
<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument" Target="word/document.xml"/>
</Relationships>"""
_DOC_RELS = """<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">
<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/styles" Target="styles.xml"/>
</Relationships>"""
_W = "http://schemas.openxmlformats.org/wordprocessingml/2006/main"
_STYLES = f"""<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<w:styles xmlns:w="{_W}">
<w:style w:type="paragraph" w:default="1" w:styleId="Normal"><w:name w:val="Normal"/>
<w:rPr><w:sz w:val="22"/></w:rPr></w:style>
<w:style w:type="paragraph" w:styleId="Title"><w:name w:val="Title"/>
<w:pPr><w:spacing w:after="240"/></w:pPr>
<w:rPr><w:b/><w:sz w:val="52"/></w:rPr></w:style>
<w:style w:type="paragraph" w:styleId="Subtitle"><w:name w:val="Subtitle"/>
<w:rPr><w:i/><w:sz w:val="28"/><w:color w:val="555555"/></w:rPr></w:style>
<w:style w:type="paragraph" w:styleId="Heading1"><w:name w:val="heading 1"/>
<w:pPr><w:spacing w:before="240" w:after="120"/><w:outlineLvl w:val="0"/></w:pPr>
<w:rPr><w:b/><w:sz w:val="34"/></w:rPr></w:style>
<w:style w:type="paragraph" w:styleId="Heading2"><w:name w:val="heading 2"/>
<w:pPr><w:spacing w:before="200" w:after="100"/><w:outlineLvl w:val="1"/></w:pPr>
<w:rPr><w:b/><w:sz w:val="28"/></w:rPr></w:style>
</w:styles>"""
def _para(text: str, style: str | None = None) -> str:
ppr = f"<w:pPr><w:pStyle w:val=\"{style}\"/></w:pPr>" if style else ""
return (f"<w:p>{ppr}<w:r><w:t xml:space=\"preserve\">"
f"{escape(text)}</w:t></w:r></w:p>")
class DocxProjector:
surface = "docx"
media_type = ("application/vnd.openxmlformats-officedocument."
"wordprocessingml.document")
ext = "docx"
modality = "text"
def _document_xml(self, doc: DocumentIR) -> str:
body: list[str] = [_para(doc.title, "Title")]
if doc.subtitle:
body.append(_para(doc.subtitle, "Subtitle"))
abstract = doc.meta.get("abstract")
if abstract is not None and abstract.sentences:
body.append(_para(abstract.text()))
for sec in doc.sections:
style = "Heading1" if sec.level <= 1 else "Heading2"
body.append(_para(sec.heading, style))
for block in sec.blocks:
t = block.text()
if t:
body.append(_para(t))
return (f"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>"
f"<w:document xmlns:w=\"{_W}\"><w:body>"
+ "".join(body)
+ "<w:sectPr><w:pgSz w:w=\"12240\" w:h=\"15840\"/>"
"<w:pgMar w:top=\"1440\" w:right=\"1440\" w:bottom=\"1440\" "
"w:left=\"1440\"/></w:sectPr></w:body></w:document>")
def project(self, doc: DocumentIR) -> bytes:
buf = io.BytesIO()
with zipfile.ZipFile(buf, "w", zipfile.ZIP_DEFLATED) as z:
z.writestr("[Content_Types].xml", _CONTENT_TYPES)
z.writestr("_rels/.rels", _RELS)
z.writestr("word/_rels/document.xml.rels", _DOC_RELS)
z.writestr("word/styles.xml", _STYLES)
z.writestr("word/document.xml", self._document_xml(doc))
return buf.getvalue()
register(DocxProjector())
-45
View File
@@ -1,45 +0,0 @@
"""markdown.py — the Markdown surface projector (text facet).
The most tractable surface, and the reference implementation: reads the IR's
realized sentences and lays them out as Markdown. Introduces no content — it is
pure typography over the faithful text the realizer produced.
"""
from __future__ import annotations
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from document_ir import DocumentIR # noqa: E402
from projectors.base import register # noqa: E402
class MarkdownProjector:
surface = "markdown"
media_type = "text/markdown"
ext = "md"
modality = "text"
def render_str(self, doc: DocumentIR) -> str:
lines: list[str] = [f"# {doc.title}"]
if doc.subtitle:
lines.append(f"\n*{doc.subtitle}*")
abstract = doc.meta.get("abstract")
if abstract is not None and abstract.sentences:
lines.append("")
lines.append(abstract.text())
for sec in doc.sections:
lines.append("")
lines.append(f"{'#' * max(2, sec.level)} {sec.heading}")
for block in sec.blocks:
body = block.text()
if body:
lines.append("")
lines.append(body)
return "\n".join(lines) + "\n"
def project(self, doc: DocumentIR) -> bytes:
return self.render_str(doc).encode("utf-8")
register(MarkdownProjector())
-133
View File
@@ -1,133 +0,0 @@
"""midi.py — the MUSIC surface projector: geometry -> symbolic music (MIDI).
The first NON-TEXT surface, and the proof of the general shape. "Music is
language and it is math" (Will): symbolic music is tractable and geometry-native,
so it is the natural efferent twin to try first after text.
CRUCIALLY this projector does NOT read the realized sentences. It reads the IR's
GEOMETRY facet — ``block.provenance`` — and DECODES each edge onto a musical
surface. That is the whole thesis of the multimodal projector: the same
geometry-carrying IR drives text AND music; a text projector reads the words, a
music projector reads the meaning-geometry. The mapping is deterministic and
faithful to the geometry's structure:
relation lemma -> scale degree (same relation -> same pitch class;
meaning has a consistent sonic form)
polarity -> mode (aff = major third above; neg = minor
third / lowered — SACRED polarity is
audible, a negated edge sounds negated)
confidence -> note duration (stronger grounding rings longer)
importance -> velocity (more important source = louder)
section -> phrase + register shift (structure becomes musical form)
Own-the-core: a Standard MIDI File is a header chunk + a track chunk of
delta-timed events. We emit the raw bytes with ``struct`` — no external MIDI
library. Format 0, one track.
"""
from __future__ import annotations
import io
import os
import struct
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from document_ir import DocumentIR, Provenance # noqa: E402
from projectors.base import TwoStageProjector, register # noqa: E402
_TICKS = 480 # ticks per quarter note
_C_MAJOR = [0, 2, 4, 5, 7, 9, 11] # semitone offsets of a diatonic scale
def _vlq(n: int) -> bytes:
"""MIDI variable-length quantity encoding of a delta time."""
if n == 0:
return b"\x00"
out = bytearray()
out.append(n & 0x7F)
n >>= 7
while n:
out.insert(0, (n & 0x7F) | 0x80)
n >>= 7
return bytes(out)
def _degree_for(relation: str) -> int:
"""Stable scale degree for a relation lemma (same relation -> same pitch)."""
if not relation:
return 0
return sum(ord(c) for c in relation.lower()) % len(_C_MAJOR)
def _note_for(p: Provenance, base: int) -> tuple[int, int, int]:
"""(pitch, velocity, duration_ticks) for one geometry edge."""
root = base + _C_MAJOR[_degree_for(p.relation)]
# polarity -> mode: affirmed edges take the bright major third, negated edges
# take the darker minor third. The negation is AUDIBLE and never dropped.
third = 4 if p.polarity == "aff" else 3
pitch = max(24, min(96, root + (third if p.confidence >= 0.5 else 0)))
velocity = int(56 + 60 * min(1.0, max(0.0, p.importance)))
velocity = max(40, min(120, velocity))
# confidence -> duration: quarter .. dotted-half
dur = int(_TICKS * (0.5 + 1.5 * min(1.0, max(0.0, p.confidence))))
return pitch, velocity, dur
# a mode-profile: the pluggable musical knob (the peer's mode_profile). Scale +
# tempo. Swapping this profile re-voices the SAME geometry — surface as parameter.
_DEFAULT_PROFILE = {"scale": _C_MAJOR, "tempo_us": 500000,
"registers": [60, 55, 64, 50, 67, 48], "program": 0}
class MidiProjector(TwoStageProjector):
"""geometry -> symbolic music, in the shared two-stage shape.
``plan(frame)`` -> a music_spec: an ordered list of note dicts derived
deterministically from the frame's provenance geometry
(the peer's ``plan(frame, profile) -> spec``).
``realize(spec)`` -> Standard MIDI File bytes (the peer's
``realize(spec, profile) -> surface``; here the surface
is symbolic MIDI, the minimal audio proof — a richer
additive-synth audio projector conforms identically).
"""
surface = "midi"
media_type = "audio/midi"
ext = "mid"
modality = "audio"
def __init__(self, profile: dict | None = None):
self.profile = profile or _DEFAULT_PROFILE
# -- stage 1: meaning-geometry -> music_spec (reads the GEOMETRY facet) -- #
def plan(self, doc: DocumentIR) -> list[dict]:
registers = self.profile["registers"]
spec: list[dict] = []
for si, sec in enumerate(doc.sections):
base = registers[si % len(registers)]
provs = [p for p in sec.all_provenance()
if p.kind in ("fact", "interpretation")]
for i, p in enumerate(provs):
pitch, vel, dur = _note_for(p, base)
spec.append({"pitch": pitch, "velocity": vel, "dur": dur,
"rest_before": (_TICKS // 2) if (si > 0 and i == 0) else 0,
"relation": p.relation, "polarity": p.polarity})
return spec
# -- stage 2: music_spec -> MIDI bytes (own-core, no library) ------------ #
def realize(self, spec: list[dict]) -> bytes:
ev = bytearray()
ev += _vlq(0) + b"\xFF\x51\x03" + struct.pack(">I", self.profile["tempo_us"])[1:]
ev += _vlq(0) + bytes([0xC0, self.profile["program"] & 0x7F])
for note in spec:
ev += _vlq(note["rest_before"]) + bytes([0x90, note["pitch"], note["velocity"]])
ev += _vlq(note["dur"]) + bytes([0x80, note["pitch"], 0])
ev += _vlq(0) + b"\xFF\x2F\x00"
track = bytes(ev)
buf = io.BytesIO()
buf.write(b"MThd" + struct.pack(">IHHH", 6, 0, 1, _TICKS))
buf.write(b"MTrk" + struct.pack(">I", len(track)) + track)
return buf.getvalue()
register(MidiProjector())
-60
View File
@@ -1,60 +0,0 @@
"""seams.py — documented efferent seams for IMAGE and VIDEO surfaces.
These are NOT implemented (per the build rails: architect, do not overbuild).
They are registered as first-class seams so the interface PROVES it accepts
future non-text projectors without any upstream change. Each documents exactly
what its decoder would read from the geometry-carrying IR, making the multimodal
generalization concrete rather than hand-wavy.
The symmetry that guarantees these are possible, not moonshots: they are the
efferent twins of multimodal INGEST. If meaning can HOLD an image (ingest as
first-class geometry), meaning can PROJECT one back. Video = image x sound x
TIME, and the engram already stores time (chronoception). So video falls out of
an image projector + the music projector + the stored temporal ordering.
"""
from __future__ import annotations
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from document_ir import DocumentIR # noqa: E402
from projectors.base import register # noqa: E402
class _Seam:
"""A registered-but-unimplemented projector. Names its decoder contract."""
def project(self, doc: DocumentIR) -> bytes: # pragma: no cover - seam
raise NotImplementedError(
f"{self.surface!r} projector is a documented seam, not yet built. "
f"Decoder contract: {self.decoder_contract}")
class ImageProjector(_Seam):
surface = "image"
media_type = "image/png"
ext = "png"
modality = "image"
decoder_contract = (
"reads block.provenance as a spatial layout — nodes become regions, edges "
"become adjacencies; salience/importance drive size/contrast; polarity "
"drives figure/ground. The efferent twin of image ingest (a geometry->raster "
"decoder, learned or engineered), exactly mirroring the embedder that turned "
"the image INTO geometry.")
class VideoProjector(_Seam):
surface = "video"
media_type = "video/mp4"
ext = "mp4"
modality = "video"
decoder_contract = (
"image x sound x TIME. Composes the image projector (per-keyframe geometry "
"layout) with the midi/music projector (score) along the geometry's stored "
"temporal ordering (chronoception). Needs no new principle once image + music "
"exist — only a muxer.")
register(ImageProjector())
register(VideoProjector())
-63
View File
@@ -1,63 +0,0 @@
"""provenance.py — the faithfulness audit + geometry->section trace.
A document projected from geometry is only worth anything if every claim traces
back. This module walks the DocumentIR and proves the discipline held:
* ZERO ungrounded claims (every fact/interpretation has a real node id),
* every emitted sentence maps to a geometry edge (or is a marked connective),
* SACRED polarity survived (negations are reported, never silently dropped),
* COHERE introduced no new geometry (connectives carry no claim).
It emits both a machine verdict and a human-readable geometry->section table.
"""
from __future__ import annotations
from document_ir import DocumentIR
def audit(doc: DocumentIR) -> dict:
provs = doc.all_provenance()
facts = [p for p in provs if p.kind in ("fact", "interpretation")]
connectives = [p for p in provs if p.kind == "connective"]
ungrounded = [p for p in facts if not p.node_id]
negations = [p for p in facts if p.polarity == "neg"]
node_ids = sorted({p.node_id for p in facts if p.node_id})
return {
"claims": len(facts),
"connectives": len(connectives),
"ungrounded_claims": len(ungrounded),
"negations_preserved": len(negations),
"distinct_source_nodes": len(node_ids),
"faithful": len(ungrounded) == 0,
"source_nodes": node_ids,
}
def trace_table(doc: DocumentIR) -> str:
"""Human-readable geometry -> section -> claim provenance table."""
lines = ["# Provenance — every claim traces geometry", ""]
lines.append(f"**Document:** {doc.title}")
a = audit(doc)
lines.append(f"**Claims:** {a['claims']} · **Ungrounded:** "
f"{a['ungrounded_claims']} · **Negations preserved:** "
f"{a['negations_preserved']} · **Source nodes:** "
f"{a['distinct_source_nodes']} · **Faithful:** "
f"{'YES' if a['faithful'] else 'NO'}")
lines.append("")
for si, sec in enumerate(doc.sections, 1):
lines.append(f"## {si}. {sec.heading}")
lines.append(f"_seed nodes: {', '.join(i[:8] for i in sec.seed_ids)}_")
lines.append("")
lines.append("| # | realized claim | traces geometry edge |")
lines.append("|---|----------------|----------------------|")
n = 0
for block in sec.blocks:
for sent, prov in zip(block.sentences, block.provenance):
if prov.kind == "connective":
continue
n += 1
edge = prov.trace().replace("|", "\\|")
s = sent.replace("|", "\\|")
lines.append(f"| {n} | {s} | {edge} |")
lines.append("")
return "\n".join(lines) + "\n"
-112
View File
@@ -1,112 +0,0 @@
"""realize.py — REALIZE stage: fill each planned section with faithful passages.
Scales the PROVEN realizer from a single assertion to a passage. For each
planned proposition we build a realizer-ready clause (the proven
``_prop_to_clause`` mapping) and run it through the proven engine
(``engine.realize``), which is a deterministic grammar with the SACRED negation
contract — it never invents. Each realized sentence is paired with a
:class:`Provenance` that pins it to the exact geometry edge it came from.
"Passage, not a list of sentences": within a section we lightly vary sentence
openings and group related claims, but we add NO content the geometry did not
assert. The only non-geometry words are function words the grammar already owns
(articles, "and", conjunction of same-subject claims). Document-level flow is
COHERE's job; this stage owns intra-section fluency + fidelity.
"""
from __future__ import annotations
import os
import sys
_NT = os.path.expanduser("~/Desktop/neuron-talk")
_LR = os.path.expanduser("~/Desktop/lang-realizers")
for _p in (_NT, _LR):
if _p not in sys.path:
sys.path.insert(0, _p)
import engine # noqa: E402 (the proven no-LLM realizer)
from dialogue import _prop_to_clause # noqa: E402 (proven prop -> clause)
from document_ir import Block, DocumentIR, Provenance, Section # noqa: E402
def _provenance_from(p, kind: str = "fact") -> Provenance:
return Provenance(
subj_id=p.source_node_id, subject=p.subject, relation=p.predicate,
obj=p.object, polarity=p.polarity, confidence=round(float(p.confidence), 3),
node_id=p.source_node_id, kind=kind,
importance=float(getattr(p, "node_importance", 0.0) or 0.0),
salience=0.0,
)
import re as _re
# a well-formed declarative opens with a determiner, a proper noun, "I", or a
# capitalized head — not a mis-parsed object pronoun or a copula fragment.
_BAD_OPENERS = _re.compile(r"^(Me |It is I|There is|This is it|That is it)\b")
_VACUOUS = _re.compile(r"^\w+ (is|are|was|were) (it|no|nothing|empty|those|this|that)\.?$",
_re.I)
def _good_sentence(text: str) -> bool:
"""Fluency gate — drops degenerate realizations. NEVER loosens faithfulness;
it only refuses to SPEAK a claim whose surface came out malformed."""
words = text.rstrip(".").split()
if len(words) < 3:
return False
if _BAD_OPENERS.search(text):
return False
if _VACUOUS.match(text):
return False
# a sentence that is mostly one-letter/two-letter tokens is a parse artifact
short = sum(1 for w in words if len(w.strip(".,'")) <= 2)
if short > len(words) / 2:
return False
return True
def _realize_prop(p, lang: str = "en") -> tuple[str, Provenance] | None:
"""One proposition -> (faithful sentence, provenance) or None if it drops."""
clause = _prop_to_clause(p)
text = engine.realize(clause, lang)
if not text or not text.strip():
return None
text = text.strip()
if not text.endswith((".", "!", "?")):
text += "."
# capitalize first character (proper nouns / "I" already handled by grammar)
text = text[0].upper() + text[1:]
if not _good_sentence(text):
return None
return text, _provenance_from(p)
def realize_document(doc: DocumentIR, lang: str = "en") -> DocumentIR:
"""Fill every planned section's blocks with faithful, realized passages."""
for sec in doc.sections:
planned = sec.__dict__.get("_planned_props", [])
block = Block(role="body")
summary_bits: list[str] = []
for p in planned:
r = _realize_prop(p, lang)
if r is None:
continue
text, prov = r
block.sentences.append(text)
block.provenance.append(prov)
if len(summary_bits) < 1:
# a short grounded gloss for TOC / pptx bullets
obj = (prov.obj or "").strip().rstrip(".")
if obj:
summary_bits.append(obj)
if block.sentences:
sec.blocks.append(block)
sec.summary = summary_bits[0] if summary_bits else ""
# drop the transient planning payload; the IR is now self-contained
sec.__dict__.pop("_planned_props", None)
sec.__dict__.pop("_node", None)
# prune sections that realized to nothing
doc.sections = [s for s in doc.sections if s.blocks]
return doc
-136
View File
@@ -1,136 +0,0 @@
// accent.el - A British-RP ACCENT as an INGESTED TRANSFORM-GEOMETRY, composed
// onto the voice (voice (+) accent, SEPARABLE). Reads elp/data/british-accent.psv
// into an accent MANIFOLD in the engram (override nodes + a shared accent hub),
// and the render reads the RP formant overrides + the non-rhotic rule back from
// that geometry. NO accent targets live in code same discipline as the base
// phonetics. PROVENANCE NOTE: the RP Hz values are PROVISIONAL (reconstructed-
// from-knowledge approximations, cite Deterding1997 / Hawkins&Midgley2005 /
// Wells1982) pending transcription from the published tables the PIPELINE is
// the deliverable; exact values are being source-verified separately.
fn ingest_accent(path: String) -> [String] {
let content: String = fs_read(path)
let lines: [String] = str_split(content, "\n")
let nl: Int = native_list_len(lines)
let amap: [String] = native_list_empty()
let hub: String = engram_node("accent british-rp prov=PROVISIONAL cite=Deterding1997-HawkinsMidgley2005-Wells1982", "Accent", 80)
let li: Int = 0
while li < nl {
let line: String = native_list_get(lines, li)
let ll: Int = str_len(line)
let skip: Int = 0
if ll < 3 {
skip = 1
}
if skip == 0 {
let first: Int = str_char_code(line, 0)
if first == 35 {
skip = 1
}
}
if skip == 0 {
let f: [String] = str_split(line, "|")
let nf: Int = native_list_len(f)
if nf >= 6 {
let key: String = native_list_get(f, 0)
let f1: String = native_list_get(f, 1)
let f2: String = native_list_get(f, 2)
let f3: String = native_list_get(f, 3)
let kind: String = native_list_get(f, 4)
let set: String = native_list_get(f, 5)
let cont: String = "accent british-rp " + key + " f1=" + f1 + " f2=" + f2 + " f3=" + f3 + " kind=" + kind + " set=" + set + " prov=PROVISIONAL cite=Deterding1997-HawkinsMidgley2005-Wells1982"
let id: String = engram_node(cont, "AccentTarget", 80)
amap = native_list_append(amap, key)
amap = native_list_append(amap, cont)
engram_connect(id, hub, 80, "of_accent")
}
}
li = li + 1
}
return amap
}
// RP formant override for a phoneme, read from the accent manifold. Returns
// [f1,f2,f3] for a vowel_override record, or an empty list if none / a rule.
fn accent_formants(amap: [String], code: String) -> [Int] {
let out: [Int] = native_list_empty()
let id: String = sp_map_get(amap, code)
if str_eq(id, "") {
return out
}
let j: String = id
let isrule: Int = str_index_of(j, "drop_coda")
if isrule >= 0 {
return out
}
let f1: Int = parse_uint_from(j, "f1=")
if f1 <= 0 {
return out
}
let out = native_list_append(out, f1)
let out = native_list_append(out, parse_uint_from(j, "f2="))
let out = native_list_append(out, parse_uint_from(j, "f3="))
return out
}
// Is this accent non-rhotic? (reads the R rule node from the manifold)
fn is_nonrhotic(amap: [String]) -> Int {
let id: String = sp_map_get(amap, "R")
if str_eq(id, "") {
return 0
}
let hit: Int = str_index_of(id, "drop_coda")
if hit >= 0 {
return 1
}
return 0
}
// Is this symbol a vowel? Membership in the vowel-set derived from the phonetics
// source's class column (phonological structure the FORMANT NUMBERS still come
// from the organ manifold; this is only the categorical class for the rule).
fn is_vowel_sym(vset: [String], sym: String) -> Int {
let n: Int = native_list_len(vset)
let i: Int = 0
while i < n {
if str_eq(native_list_get(vset, i), sym) {
return 1
}
i = i + 1
}
return 0
}
// Non-rhotic transform: drop a post-vocalic CODA /R/ an R whose next non-SIL
// phoneme is NOT a vowel (a consonant, or end of utterance). Keep INTERVOCALIC/
// onset R (next non-SIL phoneme is a vowel, e.g. the medial R in N UW R AA N).
fn apply_rhoticity(codes: [String], vset: [String]) -> [String] {
let n: Int = native_list_len(codes)
let out: [String] = native_list_empty()
let i: Int = 0
while i < n {
let c: String = native_list_get(codes, i)
let keep: Int = 1
if str_eq(c, "R") {
let jx: Int = i + 1
let nextv: Int = 0
while jx < n {
let ncode: String = native_list_get(codes, jx)
if str_eq(ncode, "SIL") {
jx = jx + 1
} else {
nextv = is_vowel_sym(vset, ncode)
jx = n + 1000
}
}
if nextv == 0 {
keep = 0
}
}
if keep == 1 {
out = native_list_append(out, c)
}
i = i + 1
}
return out
}
-73
View File
@@ -1,73 +0,0 @@
// audio-demo.el - Drive the native audio surface: render a tone per instrument
// from its LEARNED signature, then render a small meaning-phrase "piece".
// Entry point: top-level statement calls main() (same convention as the
// examples' top-level println(run_test())).
fn micros_to_str(xs: [Int]) -> String {
let n: Int = native_list_len(xs)
let out: String = ""
let i: Int = 0
while i < n {
if i > 0 { let out: String = out + "," }
let out: String = out + int_to_str(native_list_get(xs, i))
let i: Int = i + 1
}
return out
}
// Render a 1.0s A4 (midi 69) tone from a signature file, print the parsed
// partials (proving the numbers came from the engram .sig), write the WAV.
fn render_tone(name: String, sigpath: String, outpath: String, table: [Int]) -> Int {
let lines: [String] = sig_load(sigpath)
let partials: [Int] = parse_micros(sig_field(lines, "partials"))
println("[" + name + "] partials_n=" + sig_field(lines, "partials_n") + " parsed_partials_micro(scale 1e6)=" + micros_to_str(partials))
println("[" + name + "] raw partials line from .sig = " + sig_field(lines, "partials"))
let freq: Int = freq_of_midi(69)
let note: [Int] = synth_from_sig(lines, freq, 1000, 900, 44100, table)
let n: Int = native_list_len(note)
let ok: Int = wav_write(outpath, note, n, 44100)
println("[" + name + "] rendered " + int_to_str(n) + " samples -> " + outpath + " (write_ok=" + int_to_str(ok) + ")")
return n
}
fn run_demo() -> Int {
let table: [Int] = sin_table()
fs_mkdir("/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out")
println("=== TONES: render A4 (midi 69) from each learned signature ===")
render_tone("flute", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/flute.sig", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/tone-flute.wav", table)
render_tone("clarinet", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/clarinet.sig", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/tone-clarinet.wav", table)
render_tone("violin", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/violin.sig", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/tone-violin.wav", table)
render_tone("piano", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/piano.sig", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/tone-piano.wav", table)
render_tone("organ", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/organ.sig", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/tone-organ.wav", table)
println("")
println("=== PIECE: a 6-frame meaning phrase (incl. a NEG frame) ===")
let frames: [[String]] = native_list_empty()
let frames: [[String]] = native_list_append(frames, audio_frame("agent", "aff", "0.9", "0.8", "0", "s1"))
let frames: [[String]] = native_list_append(frames, audio_frame("theme", "aff", "0.7", "0.6", "0", "s2"))
let frames: [[String]] = native_list_append(frames, audio_frame("cause", "aff", "0.8", "0.9", "1", "s3"))
let frames: [[String]] = native_list_append(frames, audio_frame("negation", "neg", "0.85", "0.7", "0", "s4"))
let frames: [[String]] = native_list_append(frames, audio_frame("goal", "aff", "0.6", "0.5", "1", "s5"))
let frames: [[String]] = native_list_append(frames, audio_frame("result", "aff", "0.95", "1.0", "0", "s6"))
// Print the plan so the NEG frame's minor third (+3) vs major (+4) is visible.
let nf: Int = native_list_len(frames)
let fi: Int = 0
while fi < nf {
let frame: [String] = native_list_get(frames, fi)
let plan: [Int] = plan_note(frame)
let pol: String = surface_get(frame, "polarity")
let third_name: String = "major(+4)"
if str_eq(pol, "neg") { let third_name: String = "MINOR(+3)" }
println("frame " + int_to_str(fi) + " relation=" + surface_get(frame, "relation") + " polarity=" + pol + " -> midi=" + int_to_str(native_list_get(plan, 0)) + " dur_ms=" + int_to_str(native_list_get(plan, 1)) + " amp_pm=" + int_to_str(native_list_get(plan, 2)) + " third=" + third_name)
let fi: Int = fi + 1
}
let piano_lines: [String] = sig_load("/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/piano.sig")
let total: Int = realize_audio(frames, piano_lines, "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/piece.wav", 44100, table)
println("PIECE rendered " + int_to_str(total) + " samples -> /Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/piece.wav")
return total
}
println("audio-demo main returned samples=" + int_to_str(run_demo()))
-400
View File
@@ -1,400 +0,0 @@
// audio-surface.el - Native own-core additive-synthesis audio surface.
//
// The AUDIO efferent seam, native, no Python and no library. This renders real
// PCM .wav bytes from instrument SIGNATURES read from engram-sourced .sig data
// files (elp/faculty/sig/*.sig) - the partial amplitudes are NEVER literals in
// this source; they are parsed from the learned signature at run time. That is
// the whole proof: render-from-learned-signatures.
//
// EL has no float arithmetic operator (codegen emits raw int64 ops for + - * /
// on the shared 64-bit slot) and no float-arithmetic natives - so ALL synthesis
// math here is own-core INTEGER fixed-point. Angles use a quarter-wave sine
// table (scale 10000) from a fixed-point Taylor series; amplitudes are parsed to
// micro (scale 1e6) straight from the .sig text; frequencies are milliHz ints.
//
// Pipeline mirrors the two-stage projector (midi.py): plan_note(frame) reads a
// frame's meaning-geometry slot-map and derives (pitch, duration, amplitude);
// realize_audio SUPERPOSES the signature's partials (the compose op) and
// serialises RIFF/WAVE. Same frame -> midi OR audio.
// -- integer decimal + string helpers -----------------------------------------
fn str_to_int_el(s: String) -> Int {
let n: Int = str_len(s)
let i: Int = 0
let v: Int = 0
let neg: Bool = false
while i < n {
let c: Int = str_char_code(s, i)
if c == 45 { let neg: Bool = true }
if c >= 48 {
if c < 58 {
let v: Int = v * 10 + (c - 48)
}
}
let i: Int = i + 1
}
if neg { return 0 - v }
return v
}
fn parse_micro(s: String) -> Int {
let dot: Int = str_index_of(s, ".")
if dot < 0 {
return str_to_int_el(s) * 1000000
}
let n: Int = str_len(s)
let ipart: String = str_slice(s, 0, dot)
let fpart: String = str_slice(s, dot + 1, n)
let iv: Int = str_to_int_el(ipart)
let fv: Int = 0
let scale: Int = 100000
let fn2: Int = str_len(fpart)
let i: Int = 0
while i < 6 {
let d: Int = 0
if i < fn2 {
let d: Int = str_char_code(fpart, i) - 48
}
let fv: Int = fv + d * scale
let scale: Int = scale / 10
let i: Int = i + 1
}
return iv * 1000000 + fv
}
// -- signature (engram data file) loader ---------------------------------------
fn sig_load(path: String) -> [String] {
let text: String = fs_read(path)
return str_split(text, "\n")
}
fn sig_field(lines: [String], key: String) -> String {
let pref: String = key + ": "
let n: Int = native_list_len(lines)
let plen: Int = str_len(pref)
let i: Int = 0
while i < n {
let ln: String = native_list_get(lines, i)
if str_starts_with(ln, pref) {
return str_slice(ln, plen, str_len(ln))
}
let i: Int = i + 1
}
return ""
}
fn parse_micros(csv: String) -> [Int] {
let parts: [String] = str_split(csv, ",")
let n: Int = native_list_len(parts)
let out: [Int] = native_list_empty()
let i: Int = 0
while i < n {
let out: [Int] = native_list_append(out, parse_micro(native_list_get(parts, i)))
let i: Int = i + 1
}
return out
}
// -- fixed-point sine (own-core, quarter-wave Taylor table, scale 10000) --------
fn sin_table() -> [Int] {
let HP: Int = 1570796
let t: [Int] = native_list_empty()
let q: Int = 0
while q < 257 {
let x: Int = q * HP / 256
let x2: Int = x * x / 1000000
let x3: Int = x2 * x / 1000000
let x5: Int = x3 * x2 / 1000000
let x7: Int = x5 * x2 / 1000000
let x9: Int = x7 * x2 / 1000000
let s: Int = x - x3 / 6 + x5 / 120 - x7 / 5040 + x9 / 362880
let t: [Int] = native_list_append(t, s / 100)
let q: Int = q + 1
}
return t
}
fn sin_lookup(t: [Int], phase: Int) -> Int {
let p: Int = phase % 1024
if p < 0 { let p: Int = p + 1024 }
let quad: Int = p / 256
let r: Int = p % 256
if quad == 0 { return native_list_get(t, r) }
if quad == 1 { return native_list_get(t, 256 - r) }
if quad == 2 { return 0 - native_list_get(t, r) }
return 0 - native_list_get(t, 256 - r)
}
fn isqrt_int(n: Int) -> Int {
if n <= 0 { return 0 }
let x: Int = n
let y: Int = (x + 1) / 2
while y < x {
let x: Int = y
let y: Int = (x + n / x) / 2
}
return x
}
// freq_of_midi: equal-tempered frequency in milliHz. 440000 mHz at midi 69.
fn freq_of_midi(m: Int) -> Int {
let f: Int = 440000
if m > 69 {
let k: Int = m - 69
let i: Int = 0
while i < k {
let f: Int = f * 1059463 / 1000000
let i: Int = i + 1
}
return f
}
if m < 69 {
let k: Int = 69 - m
let i: Int = 0
while i < k {
let f: Int = f * 1000000 / 1059463
let i: Int = i + 1
}
return f
}
return f
}
// -- envelope (ADSR), scale 1000 -----------------------------------------------
fn adsr_env(i: Int, total: Int, atk_n: Int, dec_n: Int, sus_pm: Int, rel_n: Int) -> Int {
if i < atk_n {
if atk_n == 0 { return 1000 }
return 1000 * i / atk_n
}
if i < atk_n + dec_n {
if dec_n == 0 { return sus_pm }
return 1000 - (1000 - sus_pm) * (i - atk_n) / dec_n
}
let rel_start: Int = total - rel_n
if i < rel_start {
return sus_pm
}
if rel_n == 0 { return 0 }
let left: Int = total - i
return sus_pm * left / rel_n
}
// -- note synthesis: SUPERPOSE the learned partials -> [Int] samples -----------
fn note_samples(freq_mHz: Int, dur_ms: Int, rate: Int, partials: [Int], sumP: Int, b_micro: Int, vib_rate: Int, vib_cents: Int, atk_ms: Int, dec_ms: Int, sus_pm: Int, rel_ms: Int, amp_pm: Int, table: [Int]) -> [Int] {
let total: Int = dur_ms * rate / 1000
let atk_n: Int = atk_ms * rate / 1000
let dec_n: Int = dec_ms * rate / 1000
let rel_n: Int = rel_ms * rate / 1000
let np: Int = native_list_len(partials)
let half_mhz: Int = rate * 1000 / 2
let out: [Int] = native_list_empty()
let i: Int = 0
while i < total {
let acc: Int = 0
let k: Int = 0
while k < np {
let harm: Int = k + 1
let amp_k: Int = native_list_get(partials, k)
let factor: Int = 1000000
if b_micro > 0 {
let val: Int = 1000000 + b_micro * harm * harm
let factor: Int = isqrt_int(val * 1000000)
}
let fn_mhz: Int = freq_mHz * harm
let fn_mhz: Int = fn_mhz * factor / 1000000
if vib_cents > 0 {
if vib_rate > 0 {
let vphase: Int = i * vib_rate * 1024 / rate
let vs: Int = sin_lookup(table, vphase)
let vibf: Int = 1000000 + (vib_cents * vs * 833) / 10000
let fn_mhz: Int = fn_mhz * vibf / 1000000
}
}
if fn_mhz <= half_mhz {
let phase: Int = i * fn_mhz * 1024 / (rate * 1000)
let sv: Int = sin_lookup(table, phase)
let acc: Int = acc + sv * amp_k / 1000000
}
let k: Int = k + 1
}
let env: Int = adsr_env(i, total, atk_n, dec_n, sus_pm, rel_n)
let s16: Int = acc * 2800000 / sumP
let s16: Int = s16 * env / 1000
let s16: Int = s16 * amp_pm / 1000
if s16 > 32767 { let s16: Int = 32767 }
if s16 < 0 - 32767 { let s16: Int = 0 - 32767 }
let out: [Int] = native_list_append(out, s16)
let i: Int = i + 1
}
return out
}
fn synth_from_sig(lines: [String], freq_mHz: Int, dur_ms: Int, amp_pm: Int, rate: Int, table: [Int]) -> [Int] {
let partials: [Int] = parse_micros(sig_field(lines, "partials"))
let np: Int = native_list_len(partials)
let sumP: Int = 0
let j: Int = 0
while j < np {
let pj: Int = native_list_get(partials, j)
let sumP: Int = sumP + pj
let j: Int = j + 1
}
if sumP <= 0 { let sumP: Int = 1000000 }
let adsr: [String] = str_split(sig_field(lines, "adsr"), ",")
let atk_ms: Int = parse_micro(native_list_get(adsr, 0)) / 1000
let dec_ms: Int = parse_micro(native_list_get(adsr, 1)) / 1000
let sus_pm: Int = parse_micro(native_list_get(adsr, 2)) / 1000
let rel_ms: Int = parse_micro(native_list_get(adsr, 3)) / 1000
let b_micro: Int = parse_micro(sig_field(lines, "inharmonicity_B"))
let vib_rate: Int = str_to_int_el(sig_field(lines, "vibrato_rate_hz"))
let vib_cents: Int = str_to_int_el(sig_field(lines, "vibrato_depth_cents"))
return note_samples(freq_mHz, dur_ms, rate, partials, sumP, b_micro, vib_rate, vib_cents, atk_ms, dec_ms, sus_pm, rel_ms, amp_pm, table)
}
// -- byte-buffer helpers (own-core, no library) --------------------------------
fn put_tag(buf: String, pos: Int, s: String) -> String {
let n: Int = str_len(s)
let i: Int = 0
while i < n {
let buf: String = __str_set_char(buf, pos + i, str_char_code(s, i))
let i: Int = i + 1
}
return buf
}
fn put_u32le(buf: String, pos: Int, v: Int) -> String {
let buf: String = __str_set_char(buf, pos, v % 256)
let buf: String = __str_set_char(buf, pos + 1, (v / 256) % 256)
let buf: String = __str_set_char(buf, pos + 2, (v / 65536) % 256)
let buf: String = __str_set_char(buf, pos + 3, (v / 16777216) % 256)
return buf
}
fn put_u16le(buf: String, pos: Int, v: Int) -> String {
let buf: String = __str_set_char(buf, pos, v % 256)
let buf: String = __str_set_char(buf, pos + 1, (v / 256) % 256)
return buf
}
// -- WAV serializer: own-core RIFF/WAVE, PCM mono 16-bit -----------------------
fn wav_write(path: String, samples: [Int], n: Int, rate: Int) -> Int {
let data_len: Int = n * 2
let total: Int = 44 + data_len
let buf: String = __str_alloc(total)
let buf: String = put_tag(buf, 0, "RIFF")
let buf: String = put_u32le(buf, 4, 36 + data_len)
let buf: String = put_tag(buf, 8, "WAVE")
let buf: String = put_tag(buf, 12, "fmt ")
let buf: String = put_u32le(buf, 16, 16)
let buf: String = put_u16le(buf, 20, 1)
let buf: String = put_u16le(buf, 22, 1)
let buf: String = put_u32le(buf, 24, rate)
let buf: String = put_u32le(buf, 28, rate * 2)
let buf: String = put_u16le(buf, 32, 2)
let buf: String = put_u16le(buf, 34, 16)
let buf: String = put_tag(buf, 36, "data")
let buf: String = put_u32le(buf, 40, data_len)
let i: Int = 0
while i < n {
let v: Int = native_list_get(samples, i)
if v < 0 { let v: Int = v + 65536 }
let buf: String = __str_set_char(buf, 44 + i * 2, v % 256)
let buf: String = __str_set_char(buf, 44 + i * 2 + 1, (v / 256) % 256)
let i: Int = i + 1
}
let ok: Int = fs_write_bytes(path, buf, total)
return ok
}
// -- plan: frame slot-map -> note atom (pitch, duration, amplitude) ------------
fn audio_frame(relation: String, polarity: String, confidence: String, importance: String, salience: String, subj_id: String) -> [String] {
let f: [String] = native_list_empty()
let f: [String] = native_list_append(f, "relation")
let f: [String] = native_list_append(f, relation)
let f: [String] = native_list_append(f, "polarity")
let f: [String] = native_list_append(f, polarity)
let f: [String] = native_list_append(f, "confidence")
let f: [String] = native_list_append(f, confidence)
let f: [String] = native_list_append(f, "importance")
let f: [String] = native_list_append(f, importance)
let f: [String] = native_list_append(f, "salience")
let f: [String] = native_list_append(f, salience)
let f: [String] = native_list_append(f, "subj_id")
let f: [String] = native_list_append(f, subj_id)
return f
}
fn degree_offset(deg: Int) -> Int {
if deg == 0 { return 0 }
if deg == 1 { return 2 }
if deg == 2 { return 4 }
if deg == 3 { return 5 }
if deg == 4 { return 7 }
if deg == 5 { return 9 }
return 11
}
// returns [midi, dur_ms, amp_pm]
fn plan_note(frame: [String]) -> [Int] {
let relation: String = surface_get(frame, "relation")
let polarity: String = surface_get(frame, "polarity")
let confidence: String = surface_get(frame, "confidence")
let importance: String = surface_get(frame, "importance")
let salience: String = surface_get(frame, "salience")
let rn: Int = str_len(relation)
let csum: Int = 0
let i: Int = 0
while i < rn {
let cc: Int = str_char_code(relation, i)
let csum: Int = csum + cc
let i: Int = i + 1
}
let deg: Int = csum % 7
let third: Int = 4
if str_eq(polarity, "neg") { let third: Int = 3 }
let sal_oct: Int = str_to_int_el(salience)
let doff: Int = degree_offset(deg)
let midi: Int = 60 + sal_oct * 12 + doff + third
let conf_micro: Int = parse_micro(confidence)
let dur_ms: Int = 200 + conf_micro / 1000
let imp_micro: Int = parse_micro(importance)
let amp_pm: Int = 400 + imp_micro / 2000
let out: [Int] = native_list_empty()
let out: [Int] = native_list_append(out, midi)
let out: [Int] = native_list_append(out, dur_ms)
let out: [Int] = native_list_append(out, amp_pm)
return out
}
fn realize_audio(frames: [[String]], sig_lines: [String], path: String, rate: Int, table: [Int]) -> Int {
let nf: Int = native_list_len(frames)
let all: [Int] = native_list_empty()
let count: Int = 0
let fi: Int = 0
while fi < nf {
let frame: [String] = native_list_get(frames, fi)
let plan: [Int] = plan_note(frame)
let midi: Int = native_list_get(plan, 0)
let dur_ms: Int = native_list_get(plan, 1)
let amp_pm: Int = native_list_get(plan, 2)
let freq: Int = freq_of_midi(midi)
let note: [Int] = synth_from_sig(sig_lines, freq, dur_ms, amp_pm, rate, table)
let nn: Int = native_list_len(note)
let j: Int = 0
while j < nn {
let all: [Int] = native_list_append(all, native_list_get(note, j))
let j: Int = j + 1
}
let count: Int = count + nn
let fi: Int = fi + 1
}
let ok: Int = wav_write(path, all, count, rate)
return count
}
File diff suppressed because it is too large Load Diff
-16
View File
@@ -1,16 +0,0 @@
// comprehend.elh — public surface of the ELP comprehension front-end.
// text → meaning-spec (the input half of the ELP; inverse of the realizer).
extern fn parse_spec(text: String) -> [String]
extern fn parse_spec_lang(text: String, lang: String) -> [String]
extern fn parse_json(text: String) -> String
extern fn parse_json_lang(text: String, lang: String) -> String
// Analysis primitives (invertible morphology + deterministic grammar helpers):
extern fn cp_tokenize(text: String) -> [String]
extern fn cp_pron_concept(w: String) -> String
extern fn cp_is_negation(w: String) -> Bool
extern fn cp_is_neg_adverb(w: String) -> Bool
extern fn cp_irr2(surface: String) -> [String]
extern fn cp_reg_verb(w: String) -> [String]
extern fn cp_analyze_verb(surface: String) -> [String]
extern fn cp_verb_start(toks: [String], end: Int) -> Int
extern fn cp_subord_start(toks: [String], n: Int) -> Int
-1
View File
@@ -1 +0,0 @@
import "language-profile.el"
-2
View File
@@ -1,2 +0,0 @@
import "language-profile.el"
import "dedup_test_a_nodedup.el"
-2
View File
@@ -1,2 +0,0 @@
import "language-profile.el"
extern fn fn_a(x: String) -> String
-1
View File
@@ -1 +0,0 @@
extern fn fn_a(x: String) -> String
-2
View File
@@ -1,2 +0,0 @@
import "language-profile.el"
extern fn fn_a(x: String) -> String
-6
View File
@@ -1,6 +0,0 @@
import "language-profile.el"
import "dedup_test_a.el"
fn main_fn(x: String) -> String {
return x
}
-2
View File
@@ -1,2 +0,0 @@
import "language-profile.el"
import "dedup_test_a.el"
-2
View File
@@ -1,2 +0,0 @@
import "language-profile.el"
import "dedup_test_a_notail.el"
-287
View File
@@ -1,287 +0,0 @@
// dialogue.el SUMMON-THROUGH-SELF, native el. Port of dialogue.py's core.
//
// THE WHOLE DIALOGUE IS ONE OPERATION. A fact is never merely *fetched*: the
// query is PROJECTED into the engram's self + memory geometry, LANDS in a region,
// and the reply is READ OUT / the region MATERIALIZED from wherever it landed.
//
// project(query) -> land on a region -> read out from that region
//
// lands in the SELF region -> grounded identity/presence, read out of
// the real self nodes (self_region.el)
// lands on a memory NEIGHBORHOOD -> MATERIALIZE it: walk the neighborhood
// (engram_neighbors_json) and read out the
// region's connected members
// lands nowhere close -> HONEST ABSENCE (an empty region, not a
// fabricated answer, not an error)
//
// CRITICAL INVARIANTS (enforced structurally, not by convention):
// * ONE operation there is NO intent classifier and NO separate
// fact-retrieval branch. Identity is nearest-region proximity, not a switch.
// * MATERIALIZE by walking the neighborhood, never by fetching top-props.
// * HONEST ABSENCE when the region is thin.
// * NEGATION is SACRED: the readout is the stored prose VERBATIM, so a negated
// memory stays negated we never paraphrase a polarity away.
// * NO ECHO: the old "I noted that X. That relates to Y." template is gone.
// The summon path materializes or honestly declines it never echoes.
// * DIRECTIVE OVERRIDE: a meta-directive ("answer in English") overrides the
// reply language while the content language is still auto-detected.
//
// Depends on: comprehend (parse_spec_lang, cp_tokenize), multilingual (ml_detect,
// ml_tr, ml_term), propositions (prop_split_sentences), self_region
// (sr_available, sr_readout), the engram + json runtime builtins.
// directive override
// Return [target_lang, content]. target_lang is "" when no directive is present.
// A directive names an output language; we strip it and keep the remaining text
// as the content (whose OWN language is still auto-detected downstream).
fn dlg_dir_hit(low: String, phrase: String) -> Bool {
return str_contains(low, phrase)
}
fn dlg_parse_directive(text: String) -> [String] {
let low: String = str_to_lower(text)
let lang: String = ""
let phrase: String = ""
// English target
if dlg_dir_hit(low, "in english") { let lang = "en"; let phrase = "in english" }
if dlg_dir_hit(low, "em inglês") { let lang = "en"; let phrase = "em inglês" }
if dlg_dir_hit(low, "em ingles") { let lang = "en"; let phrase = "em ingles" }
if dlg_dir_hit(low, "en inglés") { let lang = "en"; let phrase = "en inglés" }
// Portuguese target
if dlg_dir_hit(low, "in portuguese") { let lang = "pt"; let phrase = "in portuguese" }
if dlg_dir_hit(low, "em português") { let lang = "pt"; let phrase = "em português" }
// Spanish target
if dlg_dir_hit(low, "in spanish") { let lang = "es"; let phrase = "in spanish" }
if dlg_dir_hit(low, "en español") { let lang = "es"; let phrase = "en español" }
// Italian target
if dlg_dir_hit(low, "in italian") { let lang = "it"; let phrase = "in italian" }
let content: String = text
if !str_eq(phrase, "") {
// strip the directive phrase (and a common "answer"/"responda" lead-in),
// leaving the real question as content.
let idx: Int = str_index_of(low, phrase)
if idx >= 0 {
let before: String = str_slice(text, 0, idx)
let after: String = str_slice(text, idx + str_len(phrase), str_len(text))
let content = str_trim(before + " " + after)
}
// trim a leading "answer"/"responda"/"reply" and stray colon/comma.
let cl: String = str_to_lower(content)
if str_starts_with(cl, "answer") { let content = str_trim(str_slice(content, 6, str_len(content))) }
if str_starts_with(cl, "responda") { let content = str_trim(str_slice(content, 8, str_len(content))) }
if str_starts_with(cl, "reply") { let content = str_trim(str_slice(content, 5, str_len(content))) }
if str_starts_with(content, ":") { let content = str_trim(str_slice(content, 1, str_len(content))) }
if str_starts_with(content, ",") { let content = str_trim(str_slice(content, 1, str_len(content))) }
}
let r: [String] = native_list_empty()
let r = native_list_append(r, lang)
let r = native_list_append(r, content)
return r
}
// identity landing (a region proximity, not a classifier switch)
// The query lands in the SELF region when it takes an identity/presence shape.
// Cross-lingual forms are included because the engram's lexical probe is
// English-leaning. This is the SELF attractor of the single operation.
fn dlg_is_identity(content: String) -> Bool {
let low: String = str_to_lower(str_trim(content))
if str_contains(low, "who are you") { return true }
if str_contains(low, "what are you") { return true }
if str_contains(low, "who i am") { return true }
if str_contains(low, "your name") { return true }
if str_contains(low, "about yourself") { return true }
if str_contains(low, "are you conscious") { return true }
if str_contains(low, "are you there") { return true }
// cross-lingual identity question-forms
if str_contains(low, "quem é você") { return true }
if str_contains(low, "quem es voce") { return true }
if str_contains(low, "quién eres") { return true }
if str_contains(low, "quien eres") { return true }
if str_contains(low, "chi sei") { return true }
if str_contains(low, "qui es-tu") { return true }
if str_contains(low, "wer bist du") { return true }
return false
}
// readout helpers
fn dlg_first_sentence(content: String) -> String {
let sents: [String] = prop_split_sentences(content)
let n: Int = native_list_len(sents)
let i: Int = 0
while i < n {
let s: String = str_trim(native_list_get(sents, i))
// drop a leading markdown heading marker for a clean read-out line
if str_starts_with(s, "# ") { let s = str_trim(str_slice(s, 2, str_len(s))) }
if str_len(s) > 0 { return s }
let i = i + 1
}
return str_trim(content)
}
// strip trailing/leading punctuation from a token.
fn dlg_clean_tok(w: String) -> String {
let s: String = str_trim(w)
let s = str_strip_suffix(s, ".")
let s = str_strip_suffix(s, ",")
let s = str_strip_suffix(s, "?")
let s = str_strip_suffix(s, "!")
let s = str_strip_suffix(s, ":")
let s = str_strip_suffix(s, ";")
return str_trim(s)
}
// closed-class across the supported languages (union) a word we must NOT treat
// as a retrieval topic. Also drops the meta verbs of a request ("tell", "prove",
// "show") so the TOPIC, not the speech act, is what projects into memory.
fn dlg_is_stop(w: String) -> Bool {
if ml_stop_en(w) { return true }
if ml_stop_es(w) { return true }
if ml_stop_pt(w) { return true }
if ml_stop_it(w) { return true }
if str_eq(w, "tell") { return true }
if str_eq(w, "show") { return true }
if str_eq(w, "about") { return true }
if str_eq(w, "sobre") { return true }
if str_eq(w, "acerca") { return true }
return false
}
// The CONTENT TERMS the query projects into memory: content words only, cleaned,
// cross-lingually mapped to the engram's English vocabulary, 3 chars. This is
// the geometry probe the speech-act verbs and function words are stripped so a
// PP topic ("tell me ABOUT Lisbon") projects on "lisbon", not "tell"/"me".
fn dlg_content_terms(content: String, lang: String) -> [String] {
let toks: [String] = cp_tokenize(content)
let n: Int = native_list_len(toks)
let out: [String] = native_list_empty()
let i: Int = 0
while i < n {
let w: String = str_to_lower(dlg_clean_tok(native_list_get(toks, i)))
if str_len(w) >= 3 {
if !dlg_is_stop(w) {
let out = native_list_append(out, ml_term(w, lang))
}
}
let i = i + 1
}
return out
}
// Does this landed node lexically overlap the query's content terms? This is the
// RELEVANCE FLOOR: activation always returns the store's most salient nodes, so
// without this a query about nothing would "land" on the self/top node. A node
// that shares no content term with the query is "nowhere close" -> honest absence.
fn dlg_node_matches(node: String, terms: [String]) -> Bool {
let hay: String = str_to_lower(json_get_string(node, "content") + " " + json_get_string(node, "label"))
let n: Int = native_list_len(terms)
let i: Int = 0
while i < n {
let t: String = native_list_get(terms, i)
if str_len(t) >= 3 {
if str_contains(hay, t) { return true }
}
let i = i + 1
}
return false
}
// MATERIALIZE the landed region: read out the landed fact, then WALK the
// neighborhood and read out its connected members (real edges, not top-props).
fn dlg_materialize(top_node: String, reply_lang: String) -> String {
let id: String = json_get_string(top_node, "id")
let content: String = json_get_string(top_node, "content")
let lead: String = dlg_first_sentence(content)
let nb: String = engram_neighbors_json(id, 2, "both")
let m: Int = json_array_len(nb)
let parts: [String] = native_list_empty()
let parts = native_list_append(parts, lead)
let added: Int = 0
let i: Int = 0
while i < m {
if added < 3 {
let rec: String = json_array_get(nb, i)
let node: String = json_get_raw(rec, "node")
let nc: String = json_get_string(node, "content")
if !str_eq(nc, "") {
let sent: String = dlg_first_sentence(nc)
if !str_eq(sent, "") {
let parts = native_list_append(parts, sent)
let added = added + 1
}
}
}
let i = i + 1
}
// The readout is the region's OWN prose, verbatim negation SACRED, no echo.
return str_join(parts, " ")
}
// THE single operation
fn dlg_respond(text: String) -> String {
// directive override: reply language may differ from content language.
let dir: [String] = dlg_parse_directive(text)
let target_lang: String = native_list_get(dir, 0)
let content: String = native_list_get(dir, 1)
let content_lang: String = ml_detect(content)
let reply_lang: String = content_lang
if !str_eq(target_lang, "") { let reply_lang = target_lang }
// comprehend the content (SACRED polarity carried in the spec).
let spec: [String] = parse_spec_lang(content, content_lang)
// PROJECT + LAND: SELF region
// Identity/presence shape lands in the self region; read out the REAL self
// nodes (self_region.el), never a template. Same single operation this is
// just the self attractor winning the landing.
if dlg_is_identity(content) {
if sr_available() {
// read out the REAL self nodes when replying in their own language
// (the soul's prose is English); for another reply language we cannot
// translate real content without an LLM, so we answer with the
// localized SACRED identity anchor honest, in-language, no fabrication.
if str_eq(reply_lang, "en") { return sr_readout("en") }
return ml_tr("identity", reply_lang)
}
// self region thin honest localized identity (logged fallback shape).
return ml_tr("identity", reply_lang)
}
// PROJECT into MEMORY geometry
let terms: [String] = dlg_content_terms(content, content_lang)
let qterm: String = str_join(terms, " ")
let act: String = engram_activate_json(qterm, 12)
let n: Int = json_array_len(act)
// LAND: the highest-activation node that ACTUALLY overlaps the query's
// content terms (the relevance floor). Activation always returns the most
// salient nodes, so we walk the ranked list and take the first that is
// genuinely "close"; if none is, the query landed nowhere. ───────────────
let landing: String = ""
let i: Int = 0
while i < n {
if str_eq(landing, "") {
let rec: String = json_array_get(act, i)
let node: String = json_get_raw(rec, "node")
if dlg_node_matches(node, terms) {
let landing = node
}
}
let i = i + 1
}
// HONEST ABSENCE: nothing close an empty region, not a fabricated answer,
// not an "I noted that" echo.
if str_eq(landing, "") {
return ml_tr("no_memory", reply_lang)
}
// MATERIALIZE the landing by WALKING its neighborhood.
return dlg_materialize(landing, reply_lang)
}
-172
View File
@@ -1,172 +0,0 @@
// elp.el - Engram Language Protocol public API.
//
// Output half of the ELP: Engram semantic form natural language surface text.
// 31 languages. Ties together language-profile, vocabulary, morphology,
// grammar, realizer, and semantics into a single entry point.
//
// Import chain (mirrors manifest.el dependency order):
// language-profile (no deps)
// vocabulary (no deps)
// morphology (depends on: language-profile)
// morphology-XX (depends on: morphology) all language engines
// grammar (depends on: language-profile)
// realizer (depends on: morphology, grammar, language-profile)
// semantics (depends on: grammar, realizer, language-profile)
//
// When elc processes a source that imports this file, it resolves all
// transitive imports via depth-first deduplication each module is
// inlined exactly once regardless of how many importers reference it.
// Base layers
import "language-profile.el"
import "vocabulary.el"
// Morphology: base engine
import "morphology.el"
// Morphology: living languages
import "morphology-es.el"
import "morphology-fr.el"
import "morphology-de.el"
import "morphology-ru.el"
import "morphology-ja.el"
import "morphology-fi.el"
import "morphology-ar.el"
import "morphology-hi.el"
import "morphology-sw.el"
// Morphology: ancient / classical
import "morphology-la.el"
import "morphology-he.el"
// Morphology: dead languages
import "morphology-grc.el"
import "morphology-ang.el"
import "morphology-sa.el"
import "morphology-got.el"
import "morphology-non.el"
import "morphology-enm.el"
import "morphology-pi.el"
import "morphology-fro.el"
import "morphology-goh.el"
import "morphology-sga.el"
import "morphology-txb.el"
import "morphology-peo.el"
import "morphology-akk.el"
import "morphology-uga.el"
import "morphology-egy.el"
import "morphology-sux.el"
import "morphology-gez.el"
import "morphology-cop.el"
// Higher layers
import "grammar.el"
import "realizer.el"
import "semantics.el"
// Comprehension front-end (input half: text meaning-spec)
import "comprehend.el"
//
// Entry points:
//
// generate(semantic_form_json) -> String
// Low-level JSON-based API, defaults to English. SemanticForm JSON fields:
// intent - "assert" | "question" | "command"
// agent - subject (pronoun or noun phrase, optional for commands)
// predicate - verb base form
// patient - object noun phrase (optional)
// location - prepositional phrase e.g. "in the park" (optional)
// tense - "present" | "past" | "future" (default: "present")
// aspect - "simple" | "progressive" | "perfect" (default: "simple")
// lang - ISO 639-1 code (default: "en")
//
// generate_lang(semantic_form_json, lang_code) -> String
// JSON-based API with explicit language code (overrides any "lang" in JSON).
//
// generate_frame(frame: SemFrame) -> String
// High-level SemFrame API. Language from frame's "lang" field (default "en").
// Intents: "assert" | "query" | "describe" | "greet".
//
// generate_frame_lang(frame: SemFrame, lang_code: String) -> String
// High-level SemFrame API with explicit language code override.
// JSON helpers
fn sem_get(json: String, key: String) -> String {
let val: String = json_get(json, key)
return val
}
// Public API: SemFrame
// Generate text from a SemFrame in the language embedded in the frame (default "en").
fn generate_frame(frame: [String]) -> String {
return sem_realize(frame)
}
// Generate text from a SemFrame in the specified language.
fn generate_frame_lang(frame: [String], lang_code: String) -> String {
return sem_realize_lang(frame, lang_code)
}
// Public API: JSON
// Build a realizer slot map from JSON fields and an explicit lang code.
fn build_form_from_json(semantic_form_json: String, lang_code: String) -> [String] {
let intent: String = sem_get(semantic_form_json, "intent")
let agent: String = sem_get(semantic_form_json, "agent")
let predicate: String = sem_get(semantic_form_json, "predicate")
let patient: String = sem_get(semantic_form_json, "patient")
let location: String = sem_get(semantic_form_json, "location")
let tense: String = sem_get(semantic_form_json, "tense")
let aspect: String = sem_get(semantic_form_json, "aspect")
let polarity: String = sem_get(semantic_form_json, "polarity")
let neg_word: String = sem_get(semantic_form_json, "neg_word")
let iobj: String = sem_get(semantic_form_json, "iobj")
let form: [String] = native_list_empty()
let form = native_list_append(form, "intent")
let form = native_list_append(form, intent)
let form = native_list_append(form, "agent")
let form = native_list_append(form, agent)
let form = native_list_append(form, "predicate")
let form = native_list_append(form, predicate)
let form = native_list_append(form, "patient")
let form = native_list_append(form, patient)
let form = native_list_append(form, "iobj")
let form = native_list_append(form, iobj)
let form = native_list_append(form, "location")
let form = native_list_append(form, location)
let form = native_list_append(form, "tense")
let form = native_list_append(form, tense)
let form = native_list_append(form, "aspect")
let form = native_list_append(form, aspect)
// SACRED: polarity crosses the JSON boundary and is never inferred away.
let form = native_list_append(form, "polarity")
let form = native_list_append(form, polarity)
let form = native_list_append(form, "neg_word")
let form = native_list_append(form, neg_word)
let form = native_list_append(form, "lang")
let form = native_list_append(form, lang_code)
return form
}
// Generate text from a JSON semantic form. Language defaults to "en" unless
// the JSON contains a "lang" field.
fn generate(semantic_form_json: String) -> String {
let lang_in_json: String = sem_get(semantic_form_json, "lang")
let lang_code: String = lang_in_json
if str_eq(lang_code, "") {
let lang_code = "en"
}
let form: [String] = build_form_from_json(semantic_form_json, lang_code)
return realize(form)
}
// Generate text from a JSON semantic form in the specified language.
// lang_code overrides any "lang" field present in the JSON.
fn generate_lang(semantic_form_json: String, lang_code: String) -> String {
let form: [String] = build_form_from_json(semantic_form_json, lang_code)
return realize(form)
}

Some files were not shown because too many files have changed in this diff Show More