Commit Graph

249 Commits

Author SHA1 Message Date
will.anderson edbde5ef51 ci: rebuild ci-base on SDK release; publish elb + el_runtime.js to Artifact Registry
El SDK CI - dev / build-and-test (pull_request) Successful in 3m45s
- sdk-release.yaml: add elb and el_runtime.js to foundation-prod uploads
- sdk-release.yaml: add 'Rebuild ci-base' step — patches ci-base:latest with
  freshly built El SDK after each main branch release (pull → overlay → push)
- sdk-release.yaml: add neuron-web to el-sdk-updated dispatch so downstream
  CI rebuilds automatically on SDK update
- ci-dev.yaml: add elb build step and publish elb + el_runtime.js to
  foundation-dev alongside elc and runtime
2026-05-07 02:25:19 -05:00
will.anderson 2e529bd0fe Merge pull request 'Remove Cargo.toml and .rs bootstrap files from el-ui vessels' (#30) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Successful in 3m49s
2026-05-07 05:49:10 +00:00
will.anderson 5d9299a472 Remove all .rs bootstrap files from el-ui vessels
El SDK CI - dev / build-and-test (pull_request) Successful in 3m33s
el-ui vessels are El. The Rust bootstrap implementations were added as
a stopgap but don't belong here — everything should be El source.
Each vessel's src/main.el and manifest.el are the source of truth.
2026-05-06 23:12:25 -05:00
will.anderson e8b01583d8 Remove Cargo.toml files from el-ui — vessels use manifest.el
All package management is through manifest.el / epm. Cargo.toml files
were incorrectly added to vessels and the root. Removed root workspace
Cargo.toml + Cargo.lock and all vessel-level Cargo.toml files.

el-graph and el-html were already correct (no Cargo.toml).
2026-05-06 22:21:47 -05:00
will.anderson b19dd5608f Merge pull request 'ci: use elb to build epm and el-install' (#27) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Successful in 3m50s
El SDK CI - stage / build-and-test (pull_request) Successful in 3m33s
ci: use elb to build epm and el-install
2026-05-07 02:37:49 +00:00
will.anderson 94d6eace94 ci: use elb to build epm and el-install (cd into project dir, use --elc flag)
El SDK CI - dev / build-and-test (pull_request) Successful in 3m35s
2026-05-06 21:33:05 -05:00
will.anderson f1dfc394e3 Merge pull request 'fix: add __http_do_map_to_file runtime primitive' (#24) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Successful in 3m51s
El SDK CI - stage / build-and-test (pull_request) Successful in 3m25s
2026-05-07 02:06:34 +00:00
will.anderson 61bf501b84 fix: add __http_do_map_to_file runtime primitive
El SDK CI - dev / build-and-test (pull_request) Successful in 3m41s
el-install.el generates calls to __http_do_map_to_file (HTTP request
with JSON headers map, streaming response to file). Add it to both
the HAVE_CURL implementation and the no-curl stub section.
2026-05-06 21:01:46 -05:00
will.anderson 2fd298df55 Merge pull request 'fix: add __-prefixed runtime primitives for El compiler' (#21) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Successful in 3m43s
El SDK CI - stage / build-and-test (pull_request) Successful in 3m22s
2026-05-07 01:40:37 +00:00
will.anderson 254cbe0ac2 fix: add __-prefixed runtime primitives expected by El compiler
El SDK CI - dev / build-and-test (pull_request) Successful in 3m22s
The El compiler generates calls to __-prefixed C primitives from within
El stdlib compiled code (e.g. __println, __str_len, __json_get, etc).
These were absent from el_runtime.c, causing linker failures when
building el-install, elb, or epm with the current compiler.

Add 46 __-prefixed aliases/implementations in el_runtime.c covering:
- I/O: __println, __print, __readline
- String: __str_len, __str_cmp, __str_ncmp, __str_alloc, __str_set_char,
  __str_concat_raw, __str_slice_raw, __str_char_at, plus numeric converters
- FS: __fs_read, __fs_write, __fs_exists, __fs_mkdir, __fs_list_raw, etc
- HTTP: __http_do, __http_do_map, __http_serve, __http_serve_v2,
  __http_response, __http_sse_* (weak stubs)
- JSON: __json_get, __json_set, __json_parse_map, __json_stringify_val, etc
- State, env, exec, uuid, sha256, args
2026-05-06 20:36:49 -05:00
will.anderson bcfb33ea83 Merge pull request 'fix: align runtime return types with El compiler output' (#18) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Successful in 3m36s
El SDK CI - stage / build-and-test (pull_request) Successful in 3m16s
2026-05-07 01:04:29 +00:00
will.anderson 60ad7f2f6b fix: align runtime function return types with El compiler output
El SDK CI - dev / build-and-test (pull_request) Successful in 3m16s
El compiler generates calls to println, print, exit_program,
http_set_handler, http_serve, http_set_handler_v2, and http_serve_v2
as el_val_t-returning functions. The runtime declared them void,
causing conflicting-type errors when el-install.c was compiled.

Change all seven to return el_val_t (side-effect functions return 0).
Also update el_runtime.h declarations to match.
2026-05-06 20:00:40 -05:00
will.anderson 231cb5eddd Merge pull request 'fix: add missing runtime functions (native_str_to_int, http_post_json_with_headers)' (#15) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Successful in 3m37s
El SDK CI - stage / build-and-test (pull_request) Successful in 3m39s
2026-05-07 00:35:28 +00:00
will.anderson 54de7d3f3f fix: add missing runtime functions for epm.el
El SDK CI - dev / build-and-test (pull_request) Successful in 3m19s
Add native_str_to_int (El compiler alias for str_to_int) and
http_post_json_with_headers (JSON POST with additional headers map)
which epm.el generates calls to but were absent from el_runtime.c.
2026-05-06 19:31:35 -05:00
will.anderson 77100649c3 Merge pull request 'fix: use GIT_TOKEN secret in sdk-release.yaml' (#13) from fix/ci-openssl-linker into dev
El SDK CI - stage / build-and-test (pull_request) Successful in 3m46s
El SDK CI - dev / build-and-test (push) Successful in 4m9s
2026-05-07 00:21:20 +00:00
will.anderson b0570656b1 fix: use GIT_TOKEN secret (GITEA_ prefix is reserved)
El SDK CI - dev / build-and-test (pull_request) Successful in 3m25s
2026-05-06 19:17:16 -05:00
will.anderson a79b421578 Merge pull request 'fix: use GITHUB_SHA for artifact version' (#11) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Successful in 3m47s
El SDK CI - stage / build-and-test (pull_request) Successful in 3m26s
2026-05-07 00:09:18 +00:00
will.anderson 0ab9361fab fix: use GITHUB_SHA instead of GITEA_SHA for artifact version
El SDK CI - dev / build-and-test (pull_request) Successful in 3m19s
GITEA_SHA is not set in the runner container environment; GITHUB_SHA is.
Empty version string caused INVALID_ARGUMENT from Artifact Registry.
2026-05-06 19:05:21 -05:00
will.anderson f7953eb73a Merge pull request 'fix: use valid Artifact Registry package IDs (no slashes)' (#10) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Failing after 3m36s
2026-05-07 00:00:20 +00:00
will.anderson 9c350e9f2f fix: use valid Artifact Registry package IDs (no slashes)
El SDK CI - dev / build-and-test (pull_request) Successful in 3m20s
Package IDs must contain only letters, numbers, periods, hyphens and
underscores. el/elc → el-elc, el/el_runtime.c → el-runtime-c, etc.
2026-05-06 18:56:23 -05:00
will.anderson e93c899d1f Merge pull request 'fix: use trusted=yes for gcloud apt source, drop GPG key dance' (#9) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Failing after 3m56s
2026-05-06 23:50:52 +00:00
will.anderson 0b50e61f98 fix: use trusted=yes for gcloud apt source, drop GPG key dance
El SDK CI - dev / build-and-test (pull_request) Successful in 3m18s
The packages.cloud.google.com key format has changed and signature
verification keeps failing in CI. trusted=yes bypasses the ceremony —
we're downloading from a known Google URL so it's fine.
2026-05-06 18:47:05 -05:00
will.anderson f2741e4bdb Merge pull request 'fix: download GCP apt key directly without gpg --dearmor' (#8) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Failing after 3m59s
2026-05-06 23:39:06 +00:00
will.anderson 7fd01b8a8d fix: download GCP apt key directly, no gpg --dearmor
El SDK CI - dev / build-and-test (pull_request) Successful in 6m38s
packages.cloud.google.com now serves the key in binary format; piping
through gpg --dearmor fails with 'no valid OpenPGP data found'.
2026-05-06 18:28:38 -05:00
will.anderson d2940f5d1d Merge pull request 'fix: add --batch to gpg --dearmor in CI publish steps' (#7) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Failing after 10m1s
2026-05-06 23:17:12 +00:00
will.anderson dca741f915 fix: add --batch to gpg --dearmor in publish steps
El SDK CI - dev / build-and-test (pull_request) Successful in 8m50s
gpg tries to open /dev/tty for passphrase input when no TTY is present
in CI, causing the GCP key setup to fail. --batch suppresses interactive
prompts and dearmoring doesn't require one anyway.
2026-05-06 17:58:16 -05:00
will.anderson 9862f4d6e1 Merge pull request 'fix: add -lssl -lcrypto to all CI gcc linker commands' (#6) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Failing after 3m25s
2026-05-06 22:53:37 +00:00
will.anderson 8b074d2e39 fix: normalize NaN to 'nan' in float_to_str regardless of sign bit
El SDK CI - dev / build-and-test (pull_request) Successful in 3m18s
0.0/0.0 can produce -nan on Linux/x86_64 (%g gives '-nan'),
causing the no-cycle calendar test to fail. Explicitly check isnan()
and emit 'nan' so behavior is platform-independent.
2026-05-06 17:49:35 -05:00
will.anderson ec9c322cc7 fix: use elc-linux-amd64 as bootstrap seed instead of elc-bootstrap.c
El SDK CI - dev / build-and-test (pull_request) Failing after 1m57s
elc-bootstrap.c is stale and produces a broken gen2 that can't correctly
compile current El source (generates C without main() for user programs).
The committed elc-linux-amd64 binary is the current, correct seed for
linux CI. This removes the gen2-from-C step entirely.
2026-05-06 17:45:57 -05:00
will.anderson 702093e043 fix: add -lssl -lcrypto -lm to all test runner gcc commands
El SDK CI - dev / build-and-test (pull_request) Failing after 1m7s
Same OpenSSL/math linker flags needed everywhere el_runtime.c is linked.
2026-05-06 17:41:55 -05:00
will.anderson 95b6fac094 fix: use elc-cli.el as gen3 entry point, not compiler.el directly
El SDK CI - dev / build-and-test (pull_request) Failing after 1m8s
compiler.el imports lexer.el/parser.el/codegen.el with bare names; those
resolve relative to the source file's directory only when the entry point
is elc-cli.el (which imports el-compiler/src/compiler.el by full path).
Compiling compiler.el directly leaves lex/parse/codegen as undefined refs.
2026-05-06 17:39:38 -05:00
will.anderson af66cebfd3 fix: add -lssl -lcrypto to all CI gcc linker commands
El SDK CI - dev / build-and-test (pull_request) Failing after 28s
el_runtime.c now uses OpenSSL EVP AEAD encryption; all gcc commands
in all three workflow files need -lssl -lcrypto to link correctly.
2026-05-06 17:36:48 -05:00
will.anderson 1b9bc049de ci: trigger test run 2026-05-06 17:15:37 -05:00
Will Anderson 19a7430ff8 ci-dev: add PR trigger, gate publish on push (post-merge only) 2026-05-06 17:11:28 -05:00
Will Anderson 6b0a77a1dd Restructure CI/CD into proper dev -> stage -> main gated pipeline
- ci-dev.yaml: push to dev only (remove stale PR trigger)
- ci-stage.yaml: PR from dev validates, push to stage publishes to foundation-stage;
  add -lm/-Wl,--allow-multiple-definition flags and all 9 native --test suites
- sdk-release.yaml: add PR to main trigger for validation, gate publish/release/dispatch
  on push (post-merge) only; add -lm flags and all 9 native --test suites to main as well
2026-05-06 17:05:49 -05:00
Will Anderson 17e204ff2b Merge branch 'main' into dev 2026-05-06 17:01:03 -05:00
Will Anderson fc6d496937 fix: correct if-stmt parser test assertions — if is an expression in El 2026-05-06 16:45:01 -05:00
Will Anderson 2f713d64d4 Merge PR #4: perf: 81% RSS reduction in elc compiler + --test mode 2026-05-06 14:35:54 -05:00
Will Anderson 0a2a084d65 Merge PR #3: feat: port remaining foundation El source into monorepo 2026-05-06 14:35:49 -05:00
Will Anderson 7116610ffd Merge PR #2: feat: port el-ui vessels — rename crates→vessels, add El source + manifests 2026-05-06 14:35:45 -05:00
Will Anderson 810d8107da Merge PR #1: ci: fix gen2/gen3 gcc flags and step name formatting 2026-05-06 14:35:30 -05:00
will.anderson 607d5661d8 Merge pull request 'feat: port el-ui vessels — rename crates→vessels, add El source + manifests' (#2) from feat/el-ui-html-emit into dev 2026-05-06 19:34:00 +00:00
will.anderson c4a569f29b Merge pull request 'ci: fix gen2/gen3 gcc flags and step name formatting' (#1) from fix/ci-workflow-flags into dev 2026-05-06 19:33:56 +00:00
Will Anderson ec889e1e53 Add --test mode to elc with Assert stmt and full native test suite passing
Implement compile_test() entry point that emits a C test harness instead
of a normal program. Test blocks (previously skipped) now compile to
static functions with per-assertion pass/fail tracking. Assert statement
added to parser and codegen. Runtime extended with now_ns, fs_list_json,
json_build_object, json_build_array, json_escape_string, state_has,
state_get_or. Fix float negation codegen, float equality comparisons,
time_to_parts return type (JSON string), time_format empty-fmt, json_set
raw-value semantics, state_keys JSON array return. All 310 native tests
pass across 9 suites (core, text, string, math, env, state, json, time, fs).
2026-05-06 14:33:47 -05:00
Will Anderson 6ced0f8009 fix: double-free in engram_neighbors_json BFS + rebuild engram.c
el_strdup tracks pointers in the arena. The BFS arrays in
engram_neighbors_json are manually freed — using el_strdup caused a
double-free when the arena was later popped. Changed to plain strdup
for those allocations.

engram/dist/engram.c rebuilt from engram/src/server.el with current
elc (minor codegen diff: parenthesisation and _argc/_argv rename).
2026-05-06 14:11:40 -05:00
Will Anderson bd7303447b fix: skip test blocks in codegen to prevent OOM on test files
test "name" { ... } blocks were not recognized by the self-hosted
compiler. The body { } was parsed as a Map literal, creating a huge
AST with O(n²) string concatenation in the toplevel_exec_stmts loop
(which had no arena scope). A 272-line test file would consume 400MB+
and a 720-line file importing the full compiler source caused 150GB
usage and crashed the machine.

Two fixes:
1. Skip Test tokens in codegen_streaming before parse_one() —
   advance past "name" and skip_to_rbrace on the body block.
   Test blocks are never compiled; self-hosted compiler has no test runner.

2. Add per-statement arena scope to toplevel_exec_stmts emission loop,
   matching the el_main_body loop. Frees intermediate strings after
   each statement to prevent O(n²) accumulation from any unrecognized
   construct that reaches that path.

Result: test_string.el (272 lines, 27 test blocks): 0MB peak (was 400MB+).
        test_compiler.el (720 lines + 8728 imported): 15MB peak (was 150GB).
2026-05-06 13:34:03 -05:00
Will Anderson e8f6765750 fix: arena leak in compile() — token/sig strings now tracked
Wrapped compile() body in el_arena_push/pop so the arena is active
before lex() and scan_fn_sigs(). Previously both ran with
_tl_arena_active=0, leaking all token and signature strings permanently.
Also prevents inner pop(mark=0) calls from deactivating the arena
between per-function scopes. Verified: self-host PASS, RSS stable.
2026-05-06 10:53:12 -05:00
Will Anderson 3726f69435 perf: 81% RSS reduction — el_release, arena scoping, streaming codegen, libcurl stub
Chain of optimizations from swarm rounds 4-7:
- Flat stride-2 token list: eliminate per-token Map allocation (~112B each × N tokens)
- Systematic el_release() in parser.el: eagerly free intermediate parse result maps
- Per-function and per-statement arena scoping in codegen_streaming()
- Streaming codegen pipeline: parse one fn at a time, emit C, discard AST
- HAVE_CURL guard: elc CLI binary drops libcurl, eliminating SSL/TLS init overhead
- HTML codegen parts-list: O(n) instead of O(n²) string growth for nested templates
- Batch c_escape: str_slice clean runs instead of char-at per byte

Result: 33.4MB → 6.5MB RSS on web/src/main.el (-81%). Self-host: PASS.
2026-05-05 20:39:38 -05:00
Will Anderson ee86736eab merge round-4-delta: flat stride-2 token list + str_char_code dispatch + batch c_escape
- Flat token list: lexer emits [kind0, val0, kind1, val1, ...] instead of [{kind,val}, ...]
  Eliminates per-token ElMap allocation (~112B × N tokens)
- str_char_code hot loop: char classification via Int codes, no strdup per char
- Batch c_escape: str_slice clean runs instead of char-at per byte
- Parser updated to use tok_at/tok_kind/tok_value stride-2 accessors
2026-05-05 20:29:35 -05:00
Will Anderson eb52be4ade runtime: add EL_TRUE/EL_FALSE macros and scoped arena for CLI
Adds EL_TRUE/EL_FALSE convenience macros to el_runtime.h alongside the
existing EL_NULL, making boolean-returning builtins readable without
raw (el_val_t) casts. Documents all value macros in the header comment.

Also lands el_arena_push/el_arena_pop — a scoped string arena for CLI
programs that never call el_request_start/end. The compiler can push a
mark before a compilation unit and pop it after to free intermediate
strings, reducing peak RSS during long compile runs.
2026-05-05 19:15:49 -05:00