Commit Graph

118 Commits

Author SHA1 Message Date
Will Anderson e8b22e16a2 add el-tests vessel: manifest-based test suite under tests/suite/
Restructures the test suite as a proper El vessel with manifest.el and
src/ layout, eliminating the bash run.sh harness. CI runs the suite with
two commands: `cd tests/suite && elb && ./dist/el-tests`. Exit code is
the fail count (0 = all pass).

163 test cases across 7 modules: string (52), math (13), json (26),
state (11), time (25), fs (16), collections (19).
2026-05-04 19:53:35 -05:00
Will Anderson 3086a56b5c enforce source branch in CI: stage←dev, main←stage 2026-05-04 19:34:34 -05:00
Will Anderson 7c644b8d89 Add dev/stage CI pipelines and expand sdk-release to full prod pipeline
- Add ci-dev.yaml: builds elc gen2→gen3, runs 4 test suites, publishes
  el/elc + el_runtime.c + el_runtime.h to foundation-dev Artifact Registry
- Add ci-stage.yaml: same as dev but targets foundation-stage registry
- Update sdk-release.yaml: publish 3 SDK artifacts to foundation-prod
  Artifact Registry after Gitea release; expand dispatch list from 2 to 6
  downstream repos (el-ui, elp, elql, el-ide added alongside engram/forge)
2026-05-04 19:32:05 -05:00
Will Anderson 9e8d23bcd9 add epm — El Package Manager
Introduces epm/, a new component written entirely in native El.
epm manages vessels (El's deployable package format): publish to Engram,
install with full dependency resolution, list registry contents, and
inspect vessel metadata.

- epm/manifest.el         — package manifest
- epm/src/manifest.el     — vessel/package manifest parser (line-by-line,
                            same approach as elb.el)
- epm/src/registry.el     — Engram-backed vessel registry (POST /api/nodes,
                            GET /api/search); vessels stored as Entity nodes
                            with label "vessel:<name>:<version>"
- epm/src/install.el      — topological dependency resolver with cycle
                            detection; installs to .epm/vessels/<name>/
- epm/src/epm.el          — main entry point: publish / install / list / info
2026-05-04 19:31:24 -05:00
Will Anderson 0791fda43e elb: add -lm to link flags — el_runtime.c uses math.h functions
el_runtime.c includes <math.h> and calls pow(), sqrt(), log() in several
places (math operations, engram dampening, float formatting). Without -lm
the linker fails on Linux when linking programs built with elb.
2026-05-04 19:14:09 -05:00
Will Anderson 53f2df500d runtime: add dharma-required functions to el_runtime.c and runtime/*.el
Add the following functions that dharma registry calls but were missing
from the El runtime:

el_runtime.c (consumed by the old build system via released SDK):
  - list_len, list_get — aliases for el_list_len/el_list_get (handlers.el)
  - json_array_push — append pre-encoded element to JSON array string
  - now_millis, unix_timestamp_ms, time_now_ms — ms-since-epoch aliases
  - log_info, log_warn — structured stderr log helpers
  - config — reads config from environment (alias for getenv)
  - http_patch — HTTP PATCH with Content-Type: application/json
  - http_post_engram — HTTP POST with optional X-API-Key header
  - http_get_engram — HTTP GET with optional X-API-Key header
  - str_to_bytes — encode string as JSON byte array [72,101,...]
  - bytes_to_str — decode JSON byte array back to string
  - hash_sha256 — SHA-256 hex digest using built-in sha256 impl

runtime/*.el (consumed by the new build system):
  - http.el: http_patch, http_post_engram, http_get_engram
  - time.el: now_millis, unix_timestamp_ms, time_now_ms
  - env.el: config, log_info, log_warn, list_len, list_get
  - json.el: json_array_push, bytes_to_str
  - string.el: str_to_bytes, hash_sha256 (via __sha256_hex seed)

el_seed.h / el_seed.c:
  - __sha256_hex primitive with self-contained SHA-256 implementation
2026-05-04 19:07:08 -05:00
Will Anderson 245eb2898e runtime: declare __thread_create and __thread_join in header for C99 compliance 2026-05-03 18:00:47 -05:00
Will Anderson 7bfe30b767 elb: raise clang bracket depth to 1024 — fixes compile failure for large JS string renders 2026-05-03 17:37:56 -05:00
Will Anderson 6ede9e4379 runtime: restore el_runtime.c as build shim; fix el_seed.h self-contained types
el_runtime.c was deleted prematurely — elb still resolves the runtime at build time
via a hardcoded relative path, and the elc code generator still emits
#include "el_runtime.h" in generated C.

Restoring el_runtime.c + el_runtime.h as the working build runtime until the
compiler is updated to emit #include "el_seed.h" and link against el_seed.c
directly.

el_seed.h: remove #include "el_runtime.h" that broke after el_runtime.h deletion;
add inline el_val_t typedef + macros + float cast helpers so el_seed.h is fully
self-contained.
2026-05-03 17:35:14 -05:00
Will Anderson 4ae42ee7db runtime: native SSE streaming — http_sse_open/send/close
Add Server-Sent Events support to the El runtime. El v2 handlers can now
hold HTTP connections open and push events in real time.

New builtins in el_seed.c:
  __http_conn_fd()          — retrieve raw fd from thread-local set by worker
  __http_sse_open(fd)       — send SSE headers (text/event-stream), keep-alive
  __http_sse_send(fd, data) — write "data: <data>\n\n" frame
  __http_sse_close(fd)      — close the connection fd

http_worker_v2 in legacy/el_runtime.c now:
  - stashes the fd via el_seed_set_http_conn_fd() before calling the handler
  - detects the "__sse__" sentinel return value to skip http_send_response
    and skip close(fd) — SSE handler took ownership of the fd
  - clears the thread-local after the handler returns

El wrappers added to runtime/http.el:
  http_conn_fd() http_sse_open(fd) http_sse_send(fd, data)
  http_sse_close(fd) http_sse_sentinel()
2026-05-03 17:15:37 -05:00
Will Anderson 3e5130e98d remove el_runtime.c — runtime is 100% native El
el_runtime.c and el_runtime.h removed from the active runtime directory
(archived copies remain in el-compiler/runtime/legacy/).
tools/lsp/build.sh removed as it depended on el_runtime.c directly.
AGENTS.md updated to reflect el_seed.c as the sole C dependency.
2026-05-03 17:10:04 -05:00
Will Anderson beb4e436e1 archive el_runtime.c — native El runtime complete, seed is self-contained
el_seed.c now defines el_request_start/el_request_end directly (delegating
to its own seed arena) rather than declaring them as externs from el_runtime.c.
Header comment updated to reflect self-contained build.
2026-05-03 17:08:49 -05:00
Will Anderson 71a1e41f93 lsp: type-aware field completions — scan type/let/param decls, complete on dot-access
When a document is opened or changed, scan for `type Name { field: Type }` blocks
and `let var: Type` / `fn foo(param: Type)` annotations. On completion requests,
if the text before the cursor ends with `identifier.`, look up the variable's type
and return its fields as Field (kind=5) completion items instead of the full list.
2026-05-03 16:16:11 -05:00
Will Anderson 0676725cb7 Bundle El runtime as installable framework
- runtime/stdlib.el: master import file for the full El standard library
  in correct dependency order (string→math→time→env→fs→exec→json→http→
  state→thread→channel→engram→manifest); test.el excluded (dev-only)

- tools/install.sh: installs El to a prefix (default /usr/local/el);
  copies elc binary, runtime .el files, headers, compiles libel.a from
  el_seed.c + el_runtime.c, generates an installed stdlib.el with absolute
  paths

- tools/new-project.sh: scaffolds a new El project with src/main.el,
  build.sh (auto-discovers local elc/runtime), README.md, .gitignore;
  verified working end-to-end

- AGENTS.md: fix elc rebuild docs — elc writes to stdout, not to its second
  argument; correct usage is ./dist/platform/elc elc-cli.el > elc-new.c
2026-05-03 16:04:26 -05:00
Will Anderson d98a968f89 rebuild elc — add __read_n and __print_raw for LSP stdin framing (self-hosting verified) 2026-05-03 16:03:34 -05:00
Will Anderson 0c9154551f merge tools/lsp (full) — 184 completions, hover, go-to-def, live diagnostics, enhanced VSCode extension 2026-05-03 16:01:11 -05:00
Will Anderson 9aa0c49d0c add full El LSP — completions, hover, go-to-def, diagnostics, VSCode extension 2026-05-03 15:59:42 -05:00
Will Anderson 59b96e3324 rebuild elc from sprint-integrated source — all language features verified
Self-hosting sprint merges compiled in:
- compiler: break/continue, % modulo, for-range loops, match-stmt codegen, string interpolation
- runtime: threading, http, json, fs/exec/env, time/math/state, engram, string (57 fns), channels
- seed: el_seed.c minimal C OS boundary (968 lines)
- tools: LSP skeleton with JSON-RPC, completions, hover, go-to-def, VSCode extension
- tests: El test framework with state-backed runner + string test suite

Self-hosting verified: elc-new → C → binary → same C output (idempotent).
Provenance snapshot: dist/platform/elc.20260503-1555-post-sprint
2026-05-03 15:55:50 -05:00
Will Anderson 287b39f8e6 merge tools/lsp — El LSP skeleton: JSON-RPC, completions, hover, go-to-def, VSCode extension 2026-05-03 15:53:39 -05:00
Will Anderson c45744d8ca merge runtime/seed — el_seed.c minimal C OS boundary (968 lines) 2026-05-03 15:52:21 -05:00
Will Anderson 2e778ca664 merge compiler/string-interp — string interpolation via lexer desugaring 2026-05-03 15:52:21 -05:00
Will Anderson 641227a7d3 merge runtime/channels — MPMC buffered channels, channel_pipeline, channel_fan_out 2026-05-03 15:52:21 -05:00
Will Anderson 805318e2d0 merge runtime/test — El test framework with state-backed runner and string test suite 2026-05-03 15:52:21 -05:00
Will Anderson cefff5b891 add El LSP skeleton — language server, VSCode extension, syntax highlighting
Implements a Language Server Protocol server for El files over stdin/stdout
JSON-RPC. Provides completions (builtins + keywords + document functions),
hover documentation, go-to-definition, and full document sync.

Also adds a VSCode extension that launches the binary as a child process and
a TextMate grammar for .el syntax highlighting.

NOTE: el-lsp.el calls __read_n(n: Int) -> String, a seed primitive not yet
in el_runtime.c. Build.sh documents the required C implementation; the seed
agent must add it before the binary will link.
2026-05-03 15:52:10 -05:00
Will Anderson ce9a2caff4 add string interpolation to El ("hello ${name}")
Lexer gains scan_interp_string which replaces scan_string in the main
lex loop. When no ${ is found it behaves identically to before (single
Str token). When interpolations are present it emits a flat token
sequence — Str, Plus, (expr tokens), Plus, Str, … — that the existing
parse_binop / cg_expr BinOp-Plus-string path assembles into nested
el_str_concat calls with zero parser or codegen changes.

Key design choices:
- scan_interp_brace tracks { depth so fn(a, b) inside ${} is safe
- inner expr tokens are wrapped in ( ) so operators like + in ${n+1}
  do not associate with the surrounding concat Plus tokens
- \$ escapes to a literal dollar sign; bare $ not before { passes through
- empty ${} emits an empty string segment
2026-05-03 15:50:23 -05:00
Will Anderson d1af4b0f8b add channels to El — buffered MPMC channel with send/recv/close
Introduces Go-style channels as El's mid-flight communication primitive,
completing the threading model: threads can now not only spawn/join but
also communicate while running.

Part 1 — seed layer (el_runtime.c / el_runtime.h):
- Add __thread_create/__thread_join/__mutex_new/__mutex_lock/__mutex_unlock
  as C seed primitives (dlsym-based thread dispatch, pthread mutex table)
- Add __channel_new/__channel_send/__channel_recv/__channel_try_recv/__channel_close
  as MPMC channel seed primitives backed by mutex + condvar + circular buffer
- Bounded channels (cap > 0): circular buffer, sender blocks when full
- Unbounded channels (cap == 0): dynamic array, grows on demand, never blocks
- channel_close wakes all blocked recvers/senders; recv drains then returns ""

Part 2 — El API (runtime/channel.el):
- channel_new/send/recv/try_recv/close — thin wrappers over seed layer
- channel_pipeline — spawn N worker threads reading from in_ch, applying
  fn_name, writing to out_ch; workers exit on "" sentinel from close
- channel_drain — collect all messages from a closed channel into [String]
- channel_fan_out — send a [String] list into a channel then close it

Part 3 — codegen.el:
- Register all 10 seed builtins (__thread_* + __channel_*) in builtin_arity
  so the arity checker validates call sites at compile time
2026-05-03 15:50:13 -05:00
Will Anderson 282df712a8 add runtime/test.el — El test framework with assertions and runner
Provides assert_true/false, assert_eq, assert_int_eq, assert_neq,
assert_contains, assert_starts_with, assert_ends_with, and fail.
Test cases are registered by name+fn_name, executed sequentially via
the thread/dlsym dispatch mechanism, with results tracked in state_.
Includes tests/runtime/string_test.el covering all 23 string.el exports.
2026-05-03 15:49:01 -05:00
Will Anderson cfcedff7f4 implement match statement codegen in El
Add cg_match_stmt() to lower match-as-statement to proper C if/else if/else
chains. Previously, match in statement position fell through to cg_expr() which
emitted a GCC statement-expression — fine for expression arms but wrong for the
statement form. Now matched using the same dispatch pattern as If and For in the
Expr handler of cg_stmt().

Pattern dispatch mirrors cg_match (expression form):
  LitStr  -> str_eq(subj, EL_STR("..."))
  LitInt  -> subj == N
  LitBool -> subj == 1 / 0
  Binding -> else { el_val_t name = subj; body; }
  Wildcard -> else { body; }

Subject is evaluated once into a scoped temporary to avoid double evaluation.
2026-05-03 15:47:50 -05:00
Will Anderson eab483ed4f add el_seed.c — minimal C OS boundary for El runtime migration
Introduces el_seed.c / el_seed.h as the clean OS-boundary layer for new-generation
El programs. All public symbols use the __ prefix convention; el_val_t (int64_t) is
the universal value type throughout.

Key additions over el_runtime.c:
- __thread_create / __thread_join: pthreads + dlsym(RTLD_DEFAULT) parallelism
  foundation. Static ElThread table (64 slots); worker resolves El fn symbols at
  runtime, stores result string for join to return.
- __mutex_new / __mutex_lock / __mutex_unlock: pooled pthread_mutex_t handles
- __http_do: unified curl call with JSON headers string (vs ElMap) and explicit
  timeout_ms parameter
- __fs_list_raw: returns newline-separated filename string (not ElList)
- __str_char_at: returns Int byte value (not single-char String)
- __args_json: CLI args as JSON array string; seeded by el_seed_init_args()

JSON, state, engram, HTML/URL, serve — thin __ wrappers over el_runtime.c.
Private seed arena (parallel to el_runtime.c arena) for standalone use.
2026-05-03 15:45:52 -05:00
Will Anderson cfa4301026 merge compiler features: break/continue, % modulo, for-range loops 2026-05-03 15:45:45 -05:00
Will Anderson f271f9d9d8 add for-range loops to El (for i in 0..n)
Adds `for i in start..end` (exclusive) and `for i in start..=end`
(inclusive) range loop syntax. Existing `for item in list` iteration
is preserved; the parser branches on DotDot/DotDotEq presence after
the start expression. Lexer adds DotDot and DotDotEq tokens with
longer-match-first priority. Codegen emits a C `for` loop with the
loop variable scoped to the statement; inclusive uses `<=`, exclusive `<`.
2026-05-03 15:44:58 -05:00
Will Anderson 49a8a1c24b add % modulo operator to El lexer, parser, codegen
Lexer and parser already had Percent token and precedence on the
compiler/string-interp branch. This commit adds the missing is_int_expr
case for Percent so that modulo expressions over Int operands are
correctly typed as Int (enabling arithmetic dispatch rather than
falling through to string concat or untyped paths).

binop_to_c already mapped Percent -> % at HEAD; only is_int_expr
needed the Percent arm.
2026-05-03 15:44:19 -05:00
Will Anderson 252ad04c96 add break and continue statements to El 2026-05-03 15:43:20 -05:00
Will Anderson 3cc9b1cc3d merge runtime/util — time, math, state in El 2026-05-03 15:40:55 -05:00
Will Anderson 5678745381 add runtime/time.el, math.el, state.el — time, math, and state in El
Migrates the time, math/float, and in-process state surfaces from
el-compiler/runtime/legacy/el_runtime.c to self-hosted El source:

- runtime/time.el: time_now, sleep_secs/ms, time_to_parts (via pure-El
  Gregorian civil_from_days decomposition), time_format (ISO + strftime
  subset), time_add, time_diff, time_from_parts; full Instant/Duration
  nanosecond API (now, unix_seconds/millis, duration_seconds/millis,
  instant_to_iso8601, sleep_duration); TTL cache (ttl_cache_set/get/age
  backed by state); uuid_new / uuid_v4 via __uuid_v4 seed.

- runtime/math.el: el_abs, el_max, el_min (Int); math_sqrt/log/ln/sin/cos/pi
  (Float seed wrappers); float_to_str, int_to_float, float_to_int, str_to_float,
  format_float (__format_float seed), decimal_round (half-away-from-zero via
  pure-El _pow10/_floor_f helpers).

- runtime/state.el: state_set/get/del/keys thin wrappers over __state_* seeds;
  convenience helpers state_has and state_get_or.
2026-05-03 15:39:48 -05:00
Will Anderson e853f4a25c merge runtime/string — all string operations in El (57 functions) 2026-05-03 15:37:51 -05:00
Will Anderson 5b6915ec9e merge runtime/engram-build — engram wrappers, manifest, seed arity table 2026-05-03 15:37:42 -05:00
Will Anderson b0e38a245a merge runtime/fs-exec-env — filesystem, subprocess, environment in El 2026-05-03 15:37:42 -05:00
Will Anderson 84b5355fce merge runtime/json — JSON operations in El 2026-05-03 15:37:41 -05:00
Will Anderson b547d1daf8 merge runtime/http — HTTP client and server in El 2026-05-03 15:37:41 -05:00
Will Anderson cc6522c69d merge runtime/thread — native El threading model, parallel_map 2026-05-03 15:37:41 -05:00
Will Anderson 5807de835e add runtime/string.el — string operations implemented in El
All string, I/O, math, classification, splitting, joining, counting, padding,
and URL encoding functions from el_runtime.c implemented in El using seed
primitives. No C required; compiles via the normal El pipeline.
2026-05-03 15:37:33 -05:00
Will Anderson 33af4ed09e add runtime/engram.el, manifest.el; register seed builtins in codegen arity table
- runtime/engram.el: thin El wrappers over all __engram_* and __generate
  seed primitives (16 functions), matching the el_seed.c API exactly
- runtime/manifest.el: build manifest documenting module load order and
  the cat+compile+cc command for runtime builds
- el-compiler/src/codegen.el: add 77 __-prefix seed primitive entries to
  builtin_arity, covering str, fs, http, thread, exec, env, time, uuid,
  math, state, html, json, and engram seeds
2026-05-03 15:37:05 -05:00
Will Anderson f2c63f95fd add runtime/fs.el, exec.el, env.el — filesystem, subprocess, environment in El
Migrates fs_read/write/exists/mkdir/write_bytes/list, exec/exec_bg/exec_command/exec_capture,
env/args/exit_program, state_set/get/del/keys, uuid_new/v4, and list helpers get/len from
el_runtime.c into El source as thin wrappers over seed primitives.
2026-05-03 15:35:45 -05:00
Will Anderson 01849c2033 add runtime/thread.el — native El threading model with parallel_map
Introduces El's first-class threading primitives built on the seed layer's
__thread_create/__thread_join/mutex ops. parallel_map is the key deliverable:
spawns one thread per item, joins in order — replaces bash fan-out for room
dispatch and any other concurrent HTTP workload.
2026-05-03 15:35:31 -05:00
Will Anderson 56724325ed add runtime/json.el — JSON operations in El
Thin El wrappers over seed JSON primitives (json_get, json_get_raw,
json_parse, json_stringify, json_set, json_array_len, json_array_get,
json_array_get_string) plus typed extractors (json_get_string/int/float/bool)
and pure-El builders (json_build_object, json_build_array,
json_escape_string) that require no seed call.
2026-05-03 15:35:20 -05:00
Will Anderson fea830cca2 add runtime/http.el — HTTP client and server in El
Thin El wrappers over seed primitives that form the public HTTP API for
El programs. Covers GET/POST/DELETE, header-map variants, binary streaming
to file, form-auth, v1/v2 server dispatch, and http_response envelope
construction. Documents two new seed primitives needed: __http_do_map and
__http_do_map_to_file (ElMap-accepting variants to avoid needing map
iteration in El).
2026-05-03 15:35:04 -05:00
Will Anderson f9cfe43f05 preserve original el_runtime.c/h in legacy/ for reference 2026-05-03 15:31:35 -05:00
Will Anderson f97354e96b add exec() and exec_bg() builtins to El runtime
- exec(cmd) -> String: runs shell command, captures stdout, 30s timeout
- exec_bg(cmd) -> String: forks command in background, returns PID string
- add both to codegen arity table (builtin_arity)
- rebuild elc with updated arity table (self-hosting, identity-verified)
- update release snapshot at releases/v1.0.0-20260501/
2026-05-03 02:57:53 -05:00
Will Anderson 9d0e1f64d4 fix elb: cp instead of mv for .elh files, preserves headers for downstream modules v1.2.1 2026-05-03 01:19:58 -05:00