Commit Graph

13 Commits

Author SHA1 Message Date
will.anderson e08150d6e7 elb: fix monolithic link, capability violations, liboqs detection
- Link only the entry-point .c (monolithic) instead of all module .c
  files — prevents duplicate symbol errors from inlined imports
- Strip capability-violation #error guards post-compile; they fire
  incorrectly when modules are compiled in isolation but linked into
  a CGI program
- Add liboqs (post-quantum) include/lib detection via brew, matching
  the existing OpenSSL detection pattern
2026-06-05 11:34:28 -05:00
will.anderson 34249b39a3 self-review 2026-05-23: ISE recent-first ordering + http_serve_async builtin
Two improvements:

1. ISE scan ordering — engram_scan_nodes_by_type_json now sorts InternalStateEvent
   nodes by created_at DESC (most-recent-first) instead of salience DESC. Old
   high-salience ISEs (session-start, wm-promotion) no longer dominate offset 0,
   burying recent heartbeat and curiosity_scan events at offsets 20000+. New
   behavior: ?limit=10 returns the 10 most recent ISEs regardless of salience.
   All other node types retain existing salience-sorted behavior.

2. http_serve_async registered as elc builtin — added to builtin_arity() in both
   codegen.el (EL compiler source) and dist/platform/elc.c (compiled C). Also
   rebuilt elc binary from updated elc.c. This closes the fragile-patch gap from
   2026-05-21: elc previously treated http_serve_async as an unknown identifier,
   and the gap description noted elc would 'silently revert to blocking http_serve'
   on next soul rebuild. Now http_serve_async has a proper 2-arg arity entry and
   will survive all future soul recompiles without a manual neuron.c patch.
2026-06-05 11:34:27 -05:00
will.anderson a3ead6552e feat(engram): ML-KEM-1024 PQC encryption, nomic embeddings, MCP routes, checkpoint-per-write
- Add ML-KEM-1024 + AES-256-GCM binary persistence to el_runtime.c with
  two-key scheme (Neuron master + user key); SHAKE-256 key derivation
- Add nomic-embed-text 768-dim float32 embeddings on every node write
  via Ollama; graceful fallback when Ollama is not running
- Wire all /api/neuron/* MCP routes directly into Engram (server.el),
  eliminating the Kotlin server as the MCP backend
- Set ENGRAM_CHECKPOINT_INTERVAL = 1 (write binary on every node write,
  not every 50)
- Add el_runtime.h declarations for engram_write_binary_el and
  engram_load_binary_el builtins
2026-06-05 11:34:27 -05:00
will.anderson c2cd5e01e1 fix: elb macOS OpenSSL + C master declarations header; add ELP missing imports
El SDK CI - dev / build-and-test (pull_request) Successful in 3m34s
elb.el:
- Auto-detect Homebrew OpenSSL (-L$(brew --prefix openssl)/lib) so -lssl
  resolves on macOS without manual flags; no-op on Linux
- Add -include elp-c-decls.h when present in out_dir: resolves undeclared
  cross-module calls in packages like ELP that lack explicit imports

ELP source:
- Add import "morphology.el" to all 29 language morphology modules
- Add language module imports to morphology.el (all langs it dispatches to)
  These were missing since ELP was originally built as a monolithic unit
2026-05-08 19:44:31 -05:00
will.anderson 1f4b594ae7 feat(elb): c_source manifest directive + macOS OpenSSL path detection
Add `c_source "path"` in manifest.el build block — lets packages link
extra C files (platform stubs, native glue) without touching elb source.

On macOS, homebrew OpenSSL isn't on the default linker path. Detect it
via `brew --prefix` and inject -L/-I flags; no-op on Linux.

Rebuild elb binary; remove elc-new binary (elc is now canonical).
2026-05-08 11:31:36 -05:00
will.anderson f5dcca0386 build: update dist/platform/elc with OOM fix and memory guard
El SDK CI - dev / build-and-test (pull_request) Successful in 4m16s
Rebuilt from fix/elc-oom-checkout: scan_fn_sigs_el() --emit-header path
+ el_mem_check() guard. Verified on checkout.el: all 3 sigs in .elh,
clean exit under normal load, exit(1) on memory limit exceeded.
2026-05-08 08:23:07 -05:00
Will Anderson ec889e1e53 Add --test mode to elc with Assert stmt and full native test suite passing
Implement compile_test() entry point that emits a C test harness instead
of a normal program. Test blocks (previously skipped) now compile to
static functions with per-assertion pass/fail tracking. Assert statement
added to parser and codegen. Runtime extended with now_ns, fs_list_json,
json_build_object, json_build_array, json_escape_string, state_has,
state_get_or. Fix float negation codegen, float equality comparisons,
time_to_parts return type (JSON string), time_format empty-fmt, json_set
raw-value semantics, state_keys JSON array return. All 310 native tests
pass across 9 suites (core, text, string, math, env, state, json, time, fs).
2026-05-06 14:33:47 -05:00
Will Anderson bd7303447b fix: skip test blocks in codegen to prevent OOM on test files
test "name" { ... } blocks were not recognized by the self-hosted
compiler. The body { } was parsed as a Map literal, creating a huge
AST with O(n²) string concatenation in the toplevel_exec_stmts loop
(which had no arena scope). A 272-line test file would consume 400MB+
and a 720-line file importing the full compiler source caused 150GB
usage and crashed the machine.

Two fixes:
1. Skip Test tokens in codegen_streaming before parse_one() —
   advance past "name" and skip_to_rbrace on the body block.
   Test blocks are never compiled; self-hosted compiler has no test runner.

2. Add per-statement arena scope to toplevel_exec_stmts emission loop,
   matching the el_main_body loop. Frees intermediate strings after
   each statement to prevent O(n²) accumulation from any unrecognized
   construct that reaches that path.

Result: test_string.el (272 lines, 27 test blocks): 0MB peak (was 400MB+).
        test_compiler.el (720 lines + 8728 imported): 15MB peak (was 150GB).
2026-05-06 13:34:03 -05:00
Will Anderson e8f6765750 fix: arena leak in compile() — token/sig strings now tracked
Wrapped compile() body in el_arena_push/pop so the arena is active
before lex() and scan_fn_sigs(). Previously both ran with
_tl_arena_active=0, leaking all token and signature strings permanently.
Also prevents inner pop(mark=0) calls from deactivating the arena
between per-function scopes. Verified: self-host PASS, RSS stable.
2026-05-06 10:53:12 -05:00
Will Anderson 3726f69435 perf: 81% RSS reduction — el_release, arena scoping, streaming codegen, libcurl stub
Chain of optimizations from swarm rounds 4-7:
- Flat stride-2 token list: eliminate per-token Map allocation (~112B each × N tokens)
- Systematic el_release() in parser.el: eagerly free intermediate parse result maps
- Per-function and per-statement arena scoping in codegen_streaming()
- Streaming codegen pipeline: parse one fn at a time, emit C, discard AST
- HAVE_CURL guard: elc CLI binary drops libcurl, eliminating SSL/TLS init overhead
- HTML codegen parts-list: O(n) instead of O(n²) string growth for nested templates
- Batch c_escape: str_slice clean runs instead of char-at per byte

Result: 33.4MB → 6.5MB RSS on web/src/main.el (-81%). Self-host: PASS.
2026-05-05 20:39:38 -05:00
Will Anderson 7f295bffe9 fix: codegen O(n²) HTML memory leak + elb stderr surface + runtime dir path 2026-05-05 14:40:15 -05:00
Will Anderson 962c8cbe57 dist: add linux/amd64 binaries and el_runtime.js 2026-05-05 09:44:25 -05:00
Will Anderson 1ae68962cf restructure: move el compiler content into lang/ 2026-05-05 01:38:51 -05:00