runtime: ARC scaffolding + indirection so el_list_append amortizes O(1)

The compiler used to OOM at ~8.7 GB on 4325-line inputs because every
el_list_append allocated a fresh ElList header + elements array. That
was the workaround for an aliasing bug in cg_if_stmt — codegen held a
stale pointer through a realloc. Persistent semantics fixed the bug
but turned every accumulator (decl in cg_stmts, AST construction, the
__int_names CSV) into O(N²) memory.

Real fix in two coordinated parts:

1. Runtime — ElList and ElMap now carry a magic-tagged ElHeader at
   offset 0 (uint32 magic, uint32 refcount). The payload arrays live in
   separate heap allocations behind a stable header pointer, so realloc-
   grow on append never invalidates the caller's reference. el_list_append
   and el_map_set mutate in place when refcount <= 1 (the common single-
   owner case, amortized O(1)) and copy-on-write when shared. Adds
   el_list_clone for explicit shallow copies, plus el_retain/el_release
   no-op-on-non-pointers so codegen can emit them on every let-binding
   without tracking types. The magic words (0xE1xxxxxx) live above the
   printable-ASCII range so they can never collide with a string's first
   byte, and looks_like_string in json_stringify already rejects them.

2. Codegen — every place that delegates to a child C scope now clones
   `declared` before passing it down: cg_if_stmt for both then/else
   branches, cg_for_body for the loop body (which also picks up the
   loop variable via append), and cg_stmt's While case. Without the
   clones, mutation-in-place would let a sibling scope's let-bindings
   leak into the parent's declared list and the parent would emit
   `x = ...` against an undeclared name. The clones are cheap shallow
   copies of a list of strings.

Result on the landing-combined.el (4325 lines): 8.7 GB → 3.5 GB peak,
0.26s wall clock, compile completes successfully where it previously
OOM'd. Self-hosting fixpoint reached: dist/platform/elc compiled from
elc-combined.el reproduces dist/platform/elc.c byte-for-byte on a
second pass through itself.

Strings still allocate fresh on every concat; that's the next layer of
optimization (probably an arena tied to function scope) but isn't
blocking. The persistent-list aliasing bug remains structurally fixed —
clones are explicit at the codegen sites where the persistence
guarantee matters; everywhere else the compiler runs at mutation speed.
This commit is contained in:
Will Anderson
2026-04-30 15:05:02 -05:00
parent 04ecd1aafe
commit 23bbc99e43
6 changed files with 270 additions and 56 deletions
+6 -4
View File
@@ -2185,7 +2185,7 @@ el_val_t cg_stmt(el_val_t stmt, el_val_t indent, el_val_t declared) {
el_val_t cond_c = cg_expr(cond);
cond_c = strip_outer_parens(cond_c);
emit_line(el_str_concat(el_str_concat(el_str_concat(indent, EL_STR("while (")), cond_c), EL_STR(") {")));
cg_stmts(body, el_str_concat(indent, EL_STR(" ")), declared);
cg_stmts(body, el_str_concat(indent, EL_STR(" ")), native_list_clone(declared));
emit_line(el_str_concat(indent, EL_STR("}")));
return declared;
}
@@ -2266,10 +2266,10 @@ el_val_t cg_if_stmt(el_val_t expr, el_val_t indent, el_val_t declared) {
el_val_t cond_c = cg_expr(cond);
cond_c = strip_outer_parens(cond_c);
emit_line(el_str_concat(el_str_concat(el_str_concat(indent, EL_STR("if (")), cond_c), EL_STR(") {")));
cg_stmts(then_stmts, el_str_concat(indent, EL_STR(" ")), declared);
cg_stmts(then_stmts, el_str_concat(indent, EL_STR(" ")), native_list_clone(declared));
if (has_else) {
emit_line(el_str_concat(indent, EL_STR("} else {")));
cg_stmts(else_stmts, el_str_concat(indent, EL_STR(" ")), declared);
cg_stmts(else_stmts, el_str_concat(indent, EL_STR(" ")), native_list_clone(declared));
}
emit_line(el_str_concat(indent, EL_STR("}")));
return 0;
@@ -2285,7 +2285,9 @@ el_val_t cg_for_body(el_val_t item, el_val_t list_expr, el_val_t body, el_val_t
emit_line(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(indent, EL_STR(" el_val_t ")), len_tmp), EL_STR(" = el_list_len(")), list_tmp), EL_STR(");")));
emit_line(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(indent, EL_STR(" for (el_val_t ")), idx), EL_STR(" = 0; ")), idx), EL_STR(" < ")), len_tmp), EL_STR("; ")), idx), EL_STR("++) {")));
emit_line(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(el_str_concat(indent, EL_STR(" el_val_t ")), item), EL_STR(" = el_list_get(")), list_tmp), EL_STR(", ")), idx), EL_STR(");")));
cg_stmts(body, el_str_concat(indent, EL_STR(" ")), declared);
el_val_t body_decl = native_list_clone(declared);
body_decl = native_list_append(body_decl, item);
cg_stmts(body, el_str_concat(indent, EL_STR(" ")), body_decl);
emit_line(el_str_concat(indent, EL_STR(" }")));
emit_line(el_str_concat(indent, EL_STR("}")));
return 0;