runtime: ARC scaffolding + indirection so el_list_append amortizes O(1)

The compiler used to OOM at ~8.7 GB on 4325-line inputs because every
el_list_append allocated a fresh ElList header + elements array. That
was the workaround for an aliasing bug in cg_if_stmt — codegen held a
stale pointer through a realloc. Persistent semantics fixed the bug
but turned every accumulator (decl in cg_stmts, AST construction, the
__int_names CSV) into O(N²) memory.

Real fix in two coordinated parts:

1. Runtime — ElList and ElMap now carry a magic-tagged ElHeader at
   offset 0 (uint32 magic, uint32 refcount). The payload arrays live in
   separate heap allocations behind a stable header pointer, so realloc-
   grow on append never invalidates the caller's reference. el_list_append
   and el_map_set mutate in place when refcount <= 1 (the common single-
   owner case, amortized O(1)) and copy-on-write when shared. Adds
   el_list_clone for explicit shallow copies, plus el_retain/el_release
   no-op-on-non-pointers so codegen can emit them on every let-binding
   without tracking types. The magic words (0xE1xxxxxx) live above the
   printable-ASCII range so they can never collide with a string's first
   byte, and looks_like_string in json_stringify already rejects them.

2. Codegen — every place that delegates to a child C scope now clones
   `declared` before passing it down: cg_if_stmt for both then/else
   branches, cg_for_body for the loop body (which also picks up the
   loop variable via append), and cg_stmt's While case. Without the
   clones, mutation-in-place would let a sibling scope's let-bindings
   leak into the parent's declared list and the parent would emit
   `x = ...` against an undeclared name. The clones are cheap shallow
   copies of a list of strings.

Result on the landing-combined.el (4325 lines): 8.7 GB → 3.5 GB peak,
0.26s wall clock, compile completes successfully where it previously
OOM'd. Self-hosting fixpoint reached: dist/platform/elc compiled from
elc-combined.el reproduces dist/platform/elc.c byte-for-byte on a
second pass through itself.

Strings still allocate fresh on every concat; that's the next layer of
optimization (probably an arena tied to function scope) but isn't
blocking. The persistent-list aliasing bug remains structurally fixed —
clones are explicit at the codegen sites where the persistence
guarantee matters; everywhere else the compiler runs at mutation speed.
This commit is contained in:
Will Anderson
2026-04-30 15:05:02 -05:00
parent 04ecd1aafe
commit 23bbc99e43
6 changed files with 270 additions and 56 deletions
+17 -4
View File
@@ -600,7 +600,10 @@ fn cg_stmt(stmt: Map<String, Any>, indent: String, declared: [String]) -> [Strin
let cond_c: String = cg_expr(cond)
let cond_c = strip_outer_parens(cond_c)
emit_line(indent + "while (" + cond_c + ") {")
cg_stmts(body, indent + " ", declared)
// Body lives in its own C block clone so let-bindings inside the
// loop don't leak into the parent's `declared` list (which would make
// a sibling scope's `let x` emit assignment on an undeclared name).
cg_stmts(body, indent + " ", native_list_clone(declared))
emit_line(indent + "}")
return declared
}
@@ -670,10 +673,15 @@ fn cg_if_stmt(expr: Map<String, Any>, indent: String, declared: [String]) -> Voi
let cond_c: String = cg_expr(cond)
let cond_c = strip_outer_parens(cond_c)
emit_line(indent + "if (" + cond_c + ") {")
cg_stmts(then_stmts, indent + " ", declared)
// Each branch gets its own clone of `declared` variables let-bound
// inside the then/else block live only in that C scope, and must not
// leak back to the parent (or to the sibling branch) through shared
// list mutation. Cheap shallow copy; the entries (variable name strings)
// are shared.
cg_stmts(then_stmts, indent + " ", native_list_clone(declared))
if has_else {
emit_line(indent + "} else {")
cg_stmts(else_stmts, indent + " ", declared)
cg_stmts(else_stmts, indent + " ", native_list_clone(declared))
}
emit_line(indent + "}")
}
@@ -688,7 +696,12 @@ fn cg_for_body(item: String, list_expr: Map<String, Any>, body: [Map<String, Any
emit_line(indent + " el_val_t " + len_tmp + " = el_list_len(" + list_tmp + ");")
emit_line(indent + " for (el_val_t " + idx + " = 0; " + idx + " < " + len_tmp + "; " + idx + "++) {")
emit_line(indent + " el_val_t " + item + " = el_list_get(" + list_tmp + ", " + idx + ");")
cg_stmts(body, indent + " ", declared)
// Body lives inside its own C block; the loop variable and any locally
// let-bound names go out of scope at the closing brace, so we mustn't
// pollute the parent's `declared` with them.
let body_decl = native_list_clone(declared)
let body_decl = native_list_append(body_decl, item)
cg_stmts(body, indent + " ", body_decl)
emit_line(indent + " }")
emit_line(indent + "}")
}