parser: bound token reads to Eof so malformed input errors instead of OOMing
El SDK Release / build-and-release (pull_request) Failing after 16s
El SDK Release / build-and-release (pull_request) Failing after 16s
Out-of-range tok_kind/tok_value reads returned runtime null (el_list_get OOB -> 0) rather than the Eof sentinel, so the inner parse loops (parse_block, call-arg, array-literal, match-arm) that terminate only on their close delimiter or k=="Eof" never saw Eof once the cursor ran past the single trailing Eof token. On unclosed-delimiter input the parser then appended AST nodes forever -> unbounded allocation -> ~700GB -> OOM (observed compiling neuron/sessions.el). Fix at the choke point: tok_kind returns "Eof" and tok_value returns "" for out-of-range positions, restoring the parser-wide contract that reads at/after the end yield Eof. expect() no longer steps past the Eof sentinel on mismatch. This terminates every overrun loop simultaneously; a malformed program now surfaces as a normal (best-effort) parse end instead of exhausting memory. Requires a self-hosted bootstrap rebuild of elc to take effect.
This commit is contained in:
@@ -23,10 +23,29 @@ fn tok_at(tokens: [Any], pos: Int) -> Map<String, Any> {
|
||||
}
|
||||
|
||||
fn tok_kind(tokens: [Any], pos: Int) -> String {
|
||||
// Out-of-range reads must report the Eof sentinel so every `== "Eof"`
|
||||
// termination guard in the parser fires. Without this, reading past the
|
||||
// single trailing Eof token returns runtime null (el_list_get OOB -> 0),
|
||||
// which matches no delimiter, letting inner parse loops append AST nodes
|
||||
// forever on malformed input -> unbounded allocation -> OOM.
|
||||
let n: Int = native_list_len(tokens) / 2
|
||||
if pos < 0 {
|
||||
return "Eof"
|
||||
}
|
||||
if pos >= n {
|
||||
return "Eof"
|
||||
}
|
||||
native_list_get(tokens, pos * 2)
|
||||
}
|
||||
|
||||
fn tok_value(tokens: [Any], pos: Int) -> String {
|
||||
let n: Int = native_list_len(tokens) / 2
|
||||
if pos < 0 {
|
||||
return ""
|
||||
}
|
||||
if pos >= n {
|
||||
return ""
|
||||
}
|
||||
native_list_get(tokens, pos * 2 + 1)
|
||||
}
|
||||
|
||||
@@ -35,7 +54,12 @@ fn expect(tokens: [Any], pos: Int, kind: String) -> Int {
|
||||
if k == kind {
|
||||
return pos + 1
|
||||
}
|
||||
// On mismatch just advance; error recovery is best-effort
|
||||
// On mismatch, error recovery is best-effort. But never step PAST the Eof
|
||||
// sentinel: once at Eof a mismatch means the input ended early, and
|
||||
// advancing would run the cursor off the token list.
|
||||
if k == "Eof" {
|
||||
return pos
|
||||
}
|
||||
pos + 1
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user