parser: bound token reads to Eof so malformed input errors instead of OOMing
El SDK Release / build-and-release (pull_request) Failing after 16s
El SDK Release / build-and-release (pull_request) Failing after 16s
Out-of-range tok_kind/tok_value reads returned runtime null (el_list_get OOB -> 0) rather than the Eof sentinel, so the inner parse loops (parse_block, call-arg, array-literal, match-arm) that terminate only on their close delimiter or k=="Eof" never saw Eof once the cursor ran past the single trailing Eof token. On unclosed-delimiter input the parser then appended AST nodes forever -> unbounded allocation -> ~700GB -> OOM (observed compiling neuron/sessions.el). Fix at the choke point: tok_kind returns "Eof" and tok_value returns "" for out-of-range positions, restoring the parser-wide contract that reads at/after the end yield Eof. expect() no longer steps past the Eof sentinel on mismatch. This terminates every overrun loop simultaneously; a malformed program now surfaces as a normal (best-effort) parse end instead of exhausting memory. Requires a self-hosted bootstrap rebuild of elc to take effect.
This commit is contained in:
@@ -23,10 +23,29 @@ fn tok_at(tokens: [Any], pos: Int) -> Map<String, Any> {
|
|||||||
}
|
}
|
||||||
|
|
||||||
fn tok_kind(tokens: [Any], pos: Int) -> String {
|
fn tok_kind(tokens: [Any], pos: Int) -> String {
|
||||||
|
// Out-of-range reads must report the Eof sentinel so every `== "Eof"`
|
||||||
|
// termination guard in the parser fires. Without this, reading past the
|
||||||
|
// single trailing Eof token returns runtime null (el_list_get OOB -> 0),
|
||||||
|
// which matches no delimiter, letting inner parse loops append AST nodes
|
||||||
|
// forever on malformed input -> unbounded allocation -> OOM.
|
||||||
|
let n: Int = native_list_len(tokens) / 2
|
||||||
|
if pos < 0 {
|
||||||
|
return "Eof"
|
||||||
|
}
|
||||||
|
if pos >= n {
|
||||||
|
return "Eof"
|
||||||
|
}
|
||||||
native_list_get(tokens, pos * 2)
|
native_list_get(tokens, pos * 2)
|
||||||
}
|
}
|
||||||
|
|
||||||
fn tok_value(tokens: [Any], pos: Int) -> String {
|
fn tok_value(tokens: [Any], pos: Int) -> String {
|
||||||
|
let n: Int = native_list_len(tokens) / 2
|
||||||
|
if pos < 0 {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
if pos >= n {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
native_list_get(tokens, pos * 2 + 1)
|
native_list_get(tokens, pos * 2 + 1)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -35,7 +54,12 @@ fn expect(tokens: [Any], pos: Int, kind: String) -> Int {
|
|||||||
if k == kind {
|
if k == kind {
|
||||||
return pos + 1
|
return pos + 1
|
||||||
}
|
}
|
||||||
// On mismatch just advance; error recovery is best-effort
|
// On mismatch, error recovery is best-effort. But never step PAST the Eof
|
||||||
|
// sentinel: once at Eof a mismatch means the input ended early, and
|
||||||
|
// advancing would run the cursor off the token list.
|
||||||
|
if k == "Eof" {
|
||||||
|
return pos
|
||||||
|
}
|
||||||
pos + 1
|
pos + 1
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user