thread provenance through resolve_imports
The module question ended with a limit: textual inlining destroys file
provenance, so a duplicate-definition message could name the symbol but not the
files. Threading it exposed a bigger absence first.
TOKENS HAD NO POSITION AT ALL. A token was a flat (kind, value) pair, so NO
diagnostic in El could name a place -- every error named a symbol and never a
line. That is the prerequisite the module question was resting on.
THE CHAIN, end to end
lexer counts newlines; tok_append mints (kind, value, line)
parser stride 2 -> 3; tok_line added; FnDef carries its line
codegen records <fn> defines_at:<line>
resolve_imports publishes <file> spans <start> <end> for the combined source
checker maps a combined line back to file:line-within-that-file
duplicate definition: 'helper' is defined 2 times — El has no namespacing,
so imported modules share one global scope
/tmp/modtest/a.el:1
/tmp/modtest/b.el:1
PREDICTIONS AND RESULTS
P1 15 stride sites, encapsulated in tok_kind/tok_value TRUE, but see below
P2 adding a line field is mechanical TRUE
P3 the lexer must count newlines TRUE
P4 resolve_imports can record per-file line ranges TRUE
P5 the message can then name both files TRUE
P6 token memory grows TRUE, 25.0 -> 33.9 MB (+36%)
FOUR DEFECTS, EACH FOUND BY RUNNING AND NOT BY READING
1. interp_tokens_append_all walks the token list DIRECTLY with its own copy of
the stride. Gen1 built fine and gen2 emitted corrupt C, because the
compiler's own source uses string interpolation. My search missed it because
I grepped for the variable name `tokens`; it is called `dst`/`result`.
Searching by name instead of by shape -- third time today.
2. tok_count in test_compiler.el carried the stride too. I had scoped the search
to compiler sources and it had escaped into the tests.
3. Nested resolve_imports calls accumulated spans into shared state, so each
republished meaningless line ranges under the parent's name. Making the
buffer local fixed it; guarding the WRITE did not, which is what I tried
first.
4. The first working version reported b.el:3 -- the COMBINED line against a
filename that has no line 3. A file:line that does not match the file is
worse than no line at all.
105/105 native, 37/37 integration, fixpoint ok, compiler self-checks clean.
This commit is contained in:
@@ -414,6 +414,12 @@ fn parse_import_line(trimmed: String, dir: String) -> String {
|
||||
// Accumulates chunks into lists and joins once at the end to avoid the O(n²)
|
||||
// memory growth caused by repeated `prefix = prefix + chunk` concatenation.
|
||||
fn resolve_imports(src_path: String) -> String {
|
||||
// Only the OUTERMOST call publishes provenance. Nested calls number their
|
||||
// lines from 1 within themselves, so their spans are meaningless once the
|
||||
// text is spliced into the parent.
|
||||
let depth: String = state_get("__elc_prov_depth")
|
||||
if str_eq(depth, "") { state_set("__elc_prov_depth", "1") }
|
||||
let is_top: Bool = str_eq(depth, "")
|
||||
let seen_key: String = "__elc_imp__:" + src_path
|
||||
let already: String = state_get(seen_key)
|
||||
if !str_eq(already, "") { return "" }
|
||||
@@ -443,6 +449,7 @@ fn resolve_imports(src_path: String) -> String {
|
||||
// Collect chunks into lists — O(1) amortized per append.
|
||||
// Join once at the end — O(n) single pass.
|
||||
let prefix_chunks: [String] = native_list_empty()
|
||||
let prefix_paths: [String] = native_list_empty()
|
||||
let body_chunks: [String] = native_list_empty()
|
||||
let i: Int = 0
|
||||
while i < n {
|
||||
@@ -454,21 +461,54 @@ fn resolve_imports(src_path: String) -> String {
|
||||
// Only check .elh for imported files — never for the entry file itself.
|
||||
let imp_elh_path: String = str_slice(imp_path, 0, str_len(imp_path) - 3) + ".elh"
|
||||
let imp_elh: String = fs_read(imp_elh_path)
|
||||
// Provenance: record which line range of the combined source came
|
||||
// from which file, so a diagnostic can name the FILE and not just a
|
||||
// line in a string that no longer exists on disk.
|
||||
if !str_eq(imp_elh, "") {
|
||||
// Header exists: mark the .el as seen (so it won't be re-inlined
|
||||
// if something else also imports it) and use the header text.
|
||||
let seen_imp_key: String = "__elc_imp__:" + imp_path
|
||||
state_set(seen_imp_key, "1")
|
||||
let prefix_chunks = native_list_append(prefix_chunks, imp_elh)
|
||||
let prefix_paths = native_list_append(prefix_paths, imp_path)
|
||||
} else {
|
||||
let imp_body: String = resolve_imports(imp_path)
|
||||
let prefix_chunks = native_list_append(prefix_chunks, imp_body)
|
||||
let prefix_paths = native_list_append(prefix_paths, imp_path)
|
||||
}
|
||||
} else {
|
||||
let body_chunks = native_list_append(body_chunks, line + "\n")
|
||||
}
|
||||
let i = i + 1
|
||||
}
|
||||
// Walk the assembled chunks once and publish <file> spans <start> <end>.
|
||||
// LIMIT: nested imports return a single string, so their internal
|
||||
// boundaries are already lost by the time we see them -- a definition
|
||||
// inside a transitively imported file is attributed to the direct import.
|
||||
// Local, not accumulated in state: a nested call numbers its lines from 1
|
||||
// within itself, so letting it append to a shared buffer republishes
|
||||
// meaningless spans under the parent's name.
|
||||
let prov: String = ""
|
||||
let line_at: Int = 1
|
||||
let ci: Int = 0
|
||||
let nchunks: Int = native_list_len(prefix_chunks)
|
||||
while ci < nchunks {
|
||||
let chunk: String = native_list_get(prefix_chunks, ci)
|
||||
let nlines: Int = str_count_lines(chunk)
|
||||
let src: String = native_list_get(prefix_paths, ci)
|
||||
let prov = prov + src + " spans " + native_int_to_str(line_at) + " " + native_int_to_str(line_at + nlines - 1) + "\n"
|
||||
let line_at = line_at + nlines
|
||||
let ci = ci + 1
|
||||
}
|
||||
let prov = prov + src_path + " spans " + native_int_to_str(line_at) + " 999999\n"
|
||||
if is_top {
|
||||
let prov_out: String = env("EL_RELATIONS_OUT")
|
||||
if !str_eq(prov_out, "") {
|
||||
let existing: String = ""
|
||||
if fs_exists(prov_out) { let existing = fs_read(prov_out) }
|
||||
fs_write(prov_out, existing + prov)
|
||||
}
|
||||
}
|
||||
return str_join(prefix_chunks, "") + str_join(body_chunks, "")
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user