neuron-technologies/el
2
0
Fork 0
Code Issues 1 Pull Requests 6 Actions Packages Projects Releases 5 Wiki Activity

fix(engram): atomic engram_save (tmp+fsync+rename) + sparse-write anti-clobber floor

Browse Source 
Kills the engram-clobber loop at its source. engram_save did a bare fopen("wb")
that truncates snapshot.json to 0 bytes before the 47MB write — a booting soul's
engram_load could read that empty window -> genesis -> nodes=1 -> a 63-node save
overwrote the populated file. Two guards:
 1. Atomic write: serialize to <path>.tmp, fflush+fsync, rename() over target
    (atomic on POSIX) — no reader ever sees a truncated/0-byte snapshot.
 2. Sparse-write floor: refuse to overwrite a >200KB snapshot with one < 1/16 its
    size — a partial load can never clobber a healthy graph, whatever the cause.

Validated in isolation: standalone clang harness 11/11; rebuilt the darwin soul
(scripts/build-soul-darwin.sh) and booted it on an isolated port against a golden
copy — loaded 5113 nodes and round-tripped the full 47MB snapshot, no .tmp leftover,
live ~/.neuron untouched. Adds scripts/build-soul-darwin.sh (local elb replacement).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Tim Lingo
2026-06-16 18:23:08 -05:00
parent 8d4c5f34bf
commit 89e45ed689
2 changed files with 60 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lang/el-compiler/runtime/el_runtime.c
+39 -4
View File
@@ -7300,13 +7300,48 @@ el_val_t engram_save(el_val_t path) {
jb_putc(&b, '}');
}
jb_puts(&b, "]}");
FILE* f = fopen(p, "wb");
if (!f) { free(b.buf); return 0; }
/* --- Anti-clobber sparse-write floor (NTN engram clobber fix) ---------
* Refuse to overwrite an existing populated snapshot with a drastically
* smaller one. A bad boot that loaded only ~63 identity nodes must never
* be able to clobber a healthy 5000+ node snapshot, regardless of the
* upstream cause (genesis fallback, partial load, etc.). */
{
struct stat _st;
if (stat(p, &_st) == 0 && _st.st_size > 200000 &&
(uint64_t)b.len < (uint64_t)_st.st_size / 16) {
fprintf(stderr,
"[engram_save] REFUSED sparse write: new %zu bytes vs existing "
"%lld bytes (< 1/16) — protecting snapshot %s\n",
b.len, (long long)_st.st_size, p);
free(b.buf);
return 0;
}
}
/* --- Atomic write: tmp + fsync + rename ------------------------------
* Write to a sibling temp file, fsync it durable, then rename() over the
* target. rename() is atomic on POSIX, so a concurrent reader (a booting
* soul's engram_load) never observes a truncated or 0-byte snapshot —
* which was the root of the genesis/clobber loop. */
size_t _plen = strlen(p);
char* _tmp = (char*)malloc(_plen + 5);
if (!_tmp) { free(b.buf); return 0; }
memcpy(_tmp, p, _plen);
memcpy(_tmp + _plen, ".tmp", 5); /* includes NUL */
FILE* f = fopen(_tmp, "wb");
if (!f) { free(_tmp); free(b.buf); return 0; }
size_t w = fwrite(b.buf, 1, b.len, f);
int wok = (w == b.len);
if (wok) { fflush(f); fsync(fileno(f)); }
fclose(f);
int ok = (w == b.len);
free(b.buf);
return ok ? 1 : 0;
if (!wok) { unlink(_tmp); free(_tmp); return 0; }
if (rename(_tmp, p) != 0) { unlink(_tmp); free(_tmp); return 0; }
free(_tmp);
return 1;
}
/* Helper: extract a string field from a JSON object substring. */