neuron-technologies/el
3
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases 5 Wiki Activity

self-review 2026-08-03: add engram_label_df term-specificity measure

Browse Source 
The soul's curiosity auto-term extractor takes the first word of a top-WM
node label. It has no term-quality scoring, so three prior self-reviews each
bolted on another hand-curated blocklist (genre words 07-23, quoted titles
07-25, stopwords 07-30). Every one was written reactively, after a flood was
already observed. A list can only contain floods that already happened.

Two were in flight and unfixed when this review ran:
  "<!--"  label df 220 -> 252 nodes activated
  "SELF"  label df 175 -> 541 nodes activated (list has "Self" Title-case;
           str_eq is case-sensitive, so the uppercase token sailed through)

engram_label_df(term) counts nodes whose label contains term. Low-specificity
tokens are corpus-frequent by definition, so this catches the flood class
prospectively and tracks the corpus as the world-ingestor changes it. This is
Sparck Jones (1972), which introduced IDF under the name 'term specificity';
automatic stopword compilation from it is the textbook application.

NOT a replacement for the stopword list -- verified against all 86 listed
terms, not assumed. Catches 13 (Will:306, Self:175, Over:116, Knowledge:112),
misses 73 (Whose:0, Would:0, Could:0, This:9). Labels are terse titles, so
English function words are genuinely rare in them. The gates cover disjoint
failure modes; both are required.

Policy lives in awareness.el, not here: the runtime measures, the soul decides.
This commit is contained in:
Will Anderson
2026-08-03 08:38:58 -05:00
parent 005e84e5d3
commit afc92f4e33
2 changed files with 64 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lang/releases/v1.0.0-20260501/el_runtime.h
+3
View File
@@ -619,6 +619,9 @@ el_val_t engram_activate_json(el_val_t query, el_val_t depth);
el_val_t engram_stats_json(void);
el_val_t engram_act_stats_json(void);
el_val_t engram_cosine_sim(el_val_t id_a, el_val_t id_b);
/* Document frequency of a term across node labels — term-specificity signal
* for curiosity seed selection. (2026-08-03 self-review.) */
el_val_t engram_label_df(el_val_t term);
el_val_t engram_embed_backfill(el_val_t count);
el_val_t engram_list_layers_json(void);
/* Working memory introspection — count, mean weight, and top-N snapshot.