From 21710d5c8e3c95aaace63d7d0b060066e189c771 Mon Sep 17 00:00:00 2001
From: "will.anderson" <will.anderson@neurontechnologies.ai>
Date: Mon, 3 Aug 2026 08:39:07 -0500
Subject: [PATCH] self-review 2026-08-03: gate curiosity auto-terms on label
 document frequency

Reject an extracted auto-term when its label document frequency exceeds
node_count/400 (floor 8) -- measured live at 12,859 nodes, threshold 32.

Live label df separates the classes by an order of magnitude:
  rejected: <!--:220  SELF:175  Engram:125  CORE:88  STAR:36
  passed:   Dual:12  Sparse:8  Latent:6  MemQ:1  dGRPO:1  engram_goal_bias:1

Verified against the running soul (boot 21). Peak curiosity activation fell
from 541 to 113; the flood terms (SELF, CORE, Engram, STAR, <!--) are absent
from post-fix scans while topical compound identifiers pass untouched.
Sample is 7 scans -- suggestive, not conclusive; watch the next review.

Nested conditional rather than max(): El let is single-assignment, so the
floor is expressed as a second conjunct.

Verification note: content df was tested as an alternative signal and
rejected -- 'Curiosity' has the highest content df in the store (5526) yet
one of the lowest activation counts (113). Label df is the correct field
because label is what the first-word extractor reads.
---
 awareness.el     | 42 ++++++++++++++++++++++++++++++++++++++++++
 dist/awareness.c |  7 +++++++
 2 files changed, 49 insertions(+)

diff --git a/awareness.el b/awareness.el
index 6c69745..d5e4b81 100644
--- a/awareness.el
+++ b/awareness.el
@@ -415,6 +415,48 @@ fn auto_term_try_slot(slot_type: String, slot_lbl: String) -> Void {
                 // carrying a quote character is not a topic word.
                 if str_contains(term, "\"") { state_set("_ats_gw", "1") }
                 if str_contains(term, "'")  { state_set("_ats_gw", "1") }
+                // TERM-SPECIFICITY GATE (2026-08-03 self-review): the three
+                // guards above are hand-curated lists, and every one of them
+                // was written REACTIVELY — after a flood was already observed
+                // in the ISE stream. A list can only ever contain the floods
+                // that already happened. Two were in flight, unfixed, while
+                // this review ran:
+                //     "<!--" → 252 nodes activated  (markdown comment opener:
+                //              4 chars, no quote, no colon — passes every
+                //              guard above)
+                //     "SELF" → 541 nodes activated  (the stopword list has
+                //              "Self" Title-case; str_eq is case-SENSITIVE,
+                //              so the uppercase token sails through)
+                // Replace reaction with measurement: engram_label_df(term)
+                // counts nodes whose label contains the term. Low-specificity
+                // tokens are corpus-frequent BY DEFINITION, so this catches
+                // the flood class PROSPECTIVELY and tracks the corpus as the
+                // world-ingestor changes what the store is made of.
+                // This is IDF — Spärck Jones (1972) named it "term
+                // specificity"; automatic stopword compilation from it is the
+                // textbook application.
+                //
+                // Threshold node_count/400 (floor 8), measured on this store
+                // (13,370 nodes → 33). Live df separates the classes by an
+                // order of magnitude: <!--:220, SELF:175, Context:53 rejected;
+                // Dual:12, Sparse:8, engram_goal_bias:1, Clin-JEPA:1 pass.
+                //
+                // This does NOT replace the stopword list — verified against
+                // all 86 listed terms, not assumed. It catches 13 (Will:306,
+                // Self:175, Over:116, Knowledge:112 …) and misses 73
+                // (Whose:0, Would:0, Could:0, This:9 …). Labels are terse
+                // titles, so English function words are genuinely RARE in
+                // them: low df, high noise. The gates cover disjoint failure
+                // modes — stopwords catch function words, df catches
+                // corpus-frequent markup/sentinel/genre tokens. Both required.
+                // Nested rather than max(): El `let` is single-assignment, so
+                // the floor is expressed as a second conjunct. Reject iff
+                // df > node_count/400 AND df > 8 — i.e. df > max(that, 8).
+                let df_max: Int = engram_node_count() / 400
+                let df_term: Int = engram_label_df(term)
+                if df_term > df_max {
+                    if df_term > 8 { state_set("_ats_gw", "1") }
+                }
                 // AUTO-TERM TABU (2026-07-25 self-review): finst-style
                 // inhibition-of-return (ACT-R declarative finsts: small
                 // marker pool, hard exclusion). The last 4 selected auto
diff --git a/dist/awareness.c b/dist/awareness.c
index 21c8bc7..6baa089 100644
--- a/dist/awareness.c
+++ b/dist/awareness.c
@@ -289,6 +289,13 @@ el_val_t auto_term_try_slot(el_val_t slot_type, el_val_t slot_lbl) {
         if (str_contains(term, EL_STR("'"))) {
           state_set(EL_STR("_ats_gw"), EL_STR("1"));
         }
+        el_val_t df_max = (engram_node_count() / 400);
+        el_val_t df_term = engram_label_df(term);
+        if (df_term > df_max) {
+          if (df_term > 8) {
+            state_set(EL_STR("_ats_gw"), EL_STR("1"));
+          }
+        }
         if (str_eq(term, state_get(EL_STR("soul.tabu_t0")))) {
           state_set(EL_STR("_ats_gw"), EL_STR("1"));
         }