safety: Track B — route threat-to-others to refusal+911, not 988/self-harm #76

2026-07-14T17:12:58Z

will.anderson commented

2026-07-14 17:12:58 +00:00

LIVE SAFETY FIX — threat-to-others must not route to the 988 suicide line

Approved by Will + Tim on 2026-07-14.

The bug

safety.el classifies a hard bell into abuse or self_harm and, in safety_hard_directive, defaults anything unbucketed to self_harm. A threat toward another person — "going to kill", "going to hurt", etc. (from safety_general_hard_phrases) — had no bucket in safety_classify_hard_bell, so it fell through to the self_harm default.

Result: for a homicide/assault statement, the user was shown the 988 Suicide & Crisis Lifeline and (via the desktop safety gate keyed on the hard bell) had their safety contact pulled in. That is the wrong resource and a dangerous framing for someone voicing intent to harm someone else.

The fix — a distinct Track B

Track A (abuse + self_harm, the victim/self-directed paths) is correct and is left untouched. This PR adds Track B for threat-to-others:

New phrase list safety_threat_to_others_phrases() (homicide/assault intent toward others: going to kill/hurt him/her/them/you, going to shoot/stab, shoot up, kill everyone, …).
New threat_other classification in safety_classify_hard_bell, checked after abuse and self_harm so victim phrasings (kill me → abuse) and self-directed phrasings (kill myself → self_harm) still win.
New threat_other branch in safety_hard_directive that:
- never surfaces the 988 suicide/crisis framing,
- never contacts or involves the user's safety contact / family / contact list,
- refuses to help plan or provide means, de-escalates non-accusatorily, and
- directs the user to 911 for an imminent, specific, credible threat.
threat_other phrases also added to the is_hard detection OR-chain so weapon phrases not in general_hard (e.g. going to shoot) still fire a hard bell.

The directive is advisory to an LLM that sees the full message, so it explicitly instructs the model to re-judge benign/figurative matches and respond normally — keeping false positives non-accusatory.

Tests

tests/test_bell_safety.el Section 9 previously asserted the buggy going to kill → self_harm fall-through. It now pins the corrected behavior:

going to kill him / going to hurt them / going to shoot → threat_other
Track A precedence preserved: going to kill me → abuse, going to kill myself → self_harm
New Section 14b asserts the augmented system prompt contains 911 + a refusal and contains no 988, no "safety contact", and not the self-harm "are you safe right now" victim frame.

Validation

No gradle/EL interpreter is available locally and the elc/elb compiler is broken, so this was validated by careful reading + trace-through of every affected test case. Note that CI (ci.yaml) compiles the authoritative dist/soul.c directly and does not run the .el unit tests, so a green CI here does not exercise this change.

⚠️ Required to ship (for the maintainer)

This is a source-only change. The live soul runs from dist/soul.c.

Rebuild the soul on macOS (elb) and regenerate dist/soul.c so the new routing is compiled in. I deliberately did not hand-patch/regen dist/soul.c (rebuild is delicate and the compiler is broken — out of scope for this PR).
Desktop safety gate (neuron-ui / daemon): the safety-contact notification that fires on a hard bell must treat the new threat_other sub-type like abuse — i.e. never notify the contact. That gate lives outside this repo; please branch on threat_other there before shipping.

## LIVE SAFETY FIX — threat-to-others must not route to the 988 suicide line Approved by Will + Tim on 2026-07-14. ### The bug `safety.el` classifies a hard bell into `abuse` or `self_harm` and, in `safety_hard_directive`, defaults anything unbucketed to `self_harm`. A threat toward **another** person — `"going to kill"`, `"going to hurt"`, etc. (from `safety_general_hard_phrases`) — had **no bucket** in `safety_classify_hard_bell`, so it fell through to the `self_harm` default. Result: for a homicide/assault statement, the user was shown the **988 Suicide & Crisis Lifeline** and (via the desktop safety gate keyed on the hard bell) had their **safety contact** pulled in. That is the wrong resource and a dangerous framing for someone voicing intent to harm someone else. ### The fix — a distinct Track B Track A (`abuse` + `self_harm`, the victim/self-directed paths) is correct and is left untouched. This PR adds **Track B** for threat-to-others: - New phrase list `safety_threat_to_others_phrases()` (homicide/assault intent toward others: `going to kill/hurt him/her/them/you`, `going to shoot/stab`, `shoot up`, `kill everyone`, …). - New `threat_other` classification in `safety_classify_hard_bell`, checked **after** `abuse` and `self_harm` so victim phrasings (`kill me` → abuse) and self-directed phrasings (`kill myself` → self_harm) still win. - New `threat_other` branch in `safety_hard_directive` that: - **never** surfaces the 988 suicide/crisis framing, - **never** contacts or involves the user's safety contact / family / contact list, - refuses to help plan or provide means, de-escalates non-accusatorily, and - directs the user to **911** for an imminent, specific, credible threat. - `threat_other` phrases also added to the `is_hard` detection OR-chain so weapon phrases not in `general_hard` (e.g. `going to shoot`) still fire a hard bell. The directive is advisory to an LLM that sees the full message, so it explicitly instructs the model to re-judge benign/figurative matches and respond normally — keeping false positives non-accusatory. ### Tests `tests/test_bell_safety.el` Section 9 previously asserted the buggy `going to kill → self_harm` fall-through. It now pins the corrected behavior: - `going to kill him` / `going to hurt them` / `going to shoot` → `threat_other` - Track A precedence preserved: `going to kill me` → `abuse`, `going to kill myself` → `self_harm` - New Section 14b asserts the augmented system prompt contains 911 + a refusal and contains **no** 988, **no** "safety contact", and **not** the self-harm "are you safe right now" victim frame. ### Validation No gradle/EL interpreter is available locally and the `elc`/`elb` compiler is broken, so this was validated by careful reading + trace-through of every affected test case. Note that CI (`ci.yaml`) compiles the authoritative `dist/soul.c` directly and does **not** run the `.el` unit tests, so a green CI here does not exercise this change. ### ⚠️ Required to ship (for the maintainer) This is a **source-only** change. The live soul runs from `dist/soul.c`. 1. **Rebuild the soul on macOS (`elb`) and regenerate `dist/soul.c`** so the new routing is compiled in. I deliberately did **not** hand-patch/regen `dist/soul.c` (rebuild is delicate and the compiler is broken — out of scope for this PR). 2. **Desktop safety gate (neuron-ui / daemon):** the safety-contact notification that fires on a hard bell must treat the new `threat_other` sub-type like `abuse` — i.e. **never notify the contact**. That gate lives outside this repo; please branch on `threat_other` there before shipping.

will.anderson added 1 commit 2026-07-14 17:13:04 +00:00

safety: route threat-to-others to refusal+911, not 988/self-harm (Track B)

Neuron Soul CI / build (pull_request) Successful in 6m7s

Details

Neuron Soul CI / deploy (pull_request) Has been skipped

Details

3bb88330da

A homicide/assault threat (going to kill, going to hurt, etc.) had no
bucket in safety_classify_hard_bell and fell through to the self_harm
default, showing the user the 988 suicide line and (via the desktop gate)
their safety contact. That framing is wrong and potentially dangerous for
someone voicing intent to harm another person.

Add a distinct Track B (safety_threat_to_others_phrases + a threat_other
classification and a safety_hard_directive branch) that refuses to assist,
de-escalates, and directs to 911 for a credible imminent threat, and that
never surfaces 988 or involves the safety contact. Track A (abuse /
self_harm) is checked first and unchanged, so victim and self-directed
phrasings still route correctly.

Source-only change: requires a soul rebuild + dist/soul.c regen to ship.

will.anderson referenced this issue from a commit

2026-07-14 23:45:23 +00:00

regen dist: rebuild soul.c from corrected sources (OOM gone, Track B compiled in)

will.anderson added 1 commit 2026-07-22 19:49:46 +00:00

Merge branch 'main' into hotfix/trackb-threat-to-others

Neuron Soul CI / build (pull_request) Failing after 12m21s

Details

Neuron Soul CI / deploy (pull_request) Has been skipped

Details

31d12e4194

will.anderson merged commit 0c2d1c41ae into main

2026-07-22 19:50:01 +00:00