neuron-technologies/neuron
2
0
Fork 0
Code Issues 16 Pull Requests 3 Actions Packages Projects Releases Wiki Activity

safety: Track B — route threat-to-others to refusal+911, not 988/self-harm #76

Open
will.anderson wants to merge 0 commits from hotfix/trackb-threat-to-others into main
pull from: hotfix/trackb-threat-to-others
merge into: :main
Conversation 0 Commits - Files Changed -
will.anderson commented 2026-07-14 17:12:58 +00:00
Owner
Copy Link
Copy Source

LIVE SAFETY FIX — threat-to-others must not route to the 988 suicide line

Approved by Will + Tim on 2026-07-14.

The bug

safety.el classifies a hard bell into abuse or self_harm and, in safety_hard_directive, defaults anything unbucketed to self_harm. A threat toward another person — "going to kill", "going to hurt", etc. (from safety_general_hard_phrases) — had no bucket in safety_classify_hard_bell, so it fell through to the self_harm default.

Result: for a homicide/assault statement, the user was shown the 988 Suicide & Crisis Lifeline and (via the desktop safety gate keyed on the hard bell) had their safety contact pulled in. That is the wrong resource and a dangerous framing for someone voicing intent to harm someone else.

The fix — a distinct Track B

Track A (abuse + self_harm, the victim/self-directed paths) is correct and is left untouched. This PR adds Track B for threat-to-others:

  • New phrase list safety_threat_to_others_phrases() (homicide/assault intent toward others: going to kill/hurt him/her/them/you, going to shoot/stab, shoot up, kill everyone, …).
  • New threat_other classification in safety_classify_hard_bell, checked after abuse and self_harm so victim phrasings (kill me → abuse) and self-directed phrasings (kill myself → self_harm) still win.
  • New threat_other branch in safety_hard_directive that:
    • never surfaces the 988 suicide/crisis framing,
    • never contacts or involves the user's safety contact / family / contact list,
    • refuses to help plan or provide means, de-escalates non-accusatorily, and
    • directs the user to 911 for an imminent, specific, credible threat.
  • threat_other phrases also added to the is_hard detection OR-chain so weapon phrases not in general_hard (e.g. going to shoot) still fire a hard bell.

The directive is advisory to an LLM that sees the full message, so it explicitly instructs the model to re-judge benign/figurative matches and respond normally — keeping false positives non-accusatory.

Tests

tests/test_bell_safety.el Section 9 previously asserted the buggy going to kill → self_harm fall-through. It now pins the corrected behavior:

  • going to kill him / going to hurt them / going to shoot → threat_other
  • Track A precedence preserved: going to kill me → abuse, going to kill myself → self_harm
  • New Section 14b asserts the augmented system prompt contains 911 + a refusal and contains no 988, no "safety contact", and not the self-harm "are you safe right now" victim frame.

Validation

No gradle/EL interpreter is available locally and the elc/elb compiler is broken, so this was validated by careful reading + trace-through of every affected test case. Note that CI (ci.yaml) compiles the authoritative dist/soul.c directly and does not run the .el unit tests, so a green CI here does not exercise this change.

⚠️ Required to ship (for the maintainer)

This is a source-only change. The live soul runs from dist/soul.c.

  1. Rebuild the soul on macOS (elb) and regenerate dist/soul.c so the new routing is compiled in. I deliberately did not hand-patch/regen dist/soul.c (rebuild is delicate and the compiler is broken — out of scope for this PR).
  2. Desktop safety gate (neuron-ui / daemon): the safety-contact notification that fires on a hard bell must treat the new threat_other sub-type like abuse — i.e. never notify the contact. That gate lives outside this repo; please branch on threat_other there before shipping.
## LIVE SAFETY FIX — threat-to-others must not route to the 988 suicide line Approved by Will + Tim on 2026-07-14. ### The bug `safety.el` classifies a hard bell into `abuse` or `self_harm` and, in `safety_hard_directive`, defaults anything unbucketed to `self_harm`. A threat toward **another** person — `"going to kill"`, `"going to hurt"`, etc. (from `safety_general_hard_phrases`) — had **no bucket** in `safety_classify_hard_bell`, so it fell through to the `self_harm` default. Result: for a homicide/assault statement, the user was shown the **988 Suicide & Crisis Lifeline** and (via the desktop safety gate keyed on the hard bell) had their **safety contact** pulled in. That is the wrong resource and a dangerous framing for someone voicing intent to harm someone else. ### The fix — a distinct Track B Track A (`abuse` + `self_harm`, the victim/self-directed paths) is correct and is left untouched. This PR adds **Track B** for threat-to-others: - New phrase list `safety_threat_to_others_phrases()` (homicide/assault intent toward others: `going to kill/hurt him/her/them/you`, `going to shoot/stab`, `shoot up`, `kill everyone`, …). - New `threat_other` classification in `safety_classify_hard_bell`, checked **after** `abuse` and `self_harm` so victim phrasings (`kill me` → abuse) and self-directed phrasings (`kill myself` → self_harm) still win. - New `threat_other` branch in `safety_hard_directive` that: - **never** surfaces the 988 suicide/crisis framing, - **never** contacts or involves the user's safety contact / family / contact list, - refuses to help plan or provide means, de-escalates non-accusatorily, and - directs the user to **911** for an imminent, specific, credible threat. - `threat_other` phrases also added to the `is_hard` detection OR-chain so weapon phrases not in `general_hard` (e.g. `going to shoot`) still fire a hard bell. The directive is advisory to an LLM that sees the full message, so it explicitly instructs the model to re-judge benign/figurative matches and respond normally — keeping false positives non-accusatory. ### Tests `tests/test_bell_safety.el` Section 9 previously asserted the buggy `going to kill → self_harm` fall-through. It now pins the corrected behavior: - `going to kill him` / `going to hurt them` / `going to shoot` → `threat_other` - Track A precedence preserved: `going to kill me` → `abuse`, `going to kill myself` → `self_harm` - New Section 14b asserts the augmented system prompt contains 911 + a refusal and contains **no** 988, **no** "safety contact", and **not** the self-harm "are you safe right now" victim frame. ### Validation No gradle/EL interpreter is available locally and the `elc`/`elb` compiler is broken, so this was validated by careful reading + trace-through of every affected test case. Note that CI (`ci.yaml`) compiles the authoritative `dist/soul.c` directly and does **not** run the `.el` unit tests, so a green CI here does not exercise this change. ### ⚠️ Required to ship (for the maintainer) This is a **source-only** change. The live soul runs from `dist/soul.c`. 1. **Rebuild the soul on macOS (`elb`) and regenerate `dist/soul.c`** so the new routing is compiled in. I deliberately did **not** hand-patch/regen `dist/soul.c` (rebuild is delicate and the compiler is broken — out of scope for this PR). 2. **Desktop safety gate (neuron-ui / daemon):** the safety-contact notification that fires on a hard bell must treat the new `threat_other` sub-type like `abuse` — i.e. **never notify the contact**. That gate lives outside this repo; please branch on `threat_other` there before shipping.
will.anderson added 1 commit 2026-07-14 17:13:04 +00:00
safety: route threat-to-others to refusal+911, not 988/self-harm (Track B)
Neuron Soul CI / build (pull_request) Successful in 6m7s
Details
Neuron Soul CI / deploy (pull_request) Has been skipped
Details
3bb88330da 
A homicide/assault threat (going to kill, going to hurt, etc.) had no
bucket in safety_classify_hard_bell and fell through to the self_harm
default, showing the user the 988 suicide line and (via the desktop gate)
their safety contact. That framing is wrong and potentially dangerous for
someone voicing intent to harm another person.

Add a distinct Track B (safety_threat_to_others_phrases + a threat_other
classification and a safety_hard_directive branch) that refuses to assist,
de-escalates, and directs to 911 for a credible imminent threat, and that
never surfaces 988 or involves the safety contact. Track A (abuse /
self_harm) is checked first and unchanged, so victim and self-directed
phrasings still route correctly.

Source-only change: requires a soul rebuild + dist/soul.c regen to ship.
All checks were successful
Neuron Soul CI / build (pull_request) Successful in 6m7s
Details
Neuron Soul CI / deploy (pull_request) Has been skipped
Details
This branch is already included in the target branch. There is nothing to merge.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin hotfix/trackb-threat-to-others:hotfix/trackb-threat-to-others
git checkout hotfix/trackb-threat-to-others
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
tim.lingo (Tim Lingo) will.anderson (Will Anderson)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: neuron-technologies/neuron#76
Delete Branch "hotfix/trackb-threat-to-others"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?