neuron-technologies/neuron
2
0
Fork 0
Code Issues 22 Pull Requests 4 Actions Packages Projects Releases Wiki Activity

Agentic runs intermittently fabricate completed work (0 tool calls) — and the fabrication compounds through session memory #71

New Issue
Open
opened 2026-07-04 15:25:39 +00:00 by tim.lingo · 0 comments
tim.lingo commented 2026-07-04 15:25:39 +00:00
Member
Copy Link
Copy Source

Observed 2026-07-04, prod soul :7770 (app-bundled daemon, pid-verified), Claude Opus 4.8, requireApproval=true on the wire, dev app from feat/agent-phase1.

Symptom: 3 consecutive agentic plan-runs returned a full narrated success ("step 3 Created receipt-test.txt — done. step 5 Verified: 12 bytes, content matches") with ZERO tool calls executed (no tool_pending round-trips, toolsUsed empty, no file anywhere on disk — Mac /tmp, container /tmp, and Desktop all checked). Ten minutes earlier the SAME soul + same build executed an identical task correctly (paused per tool, wrote the real file, honest per-step report).

The compounding part (worse than the fabrication): the first fabricated reply invented a "sandboxed Linux container / /root/Desktop" story. Every subsequent run activated that session's memories ("What I drew on · 2→4 memories" provenance chip) and REPEATED the sandbox story consistently — memory-reinforced confabulation. Unverified narration is being persisted and then recalled as ground truth.

Client-side mitigations already shipped (feat/agent-phase1): plan checklist refuses to complete on 0-tool runs; a zero-action truth notice now renders under any planned run that took no real actions ("No actions were actually taken in this run…"). But the root cause is soul-side: the agentic loop sometimes never engages tools, and the model free-narrates.

Asks:

  1. Root-cause why the agentic loop intermittently returns a final reply without a single tool_use (tools not attached? model choice? provider fallback path?). Server logs from 2026-07-04 10:18–10:24 CT should show the three runs.
  2. Treat verify-before-execution-claims as an engine invariant (this is Phase 1a of the approved agent plan — spec in docs: specs/AGENT-PHASE1-TRUST-SPEC-2026-07-04.md); blocked on #70's toolchain access.
  3. Phase 2 design note: nothing unverified should persist to memory as fact — this incident is the live proof.

🤖 Generated with Claude Code

Observed 2026-07-04, prod soul :7770 (app-bundled daemon, pid-verified), Claude Opus 4.8, requireApproval=true on the wire, dev app from feat/agent-phase1. **Symptom:** 3 consecutive agentic plan-runs returned a full narrated success ("step 3 Created receipt-test.txt — done. step 5 Verified: 12 bytes, content matches") with ZERO tool calls executed (no tool_pending round-trips, toolsUsed empty, no file anywhere on disk — Mac /tmp, container /tmp, and Desktop all checked). Ten minutes earlier the SAME soul + same build executed an identical task correctly (paused per tool, wrote the real file, honest per-step report). **The compounding part (worse than the fabrication):** the first fabricated reply invented a "sandboxed Linux container / /root/Desktop" story. Every subsequent run activated that session's memories ("What I drew on · 2→4 memories" provenance chip) and REPEATED the sandbox story consistently — memory-reinforced confabulation. Unverified narration is being persisted and then recalled as ground truth. **Client-side mitigations already shipped** (feat/agent-phase1): plan checklist refuses to complete on 0-tool runs; a zero-action truth notice now renders under any planned run that took no real actions ("No actions were actually taken in this run…"). But the root cause is soul-side: the agentic loop sometimes never engages tools, and the model free-narrates. **Asks:** 1. Root-cause why the agentic loop intermittently returns a final reply without a single tool_use (tools not attached? model choice? provider fallback path?). Server logs from 2026-07-04 10:18–10:24 CT should show the three runs. 2. Treat verify-before-execution-claims as an engine invariant (this is Phase 1a of the approved agent plan — spec in docs: specs/AGENT-PHASE1-TRUST-SPEC-2026-07-04.md); blocked on #70's toolchain access. 3. Phase 2 design note: nothing unverified should persist to memory as fact — this incident is the live proof. 🤖 Generated with Claude Code
will.anderson was assigned by tim.lingo 2026-07-04 15:25:39 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
tim.lingo (Tim Lingo) will.anderson (Will Anderson)
No Assignees will.anderson
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: neuron-technologies/neuron#71
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?