Commit Graph

613 Commits

Author SHA1 Message Date
Will Anderson 4f406e13c9 add fornax-ui Argo CD application
Deploys fornax/ui:latest (nginx serving React SPA) to the media
namespace. Image is now pushed by the fornax CI pipeline.
2026-04-14 12:48:42 -05:00
Will Anderson 27d9d9c436 fornax: fix crash-looping coordinator, add UI deployment
- Fix liveness/readiness probe paths from /api/v2/app/version (qBittorrent)
  to /health and /health/ready (Fornax Express server). Pod was restarting
  every ~30s due to probe failures against wrong endpoint.
- Add fornax-ui deployment and service for React dashboard
  (image: fornax/ui:latest, nginx on port 80)
- Split ingress: /api and /health → coordinator:3000, / → ui:80
- Update kustomization to include UI resources
2026-04-14 08:45:21 -05:00
Will Anderson bdba692db2 Fornax: COORDINATOR_URL env, port-lease webhook, Grafana dashboard
- Worker deployment: add COORDINATOR_URL and WORKER_ID env vars to
  portforward-helper; webhook calls coordinator /api/v2/workers/:id/port-lease
  after each NAT-PMP renewal so coordinator tracks current forwarded port
- Worker overlays: patch WORKER_ID env var to actual worker ID (tx253/tx34)
- Grafana: fornax-grafana Argo CD app pointing to k8s/fornax/grafana/
  ConfigMap with grafana_dashboard=1 label for auto-discovery by sidecar;
  7 panels covering bandwidth, active torrents, worker health, request rate,
  and error rate
2026-04-14 05:30:01 +00:00
Will Anderson d75d1b62d8 Add fornax.neuralplatform.ai to Cloudflare tunnel ingress rules 2026-04-14 04:43:25 +00:00
Will Anderson 4a0ebededf fix: revert to protonvpn provider, use server name from secret 2026-04-14 00:07:11 +00:00
Will Anderson 1cc783e704 fix: use custom gluetun provider with endpoint IP for ProtonVPN workers 2026-04-14 00:04:50 +00:00
Will Anderson eebd0226de fix: worker gluetun endpoint IP approach, add VPN key secrets 2026-04-14 00:01:43 +00:00
Will Anderson 180fb91c95 add fornax coordinator and worker k8s manifests 2026-04-13 23:57:22 +00:00
Gitea CI 0aa1f4333c chore(neuron): deploy e102af6f86599361b911a2e7e5aa0ec737b54527 2026-04-12 19:58:46 +00:00
Gitea CI e1c15205e3 chore(neuron): deploy 61e9d262595be44c6d7956e41f8a9f275c482678 2026-04-12 16:27:34 +00:00
Gitea CI f91a25fd83 chore(neuron): deploy 0091ebfaa13ef4c58d3a88b3bb3be144ea0a2d9c 2026-04-12 16:18:28 +00:00
Will Anderson 354950a75d Remove voidstash adult media stack (DNS records, tunnel routes) 2026-04-12 04:37:18 -05:00
Will Anderson 538bba323d feat: add prowlarr.voidstash.xyz DNS + tunnel rule 2026-04-12 02:34:35 -05:00
Will Anderson db980af27a monitoring: add WireGuard key expiry reminders for voidstash GL.iNet
Posts to #infrastructure-alerts via Slack bot on escalating schedule:
- 2027-03-11: 1 month out
- 2027-03-25, 04-01, 04-04: weekly
- 2027-04-05 through 04-10: daily

Key: neural_net-US-IL-267, expires 2027-04-11
2026-04-11 13:29:38 -05:00
Will Anderson b8b33e870f voidstash.xyz: add Cloudflare zone, CNAME records, and tunnel ingress rules
Register voidstash.xyz as the private media stack domain.
- DNS delegated from Porkbun to Cloudflare nameservers
- stash.voidstash.xyz and whisparr.voidstash.xyz CNAME to legion tunnel
- Tunnel ingress rules added for both subdomains → Traefik
2026-04-11 12:47:38 -05:00
Will Anderson 4ef625ef3d qbittorrent: bake performance settings into initContainer
Switch qbt-config-patch from busybox/sed to python:3-alpine so we can
cleanly handle INI keys with backslashes. Now seeds both password hash
and high-throughput defaults (3000 max connections, 50 active downloads,
unlimited rate) on fresh PVC deployments. Existing configs are updated
in-place; API-applied values are preserved on restart.
2026-04-11 12:24:16 -05:00
Will Anderson 259b1bdade overseerr: remove Kids & Family library from Jellyfin config 2026-04-11 12:17:49 -05:00
Will Anderson 494b72229c overseerr: seed Sonarr and Radarr config in restore.js
Sonarr and Radarr service settings live in settings.json (not the SQLite
database). Extend the restore.js initContainer to seed both services on
first boot or after a PVC wipe, matching current live config.
2026-04-11 12:17:18 -05:00
Will Anderson c4cd91920a Fix qBittorrent auth: stamp known password hash via initContainer
LocalhostAuthEnabled=false isn't honored in this qBT version. Instead, initContainer
stamps the PBKDF2 hash for admin:adminadmin before startup so all three pods have
consistent credentials. portforward-helper and coordinator restored to cookie-based
SID auth.
2026-04-11 11:14:23 -05:00
Will Anderson 531df76a90 Patch qBittorrent config via initContainer before startup
Adds qbt-config-patch initContainer that sets WebUI\LocalhostAuthEnabled=false
in qBittorrent.conf before the main containers start. Prevents qBittorrent from
overwriting the setting (previously edited at runtime, lost on pod restart).
2026-04-11 11:11:23 -05:00
Will Anderson 854ccd1344 Disable qBittorrent localhost auth; remove credentials from helpers
LocalhostAuthEnabled=false set in all three qbt configs. portforward-helper and
coordinator now call the API directly without auth — no more ban risk from failed
login attempts.
2026-04-11 11:08:54 -05:00
Will Anderson ffd3068f78 Fix port forwarding: read file instead of HTTP API, fix server names
- portforward-helper now reads /tmp/gluetun/forwarded_port via shared emptyDir
  volume instead of polling gluetun HTTP API (which returned Unauthorized)
- Main qbt: SERVER_NAMES=US-IL#1 (valid entry in gluetun server list)
- Both manifests cleaned up
2026-04-11 10:12:19 -05:00
Will Anderson 69976ca172 Fix main qbt server selection; bump portforward-helper memory limit
- Main qbt: switch from pinned US-IL#149 (no PF) to SERVER_REGIONS=Illinois so
  gluetun picks any IL server with port forwarding enabled
- Fornax workers: portforward-helper 32Mi→96Mi to stop OOMKill
2026-04-11 10:03:52 -05:00
Will Anderson 0aff4523af Upgrade coordinator to least-loaded routing (checks active torrent count per worker) 2026-04-11 10:02:45 -05:00
Will Anderson ff6c1f1ee4 Add Fornax coordinator — simple round-robin qBittorrent proxy
Python proxy running on python:3-alpine via ConfigMap. Routes /api/v2/torrents/add
across all three workers in round-robin; proxies status/monitoring to primary.
Manages independent SID sessions to each backend with auto-refresh on 403.
Service: fornax.media.svc:8080 — point Sonarr/Radarr here instead of qbittorrent.
2026-04-11 10:01:03 -05:00
Will Anderson cce8c0a0d4 Fix server names to match gluetun's ProtonVPN server list
US-TX#253 and US-TX#34 aren't in gluetun's built-in list. Switching to:
- Main qbt: US-IL#149 (was US-IL#267)
- Fornax TX253: US-TX#179
- Fornax TX34: US-TX#220
Private keys are per-account and work with any ProtonVPN WireGuard server.
2026-04-11 09:59:23 -05:00
Will Anderson e1996d4396 Switch to gluetun native ProtonVPN port forwarding (VPN_PORT_FORWARDING=on)
Replaces DIY natpmpc sidecar with gluetun's built-in NAT-PMP handling for the protonvpn
provider. The natpmpc UDP response was being dropped by gluetun's firewall since conntrack
doesn't track stateless UDP from the gateway. With VPN_PORT_FORWARDING=on, gluetun handles
the NAT-PMP exchange internally and exposes the port at :8000/v1/openvpn/portforwarded.
Helper sidecar now just polls that endpoint.
2026-04-11 09:57:07 -05:00
Will Anderson 24e308202b Allow NAT-PMP response port through gluetun firewall (FIREWALL_INPUT_PORTS=5351) 2026-04-11 09:50:55 -05:00
Will Anderson 03011a73e8 Add Fornax worker pods — US-TX#253 and US-TX#34
Two initial Fornax distributed torrent workers, each a gluetun+qBittorrent+natpmpc-helper
pod on a different ProtonVPN TX server with NAT-PMP enabled. VPN private keys stored in
Vault at secret/fornax/worker-tx253 and secret/fornax/worker-tx34, surfaced via
ExternalSecrets. Workers share the media-data PVC; each has its own config PVC.

Services: fornax-worker-tx253:8080 and fornax-worker-tx34:8080 (ClusterIP, media ns)
2026-04-11 09:46:18 -05:00
Will Anderson 5f2720ae4e Add Cloudflare provider and neural-platform tunnel config to Terraform
Tunnel routes were being managed ad-hoc via Cloudflare API. Now codified
in Terraform so they can't drift — neuron.neuralplatform.ai was missing
from the tunnel config, causing 404s for Neuron MCP connections.

Also fix mcp.json: type http → sse (Neuron uses SSE transport, not
streamable HTTP). CF-Access headers were already present and correct.
2026-04-11 09:04:14 -05:00
Will Anderson c6aa006ec7 Switch VPN to US-IL#267 (NAT-PMP port forwarding supported) 2026-04-11 08:41:06 -05:00
Will Anderson 3f33a76ed2 media: natpmpc-helper sleep before apk install (VPN must be up first) 2026-04-11 08:29:54 -05:00
Will Anderson ca5f9e90da media: fix natpmpc-helper package name (libnatpmp not natpmpc) 2026-04-11 08:27:53 -05:00
Will Anderson f99965da0a media: fix natpmpc-helper OOM and API call format
Increase memory limit 32→64Mi (apk install was hitting the 32Mi ceiling).
Fix qBittorrent setPreferences call: use form-encoded json=<string> not
json body nesting.
2026-04-11 08:24:19 -05:00
Will Anderson fd08987e12 media: add natpmpc-helper sidecar for ProtonVPN port forwarding
Adds a lightweight Alpine sidecar that runs natpmpc against the ProtonVPN
NAT-PMP gateway (10.2.0.1) every 45s to maintain a forwarded port, then
updates qBittorrent's listen port via API. Without inbound port forwarding
peers can only be dialed out to, which severely limits download speeds on
a gigabit connection.
2026-04-11 08:21:37 -05:00
Will Anderson 615f455eff fix(bazarr): merge duplicate metadata blocks in pod template 2026-04-11 04:40:21 -05:00
Will Anderson 0f15063260 fix(bazarr): patch ffprobe refiner to handle BoxList audio language
Audio tracks in some video files return language as a BoxList (list of
Language objects) from the video metadata parser. The existing code
tries to add this directly to a Python set, which fails because BoxList
is not hashable — crashing subtitle search for every movie.

Fix: check if lang has __iter__ and iterate over it, or call .alpha3
directly if it's a single Language object.

Delivered via ConfigMap volumeMount so it survives pod restarts.
2026-04-11 04:39:33 -05:00
Will Anderson 7c17b8a93e fix(gluetun): remove shell wrapper, use default entrypoint
The /gluetun binary has restrictive permissions and cannot be exec'd
from a shell wrapper. The OOM fix (512Mi limit) prevents the crashes
that caused stale ip rules, so the wrapper is not needed.
2026-04-11 04:31:01 -05:00
Will Anderson 203233beca fix(gluetun): prevent OOM crash and stale ip rule on container restart
- Increase memory limit 128Mi → 512Mi (DNS block list download needs ~200MB)
- Add command wrapper that cleans stale ip rules before exec'ing gluetun
  so container restarts within the same pod don't fail with "file exists"
- Fix deprecated VPN_ENDPOINT_IP/PORT → WIREGUARD_ENDPOINT_IP/PORT
2026-04-11 04:28:32 -05:00
Will Anderson 28a108a1d8 fix(gluetun): pin to US-TX#457 server to restore downloads
The protonvpn named provider rotates through P2P servers and DNS
health checks were failing on every server it tried. Switching back
to custom provider pinned to the specific US-TX#457 server/key combo
from the ProtonVPN conf file, which is confirmed to work.

This sacrifices automatic port forwarding but restores stable downloads.
2026-04-11 04:24:35 -05:00
Will Anderson e5785f3359 gluetun: fix ip rule cleanup in init container for VPN restart stability
Delete ip rule by priority 101 before table 51820 to prevent 'file exists'
errors when gluetun internally restarts the WireGuard connection.
2026-04-11 04:18:56 -05:00
Will Anderson 9cd3dd0134 gluetun: switch to protonvpn provider with port forwarding enabled
Custom WireGuard mode doesn't support NAT-PMP port forwarding — peers
can't connect inbound so most torrents stall. Switching to protonvpn
provider with SERVER_FEATURES=p2p picks a port-forwarding-capable server
and VPN_PORT_FORWARDING=on handles the NAT-PMP handshake. Up command
updates qBittorrent listen port automatically when gluetun gets the port.
2026-04-11 04:09:29 -05:00
Will Anderson d59af5b6aa seerr: fix mediaServerType to numeric 2 (JELLYFIN enum value)
Seerr's scheduler compares mediaServerType against numeric enum values
(PLEX=1, JELLYFIN=2, EMBY=3). restore.js was setting the string 'JELLYFIN'
which never matched, so Jellyfin scan jobs were never registered in the
scheduler — causing 'Watch on Jellyfin' buttons to never appear.
2026-04-11 02:52:15 -05:00
Will Anderson ba339ee5c9 bazarr: increase liveness probe delay to 120s — startup takes 90+s
Bazarr initializes scheduler, DB, and Radarr/Sonarr connections before
serving HTTP. 30s initialDelay was too short, causing restart loops.
Also add readinessProbe and failureThreshold for resilience.
2026-04-11 02:33:41 -05:00
Will Anderson c8d240e131 fix(seerr): persist applicationUrl so Jellyfin OAuth callback works 2026-04-11 02:16:57 -05:00
Will Anderson 7225f56fbc fix(seerr): set public.initialized=true to skip setup wizard on every start 2026-04-11 02:01:54 -05:00
Will Anderson c1d4390393 fix(seerr): persist library config in ConfigMap, restore libraries on pod restart 2026-04-11 01:58:06 -05:00
Will Anderson 280924b1b3 chore(seerr): force pod restart to pick up fix for main.mediaServerType 2026-04-11 01:47:15 -05:00
Will Anderson 4031628f4b fix(seerr): add main.mediaServerType to settings.json to prevent crash
Seerr's checkOverseerrMerge calls Settings.load with raw=true, which
replaces this.data entirely with file contents. If settings.main is
absent (as in our jellyfin-only restore), it crashes at line 418:
'Cannot read properties of undefined (reading apiKey)'.

restore.js now ensures settings.main exists and mediaServerType=JELLYFIN,
which both prevents the crash and causes checkOverseerrMerge to return
early (migration already done).
2026-04-11 01:44:41 -05:00
Will Anderson 71f335c153 seerr: fix init container — use script file instead of inline shell 2026-04-11 01:36:13 -05:00