The stale apps/gitea-runner.yaml contained two Deployment manifests
that conflicted with the canonical Deployments owned by the
gitea-runner-config Argo Application (pointing at k8s/gitea-runner/).
Dual ownership caused Argo CD to fight itself — restarting runner
pods mid-job and producing the "context canceled" failures on
neuron-technologies/dharma-el CI.
Canonical Deployments (config-version 2026-05-04-cf-access-public-url,
docker.sock, CF Access env, replicas=2 for nt-runner) live in
k8s/gitea-runner/deployment.yaml and are managed by gitea-runner-config.
- dns-neuralplatform.tf: add vault.neuralplatform.ai A record → 34.54.164.21 (GCP LB)
DNS-only (not proxied) so GCP managed TLS cert can provision correctly
- main.tf: remove vault.neuralplatform.ai from Cloudflare tunnel ingress
(now served directly via GCP Global HTTPS LB)
- main.tf: remove watch.nook.family, jellyfin.nook.family, bazarr.nook.family
from tunnel ingress (nook.family media stack retired; infra is Neuron-focused)
GCE Vault cluster already initialized and running (3-node Raft, active since
2026-05-04T16:05). Secrets migrated 48/48 from k3s vault. ESO ClusterSecretStore
validated against new vault. k3s vault-0 is now superseded.
The containerd socket is mounted from the host at /run/k3s/containerd/containerd.sock
to /var/run/docker.sock inside the container, but act_runner config was pointing to
the host path directly. Fixes CrashLoopBackOff on all runner pods.
Also codifies both deployments (will org + neuron-technologies org) as git-managed
manifests so Argo CD owns them going forward.
The buildah-compiled image (imported directly into k3s) is the working
version. The registry image will catch up after the next CI build.
IfNotPresent ensures the locally-cached correct binary is used.
Soma moved to GCP Cloud Run. The soma-*.yaml files were deleted but
the kustomization still referenced them, blocking all Argo CD syncs
for neuron-prod (including the new allow-daemon-ingress NetworkPolicy).
The ExternalSecret had a stale Gitea URL (192.168.8.148:30322 — old
local IP) and runner tokens were never populated in Vault. Update the
template URL to the cluster-internal Gitea service DNS, and store the
registration tokens in Vault at secret/gitea.runner_token.
backup-verify: restic restores /dump/all-databases.sql relative to
restore target, so the check path was wrong (/all-databases.sql →
/dump/all-databases.sql). Also fix the --include filter to match.
redpanda-topics-init: redpanda is intentionally scaled to 0; the init
job will block forever waiting for a broker. Mark it suspended so Argo
CD won't keep creating a stuck job. Re-enable when redpanda is turned on.
The default-deny-all NetworkPolicy was blocking dharma from reaching
neuron-daemon port 7750 (event queue / engram endpoint). Add an explicit
ingress policy for neuron-daemon that allows traffic from the neuron-prod
namespace, so dharma and other internal services can connect at startup.
- Dockerfile for Go DHARMA server (registry.neuralplatform.ai/neuron-technologies/dharma)
- k8s manifests: deployment, service, ExternalSecret (Vault), IngressRoute
- Argo CD app: deploys to neuron-prod namespace
- Cloudflare tunnel rule for dharma.neurontechnologies.ai
- DNS CNAME record for dharma.neurontechnologies.ai → legion tunnel
- Vault secrets already populated (api_key, encryption_key, jwt_secret, engram_key, engram_url)
- Connects to neuron-daemon (engram) at neuron-daemon.neuron-prod.svc.cluster.local:7749
The registry currently allows anonymous cluster-internal pulls so the
imagePullSecrets reference was failing the externalsecret reconcile
(secret/data/mudcraft has no registry creds for this app). Pod was
pulling fine anyway. If the registry later gets locked down, add a
proper externalsecret using the path from secret/data/<app> instead of
borrowing mudcraft's.
The legion-apps Application root was hardcoded to the OLD legion's gitea
cluster IP (10.43.1.53:3000). On the new GCP legion, the gitea service
got a different cluster IP (10.43.15.98), so the umbrella has been
ComparisonError-stuck since the cluster move on 2026-04-28. That meant
new Application manifests in apps/ never got materialized in argocd —
neuron-web was the first new app to need it after the move.
All other apps reference http://gitea.git.svc.cluster.local:3000/...
(the stable DNS name). Bringing the three legion-apps spots in argocd.tf
in line with that convention so the next gitea cluster-IP shuffle
doesn't re-break the umbrella.
Stand up the elb-built native El landing server on Legion k3s alongside
the existing Cloud Run prod marketing site. Image is built on
gitea-runner-1 from elb-emitted .c files + native cc and pushed to
registry.neuralplatform.ai/neuron-web:dev-dfe41234. Deployment exposes
web-stage.neuralplatform.ai via Traefik + Cloudflare tunnel.
The point is to have a working local instance we can iterate against
while we redesign the CI/CD pipeline; nothing here replaces the Cloud
Run deploy yet.
* k8s/neuron-technologies/web/ — namespace, deployment, service, ingress,
registry-pull-secret + externalsecret pulling anthropic/supabase keys
from Vault
* apps/neuron-web.yaml — Argo CD Application
* dns-neuralplatform.tf, main.tf — CNAME + tunnel ingress for
web-stage.neuralplatform.ai
dev.neurontechnologies.ai mirrors stage in shape but looser:
broken builds OK, auto-deploys on every push to dev branch (once
CI lands), dev Supabase project (isolated), test Stripe keys,
RESEND_DRY_RUN=1 so transactional email never actually sends.
Locked behind Cloudflare Access with team policy: anyone with
@neurontechnologies.ai email plus named external collaborators
via the access app's emails list.
Initial image is marketing:fix-gallery-render-1236 (matches stage
at the time of this commit). Future deploys land via CI workflow.
- ExternalSecret: pull neuron_api_key from Vault into NEURON_API_KEY
- ConfigMap: set OTEL_EXPORTER_OTLP_ENDPOINT to alloy-otlp in-cluster service
instead of localhost (fixes 196 restart OTel connection errors)
- neuron-dev: update neuron-rest and neuron-mcp from dev-d7a587e2 to v0.18.14
- neuron-stage: update neuron-rest/mcp from stage-0f5b2328 to v0.18.14; marketing from :latest to 1e94e8ae
- swarm-alpha/beta/gamma: update neuron-rest and neuron-mcp from v0.15.3 to v0.18.14
- ci runners: fix docker socket path from /var/run/docker.sock to /run/k3s/containerd/containerd.sock (k3s node has no Docker daemon)
Expose the MCP endpoint directly at the root domain so Bearer token
clients can reach it without going through the OAuth subdomain.
The Spring Boot MCP layer already accepts ntn_* API keys via
ApiKeyAuthenticationFilter; this ingress rule enables access via
https://neurontechnologies.ai/mcp.
Dynamic PVC provisioning needed on new GCP instance.
Redis image 7.4.3-debian-12-r0 was removed from Docker Hub;
use floating 7.4-debian-12 tag instead.
Was missing from tunnel ingress rules, causing 404 on all registry
requests through Cloudflare. Disabled chunked encoding for Docker
push compatibility.
Cloudflare tunnel not yet running on new instance. Use cluster-internal
URL until tunnel is established. Revert to neuralplatform.ai once tunnel
is configured.
Single replica required for in-memory pending leads Map used by SMS
double-opt-in flow. ExternalSecret adds Resend, Twilio, and Turnstile
keys to the marketing pod environment.
Will Anderson
will.anderson