18 Commits

Author SHA1 Message Date
Will Anderson 086a7b0bf0 add DocuSeal to GKE — manifests, Terraform, Argo CD app; update ci-base Dockerfile 2026-05-05 10:38:45 -05:00
Will Anderson 422cca6962 scale gitea-runner to 0 while Autopilot DinD blocker is pending
Set replicas: 0 to prevent crash-loop restarts. The runner requires a Docker
daemon (even in host mode) which Autopilot won't provide. Scale to 1 when a
Standard node pool or external Docker TCP endpoint is available.
2026-05-05 10:22:21 -05:00
Will Anderson 9a368986b2 fix gitea-runner deployment for GKE Autopilot constraints
Switch to host-mode labels and document the DinD limitation: Autopilot's
Warden blocks privileged containers at the node level. Use internal Gitea
service URL (gitea.gitea.svc.cluster.local) to bypass Cloudflare Access.
Document three resolution paths (Standard node pool, Cloud Build, external
Docker TCP). GCE VM runner remains active in the interim.
2026-05-05 10:22:02 -05:00
Will Anderson c31edc8b83 deploy gitea CI runner to GKE as k8s pod
Replaces the GCE VM runner with a k8s Deployment in the ci namespace on
neuron-platform GKE. Uses Docker-in-Docker for build isolation since
Autopilot doesn't expose the node socket. Runner token pulled from Secret
Manager via ESO + Workload Identity.

- servers/gcp/k8s/gitea-runner/: namespace, serviceaccount, external-secrets,
  deployment manifests (ci namespace, dind sidecar, idempotent registration)
- servers/gcp/k8s/argocd-apps/gitea-runner-gke.yaml: Argo CD Application
- servers/gcp/gitea-runner.tf: gitea-runner-gke GCP SA with secretAccessor
  on gitea-runner-token, Workload Identity binding for ci/gitea-runner,
  artifactregistry.reader for pulling ci-base image
2026-05-05 10:16:49 -05:00
Will Anderson 3462ea37a4 Add ignoreDifferences for Deployment status in gitea-gke
Argo CD's bundled schema is missing terminatingReplicas from the
Deployment status spec, causing a ComparisonError on every sync.
Ignoring /status entirely is safe — Argo CD never manages status
fields; they're owned by kube-controller-manager.
2026-05-05 05:07:15 -05:00
Will Anderson 82e05429f1 Fix Argo CD child app repoURL: k3s Gitea → GKE Gitea
gitea-gke and vault-gke were pointing at the dead k3s-internal Gitea
(http://gitea.git.svc.cluster.local:3000). gke-apps already uses
https://git.neuralplatform.ai and syncs successfully — update child apps
to match. Once gke-apps picks this up, gitea-gke will sync the pending
GITEA__actions__ENABLED=true commit.
2026-05-05 05:04:01 -05:00
Will Anderson 686041946b Enable Gitea Actions (GITEA__actions__ENABLED=true) 2026-05-05 04:56:29 -05:00
Will Anderson cebaf44db3 Scale GKE Vault to 1 replica while raft standby deadlock is unresolved
Pods 1 and 2 can't unseal because port 8201 (cluster port) only opens
after barrier initialization, but the barrier key transfer requires 8201
to be reachable — deadlock. Running single-node on pod 0 so
vault.neuralplatform.ai reliably routes to the active, unsealed instance.
2026-05-05 04:37:20 -05:00
Will Anderson 2e5655c583 vault: rolling restart to fix raft join; remove sealedok from readiness probe 2026-05-05 03:59:33 -05:00
Will Anderson 804060c958 fix vault raft join addresses for GKE StatefulSet naming 2026-05-04 23:56:10 -05:00
Will Anderson 872a834989 Cut over Vault and Gitea traffic from GCE/Legion to GKE
Vault:
- Annotate vault-helm-gke Service with cloud.google.com/neg exposed_ports
  to create container-native NEGs (k8s1-bfbeff02-vault-...) in 3 zones
- Add vault-api-from-lb-gke firewall rule allowing GCP health check ranges
  (130.211.0.0/22, 35.191.0.0/16) to reach GKE pod IPs on port 8200
- Replace GCE instance group backends in google_compute_backend_service.vault
  with GKE NEG backends (RATE balancing mode, 100 req/endpoint)
- GCP Global HTTPS LB frontend unchanged — DNS stays at 34.54.164.21
- vault.neuralplatform.ai now terminates at GKE pods (all 3 NEGs HEALTHY)

Gitea:
- Change GKE Gitea Service from ClusterIP to LoadBalancer (external IP: 34.31.145.131)
- Add Cloudflare DNS A record for git.neuralplatform.ai → 34.31.145.131 (proxied)
- Remove git.neuralplatform.ai route from Legion Cloudflare tunnel config
- Add Cloudflare config rule: flexible SSL for git.neuralplatform.ai
  (origin serves HTTP/3000, CF proxies HTTPS termination)
- Scale Legion Gitea deployment to 0 replicas (PVC preserved)
- git.neuralplatform.ai now serves from GKE Gitea pod
2026-05-04 23:46:20 -05:00
Will Anderson 1c99931de6 fix(vault-gke): switch vault-1/2 to standard-rwo storage class
SSD quota (SSD_TOTAL_GB) is exhausted at 500/500 GB. pd-ssd PVCs for vault-1
and vault-2 cannot provision. Switching to standard-rwo (pd-balanced) unblocks
the Vault HA cluster immediately.

Quota increase request submitted (Cloud Quotas preference API, preference ID
0aa576b1-5050-4fa7-911d-6f408230df7f, target 1000GB). Once approved, can
optionally migrate back to premium-rwo for all-SSD Vault storage.
2026-05-04 22:41:38 -05:00
Will Anderson 3d581368c3 fix(vault-gke): relax topology spread to ScheduleAnyway while SSD quota is exhausted
vault-1 and vault-2 are stuck Pending because GCE quota (SSD_TOTAL_GB 500/500)
prevents new nodes from provisioning. ScheduleAnyway lets them land in any zone
that has capacity while the quota increase request is pending.

Also adds ESO Workload Identity IAM bindings to Terraform state (previously applied
out-of-band via gcloud; now tracked in cloud-sql.tf).
2026-05-04 22:37:17 -05:00
Will Anderson 8592a1ed74 fix(gitea-gke): configure ESO SecretStore to use workload identity with gitea SA 2026-05-04 22:05:01 -05:00
Will Anderson 9252b069a2 fix(gitea-gke): update ESO API version from v1beta1 to v1 2026-05-04 22:00:21 -05:00
Will Anderson c4f7dcc185 fix(vault-gke): raise cpu request to 500m for GKE Autopilot anti-affinity minimum 2026-05-04 21:59:51 -05:00
Will Anderson b572f5720b infra: wire GKE cluster endpoint and fix gitea SA account_id
- Replace GKE_CLUSTER_ENDPOINT placeholder with real endpoint (34.63.89.52)
  in all three Argo CD Application manifests
- Fix gitea GCP SA account_id from 'gitea' (5 chars, too short) to 'gitea-gke'
  to satisfy GCP's 6-30 char constraint
- Update serviceaccount.yaml annotation to match new SA email (gitea-gke@...)
2026-05-04 21:56:06 -05:00
Will Anderson 9330107fcc migrate Vault and Gitea to GKE Autopilot cluster
Implements Option A: move Vault (3x GCE e2-small) and Gitea (Legion k8s)
onto a new GKE Autopilot cluster (neuron-platform, us-central1) managed
through Legion Argo CD.

Terraform (servers/gcp/):
- gke.tf: GKE Autopilot cluster, Workload Identity bindings for Vault (KMS)
  and Gitea (Cloud SQL)
- cloud-sql.tf: gitea database + user on neuron-prod-pg15, gitea GCP SA,
  gitea-database-url and gitea-db-password Secret Manager secrets
- vault-nodes.tf: PENDING DECOMMISSION comment with migration checklist

k8s manifests (servers/gcp/k8s/):
- vault/: namespace.yaml
- gitea/: namespace, serviceaccount (Workload Identity annotation), pvc (50Gi
  standard-rwo), deployment (Gitea + Cloud SQL Auth Proxy sidecar), service,
  configmap (custom CSS carried from Legion), external-secrets (GCP SM provider)
- argocd-apps/: vault-gke.yaml, vault-helm-gke.yaml (Helm chart, HA Raft 3
  replicas, GCP KMS auto-unseal, topologySpread across zones, 10Gi premium-rwo),
  gitea-gke.yaml — all target GKE_CLUSTER_ENDPOINT placeholder

Legion (servers/legion/):
- apps/gke-apps.yaml: App-of-Apps entry point on Legion Argo CD that syncs the
  GKE Application manifests
- k8s/gitea-runner/Dockerfile: add system-level git insteadOf so GKE CI runners
  resolve Gitea in-cluster without Cloudflare Access headers
2026-05-04 20:40:48 -05:00