will.anderson/infrastructure
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

migrate Vault and Gitea to GKE Autopilot cluster

Browse Source 
Implements Option A: move Vault (3x GCE e2-small) and Gitea (Legion k8s)
onto a new GKE Autopilot cluster (neuron-platform, us-central1) managed
through Legion Argo CD.

Terraform (servers/gcp/):
- gke.tf: GKE Autopilot cluster, Workload Identity bindings for Vault (KMS)
  and Gitea (Cloud SQL)
- cloud-sql.tf: gitea database + user on neuron-prod-pg15, gitea GCP SA,
  gitea-database-url and gitea-db-password Secret Manager secrets
- vault-nodes.tf: PENDING DECOMMISSION comment with migration checklist

k8s manifests (servers/gcp/k8s/):
- vault/: namespace.yaml
- gitea/: namespace, serviceaccount (Workload Identity annotation), pvc (50Gi
  standard-rwo), deployment (Gitea + Cloud SQL Auth Proxy sidecar), service,
  configmap (custom CSS carried from Legion), external-secrets (GCP SM provider)
- argocd-apps/: vault-gke.yaml, vault-helm-gke.yaml (Helm chart, HA Raft 3
  replicas, GCP KMS auto-unseal, topologySpread across zones, 10Gi premium-rwo),
  gitea-gke.yaml — all target GKE_CLUSTER_ENDPOINT placeholder

Legion (servers/legion/):
- apps/gke-apps.yaml: App-of-Apps entry point on Legion Argo CD that syncs the
  GKE Application manifests
- k8s/gitea-runner/Dockerfile: add system-level git insteadOf so GKE CI runners
  resolve Gitea in-cluster without Cloudflare Access headers
This commit is contained in:
Will Anderson
2026-05-04 20:40:48 -05:00
parent cc368d2ad1
commit 9330107fcc
15 changed files with 822 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
servers/gcp/k8s/argocd-apps/gitea-gke.yaml
+38
View File
@@ -0,0 +1,38 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: gitea-gke
namespace: argocd
annotations:
# Apply to the Legion Argo CD instance after registering the GKE cluster.
# See vault-gke.yaml for the cluster registration steps.
#
# Migration checklist (Gitea from Legion to GKE):
# 1. terraform apply (creates GKE cluster, Cloud SQL gitea DB, secrets)
# 2. Register GKE cluster with Legion Argo CD (see vault-gke.yaml)
# 3. Install ESO on GKE: helm install external-secrets external-secrets/external-secrets
# --namespace external-secrets --create-namespace
# 4. Apply this Application to Legion Argo CD
# 5. Verify gitea pod is running on GKE with DB connectivity
# 6. Take a tar of /data from the Legion Gitea pod and restore to GKE PVC
# 7. Update Cloudflare Tunnel on Legion: remove git.neuralplatform.ai route
# 8. Add GKE ingress / GCP LB rule for git.neuralplatform.ai
# 9. Decommission Gitea on Legion (remove gitea*.yaml from servers/legion/apps/)
spec:
project: default
source:
repoURL: http://gitea.git.svc.cluster.local:3000/will/infrastructure.git
targetRevision: main
path: servers/gcp/k8s/gitea
destination:
# Replace GKE_CLUSTER_ENDPOINT after `terraform apply`:
# terraform -chdir=servers/gcp output -raw gke_cluster_endpoint
server: https://GKE_CLUSTER_ENDPOINT
namespace: gitea
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
- ServerSideApply=true
servers/gcp/k8s/argocd-apps/vault-gke.yaml
+33
View File
@@ -0,0 +1,33 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: vault-gke
namespace: argocd
annotations:
# Syncs plain k8s manifests for Vault to the GKE cluster.
# The Vault Helm release itself is in vault-helm-gke.yaml.
#
# Apply to Legion Argo CD after registering the GKE cluster:
# 1. gcloud container clusters get-credentials neuron-platform \
# --region us-central1 --project neuron-785695
# 2. argocd cluster add <context-name> --name gke-neuron-platform
# 3. Update GKE_CLUSTER_ENDPOINT in all argocd-apps/*.yaml:
# terraform -chdir=servers/gcp output -raw gke_cluster_endpoint
spec:
project: default
source:
repoURL: http://gitea.git.svc.cluster.local:3000/will/infrastructure.git
targetRevision: main
path: servers/gcp/k8s/vault
destination:
# Replace GKE_CLUSTER_ENDPOINT after `terraform apply`:
# terraform -chdir=servers/gcp output -raw gke_cluster_endpoint
server: https://GKE_CLUSTER_ENDPOINT
namespace: vault
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
- ServerSideApply=true
servers/gcp/k8s/argocd-apps/vault-helm-gke.yaml
+153
View File
@@ -0,0 +1,153 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: vault-helm-gke
namespace: argocd
annotations:
# Deploys the Vault Helm chart to the GKE cluster via Legion Argo CD.
# The destination.server must be updated after `terraform apply`:
# terraform -chdir=servers/gcp output -raw gke_cluster_endpoint
spec:
project: default
source:
repoURL: https://helm.releases.hashicorp.com
chart: vault
targetRevision: "0.29.1"
helm:
values: |
global:
enabled: true
injector:
enabled: false
ui:
enabled: true
server:
image:
repository: hashicorp/vault
tag: "1.19.2"
# Workload Identity — Vault pod k8s SA impersonates vault-unseal GCP SA
# for KMS auto-unseal. Binding is in servers/gcp/gke.tf.
serviceAccount:
create: true
name: vault
annotations:
iam.gke.io/gcp-service-account: vault-unseal@neuron-785695.iam.gserviceaccount.com
# GKE Autopilot: no privileged containers. Vault doesn't need privilege.
# Request IPC_LOCK so Vault can lock memory (prevents secrets swap).
securityContext:
capabilities:
add:
- IPC_LOCK
# HA mode — 3 replicas with Raft storage
ha:
enabled: true
replicas: 3
raft:
enabled: true
setNodeId: true
config: |
ui = true
listener "tcp" {
tls_disable = 1
address = "[::]:8200"
cluster_address = "[::]:8201"
}
storage "raft" {
path = "/vault/data"
retry_join {
leader_api_addr = "http://vault-0.vault-internal:8200"
}
retry_join {
leader_api_addr = "http://vault-1.vault-internal:8200"
}
retry_join {
leader_api_addr = "http://vault-2.vault-internal:8200"
}
}
seal "gcpckms" {
project = "neuron-785695"
region = "global"
key_ring = "vault"
crypto_key = "vault-unseal"
}
telemetry {
prometheus_retention_time = "30s"
disable_hostname = false
}
# Spread pods across GKE zones
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app.kubernetes.io/name: vault
component: server
# 10Gi SSD per pod — premium-rwo = pd-ssd on GKE Autopilot
dataStorage:
enabled: true
size: 10Gi
storageClass: premium-rwo
accessMode: ReadWriteOnce
readinessProbe:
enabled: true
path: "/v1/sys/health?standbyok=true&sealedok=true&uninitcode=200"
initialDelaySeconds: 5
periodSeconds: 5
failureThreshold: 3
livenessProbe:
enabled: true
path: "/v1/sys/health?standbyok=true&sealedok=true&uninitcode=200"
initialDelaySeconds: 60
periodSeconds: 10
failureThreshold: 3
# GKE Autopilot requires resource requests on all containers
resources:
requests:
memory: 256Mi
cpu: 250m
limits:
memory: 512Mi
cpu: 500m
service:
enabled: true
type: ClusterIP
port: 8200
targetPort: 8200
# Ingress disabled — Vault is exposed via GCP HTTPS LB.
# After migration, update the existing LB backend (vault-nodes.tf)
# to target a GKE NEG instead of the GCE instance groups.
# See: https://cloud.google.com/kubernetes-engine/docs/how-to/standalone-neg
ingress:
enabled: false
destination:
# Replace GKE_CLUSTER_ENDPOINT after `terraform apply`:
# terraform -chdir=servers/gcp output -raw gke_cluster_endpoint
server: https://GKE_CLUSTER_ENDPOINT
namespace: vault
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
- ServerSideApply=true
servers/gcp/k8s/gitea/configmap.yaml
+104
View File
@@ -0,0 +1,104 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: gitea-custom-css
namespace: gitea
data:
header.tmpl: |
<link rel="stylesheet" href="/assets/css/custom.css">
custom.css: |
/* ── Typography & base ── */
:root {
--color-primary: #6366f1;
--color-primary-dark: #4f46e5;
--color-secondary: #8b5cf6;
--color-bg: #0f0f17;
--color-surface: #16161f;
--color-surface-2: #1e1e2e;
--color-border: #2a2a3d;
--color-text: #e2e2f0;
--color-text-muted: #8888aa;
--color-green: #22d3a5;
--radius: 10px;
}
body {
background: var(--color-bg) !important;
color: var(--color-text) !important;
font-family: "Inter", -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif !important;
}
/* ── Top navbar ── */
.navbar, #navbar {
background: var(--color-surface) !important;
border-bottom: 1px solid var(--color-border) !important;
box-shadow: 0 1px 12px rgba(0,0,0,0.4) !important;
}
.navbar .brand svg, .navbar .brand img { filter: brightness(1.2); }
/* ── Sidebar & panels ── */
.repository, .ui.container, .ui.segment,
.ui.card, .ui.cards > .card {
background: var(--color-surface) !important;
border: 1px solid var(--color-border) !important;
border-radius: var(--radius) !important;
}
/* ── Buttons ── */
.ui.primary.button, .ui.green.button {
background: var(--color-primary) !important;
border: none !important;
border-radius: 8px !important;
}
.ui.primary.button:hover { background: var(--color-primary-dark) !important; }
/* ── Inputs ── */
input, textarea, select,
.ui.input > input, .ui.dropdown {
background: var(--color-surface-2) !important;
border: 1px solid var(--color-border) !important;
color: var(--color-text) !important;
border-radius: 8px !important;
}
/* ── Sign-in page ── */
.user.signin .ui.segment,
.user.signup .ui.segment {
background: var(--color-surface) !important;
border: 1px solid var(--color-border) !important;
border-radius: 16px !important;
box-shadow: 0 8px 32px rgba(0,0,0,0.5) !important;
padding: 2.5rem !important;
}
/* ── Repo file tree ── */
.repository.file.list .file-list {
background: var(--color-surface) !important;
border-radius: var(--radius) !important;
border: 1px solid var(--color-border) !important;
}
.repository.file.list .file-list tr:hover td {
background: var(--color-surface-2) !important;
}
/* ── Labels & badges ── */
.ui.label { border-radius: 6px !important; }
/* ── Dashboard activity feed ── */
.feeds .news { border-bottom: 1px solid var(--color-border) !important; }
/* ── Code blocks ── */
pre, code {
background: var(--color-surface-2) !important;
border: 1px solid var(--color-border) !important;
border-radius: 6px !important;
}
/* ── Muted footer ── */
#footer {
background: var(--color-surface) !important;
border-top: 1px solid var(--color-border) !important;
color: var(--color-text-muted) !important;
}
servers/gcp/k8s/gitea/deployment.yaml
+155
View File
@@ -0,0 +1,155 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: gitea
namespace: gitea
labels:
app: gitea
spec:
replicas: 1
# Recreate — only one pod can hold the RWO PVC at a time.
# Scale to multiple replicas only after adding shared storage (e.g. Filestore).
strategy:
type: Recreate
selector:
matchLabels:
app: gitea
template:
metadata:
labels:
app: gitea
spec:
serviceAccountName: gitea
containers:
- name: gitea
image: gitea/gitea:1.25.5
ports:
- name: http
containerPort: 3000
- name: ssh
containerPort: 22
env:
# Database — connect through the Cloud SQL Auth Proxy unix socket
- name: GITEA__database__DB_TYPE
value: postgres
# Unix socket path used by the Cloud SQL Auth Proxy sidecar
- name: GITEA__database__HOST
value: /cloudsql/neuron-785695:us-central1:neuron-prod-pg15
- name: GITEA__database__NAME
value: gitea
- name: GITEA__database__USER
value: gitea
- name: GITEA__database__PASSWD
valueFrom:
secretKeyRef:
name: gitea-db
key: password
# Server
- name: GITEA__server__DOMAIN
value: git.neuralplatform.ai
- name: GITEA__server__ROOT_URL
value: https://git.neuralplatform.ai
- name: GITEA__server__SSH_DOMAIN
value: git.neuralplatform.ai
- name: GITEA__server__SSH_PORT
value: "22"
- name: GITEA__server__START_SSH_SERVER
value: "false"
# Service
- name: GITEA__service__DISABLE_REGISTRATION
value: "true"
- name: GITEA__service__REQUIRE_SIGNIN_VIEW
value: "false"
# Security
- name: GITEA__security__INSTALL_LOCK
value: "true"
# Packages
- name: GITEA__packages__ENABLED
value: "true"
# Webhooks — allow calls back into the cluster and GKE VPC
- name: GITEA__webhook__ALLOWED_HOST_LIST
value: 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
# Actions
- name: GITEA__actions__DEFAULT_ACTIONS_URL
value: https://code.forgejo.org
volumeMounts:
- name: data
mountPath: /data
- name: cloudsql-socket
mountPath: /cloudsql
- name: custom-css
mountPath: /data/gitea/custom/public/assets/css/custom.css
subPath: custom.css
- name: custom-css
mountPath: /data/gitea/custom/templates/custom/header.tmpl
subPath: header.tmpl
resources:
requests:
memory: 256Mi
cpu: 100m
limits:
memory: 512Mi
cpu: 500m
readinessProbe:
httpGet:
path: /
port: 3000
initialDelaySeconds: 10
periodSeconds: 10
failureThreshold: 3
livenessProbe:
httpGet:
path: /
port: 3000
initialDelaySeconds: 30
periodSeconds: 15
failureThreshold: 3
# Cloud SQL Auth Proxy sidecar — provides unix socket at /cloudsql/
# Authenticates to Cloud SQL using Workload Identity (no key file).
- name: cloud-sql-proxy
image: gcr.io/cloud-sql-connectors/cloud-sql-proxy:2
args:
- "--structured-logs"
- "--unix-socket=/cloudsql"
- "neuron-785695:us-central1:neuron-prod-pg15"
securityContext:
runAsNonRoot: true
volumeMounts:
- name: cloudsql-socket
mountPath: /cloudsql
resources:
requests:
memory: 32Mi
cpu: 10m
limits:
memory: 128Mi
cpu: 100m
volumes:
- name: data
persistentVolumeClaim:
claimName: gitea-data
- name: cloudsql-socket
emptyDir: {}
- name: custom-css
configMap:
name: gitea-custom-css
---
apiVersion: v1
kind: Service
metadata:
name: gitea
namespace: gitea
spec:
selector:
app: gitea
ports:
- name: http
port: 3000
targetPort: 3000
- name: ssh
port: 22
targetPort: 22
type: ClusterIP
servers/gcp/k8s/gitea/external-secrets.yaml
+68
View File
@@ -0,0 +1,68 @@
---
# SecretStore for GKE — uses GCP Secret Manager directly via Workload Identity.
# On GKE we use the GCP provider instead of a Vault-backed store, since
# Vault itself may be in the process of being migrated.
# The Gitea GCP SA has secretmanager.secretAccessor on its own secret (see cloud-sql.tf).
#
# Pre-requisite: install ESO on GKE before applying this:
# helm install external-secrets external-secrets/external-secrets \
# --namespace external-secrets --create-namespace
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
name: gcp-secretmanager
namespace: gitea
spec:
provider:
gcpsm:
projectID: neuron-785695
# Workload Identity is used automatically — no explicit auth needed
# when the pod's ServiceAccount has the iam.gke.io/gcp-service-account annotation.
---
# gitea-db — Gitea database password, pulled from GCP Secret Manager.
# The Secret Manager secret "gitea-database-url" stores the full DSN, but we
# extract just the password field for use in GITEA__database__PASSWD.
#
# The full DSN format from Terraform:
# host=/cloudsql/<conn> user=gitea password=<pw> dbname=gitea sslmode=disable
#
# ESO extracts the raw secret value. Since GCP Secret Manager stores the full
# DSN as a single string, we store the password separately as "gitea-db-password"
# so Gitea can receive it as a discrete env var.
#
# Bootstrap: after `terraform apply`, run:
# PASSWORD=$(gcloud secrets versions access latest --secret=gitea-database-url \
# | grep -oP '(?<=password=)\S+')
# echo -n "$PASSWORD" | gcloud secrets create gitea-db-password \
# --data-file=- --project=neuron-785695
#
# Or simpler — let Terraform write it directly. The gitea-db-password secret
# is managed by the gitea-database-url secret version output. Use the full DSN
# secret and parse in-pod, or store password separately.
#
# For simplicity: pull the full DSN and use it as GITEA__database__PASSWD
# is wrong (it's a DSN, not a password). Instead, use the Cloud SQL proxy
# unix socket and no password — configure Gitea to use peer auth.
#
# ACTUAL APPROACH: ExternalSecret pulls the full DSN string into a k8s Secret
# key "dsn". A separate gitea-db secret provides just the password field.
# Terraform outputs both; add a gitea-db-password Secret Manager secret in cloud-sql.tf.
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: gitea-db
namespace: gitea
spec:
refreshInterval: 1h
secretStoreRef:
name: gcp-secretmanager
kind: SecretStore
target:
name: gitea-db
creationPolicy: Owner
data:
- secretKey: password
remoteRef:
# This secret is populated by Terraform (gitea-db-password in cloud-sql.tf)
# It contains just the raw database password (no DSN prefix).
key: gitea-db-password
servers/gcp/k8s/gitea/namespace.yaml
+6
View File
@@ -0,0 +1,6 @@
apiVersion: v1
kind: Namespace
metadata:
name: gitea
labels:
app.kubernetes.io/name: gitea
servers/gcp/k8s/gitea/pvc.yaml
+14
View File
@@ -0,0 +1,14 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: gitea-data
namespace: gitea
spec:
# standard-rwo = pd-balanced on GKE Autopilot (ReadWriteOnce)
# Use premium-rwo (pd-ssd) if repo performance becomes a bottleneck.
storageClassName: standard-rwo
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 50Gi
servers/gcp/k8s/gitea/serviceaccount.yaml
+10
View File
@@ -0,0 +1,10 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: gitea
namespace: gitea
annotations:
# Workload Identity — allows the Cloud SQL Auth Proxy sidecar to authenticate
# to Cloud SQL as the gitea GCP SA without a JSON key file.
# The GCP SA binding is in servers/gcp/gke.tf (gitea_workload_identity).
iam.gke.io/gcp-service-account: gitea@neuron-785695.iam.gserviceaccount.com
servers/gcp/k8s/vault/namespace.yaml
+6
View File
@@ -0,0 +1,6 @@
apiVersion: v1
kind: Namespace
metadata:
name: vault
labels:
app.kubernetes.io/name: vault