diff --git a/Agents.md b/Agents.md index 697531a..e6937b5 100644 --- a/Agents.md +++ b/Agents.md @@ -58,14 +58,11 @@ export VAULT_ADDR=http://localhost:8200 ### How secrets flow to pods ``` -Vault → TF_VAR_* (via infrastructure.env) → kubernetes_secret (Terraform) → pod env via secretKeyRef +Vault (source of truth) → ExternalSecret (in git) → ESO operator → k8s Secret → pod env ``` -Secrets in Terraform: -- `neuron-secrets` (neuron ns): GITHUB_WEBHOOK_SECRET, GITEA_WEBHOOK_SECRET, SLACK_BOT_TOKEN, SLACK_SIGNING_SECRET -- `cloudflared-secret` (neuron ns): TUNNEL_TOKEN -- `github-runner-secret` (ci ns): ACCESS_TOKEN -- `gitea-db` (git ns): password +All k8s Secrets are managed by External Secrets Operator (ESO) pulling from Vault. +No secrets are stored in Terraform. Do not add `kubernetes_secret` resources to Terraform. --- @@ -75,17 +72,28 @@ Secrets in Terraform: | Layer | Tool | What it owns | |-------|------|-------------| -| Infrastructure | **Terraform** | Namespaces, PVCs, ConfigMaps, k8s Secrets, Ingresses, Helm releases | -| Applications | **Argo CD** | Deployments, Services | +| Infrastructure | **Terraform** | Namespaces, Cloudflare DNS/tunnels, R2 storage, Vault bootstrap | +| Configuration | **Argo CD** | All k8s resources: Deployments, Services, ConfigMaps, Ingresses, PVCs, ExternalSecrets, Helm releases | +| Secrets | **ESO → Vault** | All k8s Secrets — ExternalSecret manifests in git, values pulled from Vault at runtime | **Never modify k8s resources directly with kubectl.** Always go through Terraform or Argo CD (Git push). +**Secrets flow:** +``` +Vault (source of truth) → ExternalSecret (in git) → ESO operator → k8s Secret → pod env +``` + +Adding a new secret: +1. `vault kv put secret/ field=value` +2. Add ExternalSecret manifest to `k8s//` referencing that path +3. Push → Argo CD applies ExternalSecret → ESO syncs Secret within ~60s + ### Running Terraform ```bash -cd ~/Development/infrastructure/servers/servers/legion/ -direnv exec . terraform plan # load secrets + plan -direnv exec . terraform apply # apply +cd ~/Development/infrastructure/servers/legion/ +terraform plan # uses direnv to load credentials +terraform apply ``` State is stored in Cloudflare R2 bucket `legion-terraform-state`. @@ -95,20 +103,22 @@ State is stored in Cloudflare R2 bucket `legion-terraform-state`. Root app `legion-apps` watches `servers/legion/apps/` in the `will/infrastructure` Gitea repo. Push to main → Argo CD syncs within ~30 seconds. -App manifests: `~/Development/infrastructure/servers/servers/legion/apps/*.yaml` - -Argo CD UI: https://argocd.neuralplatform.ai +- App manifests: `~/Development/infrastructure/servers/legion/apps/*.yaml` +- k8s config: `~/Development/infrastructure/servers/legion/k8s//*.yaml` +- Argo CD UI: https://argocd.neuralplatform.ai +- Gitea CLI: `tea pr ls`, `tea issue ls` (configured, default login: neuralplatform) ### Repo layout ``` ~/Development/ infrastructure/ ← this repo (local only, no GitHub remote) - Agents.md ← you are here + AGENTS.md ← you are here servers/ ← nested repo (will/infrastructure on Gitea) - servers/legion/ - *.tf ← Terraform (infrastructure layer) - apps/*.yaml ← Argo CD manifests (app layer) + legion/ + *.tf ← Terraform (infra layer: namespaces, Cloudflare, R2) + apps/*.yaml ← Argo CD Application manifests (app definitions) + k8s//*.yaml ← k8s config synced by Argo CD (PVCs, ExternalSecrets, etc.) neural-platform/ neuron/ ← github: harmonic-framework/neuron, gitea: neural-platform/neuron harmonic-framework/ @@ -237,28 +247,26 @@ Runner labels: `self-hosted,linux,x64,legion` ### Deploy a config change to an app -1. Edit `servers/servers/legion/apps/.yaml` +1. Edit `servers/legion/apps/.yaml` or `servers/legion/k8s//.yaml` 2. `git commit && git push` (from `servers/`) 3. Argo CD syncs in ~30s ### Add a new secret to a pod -1. Add `kubernetes_secret` resource to the relevant `.tf` file (value from `var.X`) -2. Add `variable "X"` to `variables.tf` (sensitive = true) -3. Add `TF_VAR_X=$(_v path field)` to `infrastructure.env` -4. Store the secret: `vault kv put secret/path field=value` -5. `direnv exec . terraform apply` -6. Reference `secretKeyRef: name: ` in the app manifest +1. Store the secret: `vault kv put secret/path field=value` +2. Add/update an ExternalSecret in `servers/legion/k8s//external-secrets.yaml` +3. Reference `secretKeyRef: name: ` in the app manifest +4. Push → ESO syncs the k8s Secret within ~60s -### Add a new Helm service (stays in Terraform) +### Add a new Helm release -Add `helm_release` resource to appropriate `.tf` file, apply. +Add an Argo CD Application manifest to `servers/legion/apps/.yaml` with `spec.source.chart` and `spec.source.helm.values`. Push to deploy. ### Rotate a secret -1. Update value in Vault: `vault kv patch secret/path field=newvalue` -2. Re-run `direnv exec . terraform apply` (updates kubernetes_secret) -3. Restart the affected pod: `kubectl rollout restart deployment/ -n ` +1. `vault kv patch secret/path field=newvalue` +2. Force ESO refresh: `kubectl annotate externalsecret -n force-sync=$(date +%s) --overwrite` +3. Restart pod if needed: `kubectl rollout restart deployment/ -n ` --- @@ -275,8 +283,10 @@ Add `helm_release` resource to appropriate `.tf` file, apply. ## What NOT To Do -- `kubectl apply` directly — use Terraform or push to Git -- `kubectl edit` — use Terraform or Git -- Hardcode any secret value — always use `var.X` → Vault +- `kubectl apply` directly — push to Git and let Argo CD sync +- `kubectl edit` — edit the file in `servers/legion/`, push to Git +- Hardcode any secret value — store in Vault, reference via ExternalSecret +- Add `kubernetes_secret` to Terraform — secrets belong in ESO/Vault, not Terraform - Commit `terraform.tfstate` — state is in R2, never local -- Call Gitea API from Mac directly — CF Access blocks it, use SSH to Legion +- Call Gitea API from Mac directly — CF Access blocks it; use `tea` CLI or SSH to Legion +- Use `var.X` pattern for secrets in Terraform — Terraform no longer owns k8s Secrets