Harden prod: security, autoscaling, observability, BuildKit CI
Security: - Drop ALL capabilities, enforce non-root, RuntimeDefault seccomp on neuron-mcp, neuron-rest, neuron-marketing pods - Add startup probes (150s window for JVM) so liveness doesn't fire early - Replace docker-sock hostPath with BuildKit rootless TCP endpoint (moby/buildkit:v0.19.0-rootless) — removes node root access from CI - Document full ESO AppRole migration path in cluster-secret-store.yaml Autoscaling & availability: - HPAs on mcp (1–6), rest (1–4), marketing (2–8) at 65–70% CPU - PodDisruptionBudgets (minAvailable: 1) on all three services - NetworkPolicy: default-deny-all in neuron-prod, explicit allow rules for Traefik ingress, intra-namespace, and egress to DNS/platform/vault Observability: - ServiceMonitors for mcp, rest, marketing (cross-namespace enabled in kube-prometheus-stack with serviceMonitorSelectorNilUsesHelmValues:false) - PrometheusRules: high error rate, high latency, crash loops, replica shortage, Postgres down/connections, backup failure, backup staleness Chart version pinning: - kube-prometheus-stack, loki, tempo, redis, alloy, postgres — all pinned to major-version ranges to block silent breaking upgrades Backup hardening: - restic:latest → restic:0.17.3 (deterministic image) - Weekly backup-verify CronJob: restores latest snapshot and validates SQL dump structure (≥5 CREATE TABLE, pg_dump header check) ArgoCD: - neuron-prod AppProject: scopes deploys to neuron-prod + platform ns, blacklists ClusterRole/ClusterRoleBinding/Namespace creation, automated sync window 2–6am UTC, manual always allowed
This commit is contained in:
@@ -0,0 +1,124 @@
|
||||
---
|
||||
# NetworkPolicies for neuron-prod
|
||||
# Default-deny all ingress and egress, then explicitly allow only what's needed.
|
||||
# Enforced by k3s's built-in NetworkPolicy controller.
|
||||
|
||||
# ── Baseline: deny all traffic in namespace ───────────────────────────────────
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-deny-all
|
||||
namespace: neuron-prod
|
||||
spec:
|
||||
podSelector: {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
---
|
||||
# ── neuron-mcp: accept traffic from Traefik (kube-system) only ───────────────
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: allow-mcp-ingress
|
||||
namespace: neuron-prod
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app: neuron-mcp
|
||||
policyTypes:
|
||||
- Ingress
|
||||
ingress:
|
||||
- from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: kube-system
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: neuron-prod
|
||||
---
|
||||
# ── neuron-rest: accept from mcp, kube-system (Traefik), monitoring (Alloy) ──
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: allow-rest-ingress
|
||||
namespace: neuron-prod
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app: neuron-rest
|
||||
policyTypes:
|
||||
- Ingress
|
||||
ingress:
|
||||
- from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: kube-system
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: neuron-prod
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: monitoring
|
||||
---
|
||||
# ── neuron-marketing: accept from Traefik only ───────────────────────────────
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: allow-marketing-ingress
|
||||
namespace: neuron-prod
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app: neuron-marketing
|
||||
policyTypes:
|
||||
- Ingress
|
||||
ingress:
|
||||
- from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: kube-system
|
||||
---
|
||||
# ── Egress: all prod pods may reach platform (postgres/redis), vault,
|
||||
# monitoring (alloy OTLP), kube-dns, and the internet (external APIs) ─
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: allow-prod-egress
|
||||
namespace: neuron-prod
|
||||
spec:
|
||||
podSelector: {}
|
||||
policyTypes:
|
||||
- Egress
|
||||
egress:
|
||||
# Kubernetes DNS
|
||||
- ports:
|
||||
- port: 53
|
||||
protocol: UDP
|
||||
- port: 53
|
||||
protocol: TCP
|
||||
# platform namespace (Postgres, Redis, accounts service)
|
||||
- to:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: platform
|
||||
# vault namespace
|
||||
- to:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: vault
|
||||
# monitoring namespace (Alloy OTLP)
|
||||
- to:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: monitoring
|
||||
# intra-namespace (mcp ↔ rest ↔ license)
|
||||
- to:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: neuron-prod
|
||||
# external internet (Stripe, OAuth providers, Cloudflare, etc.)
|
||||
- ports:
|
||||
- port: 443
|
||||
protocol: TCP
|
||||
- port: 80
|
||||
protocol: TCP
|
||||
Reference in New Issue
Block a user