Secrets (Infisical)

Infisical is the lab's single source of truth for secrets. Every credential — provider keys, broker tokens, app passwords, webhook signatures — lives there once, and everything else references it.

One source of truth, projected into the cluster by the operator, consumed by deployments — and read directly by operators via CLI and web.

Why a secrets manager on top of Kubernetes Secrets

• One place to rotate. Change a value in Infisical; the operator projects it into the cluster; the deployment picks it up. No hunting through sealed manifests and pod envs.
• Audit and access control. Who can see which secret is policy, not convention — including the humans operating the lab.
• Secrets never need to exist in plaintext in git. The git repository stores only encrypted SealedSecrets for bootstrapping; live values come from Infisical.
• Works across the boundary. The same value can be consumed by cluster workloads (via the operator) and by local machines (via the CLI) — e.g., the LiteLLM master key used by local coding harnesses.

The rotation workflow (exercised routinely)

1. Update the value in Infisical.
2. The operator updates the projected Cluster Secret.
3. Restart/roll the affected deployment.
4. Smoke-test the new value (UI login, API 200, gateway ready).
5. Re-seal the Secret into a SealedSecret manifest and commit it.
6. Log the change in the changelog.

Guardrails

• Plaintext secrets are never committed: repository rules + gitleaks scanning in CI + a dedicated sealed-secrets discipline for any Kubernetes Secret change.
• The provider broker and LiteLLM proxy mount provider credentials only from these managed Secrets — tenant gateways never reference them.
• Human-facing passwords live in Infisical/1Password, outside git entirely.