Deploying in Production

Production Centaur is a Kubernetes deployment with durable API state in Postgres, sandbox pods for agent execution, and iron-proxy for credential injection. The goal is a small working deployment with a clear operator before you add more tools, workflows, harnesses, or overlays.

Production Shape

The API saves threads, runs, and events in Postgres. The Kubernetes backend creates sandbox pods for agent work. iron-proxy handles outbound requests that need credentials:

Each pod receives the prompt files, environment, proxy CA, proxy settings, and command it needs for one assigned thread. It should not receive raw model keys or third-party API keys.

1. Choose the Operating Boundary

Before installing, decide:

Good first deployments have one narrow engineering, research, support, security, data, or operations workflow where agents can call real tools.

2. Create the Infra Secret

The Helm chart reads infrastructure values from an existing Kubernetes Secret. By default that Secret is named centaur-infra-env:

secretManager:
  existingSecretName: centaur-infra-env
  envPrefix: ""

For local development, just bootstrap-secrets creates this Secret from your shell environment. In production, create it through your normal secret delivery path before installing the chart.

Minimum keys:

The Centaur Console is the permission control plane and is a required chart component. Add these keys to the same infra Secret:

The bootstrap user is created only when the Console has no users. Do not rotate the ActiveRecord encryption keys as an ordinary credential rotation: existing encrypted records become unreadable without an explicit key-migration plan.

Generate distinct high-entropy values for each enabled ingress service key. Store them in the infra Secret. api-rs rejects duplicate configured tokens at startup.

3. Configure Harness Credentials

Store one secret per enabled harness credential:

sandbox.harnessEngine and the selectors above apply when a session is created. A session records its harness and keeps it for the life of the thread, so changing the chart value does not move existing threads onto another harness. Start a new thread to pick one up.

In normal sandbox mode, containers receive placeholder values such as OPENAI_API_KEY=OPENAI_API_KEY. iron-proxy swaps the placeholder for the real key on outbound requests, only on the hosts and headers the secret is bound to.

When ironProxy.secretSource is onepassword, iron-proxy resolves these values from op://$OP_VAULT/<SECRET_NAME>/credential. For example, store the default Codex credential in a 1Password item named OPENAI_API_KEY. To run Codex through OpenRouter, store OPENROUTER_API_KEY and set OPENROUTER_MODEL to a model slug such as openrouter/auto, or set CODEX_MODEL_PROVIDER=openrouter alongside CODEX_MODEL. Per-turn Codex model overrides with provider-style slugs such as --model anthropic/claude-fable-5 also select the OpenRouter provider even when OPENROUTER_MODEL is unset.

To run Codex through Meta AI direct, store META_AI_API_KEY and select the provider with --meta. Pair it with --model <model-id> when choosing a provider-specific model for a turn.

To register a private OpenAI-compatible Responses provider, add it once under codex.customProviders:

codex:
  customProviders:
    private_responses:
      name: Private Responses
      baseUrl: https://inference.example.com/v1
      apiKeyEnv: PRIVATE_RESPONSES_API_KEY
      defaultModel: example-model

Store the named key in the configured iron-proxy secret source. The chart sends the non-secret provider catalog to api-rs, sandboxes, chat ingresses, and the Console. api-rs registers a bearer-token replacement restricted to the baseUrl DNS host; sandboxes receive only the apiKeyEnv placeholder. Select the provider with --provider private_responses; --model overrides its configured default for that turn. If defaultModel is omitted, callers must pass --model; the Console lists only custom providers that define a default.

The backing vault can be shared across the deployment, but access is not. The Centaur Console grants each user, channel, issue, or workflow principal only the roles and secrets it should use. The infra role is assigned to new principals by default so they can run a model harness. Tool credentials are separate and should be granted deliberately. Configure that baseline in Advanced Permissioning.

Codex Auth Modes

Codex supports two authentication modes, selected per deployment with sandbox.codexAuthMode in the chart values. api-rs reads the resulting CODEX_AUTH_MODE env var to register the matching proxy credential with the console and propagates it into each sandbox, so the agent's auth.json and the injected credential always agree. Do not set CODEX_AUTH_MODE through sandbox.extraEnv: that reaches sandbox pods but not api-rs, which is the component that acts on the mode.

access_token mode routes Codex through a ChatGPT account rather than a raw API key. The console owns the refresh token as the openai-codex broker credential: its background worker refreshes it and mints short-lived access tokens, which the per-sandbox proxy injects on outbound requests so the sandbox never sees them. The refresh token is stored encrypted in the console's own database. It is not read from or synced to your secrets backend.

The commands below assume you are at the root of a Centaur checkout and have configured IRON_CONTROL_URL and IRON_CONTROL_API_KEY as described in Configure the Operator CLI.

To bootstrap access_token mode:

1. Log in locally with the dedicated ChatGPT account and force this login to use a file-backed credential store:

   codex login -c 'cli_auth_credentials_store="file"'
   CODEX_AUTH_FILE="${CODEX_HOME:-$HOME/.codex}/auth.json"
   export OPENAI_CODEX_ACCOUNT_ID="$(jq -er '.tokens.account_id' "$CODEX_AUTH_FILE")"
   export OPENAI_CODEX_REFRESH_TOKEN="$(jq -er '.tokens.refresh_token' "$CODEX_AUTH_FILE")"

   The credential-store override is intentional. Codex can otherwise use the operating system keyring, which leaves no auth.json file to read.

2. Store OPENAI_CODEX_ACCOUNT_ID in your secrets backend (1Password vault, Kubernetes Secret, etc.). iron-proxy injects this ChatGPT account UUID as the chatgpt-account-id header so the backend routes to the right workspace.

3. Create the broker credential with the refresh token from the same login:

   cargo run --manifest-path services/api-rs/Cargo.toml -p centaur-perms -- \
     broker create --foreign-id openai-codex \
     --token-endpoint https://auth.openai.com/oauth/token \
     --client-id app_EMoamEEZ73f0CkXaXp7hrann \
     --refresh-token "$OPENAI_CODEX_REFRESH_TOKEN"
   unset OPENAI_CODEX_REFRESH_TOKEN

   The client id is the Codex CLI's fixed, publicly known OAuth client id: the same for every Codex install; it is passed here, not stored in the secrets backend.

4. Start (or restart) api-rs. At startup it registers the access-token fragment with the console; if the openai-codex broker credential does not exist yet, the console rejects the registration with a 422 and api-rs fails fast, so create the credential first.

Claude Auth Modes

Claude Code supports two authentication modes, selected per deployment with sandbox.claudeCodeAuthMode in the chart values. It has the same contract as sandbox.codexAuthMode above: api-rs registers the matching proxy credential and propagates CLAUDE_CODE_AUTH_MODE into each sandbox, so do not set the env var through sandbox.extraEnv.

access_token mode routes Claude Code through a Claude.ai Pro or Max subscription rather than a raw API key. The console owns the refresh token as the anthropic-claude broker credential and mints short-lived access tokens, which the per-sandbox proxy injects as the Bearer on outbound requests so the sandbox never sees them. The sandbox entrypoint plants a dummy ~/.claude/.credentials.json so the CLI emits OAuth-shaped requests; the proxy overwrites the Bearer at request time. This mode needs no secrets-backend items.

The commands below assume you are at the root of a Centaur checkout and have configured IRON_CONTROL_URL and IRON_CONTROL_API_KEY as described in Configure the Operator CLI.

To bootstrap, run claude login locally with the dedicated Claude.ai account, then export the refresh token. On systems where Claude Code writes a credentials file, run:

export CLAUDE_CODE_REFRESH_TOKEN="$(
  jq -er '.claudeAiOauth.refreshToken' "$HOME/.claude/.credentials.json"
)"

If Claude Code used the macOS keychain instead, run:

export CLAUDE_CODE_REFRESH_TOKEN="$(
  security find-generic-password -s 'Claude Code-credentials' -w |
    jq -er '.claudeAiOauth.refreshToken'
)"

Create the broker credential from the root of the checkout:

cargo run --manifest-path services/api-rs/Cargo.toml -p centaur-perms -- \
  broker create --foreign-id anthropic-claude \
  --token-endpoint https://platform.claude.com/v1/oauth/token \
  --client-id 9d1c250a-e61b-44d9-88ed-5944d1962f5e \
  --refresh-token "$CLAUDE_CODE_REFRESH_TOKEN"
unset CLAUDE_CODE_REFRESH_TOKEN

The client id is Claude Code's fixed, publicly known OAuth client id. As with Codex, api-rs fails fast at startup if the broker credential is missing.

4. Configure Advanced Permissioning

Set console.publicUrl and configure console.ingress if operators need to reach the Console outside the cluster. Sign in as the bootstrap admin, then open System Settings before creating user sessions:

• Keep infra as a default role if every admitted user may run an agent.
• Clear default roles if even harness access must be explicitly approved.
• Set default repo-cache access to none, public, or all.
• Decide whether new sandboxes may reach observability and the api-rs control plane.

Defaults apply only to principals created later. After users or channels start their first session, assign tool roles or direct secret grants from Principals. For automated provisioning, use centaur-perms with the same secret-source policy and tool directories as api-rs.

Follow Advanced Permissioning for principal mapping, tool registration, role design, revocation, and end-to-end verification.

5. Configure Slack

Create the Slackbot app at api.slack.com/apps. Use the app page to install the bot, copy the Bot User OAuth Token for SLACK_BOT_TOKEN, and copy the Signing Secret for SLACK_SIGNING_SECRET.

1. Add the bot scopes required by the Slackbot features you enable.
2. Install the app to the workspace.
3. Store the Bot User OAuth Token as SLACK_BOT_TOKEN.
4. Store the app Signing Secret as SLACK_SIGNING_SECRET.
5. Enable Event Subscriptions.
6. Set the Request URL to https://<your-host>/api/webhooks/slack.
7. Subscribe to app_mention and to the message events you want Centaur to see: message.channels, message.groups, and message.im. To automatically join newly-created public channels, set slackbotv2.autoJoinCreatedChannels to true and subscribe to channel_created.
8. Enable Interactivity and set its Request URL to the same https://<your-host>/api/webhooks/slack URL. Block Kit actions are emitted to the workflow engine as slack.block_action.<action_id> events.

The Slackbot normalizes Slack app_mention and message events plus block_actions interactions. When SLACKBOTV2_AUTO_JOIN_CREATED_CHANNELS is enabled, it also joins newly-created public channels from subscribed channel_created events. The Slack app needs channels:read to receive those events and channels:join for the auto-join behavior. Do not rely on assistant-specific Slack event types unless the Slackbot code has explicit support for them.

Do not put Centaur API-key auth in front of /api/webhooks/slack; the Slackbot validates Slack's signature and then calls the Centaur API separately.

The Slackbot accepts Slack events at /api/webhooks/slack. It also registers compatibility paths for /api/slack/events, /api/slack/actions, /api/slack/options, and /api/slack/commands.

For Slack Connect channels, follow Enable Centaur for External Slack Channels before adding Centaur to the channel.

6. Deploy With Helm

The chart lives at contrib/chart. Select service images, iron-proxy secret source, sandbox image, and optional runtime class in your values file:

secretManager:
  existingSecretName: centaur-infra-env
  envPrefix: ""
 
api:
  executionWorkerEnabled: true
  warmPoolEnabled: true
 
ironProxy:
  secretSource: onepassword-connect
  secretTtl: 10m
 
apiRs:
  # Delete any sandbox older than this, running or suspended.
  sandboxMaxLifetimeSecs: 259200
 
onepasswordConnect:
  connect:
    create: true
    credentialsName: centaur-onepassword-connect-credentials
    credentialsKey: 1password-credentials.json
 
sandbox:
  image:
    repository: centaur-agent
    tag: latest
    pullPolicy: IfNotPresent
  runtimeClassName: gvisor

The Kubernetes sandbox backend is the active runtime backend; there is no chart switch named api.sandboxBackend.

Sandbox lifecycle has two separate timers:

• Slackbot v2 sends idle_timeout_ms on execute requests, defaulting to up to 3 hours, so api-rs can pause an idle sandbox after a turn finishes.
• api-rs deletes old sandboxes through apiRs.sandboxMaxLifetimeSecs, default 72 hours, regardless of whether the sandbox is still running or already suspended.

There is no suspended-only delete setting. If you want sandboxes gone after N hours, set apiRs.sandboxMaxLifetimeSecs to N hours in seconds.

Install or upgrade:

helm lint contrib/chart
helm upgrade --install centaur contrib/chart \
  --namespace centaur-system \
  --create-namespace \
  -f values.production.yaml

Schedule Sandboxes Onto a Dedicated Node Pool / RuntimeClass

Agent sandboxes and their paired iron-proxy pods are created at runtime by api-rs. Set sandbox.runtimeClassName, sandbox.nodeSelector, and sandbox.tolerations to pin both onto a dedicated pool and/or RuntimeClass (e.g. gVisor). The chart passes them to api-rs as SESSION_SANDBOX_RUNTIME_CLASS_NAME, SESSION_SANDBOX_NODE_SELECTOR, and SESSION_SANDBOX_TOLERATIONS.

sandbox:
  runtimeClassName: gvisor
  nodeSelector:
    workload: centaur-sandbox
  tolerations:
    - key: centaur.ai/sandbox
      operator: Exists
      effect: NoSchedule

Malformed selector/toleration JSON fails api-rs startup. Empty values keep default scheduling.

7. Verify the Deployment

Check health from inside the api-rs deployment first:

kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- \
  curl -fsS http://localhost:8080/healthz
 
kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- \
  curl -fsS http://localhost:8080/readyz | jq

Run one agent turn from inside the api-rs deployment:

THREAD_KEY=cli:production-smoke-codex
THREAD_PATH=$(jq -rn --arg v "$THREAD_KEY" '$v|@uri')
CENTAUR_API_TOKEN=$(kubectl exec -n centaur-system deploy/centaur-centaur-console -- \
  bin/rails runner 'print ApiServer::Jwt.encode_for_console_service')
 
SESSION=$(kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- curl -s -X POST "http://localhost:8080/api/session/${THREAD_PATH}" \
  -H "Authorization: Bearer ${CENTAUR_API_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{"harness_type":"codex","on_harness_conflict":"restart"}')
 
kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- curl -s -X POST "http://localhost:8080/api/session/${THREAD_PATH}/messages" \
  -H "Authorization: Bearer ${CENTAUR_API_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","parts":[{"type":"text","text":"Reply with exactly PONG."}]}]}'
 
EXECUTE=$(kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- curl -s -X POST "http://localhost:8080/api/session/${THREAD_PATH}/execute" \
  -H "Authorization: Bearer ${CENTAUR_API_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{"input_lines":["{\"type\":\"user\",\"message\":{\"content\":[{\"type\":\"text\",\"text\":\"Reply with exactly PONG.\"}]}}"]}')
EXECUTION_ID=$(printf '%s' "$EXECUTE" | jq -r '.execution_id')
 
kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- curl -s -N \
  -H "Authorization: Bearer ${CENTAUR_API_TOKEN}" \
  "http://localhost:8080/api/session/${THREAD_PATH}/events?execution_id=${EXECUTION_ID}&after_event_id=0"

Then run the same prompt through Slack:

reply with exactly PONG

Slack messages without a harness flag use Codex. Use --amp, --claude, --codex, or --pi only when you want to select a specific harness.

Inspect sandbox pods with the labels Centaur actually sets:

kubectl get pods -n centaur-system -l centaur.ai/managed=true
kubectl exec -n centaur-system <agent-sandbox-pod> -- centaur-tools list

If a run fails because the sandbox pod exits or is deleted, inspect the durable session and api-rs logs before retrying:

kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- curl -s \
  "http://localhost:8080/api/session/${THREAD_PATH}" | jq
 
kubectl logs -n centaur-system deploy/centaur-centaur-api-rs --tail=200
kubectl get pods -n centaur-system -l centaur.ai/managed=true

Centaur preserves the execution row and event trail; retry by starting a new turn after you understand whether the failure was credentials, image pull, network policy, harness startup, or the upstream model/tool call.

8. Keep the Operating Loop Small

Before expanding the deployment, record:

1. The operator.
2. Where secrets live.
3. How to restart the stack.
4. The first working Slack channel.
5. The enabled harnesses.
6. The first useful tool or workflow.
7. How to inspect logs and failed runs.

The operator's job is to leave behind a repeatable operating loop, not a one-time demo.