Skip to content

Harness integration

The lab is also the developer productivity backend: the local AI coding harnesses — Hermes, OpenAI Codex, Claude Code, OpenCode, and Pi — all consume the same LiteLLM gateway instead of individual provider keys.

The setup

Every harness is pointed at the gateway with two environment variables:

OPENAI_BASE_URL=https://gateway.example/v1    # the LiteLLM endpoint
OPENAI_API_KEY=sk-litellm-<virtual key>       # a virtual key, not a provider key

Because the endpoint is OpenAI-compatible, each harness defines a litellm provider that exposes the four stable tier aliases — keeping the model picker small instead of surfacing all 60+ aliases:

Alias Use when
chat-fast Cheap, fast, no reasoning — mechanical edits
chat-smart Default — 1M context, function calling
chat-balanced Middle ground — 1M context, function calling
research-deep Deep reasoning, long turns — reserve for hard problems

Per-harness wiring (the pattern)

Hermes (config.yaml): the primary model stays whatever the operator chose; LiteLLM aliases are available on demand and registered as the fallback chain:

model:
  aliases:
    litellm-fast:     { model: chat-fast,     provider: custom, base_url: <gateway>/v1 }
    litellm-smart:    { model: chat-smart,    provider: custom, base_url: <gateway>/v1 }
    litellm-balanced: { model: chat-balanced, provider: custom, base_url: <gateway>/v1 }
    litellm-research: { model: research-deep, provider: custom, base_url: <gateway>/v1 }
fallback_providers:
  - provider: custom
    model: smart-router
    base_url: <gateway>/v1
    api_key: ${LITELLM_API_KEY}

OpenCode (opencode.json): an OpenAI-compatible provider whose default model is the tier alias:

{
  "model": "litellm/chat-smart",
  "provider": {
    "litellm": {
      "name": "LiteLLM",
      "npm": "@ai-sdk/openai-compatible",
      "options": { "baseURL": "{env:LITELLM_BASE_URL}", "apiKey": "{env:LITELLM_API_KEY}" },
      "models": {
        "chat-fast":     { "name": "Chat Fast (cheapest, FC)" },
        "chat-smart":    { "name": "Chat Smart (1M ctx, FC)" },
        "chat-balanced": { "name": "Chat Balanced (1M ctx, FC)" },
        "research-deep": { "name": "Research Deep (reasoning, long)" }
      }
    }
  }
}

Pi (agent/models.json): the same aliases with explicit context windows and reasoning flags.

Operating rules that make it work

  • Prefer the cheapest tier that answers; escalate only on real failure — enforced by convention, alias naming, and the smart router's default.
  • Reasoning models need headroom: with tight max_tokens budgets a reasoning model can spend everything on reasoning_content and return an empty reply — keep max_tokens ≥ ~512 for the reasoning tiers.
  • Claude Code and Codex normally use their own subscriptions — but they can be pointed at the gateway through their OpenAI-compatible provider options with the same endpoint and key, which is how the lab keeps every harness with a fallback path.

Why centralize harness models?

  • One key to rotate instead of per-tool keys.
  • One catalog to evolve: a new model is available to every harness the moment it is added to the gateway — no per-tool configuration.
  • One place to observe cost: harness usage lands in the same spend telemetry as the agent fleet.
  • The smart router serves as a universal fallback: when a local default provider fails, the harness chain lands on smart-router, which still answers.

The public, MIT-licensed agent-dev-kit takes the same discipline further: a multi-runtime system for reproducible AI-assisted development — layered architecture (communication, planning, capabilities, verification, safety), adversarial PR review, prompt-injection defenses, browser QA, and planted-bug evaluations.