LLM Providers
Shiplight's AI-powered features — natural language actions, self-healing locators, and VERIFY assertions — call an LLM. Deterministic YAML steps do not, so a test made only of explicit locators and assertions runs without any provider configured.
You have two ways to supply credentials:
- Bring your own key from Anthropic, OpenAI, or Google. Simple, and you're billed by that provider.
- The Shiplight LLM proxy, using a Shiplight API token instead of a provider key. One credential for all three providers, billed with the rest of your Shiplight usage.
Either way, put the values in a .env file in your project root — the CLI discovers it automatically. Keep .env in .gitignore.
Bring your own key
At least one of these is required. The first one set determines the default model.
| Variable | Description |
|---|---|
GOOGLE_API_KEY | Google AI Studio key (Gemini models) — get a key |
ANTHROPIC_API_KEY | Anthropic API key (Claude models) — get a key |
OPENAI_API_KEY | OpenAI API key (GPT / o-series models) |
Choosing a model
These apply when you bring your own key — you pay the provider, so you choose the model. On the Shiplight LLM proxy the model is chosen by tier instead and these variables are ignored; see Choosing a tier.
| Variable | Description | Default |
|---|---|---|
WEB_AGENT_MODEL | Web agent model, e.g. claude-sonnet-5. Accepts a provider:model prefix for azure:, bedrock:, and vertex: | Detected from the first API key set |
WEB_AGENT_FALLBACK_MODELS | Comma-separated provider:model chain tried when the primary is unavailable. Set to an empty string ("") to disable it. | A built-in cross-provider chain |
COMPUTER_USE_MODEL | Model for coordinate-based computer-use actions | Auto-detected |
OpenAI-compatible endpoints
| Variable | Description | Default |
|---|---|---|
OPENAI_BASE_URL | Base URL for OpenAI-compatible APIs such as Ollama, vLLM, or LiteLLM. Applies to both the regular LLM path and the computer-use path | https://api.openai.com/v1 |
Routing through Vertex AI
Set one of the *_USE_VERTEXAI flags to route a provider through Google Vertex AI instead of calling it directly. Either flag requires GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_LOCATION, and expects GOOGLE_APPLICATION_CREDENTIALS (or Application Default Credentials) to be available.
| Variable | Description | Default |
|---|---|---|
ANTHROPIC_MODELS_USE_VERTEXAI | Route Anthropic (Claude) calls through Vertex AI | false |
GOOGLE_GENAI_USE_VERTEXAI | Route Google (Gemini) calls through Vertex AI | false |
GOOGLE_CLOUD_PROJECT | GCP project ID (required when either flag is set) | — |
GOOGLE_CLOUD_LOCATION | GCP region, e.g. us-central1 (required when either flag is set) | — |
Shiplight LLM proxy
Instead of a provider key, you can authenticate with a Shiplight API token and let Shiplight call the provider:
SHIPLIGHT_API_TOKEN=shp_pat_…Setting this is what puts the CLI in proxy mode. Create the token at app.shiplight.ai/api-tokens with the llm:invoke scope.
Provider keys take precedence
The CLI treats a provider API key as an instruction to use it directly. If ANTHROPIC_API_KEY, OPENAI_API_KEY, or GOOGLE_API_KEY is present it leaves proxy mode, even when a Shiplight token is also set. To use the proxy, remove the provider keys from your environment.
Choosing a tier
On the proxy, Shiplight picks the model and you pick a tier — whoever pays for the LLM calls chooses the model, and on the token path that is Shiplight. Set the tier with WEB_AGENT_TIER:
WEB_AGENT_TIER=standard # lite | standard | pro| Tier | Best for |
|---|---|
lite | Fastest and lowest cost |
standard | A balance of capability and cost |
pro | The most capable models, for the hardest steps |
When WEB_AGENT_TIER is unset, the run uses your organization's default tier: Enterprise organizations default to standard, and other plans default to lite. Shiplight can set a different default for your organization on request. These defaults come from your organization's settings, read at the start of each run; a run that can't load them falls back to lite. Shiplight maps each tier to a current model and manages cross-provider fallbacks for provider outages on your behalf; billing always reflects the model that actually ran. Coordinate-based computer-use actions are not tiered — Shiplight always selects that model.
WEB_AGENT_MODEL is ignored on the proxy
Because Shiplight chooses the model on the token path, WEB_AGENT_MODEL, WEB_AGENT_FALLBACK_MODELS, and COMPUTER_USE_MODEL have no effect when you authenticate with SHIPLIGHT_API_TOKEN and no provider key. The CLI prints a one-line notice when it ignores them, so the switch is never silent. To choose the model yourself, bring your own key instead — then you pick the model and pay the provider directly.
Usage visibility
Whichever option you choose, Shiplight records how many tokens your tests consumed and reports them in Usage, broken down by operation, provider, and model. What changes is whether a cost is attached:
| Routing | When it applies | In Usage |
|---|---|---|
proxy | Calls went through the Shiplight LLM proxy | Counted and billed |
byok | You supplied your own provider key | Counted, no cost |
custom_endpoint | You pointed OPENAI_BASE_URL at your own endpoint | Counted, no cost |
Routing is detected per provider, so a run that uses a direct key for one provider and the proxy for another labels each correctly. Set SHIPLIGHT_USAGE_ROUTING to proxy, byok, or custom_endpoint to override the whole run.
Shiplight Cloud turns this usage data into practical cost guidance. Ask /shiplight cloud to analyze LLM token usage and Shiplight-hosted runner compute together, identify what is driving cost, and suggest where to optimize. See Analyze usage with /shiplight cloud for details.
On CI
The same choice applies in CI. On most runners you supply credentials yourself as secrets, exactly as you would locally.
On Shiplight-hosted CI runners the proxy is the default: credentials are injected per run and revoked when it ends, so a workflow that sets no provider key uses the proxy without any configuration. Shiplight never places a provider key of its own on the machine.
You can still bring your own key. Anything your workflow sets takes precedence over the defaults Shiplight injects, so a provider key set at the workflow, job, or step level wins:
jobs:
e2e:
runs-on: shiplight-medium
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}The CLI then treats that key as a direct-provider credential and stops using the proxy for that provider.
What this costs you
LLM calls made with your own key are paid to your provider and don't draw down your Shiplight allowance. You keep the visibility either way: Shiplight still records their token counts and shows them in Usage with no cost attached. Runner compute minutes are metered normally.
Related
- CLI Reference — every environment variable the CLI reads
- API Tokens — token scopes and the proxy in detail
- CI/CD — wiring credentials into your pipeline