VS Code Extension

View as Markdown

Debug and author AI-powered web tests visually, right inside VS Code.

Shiplight lets you create, run, and debug .test.yaml test files using an interactive visual debugger — no browser tab switching required.

Features

Debug a test

Right-click any *.test.yaml file > Debug with Shiplight, or use the Command Palette.

The debugger opens in a webview panel where you can:

• Step through test statements one at a time
• Inspect and edit action entities inline
• See the browser session in real time
• Modify test steps and re-run immediately

Create a new test

Right-click a folder > Shiplight: New Test. Enter a name and starting URL — the extension scaffolds a new .test.yaml file and opens the debugger so you can start authoring immediately.

Getting Started

1. Install the Shiplight CLI

npm install -g shiplightai

2. Add an API key

Create a .env file in your test project directory:

ANTHROPIC_API_KEY=sk-ant-...
# or
GOOGLE_API_KEY=...

The extension auto-detects the right AI model based on which key you provide.

3. Debug your first test

Open a *.test.yaml file and run Debug with Shiplight from the Command Palette (Cmd+Shift+P).

Configuration

Login credentials

For tests that require authentication, place a shiplight.config.json next to your test files:

{
  "url": "https://app.example.com",
  "username": "user@example.com",
  "password": "password"
}

Organize by subdirectory to scope credentials per project:

tests/
  .env                      # API keys (shared by all projects)
  public-site/              # no config = no login
    search.test.yaml
  my-app/
    shiplight.config.json   # credentials for my-app tests
    checkout.test.yaml

Environment variables

Variable	Description
ANTHROPIC_API_KEY	Anthropic API key for Claude models
GOOGLE_API_KEY	Google API key for Gemini models
WEB_AGENT_MODEL	Override the AI model (auto-detected if not set)

Logs

Server output is available in the VS Code Output panel — select Shiplight Debugger from the dropdown.

Requirements

• VS Code 1.85+
• Shiplight CLI installed globally or in your project