# CLI Reference

<div class="view-markdown-wrapper">
<ViewMarkdown />
</div>

The `shiplightai` CLI is the entry point for every local Shiplight workflow: scaffolding a new project, running YAML and Playwright tests, debugging a test interactively, and regenerating HTML reports. It ships as an npm package (`shiplightai`) and is invoked through `npx shiplight <command>` from any project that has it installed.

This page covers how to install and keep the CLI up to date, and then every subcommand with its flags. If you are looking for the YAML test format, see [YAML Test Format](./yaml-tests). If you are looking for browser automation features, see [Browser Automation](./browser-automation).

## Installation

The CLI is meant to be installed as a **project dependency** inside the same directory as your tests — never globally. The easiest way to get started is the built-in scaffolder:

```bash
npx shiplightai@latest create ./my-tests
cd my-tests
cp .env.example .env                       # then edit .env and set an AI provider API key
npm install
npx playwright install chromium
npx shiplight debug tests/example.test.yaml --open
```

The scaffold includes `tests/example.test.yaml` — a self-contained starter test that visits [shiplight.ai](https://www.shiplight.ai) and verifies the Install Plugin link navigates to the docs. The final command opens it in the visual debugger so you can step through each statement live before writing your own tests.

The `@latest` prefix is only needed for the very first invocation, before anything is installed. Once the scaffolder has written a `package.json` with `shiplightai` as a dep and you have run `npm install`, every subsequent call can use plain `npx shiplight <command>` from inside the project.

### Why not a global install

Global installs pin once and never auto-update, so the CLI and library silently drift apart from what's live on npm. `shiplightai` refuses to run when installed globally. If you have one from an earlier attempt, remove it:

```bash
npm uninstall -g shiplightai
```

### Upgrading

Scaffolded projects pin `shiplightai` to the `latest` dist-tag in `package.json`:

```json
{
  "dependencies": {
    "shiplightai": "latest"
  }
}
```

To pick up newer versions, run `npm update` from inside your project:

```bash
npm update shiplightai
```

This moves both `package.json` and `package-lock.json` forward to whatever the `latest` dist-tag currently points at on npm. CI keeps running the version captured in your lockfile until you commit an updated lockfile, so `npm update` locally is the mechanism for propagating new releases — there is no silent auto-update in CI.

Under the hood, the `latest` dist-tag is also what gives us a safe rollback lever: if a release turns out to be broken, we can move `latest` back to the previous version, and everyone running `npm update` on their next check picks up the good version.

### Freshness warning

On every invocation, the CLI compares the running version against the latest published version on npm (cached locally for ~1 hour) and prints a yellow warning if you are behind:

```
⚠ shiplightai 0.1.49 is available (you have 0.1.47).
  Run: npm update shiplightai
```

The warning is a best-effort nag — completely silent on network failure, always skipped when `CI=true` is set in the environment (every major CI provider sets this automatically), and only prints for projects that actually have a `package-lock.json` (so the warning is always actionable by running `npm update`). The warning appears whenever the background check resolves — typically before your command output on a warm cache, or mid-run on a cold cache. Short-lived commands like `--version` may exit before the check completes, which is fine.

### Node and Playwright requirements

- **Node.js >= 22** — the CLI uses features that require this.
- **`@playwright/test`** — declared as a peer dependency of `shiplightai` (pinned to an exact version that the current CLI was built against). `npm install` picks it up automatically; you do not need to list it yourself.
- **Chromium** — only needed for browsers that use it. Install on demand with `npx playwright install chromium`.

## Commands

All commands are invoked via `npx shiplight <command>` from inside a project that has `shiplightai` installed locally. You can also use them from npm scripts without the `npx` prefix, because `npm run` automatically puts `node_modules/.bin` on `PATH`.

| Command                                       | Purpose                                                |
| --------------------------------------------- | ------------------------------------------------------ |
| [`shiplight create`](#shiplight-create)       | Scaffold a new Shiplight test project                  |
| [`shiplight test`](#shiplight-test)           | Run your YAML and Playwright test suite                |
| [`shiplight debug`](#shiplight-debug)         | Launch the interactive visual debugger for a YAML test |
| [`shiplight report`](#shiplight-report)       | Regenerate the HTML report from saved artifacts        |
| [`shiplight transpile`](#shiplight-transpile) | Transpile YAML tests to Playwright `.spec.ts` files    |
| `shiplight --version`                         | Print the installed CLI version                        |
| `shiplight --help`                            | Print top-level usage and list subcommands             |

### `shiplight create`

Scaffolds a new Shiplight test project at the given path. See [Installation](#installation) above for the full first-run recipe. Used for manual terminal onboarding — the alternative to having a coding agent scaffold via `/create_e2e_tests`.

```bash
npx shiplightai@latest create <path> [options]
```

**Generated files:**

- `package.json` — declares `shiplightai` (pinned to `latest`) and `dotenv` as direct dependencies. `@playwright/test` is installed automatically as a peer dependency of `shiplightai`, so it does not need to be listed.
- `playwright.config.ts` — uses `shiplightConfig()` from `shiplightai` and wires up the Shiplight reporter
- `.gitignore` — excludes `node_modules/`, `.env`, `test-results/`, `shiplight-report/`, transpiled `.yaml.spec.ts` files
- `.env.example` — a starter file with commented examples for each supported AI provider
- `tests/example.test.yaml` — a self-contained starter test you can run immediately to verify your install works

**Options:**

| Option          | Description                                                                                                          | Default              |
| --------------- | -------------------------------------------------------------------------------------------------------------------- | -------------------- |
| `<path>`        | Directory where the project should be created (positional)                                                           | — (required)         |
| `--name <name>` | `package.json` name field. Must be a valid lowercase npm package name (letters, digits, hyphens, underscores, dots). | basename of `<path>` |
| `--help`, `-h`  | Print usage for this subcommand                                                                                      | —                    |

The target directory must be empty (or not yet exist) — the scaffolder refuses to write into a non-empty directory to avoid silently overwriting files.

### `shiplight test`

Runs your test suite. The CLI transpiles any YAML test files (`*.test.yaml`) to `.yaml.spec.ts` on the fly so they execute alongside your regular `*.test.ts` files in the same run. Any argument you pass after `shiplight test` is forwarded to Playwright, so the full Playwright CLI surface is available:

```bash
npx shiplight test                                  # run everything
npx shiplight test --headed                         # show the browser
npx shiplight test my-saas-app/                     # run one project
npx shiplight test tests/login.test.yaml            # run one file
npx shiplight test --grep "checkout"                # run by name pattern
npx shiplight test --workers 4                      # control parallelism
```

After the run completes, open `shiplight-report/index.html` to see results with per-step screenshots, videos, and traces. See [`shiplight report`](#shiplight-report) to regenerate or merge reports, and [`shiplight transpile`](#shiplight-transpile) if you want to inspect the generated `.yaml.spec.ts` code.

See [Local Testing](./local-testing) for the full project flow — environment variables, authentication, CI/CD integration, and project structure.

### `shiplight debug`

Launches a visual debugger for a YAML test file. Opens a local web UI where you can browse tests, step through statements one at a time, inspect the page, edit YAML and re-run — all in real time against a live browser.

```bash
npx shiplight debug                                    # browse current directory
npx shiplight debug tests/                             # browse tests/ directory
npx shiplight debug tests/login.test.yaml              # open a specific test
npx shiplight debug tests/login.test.yaml --open       # also launch in browser
```

**Options:**

| Option        | Description                                    | Default |
| ------------- | ---------------------------------------------- | ------- |
| `--port <n>`  | Port for the debugger's local web server       | `6174`  |
| `--url <url>` | Starting URL for a newly created test          | —       |
| `--new`       | Create the target file if it does not exist    | —       |
| `--open`      | Auto-open the debugger in your default browser | —       |
| `--no-open`   | Do not auto-open the browser (default)         | —       |

**Creating a new test while debugging:**

```bash
npx shiplight debug tests/checkout.test.yaml --new --url https://myapp.com/checkout --open
```

This creates `tests/checkout.test.yaml` from a starter template if it does not exist, then opens the debugger pointed at it.

The debugger auto-detects your `playwright.config.ts` and uses its settings (browser, `baseURL`, storage state, etc.). If no config is found, it runs in standalone mode with a built-in browser sandbox — useful for writing tests before your project has any Playwright setup.

The [Shiplight VS Code extension](./vscode-extension) embeds the same debugger directly in VS Code. When you install the extension and open a `.test.yaml` file, you get the full debugger UI in a side panel instead of a separate browser tab. The extension reuses your project-local CLI — there is no separate install step.

### `shiplight report`

Regenerates or merges HTML reports from saved Playwright artifacts. Useful when you want to view a report without re-running tests, or to combine reports from parallel shards into one.

```bash
npx shiplight report                         # regenerate ./shiplight-report/index.html
npx shiplight report my-report               # regenerate a different folder
npx shiplight report my-report --open        # regenerate and open in browser
npx shiplight report --merge shard-1 shard-2 # merge multiple shard outputs
```

**Options:**

| Option     | Description                                                     |
| ---------- | --------------------------------------------------------------- |
| `[folder]` | Report output folder (positional, default `./shiplight-report`) |
| `--open`   | Open the generated report in your default browser               |
| `--merge`  | Merge multiple report directories into one combined report      |

The reporter is also wired up automatically by `shiplightConfig()` in your `playwright.config.ts`, so you do not normally need to run `shiplight report` after a regular `shiplight test` — the report is generated as part of the run. See [Local Testing → Report](./local-testing#report) for how to customize the reporter output during a run.

### `shiplight transpile`

Transpiles YAML test files to standalone Playwright `.yaml.spec.ts` files. You normally do not need to run this manually — `shiplight test` transpiles on the fly — but it is useful when you want to inspect the generated code, check it into source control, or run it with a vanilla `npx playwright test` command.

```bash
npx shiplight transpile                            # transpile all **/*.test.yaml
npx shiplight transpile "tests/**/*.test.yaml"     # transpile a specific glob
```

**Options:**

| Option   | Description                                               | Default          |
| -------- | --------------------------------------------------------- | ---------------- |
| `[glob]` | Glob pattern of YAML test files to transpile (positional) | `**/*.test.yaml` |

The generated `.yaml.spec.ts` files live alongside their `.test.yaml` sources. Add `*.yaml.spec.ts` to your `.gitignore` if you do not want to track them — the scaffolder does this by default.

## Environment variables

Shiplight uses an **explicit allowlist** for env vars — only the variables in this table are forwarded from your shell (or `.env` file) into the agent's internal SDK config. Any env var that is not on this list is invisible to `shiplightai` by design: user test code cannot leak arbitrary environment state through the SDK.

All of these should be set in your project's `.env` file (auto-discovered by `shiplightConfig()`) or in your shell environment before running `shiplight test`.

### LLM provider API keys

At least one is required. The first one set determines the default model (see [Model selection](#model-selection)).

| Variable            | Description                            | Default |
| ------------------- | -------------------------------------- | ------- |
| `GOOGLE_API_KEY`    | Google AI Studio key (Gemini models)   | —       |
| `ANTHROPIC_API_KEY` | Anthropic API key (Claude models)      | —       |
| `OPENAI_API_KEY`    | OpenAI API key (GPT / o-series models) | —       |

### Model selection

| Variable             | Description                                                                                                                    | Default                                  |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------- |
| `WEB_AGENT_MODEL`    | Override the web agent model (e.g. `claude-sonnet-4-6`). Supports `provider:model` prefix for `azure:`, `bedrock:`, `vertex:`. | Auto-detected from the first API key set |
| `COMPUTER_USE_MODEL` | Override the computer-use (CUA) model used for coordinate-based actions                                                        | Auto-detected                            |

### Custom OpenAI endpoint

| Variable          | Description                                                                                                                                  | Default                     |
| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- |
| `OPENAI_BASE_URL` | Custom base URL for OpenAI-compatible APIs (Ollama, vLLM, LiteLLM proxies). Used by both the regular LLM path **and** the computer-use path. | `https://api.openai.com/v1` |

### Vertex AI routing

Set one of the `*_USE_VERTEXAI` flags to route a provider through Google Vertex AI instead of the direct provider API. Either flag requires `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` to be set, and expects `GOOGLE_APPLICATION_CREDENTIALS` (or Application Default Credentials) to be available for `google-auth-library`.

| Variable                        | Description                                                    | Default |
| ------------------------------- | -------------------------------------------------------------- | ------- |
| `ANTHROPIC_MODELS_USE_VERTEXAI` | Route Anthropic (Claude) calls through Vertex AI               | `false` |
| `GOOGLE_GENAI_USE_VERTEXAI`     | Route Google (Gemini) calls through Vertex AI                  | `false` |
| `GOOGLE_CLOUD_PROJECT`          | GCP project ID (required when either flag above is set)        | —       |
| `GOOGLE_CLOUD_LOCATION`         | GCP region (e.g. `us-central1`, required when either flag set) | —       |

### Tools

| Variable          | Description                                                    | Default |
| ----------------- | -------------------------------------------------------------- | ------- |
| `MAILGUN_API_KEY` | Mailgun API key — required by the `extract_email_content` tool | —       |
| `MAILGUN_DOMAIN`  | Mailgun domain — required by `extract_email_content`           | —       |

### Logging

| Variable        | Description                                                                              | Default |
| --------------- | ---------------------------------------------------------------------------------------- | ------- |
| `SDK_LOG_LEVEL` | SDK log verbosity. One of `debug`, `info`, `warn`, `error`, `silent` (case-insensitive). | `info`  |

### Adding a new env var

If you need Shiplight to honor a new environment variable, it must be added to the allowlist in `apps/cli/src/fixture.ts` — this is the fixture that forwards vars from `process.env` into the agent's `SdkConfig.env`. Setting an env var that is not on the allowlist will have no effect at runtime, even if `sdk-core` reads it internally.

## Troubleshooting

### `shiplight: command not found`

You are running `shiplight` as a bare command from your shell, but `node_modules/.bin` is not on your `PATH`. This is expected — the CLI is intentionally not designed for global installation. Use `npx shiplight <command>` instead, or run it through an npm script (`npm run test`, etc.), which automatically puts `node_modules/.bin` on `PATH`.

### "cannot be run from a global install"

You hit the hard block on startup. The fix is in the error message itself:

```bash
npm uninstall -g shiplightai
cd <your-project>
npm install shiplightai
npx shiplight <command>
```

### "A global shiplightai install was detected"

You have a project-local install (good) but also a leftover global install (not good). The coexisting global install can shadow your project's copy in some contexts. Remove it:

```bash
npm uninstall -g shiplightai
```

The warning is non-blocking — your command still ran — but the global install is a ticking time bomb for version skew, so clean it up.

### The freshness warning keeps firing after `npm update`

If `npm update shiplightai` ran but the warning still shows the same numbers on the next invocation, check two things:

1. **Your `package.json` version range.** The scaffolder writes `"shiplightai": "latest"`, which lets `npm update` cross any version boundary. If you manually pinned to something like `"^0.1.11"`, `npm update` can only move you within that range and may silently do nothing when a newer minor ships. Change it to `"latest"` or widen the range.
2. **The local version cache.** The CLI caches the latest-version lookup for ~1 hour in `~/.shiplight/version-check.json`. If the cache is fresh but wrong, `rm ~/.shiplight/version-check.json` and try again.

### Running in CI and want the warning silenced

The warning is automatically suppressed when the `CI` environment variable is set to a truthy value — all major CI providers (GitHub Actions, GitLab, CircleCI, Jenkins, etc.) do this for you. If you are running in an environment that does not set `CI` but still want the warning silenced, set it yourself:

```bash
CI=true npx shiplight test
```

### The scaffolded project has deprecation warnings on `npm install`

Some warnings come from transitive dependencies (for example `node-domexception`) and are outside the CLI's direct control. These are non-fatal.

## Related pages

- [Local Testing](./local-testing) — project structure, authentication, environment variables, CI/CD
- [YAML Test Format](./yaml-tests) — statement types, actions, locators, variables, templates
- [Browser Automation](./browser-automation) — persistent profiles, mobile emulation, extension testing
- [VS Code Extension](./vscode-extension) — embedding the debugger into VS Code
- [Report to Cloud](./report-to-cloud) — uploading reports for sharing
- [Sync to Cloud](./sync-to-cloud) — syncing local tests with Shiplight Cloud