# CLI Reference

<div class="view-markdown-wrapper">
<ViewMarkdown />
</div>

The `shiplightai` CLI is the entry point for every local Shiplight workflow: scaffolding a new project, running YAML E2E tests alongside Playwright tests, debugging a test interactively, and regenerating HTML reports. It ships as an npm package (`shiplightai`) and is invoked through `npx shiplight <command>` from any project that has it installed.

This page covers how to install and keep the CLI up to date, and then every subcommand with its flags. If you are looking for the YAML E2E test format, see [YAML E2E Test Format](./yaml-tests/). If you are looking for browser automation features, see [Guides](/guides/).

## Installation

The CLI is meant to be installed as a **project dependency** inside the same directory as your tests. Never globally. The easiest way to get started is the built-in scaffolder:

```bash
npx shiplightai@latest create ./my-tests
cd my-tests
cp .env.example .env                       # then edit .env and set an AI provider API key
npm install
npx playwright install chromium
npx shiplight debug tests/example.test.yaml --open
```

The scaffold includes `tests/example.test.yaml`, a self-contained starter test that visits [shiplight.ai](https://www.shiplight.ai) and verifies a navigation link takes you to the docs. The final command opens it in the visual debugger so you can step through each statement live before writing your own tests.

The `@latest` prefix is only needed for the very first invocation, before anything is installed. Once the scaffolder has written a `package.json` with `shiplightai` as a dep and you have run `npm install`, every subsequent call can use plain `npx shiplight <command>` from inside the project.

### Why not a global install

Global installs pin once and never auto-update, so the CLI and library silently drift apart from what's live on npm. `shiplightai` refuses to run when installed globally. If you have one from an earlier attempt, remove it:

```bash
npm uninstall -g shiplightai
```

### Upgrading

Scaffolded projects pin `shiplightai` to the `latest` dist-tag in `package.json`:

```json
{
  "dependencies": {
    "shiplightai": "latest"
  }
}
```

To pick up newer versions, run `npm update` from inside your project:

```bash
npm update shiplightai
```

This moves both `package.json` and `package-lock.json` forward to whatever the `latest` dist-tag currently points at on npm. CI keeps running the version captured in your lockfile until you commit an updated lockfile, so `npm update` locally is the mechanism for propagating new releases. There is no silent auto-update in CI.

Under the hood, the `latest` dist-tag is also what gives us a safe rollback lever: if a release turns out to be broken, we can move `latest` back to the previous version, and everyone running `npm update` on their next check picks up the good version.

### Freshness warning

On every invocation, the CLI compares the running version against the latest published version on npm (cached locally for ~1 hour) and prints a yellow warning if you are behind:

```
⚠ shiplightai 0.1.49 is available (you have 0.1.47).
  Run: npm update shiplightai
```

The warning is a best-effort nag: completely silent on network failure, always skipped when `CI=true` is set in the environment (every major CI provider sets this automatically), and only prints for projects that actually have a `package-lock.json` (so the warning is always actionable by running `npm update`). The warning appears whenever the background check resolves: typically before your command output on a warm cache, or mid-run on a cold cache. Short-lived commands like `--version` may exit before the check completes, which is fine.

### Node and Playwright requirements

- **Node.js >= 22** — the CLI uses features that require this.
- **`@playwright/test`** — declared as a peer dependency of `shiplightai` (pinned to an exact version that the current CLI was built against). `npm install` picks it up automatically; you do not need to list it yourself.
- **Chromium** — only needed for browsers that use it. Install on demand with `npx playwright install chromium`.

## Commands

All commands are invoked via `npx shiplight <command>` from inside a project that has `shiplightai` installed locally. You can also use them from npm scripts without the `npx` prefix, because `npm run` automatically puts `node_modules/.bin` on `PATH`.

| Command                                                   | Purpose                                                        |
| --------------------------------------------------------- | -------------------------------------------------------------- |
| [`shiplight setup-api-token`](#shiplight-setup-api-token) | Create or restore a personal Shiplight API token for local use |
| [`shiplight create`](#shiplight-create)                   | Scaffold a new Shiplight test project                          |
| [`shiplight test`](#shiplight-test)                       | Run YAML E2E tests alongside Playwright tests                  |
| [`shiplight debug`](#shiplight-debug)                     | Launch the interactive visual debugger for a YAML E2E test     |
| [`shiplight report`](#shiplight-report)                   | Regenerate the HTML report from saved artifacts                |
| [`shiplight transpile`](#shiplight-transpile)             | Transpile YAML E2E tests to Playwright `.spec.ts` files        |
| `shiplight --version`                                     | Print the installed CLI version                                |
| `shiplight --help`                                        | Print top-level usage and list subcommands                     |

### `shiplight setup-api-token`

Authenticates the CLI with the Shiplight platform and writes a personal API token to the current project's `.env` file:

```bash
npx shiplight setup-api-token
```

The command opens a browser for Shiplight account authentication, asks you to select an organization, and creates or restores a personal `shp_pat_…` token as `SHIPLIGHT_API_TOKEN`. The browser-authenticated device session is used only during setup; the CLI does not persist it.

Use this command for local development only. For CI or other shared automation, create an organization token at [app.shiplight.ai/api-tokens](https://app.shiplight.ai/api-tokens) and store it in your CI secret manager.

To stop using the token in a project, remove `SHIPLIGHT_API_TOKEN` from its `.env` file. To invalidate the token everywhere, revoke it from [API Tokens](/cloud_v2/api-tokens).

### `shiplight create`

Scaffolds a new Shiplight test project at the given path. See [Installation](#installation) above for the full first-run recipe. Used for manual terminal onboarding, the alternative to having a coding agent scaffold via `/shiplight create-yaml-tests`.

```bash
npx shiplightai@latest create <path> [options]
```

**Generated files:**

- `package.json` — declares `shiplightai` (pinned to `latest`) and `dotenv` as direct dependencies. `@playwright/test` is installed automatically as a peer dependency of `shiplightai`, so it does not need to be listed.
- `playwright.config.ts` — uses `shiplightConfig()` from `shiplightai` and wires up the Shiplight reporter
- `.gitignore` — excludes `node_modules/`, `.env`, `test-results/`, `shiplight-report/`, transpiled `.yaml.spec.ts` files
- `.env.example` — a starter file with commented examples for each supported AI provider
- `tests/example.test.yaml` — a self-contained starter test you can run immediately to verify your install works

**Options:**

| Option          | Description                                                                                                          | Default              |
| --------------- | -------------------------------------------------------------------------------------------------------------------- | -------------------- |
| `<path>`        | Directory where the project should be created (positional)                                                           | — (required)         |
| `--name <name>` | `package.json` name field. Must be a valid lowercase npm package name (letters, digits, hyphens, underscores, dots). | basename of `<path>` |
| `--help`, `-h`  | Print usage for this subcommand                                                                                      | —                    |

The target directory must be empty (or not yet exist): the scaffolder refuses to write into a non-empty directory to avoid silently overwriting files.

### `shiplight test`

Runs your test suite. The CLI transpiles any YAML E2E test files (`*.test.yaml`) to `.yaml.spec.ts` on the fly so they execute alongside your regular `*.test.ts` files in the same run. Any argument you pass after `shiplight test` is forwarded to Playwright, so the full Playwright CLI surface is available:

```bash
npx shiplight test                                  # run everything
npx shiplight test --headed                         # show the browser
npx shiplight test my-saas-app/                     # run one project
npx shiplight test tests/login.test.yaml            # run one file
npx shiplight test --grep "checkout"                # run by name pattern
npx shiplight test --workers 4                      # control parallelism
```

After the run completes, open `shiplight-report/index.html` to see results with per-step screenshots, videos, and traces. See [`shiplight report`](#shiplight-report) to regenerate or merge reports, and [`shiplight transpile`](#shiplight-transpile) if you want to inspect the generated `.yaml.spec.ts` code.

See [Local Testing](/local/run-locally) for the full project flow: environment variables, authentication, CI/CD integration, and project structure.

### `shiplight debug`

Launches a visual debugger for a YAML E2E test file. Opens a local web UI where you can browse tests, step through statements one at a time, inspect the page, edit YAML and re-run, all in real time against a live browser.

```bash
npx shiplight debug                                    # browse current directory
npx shiplight debug tests/                             # browse tests/ directory
npx shiplight debug tests/login.test.yaml              # open a specific test
npx shiplight debug tests/login.test.yaml --open       # also launch in browser
```

**Options:**

| Option        | Description                                    | Default |
| ------------- | ---------------------------------------------- | ------- |
| `--port <n>`  | Port for the debugger's local web server       | `6174`  |
| `--url <url>` | Starting URL for a newly created test          | —       |
| `--new`       | Create the target file if it does not exist    | —       |
| `--open`      | Auto-open the debugger in your default browser | —       |
| `--no-open`   | Do not auto-open the browser (default)         | —       |

**Creating a new test while debugging:**

```bash
npx shiplight debug tests/checkout.test.yaml --new --url https://myapp.com/checkout --open
```

This creates `tests/checkout.test.yaml` from a starter template if it does not exist, then opens the debugger pointed at it.

The debugger auto-detects your `playwright.config.ts` and uses its settings (browser, `baseURL`, storage state, etc.). If no config is found, it runs in standalone mode with a built-in browser sandbox, useful for writing tests before your project has any Playwright setup.

### `shiplight report`

Regenerates or merges HTML reports from saved Playwright artifacts. Useful when you want to view a report without re-running tests, or to combine reports from parallel shards into one.

```bash
npx shiplight report                         # regenerate ./shiplight-report/index.html
npx shiplight report my-report               # regenerate a different folder
npx shiplight report my-report --open        # regenerate and open in browser
npx shiplight report --merge shard-1 shard-2 # merge multiple shard outputs
```

**Options:**

| Option     | Description                                                     |
| ---------- | --------------------------------------------------------------- |
| `[folder]` | Report output folder (positional, default `./shiplight-report`) |
| `--open`   | Open the generated report in your default browser               |
| `--merge`  | Merge multiple report directories into one combined report      |

The reporter is also wired up automatically by `shiplightConfig()` in your `playwright.config.ts`, so you do not normally need to run `shiplight report` after a regular `shiplight test`: the report is generated as part of the run. See [Run Locally → Report](/local/run-locally#report) for how to customize the reporter output during a run.

### `shiplight transpile`

Transpiles YAML E2E test files to standalone Playwright `.yaml.spec.ts` files. You normally do not need to run this manually: `shiplight test` transpiles on the fly, but it is useful when you want to inspect the generated code, check it into source control, or run it with a vanilla `npx playwright test` command.

```bash
npx shiplight transpile                            # transpile all **/*.test.yaml
npx shiplight transpile "tests/**/*.test.yaml"     # transpile a specific glob
```

**Options:**

| Option   | Description                                                   | Default          |
| -------- | ------------------------------------------------------------- | ---------------- |
| `[glob]` | Glob pattern of YAML E2E test files to transpile (positional) | `**/*.test.yaml` |

The generated `.yaml.spec.ts` files live alongside their `.test.yaml` sources. Add `*.yaml.spec.ts` to your `.gitignore` if you do not want to track them: the scaffolder does this by default.

## Environment variables

Shiplight reads CLI settings directly and forwards SDK settings through an **explicit allowlist**. Arbitrary shell variables are not automatically exposed to the agent's internal SDK config.

All of these should be set in your project's `.env` file (auto-discovered by `shiplightConfig()`) or in your shell environment before running `shiplight test`.

### LLM providers

Provider API keys (`GOOGLE_API_KEY`, `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`), model selection (`WEB_AGENT_MODEL`, `WEB_AGENT_FALLBACK_MODELS`, `COMPUTER_USE_MODEL`), custom OpenAI-compatible endpoints (`OPENAI_BASE_URL`), Vertex AI routing (`ANTHROPIC_MODELS_USE_VERTEXAI`, `GOOGLE_GENAI_USE_VERTEXAI`, `GOOGLE_CLOUD_PROJECT`, `GOOGLE_CLOUD_LOCATION`), and the Shiplight LLM proxy (`SHIPLIGHT_API_TOKEN`, with `WEB_AGENT_TIER` to pick a tier) are documented together on **[LLM Providers](/local/local-testing/llm-providers)**.

### Action cache

| Variable                         | Values                                                                                                                                           | Default |
| -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | ------- |
| `SHIPLIGHT_ACTION_CACHE_BACKEND` | `auto`: cloud with `CI` and a token, otherwise local. `local`: force file storage. `cloud`: force cloud storage; requires `SHIPLIGHT_API_TOKEN`. | `auto`  |

This is a CLI setting, not an SDK allowlist entry. Unknown values are errors. `.env` values override shell and CI settings. `local` stores cache under `.shiplight/action-cache/` and makes no action-cache API calls; it does not change AI routing or report uploads. See [Action cache in CI](/local/ci/action-cache) for persistence through GitHub Actions cache storage or Git commits.

### Tools

| Variable          | Description                                                   | Default |
| ----------------- | ------------------------------------------------------------- | ------- |
| `MAILGUN_API_KEY` | Mailgun API key, required by the `extract_email_content` tool | —       |
| `MAILGUN_DOMAIN`  | Mailgun domain, required by `extract_email_content`           | —       |

### Logging

| Variable        | Description                                                                              | Default |
| --------------- | ---------------------------------------------------------------------------------------- | ------- |
| `SDK_LOG_LEVEL` | SDK log verbosity. One of `debug`, `info`, `warn`, `error`, `silent` (case-insensitive). | `info`  |

### Adding a new env var

If you need the agent's SDK config to receive a new environment variable, it must be added to the allowlist in `apps/cli/src/fixture.ts`: this is the fixture that forwards vars from `process.env` into the agent's `SdkConfig.env`. Setting an env var that is not on the allowlist will have no effect at runtime, even if `sdk-core` reads it internally.

## Troubleshooting

### `shiplight: command not found`

You are running `shiplight` as a bare command from your shell, but `node_modules/.bin` is not on your `PATH`. This is expected: the CLI is intentionally not designed for global installation. Use `npx shiplight <command>` instead, or run it through an npm script (`npm run test`, etc.), which automatically puts `node_modules/.bin` on `PATH`.

### "cannot be run from a global install"

You hit the hard block on startup. The fix is in the error message itself:

```bash
npm uninstall -g shiplightai
cd <your-project>
npm install shiplightai
npx shiplight <command>
```

### "A global shiplightai install was detected"

You have a project-local install (good) but also a leftover global install (not good). The coexisting global install can shadow your project's copy in some contexts. Remove it:

```bash
npm uninstall -g shiplightai
```

The warning is non-blocking: your command still ran, but the global install is a ticking time bomb for version skew, so clean it up.

### The freshness warning keeps firing after `npm update`

If `npm update shiplightai` ran but the warning still shows the same numbers on the next invocation, check two things:

1. **Your `package.json` version range.** The scaffolder writes `"shiplightai": "latest"`, which lets `npm update` cross any version boundary. If you manually pinned to something like `"^0.1.11"`, `npm update` can only move you within that range and may silently do nothing when a newer minor ships. Change it to `"latest"` or widen the range.
2. **The local version cache.** The CLI caches the latest-version lookup for ~1 hour in `~/.shiplight/version-check.json`. If the cache is fresh but wrong, `rm ~/.shiplight/version-check.json` and try again.

### Running in CI and want the warning silenced

The warning is automatically suppressed when the `CI` environment variable is set to a truthy value: all major CI providers (GitHub Actions, GitLab, CircleCI, Jenkins, etc.) do this for you. If you are running in an environment that does not set `CI` but still want the warning silenced, set it yourself:

```bash
CI=true npx shiplight test
```

### The scaffolded project has deprecation warnings on `npm install`

Some warnings come from transitive dependencies (for example `node-domexception`) and are outside the CLI's direct control. These are non-fatal.

## Related pages

- [Local Testing](/local/run-locally) — project structure, authentication, environment variables, CI/CD
- [YAML E2E Test Format](./yaml-tests/) — statement types, actions, locators, variables, templates
- [Guides](/guides/) — persistent profiles, mobile emulation, extension testing