# GitHub Actions

<div class="view-markdown-wrapper">
<ViewMarkdown />
</div>

Run Shiplight E2E tests on **Shiplight-hosted GitHub Actions runners** — ephemeral VMs with Chromium and Playwright pre-installed. Test results are automatically uploaded to [Shiplight Cloud](https://app.shiplight.ai) for trend tracking, flaky-test detection, and team visibility.

::: tip Let your agent wire this up: `/shiplight ci`
The fastest way to author both the CI test workflow **and** the [auto-triage pipeline](#auto-triage-ci-failures-optional) is to run **`/shiplight ci`** in your coding agent. It picks the runner option, generates the workflow files, and wires the report-upload and triage steps for you. Use this page to understand what it produces and to configure it by hand when needed.
:::

## Prerequisites

1. **Install the Shiplight GitHub App** from **Settings → Integrations** at [app.shiplight.ai/settings/integrations](https://app.shiplight.ai/settings/integrations), and make sure the repository is included in the App's repository selection.
2. **Enable Shiplight Runners** — an org owner turns on the toggle at [app.shiplight.ai/runners](https://app.shiplight.ai/runners).

::: warning The repository must be owned by a GitHub organization
Runner registration tokens are minted at the organization level, so repositories under a **personal** GitHub account cannot use Shiplight runners — jobs stay queued and the dispatch fails with _Token mint failed_.
:::

See [CI Runners](/cloud_v2/ci-runners) for runner sizes, pool limits, and dispatch troubleshooting.

## Minimal Workflow Example

Create `.github/workflows/e2e.yml`:

```yaml
name: E2E Tests

on:
  push:
    branches: [main]
  pull_request:

jobs:
  e2e:
    runs-on: shiplight-small
    timeout-minutes: 30
    steps:
      - uses: actions/checkout@v5

      - name: Install dependencies
        working-directory: tests/e2e
        run: npm install

      - name: Run E2E tests
        working-directory: tests/e2e
        run: npx shiplight test

      - name: Upload results to Shiplight
        if: always()
        working-directory: tests/e2e
        run: npx shiplight report
```

That's it. The sections below explain each part in detail.

## Runner Sizes

Use `runs-on` to select the VM size for your job:

```yaml
runs-on: shiplight-small
```

When the workflow is triggered, Shiplight provisions an ephemeral VM, registers it as a GitHub self-hosted runner, and destroys it after the job completes.

| Label              | vCPUs | Memory |
| ------------------ | ----- | ------ |
| `shiplight-small`  | 4     | 16 GB  |
| `shiplight-medium` | 8     | 32 GB  |
| `shiplight-large`  | 16    | 64 GB  |
| `shiplight-xlarge` | 32    | 128 GB |

::: tip No browser install needed
The runner image ships with Chromium and Playwright pre-installed. Do not run `npx playwright install chromium` in your workflow — it's already there.
:::

## What the Runner Provides

Shiplight runners come pre-configured with everything needed to run and report E2E tests:

- **Action cache** — when a cached locator self-heals during a CI run, the updated locator is persisted back to Shiplight Cloud so future runs replay at full speed without manual intervention.
- **Chromium + Playwright** — pre-installed, no `npx playwright install` step needed.
- **Automatic result reporting** — `shiplight report` works out of the box with no tokens or API URLs to configure. Credentials are provisioned per run and revoked automatically when the run completes.
- **LLM access** — if your tests use AI-powered actions (natural language steps, self-healing locators), the runner provides LLM credentials automatically. No API keys needed in your workflow.

## Upload results

```yaml
- name: Upload results to Shiplight
  if: always()
  working-directory: tests/e2e
  run: npx shiplight report
```

Uploads test results — including per-step screenshots, videos, and traces — to [Shiplight Cloud](https://app.shiplight.ai). Results appear in the **Test Results** section, linked to your organization.

::: warning
Always use `if: always()` so results are uploaded even when tests fail. Without it, a red test run produces no cloud report.
:::

On Shiplight runners, no additional configuration is needed — credentials are provided automatically.

### On GitHub-hosted (non-Shiplight) runners

Shiplight also runs on stock `ubuntu-latest` runners, without the GitHub App. Two differences from the Shiplight-runner example: you provide the credentials yourself, and you install the browser yourself.

Set `SHIPLIGHT_API_TOKEN` (an org token from [app.shiplight.ai/api-tokens](https://app.shiplight.ai/api-tokens)) at the job's **`env` (global) scope**. `shiplight report` needs it to authenticate the upload; `shiplight test` needs it only if your tests use AI-powered actions (natural-language steps, self-healing locators) that run through the Shiplight LLM proxy — deterministic YAML tests don't. Setting it at the job scope covers both cases. `shiplight report` additionally needs `SHIPLIGHT_REPORT_TO_CLOUD=1` to enable the upload (Shiplight runners set this automatically; stock runners don't).

```yaml
jobs:
  e2e:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    env:
      SHIPLIGHT_API_TOKEN: ${{ secrets.SHIPLIGHT_API_TOKEN }} # report (upload) always; test only for AI-powered steps
    steps:
      - uses: actions/checkout@v5

      - name: Install dependencies
        working-directory: tests/e2e
        run: npm install

      - name: Install Playwright browser
        working-directory: tests/e2e
        run: npx playwright install --with-deps chromium

      - name: Run E2E tests
        working-directory: tests/e2e
        run: npx shiplight test

      - name: Upload results to Shiplight
        if: always()
        working-directory: tests/e2e
        env:
          SHIPLIGHT_REPORT_TO_CLOUD: "1" # enable the upload on non-Shiplight runners
        run: npx shiplight report
```

Store the token as a repository or organization secret named `SHIPLIGHT_API_TOKEN` — never hardcode it in the workflow file, which is committed to your repo. For non-GitHub providers (GitLab, Jenkins, CircleCI, …), see the [CI/CD overview](/local/ci).

### Sharded runs (GHA matrix)

When using [Playwright sharding](https://playwright.dev/docs/test-sharding) with a GitHub Actions matrix, each shard produces its own report. Use [`shiplight report --merge`](/local/cli-reference#shiplight-report) to combine them into a single upload so the cloud dashboard shows one unified run instead of N separate ones.

## Auto-triage CI failures (optional)

When an E2E workflow goes red, Shiplight can have an AI agent diagnose the failure and — for fixable spec issues — repair it automatically. The pipeline lives in the reusable workflow [`ShiplightAI/ci-triage`](https://github.com/ShiplightAI/ci-triage). On a failed run it reads the run logs **and** the uploaded report artifacts (screenshots, traces), posts a diagnosis to Slack, and for failures it classifies as fixable spec issues it applies the fix, re-runs the test, and opens a PR. It never auto-merges.

::: tip GitHub Actions only
Auto-triage is built on the GitHub `workflow_run` trigger and reusable workflows, so it is available on GitHub Actions only.
:::

Set this up only once a test workflow (from the examples above) exists — triage triggers off that workflow's completion. Wiring is two steps.

### Step 1 — upload the report artifact from the test workflow

Triage reads failure evidence from a GitHub artifact, which is separate from the cloud upload. Add this step to your `e2e.yml`, after the test step:

```yaml
- name: Upload test report (for triage)
  if: ${{ !cancelled() }}
  uses: ShiplightAI/ci-triage/upload-report@v1
  # sharded/matrix jobs: give each shard a unique name
  # with:
  #   name: test-report-shard-${{ matrix.shardIndex }}
  #   retention-days: "1"
```

The helper bakes in the `shiplight-report/` path and drops the heavy Playwright traces (`*.zip`) and videos (`*.webm`) the agent never reads (~80% of the artifact size). Keep your `npx shiplight report` step too — the cloud report still gets full traces and videos for humans.

### Step 2 — add the caller workflow

The `workflow_run` trigger and the per-repo credential mapping **must** live in your repo — a `workflow_run` trigger is illegal inside a reusable workflow, and secret names differ per repo. Everything else lives in the reusable workflow. Create `.github/workflows/ci-failure-triage.yml`:

```yaml
name: CI Failure Triage

on:
  workflow_run:
    workflows: [E2E Tests] # exact `name:` of each test workflow to watch
    types: [completed]

jobs:
  triage:
    uses: ShiplightAI/ci-triage/.github/workflows/triage.yml@v1 # pin to an immutable tag, not @main
    permissions:
      contents: write
      pull-requests: write
      actions: read
    with:
      triage-runner: ubuntu-latest # read-only diagnosis job
      autofix-runner: shiplight-medium # re-runs tests, so needs browsers/network
      node-version: "20"
      allowed-paths: "tests templates" # top-level dirs the autofix agent may edit (hard guard)
      slack-channel: ${{ vars.SLACK_CHANNEL_ID || '' }} # empty disables Slack
    secrets:
      claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
      openai_api_key: ${{ secrets.OPENAI_API_KEY }} # Codex fallback when Claude is unavailable
      autofix_github_token: ${{ secrets.AUTOFIX_GITHUB_TOKEN }} # PAT/App token to open the PR; falls back to GITHUB_TOKEN
      slack_bot_token: ${{ secrets.SLACK_BOT_TOKEN }}
      # Per-repo credential mapping → generic env for the autofix re-run.
      # One KEY=VALUE per line; values must be single-line.
      extra_env: |
        BASE_URL=${{ vars.BASE_URL || 'https://example.com' }}
        MY_TEST_USER_PASSWORD=${{ secrets.MY_TEST_USER_PASSWORD }}
        MY_TEST_USER_2FA_SECRET=${{ secrets.MY_TEST_USER_2FA_SECRET }}
```

Notes:

- `workflows:` must list the exact `name:` of each test workflow to watch. Never list the triage workflow itself there.
- `extra_env` maps your repo's secret names onto the generic env the autofix job uses, so `npx shiplight test` and the MCP browser can authenticate during the re-run. Mirror the `env:` block from your test workflow.
- Provide at least one model credential (`claude_code_oauth_token` or `anthropic_api_key`); `openai_api_key` enables the Codex fallback. `autofix_github_token` and `slack_bot_token` are optional.
- `autofix-runner` re-runs the failing test, so it needs browsers and network — use a Shiplight runner, or install Chromium on a stock runner the same way your test workflow does.
- These examples assume the Shiplight project lives at the repository root. If it lives in a subdirectory, adjust the report path and add `working-directory` to the relevant steps.

::: warning
The triage job runs privileged — `contents: write` plus live credentials for the autofix re-run. Pin `uses:` to an immutable tag (`@v1.x.y`), never a moving branch like `@main`.
:::

## Troubleshooting

### Job stays queued

- Confirm Shiplight Runners are enabled at [app.shiplight.ai/runners](https://app.shiplight.ai/runners), and check the **Dispatches** tab there for the failure reason. See [CI Runners](/cloud_v2/ci-runners#job-stays-queued).
- Verify the Shiplight GitHub App is installed on the repository with the required permissions.
- Check whether your organization's runner concurrency limit has been reached. Contact your Shiplight admin to raise the limit if needed.

### Test results not appearing in the dashboard

- Make sure the `shiplight report` step has `if: always()` so it runs after failures.
- Check the step logs for errors.

### Chromium is downloading during CI

The runner has Chromium pre-installed. Do not add `npx playwright install chromium` to your workflow. If Playwright still downloads a browser, check that your `package.json` does not override the Playwright version bundled with `shiplightai`.

## Related

- [CI/CD overview](/local/ci) — how the pieces fit together, and other providers
- [CLI Reference](/local/cli-reference) — full `shiplight test`, `shiplight report`, and other command documentation
- [Local Testing](/local/run-locally) — project structure, authentication, and configuration