# GitHub Actions

<div class="view-markdown-wrapper">
<ViewMarkdown />
</div>

Run Shiplight E2E tests on **Shiplight-hosted GitHub Actions runners** — ephemeral VMs with Chromium and Playwright pre-installed. Test results are automatically uploaded to [Shiplight Cloud](https://app.shiplight.ai) for trend tracking, flaky-test detection, and team visibility.

::: tip Let your agent wire this up: `/shiplight ci`
The fastest way to author both the CI test workflow **and** the [auto-triage pipeline](#auto-triage-ci-failures-optional) is to run **`/shiplight ci`** in your coding agent. It picks the runner option, generates the workflow files, and wires the report-upload and triage steps for you. Use this page to understand what it produces and to configure it by hand when needed.
:::

## Prerequisites

1. **Install the Shiplight GitHub App** from **Settings → Integrations** at [app.shiplight.ai/settings/integrations](https://app.shiplight.ai/settings/integrations), and make sure the repository is included in the App's repository selection.
2. **Enable Shiplight Runners** — an org owner turns on the toggle at [app.shiplight.ai/runners](https://app.shiplight.ai/runners).

::: warning The repository must be owned by a GitHub organization
Runner registration tokens are minted at the organization level, so repositories under a **personal** GitHub account cannot use Shiplight runners — jobs stay queued and the dispatch fails with _Token mint failed_.
:::

See [CI Runners](/cloud_v2/ci-runners) for runner sizes, pool limits, and dispatch troubleshooting.

## Minimal Workflow Example

In your existing E2E workflow, switch the runner and add Shiplight's test and report commands:

```yaml
jobs:
  e2e:
    runs-on: ubuntu-latest # [!code --]
    runs-on: shiplight-small # [!code ++]
    steps:
      # Keep your existing checkout and dependency-install steps.

      - name: Run E2E tests # [!code ++]
        working-directory: tests/e2e # [!code ++]
        run: npx shiplight test # [!code ++]

      - name: Upload results to Shiplight # [!code ++]
        if: always() # [!code ++]
        working-directory: tests/e2e # [!code ++]
        run: npx shiplight report # [!code ++]
```

The highlighted lines are the only Shiplight-specific changes. Adjust `working-directory` if your tests live somewhere else.

## Runner Sizes

Use `runs-on` to select the VM size for your job:

```yaml
runs-on: shiplight-small
```

When the workflow is triggered, Shiplight provisions an ephemeral VM, registers it as a GitHub self-hosted runner, and destroys it after the job completes.

| Label              | vCPUs | Memory |
| ------------------ | ----- | ------ |
| `shiplight-small`  | 4     | 16 GB  |
| `shiplight-medium` | 8     | 32 GB  |
| `shiplight-large`  | 16    | 64 GB  |
| `shiplight-xlarge` | 32    | 128 GB |

::: tip No browser install needed
The runner image ships with Chromium and Playwright pre-installed. Do not run `npx playwright install chromium` in your workflow — it's already there.
:::

## What the Runner Provides

Shiplight runners come pre-configured with everything needed to run and report E2E tests:

- **Action cache** — when a cached locator self-heals during a CI run, the updated locator is persisted back to Shiplight Cloud so future runs replay at full speed without manual intervention.
- **Chromium + Playwright** — pre-installed, no `npx playwright install` step needed.
- **Automatic result reporting** — `shiplight report` works out of the box with no tokens or API URLs to configure. Credentials are provisioned per run and revoked automatically when the run completes.
- **LLM access** — if your tests use AI-powered actions (natural language steps, self-healing locators), the runner provides LLM credentials automatically. No API keys needed in your workflow.

## Upload results

```yaml
- name: Upload results to Shiplight
  if: always()
  working-directory: tests/e2e
  run: npx shiplight report
```

Uploads test results — including per-step screenshots, videos, and traces — to [Shiplight Cloud](https://app.shiplight.ai). Results appear in the **Test Results** section, linked to your organization.

::: warning
Always use `if: always()` so results are uploaded even when tests fail. Without it, a red test run produces no cloud report.
:::

On Shiplight runners, no additional configuration is needed — credentials are provided automatically.

### On GitHub-hosted (non-Shiplight) runners

Shiplight also runs on stock `ubuntu-latest` runners, without the GitHub App. In your existing GitHub Actions workflow, make these changes:

Set `SHIPLIGHT_API_TOKEN` (an org token from [app.shiplight.ai/api-tokens](https://app.shiplight.ai/api-tokens)) at the job's **`env` (global) scope**. `shiplight report` needs it to authenticate the upload; `shiplight test` needs it only if your tests use AI-powered actions (natural-language steps, self-healing locators) that run through the Shiplight LLM proxy — deterministic YAML tests don't. Setting it at the job scope covers both cases. `shiplight report` additionally needs `SHIPLIGHT_REPORT_TO_CLOUD=1` to enable the upload (Shiplight runners set this automatically; stock runners don't).

GitHub Actions sets `CI=true`, so the same token also enables the shared Shiplight Cloud action cache when it has `action-cache:read` and `action-cache:write`. Healed locators from passing tests are persisted for later runs; this cache is not limited to Shiplight-hosted runners.

```yaml
jobs:
  e2e:
    runs-on: ubuntu-latest
    env: # [!code ++]
      SHIPLIGHT_API_TOKEN: ${{ secrets.SHIPLIGHT_API_TOKEN }} # [!code ++]
    steps:
      # Keep your existing checkout, Node.js setup, dependency-install,
      # and Playwright browser-install steps.

      - name: Run E2E tests
        working-directory: tests/e2e
        run: npx playwright test # [!code --]
        run: npx shiplight test # [!code ++]

      - name: Upload results to Shiplight # [!code ++]
        if: always() # [!code ++]
        working-directory: tests/e2e # [!code ++]
        env: # [!code ++]
          SHIPLIGHT_REPORT_TO_CLOUD: "1" # [!code ++]
        run: npx shiplight report # [!code ++]
```

The highlighted lines are the only Shiplight-specific changes. Keep your existing stock-runner setup, including the Node.js and Playwright browser installation steps, and adjust `working-directory` if your tests live somewhere else.

Store the token as a repository or organization secret named `SHIPLIGHT_API_TOKEN` — never hardcode it in the workflow file, which is committed to your repo. For non-GitHub providers (GitLab, Jenkins, CircleCI, …), see the [CI/CD overview](/local/ci).

### Sharded runs (GHA matrix)

When using [Playwright sharding](https://playwright.dev/docs/test-sharding) with a GitHub Actions matrix, each shard produces its own report. Use [`shiplight report --merge`](/local/cli-reference#shiplight-report) to combine them into a single upload so the cloud dashboard shows one unified run instead of N separate ones.

## Auto-triage CI failures (optional)

When an E2E workflow goes red, Shiplight can have an AI agent diagnose the failure and — for fixable spec issues — repair it automatically. The pipeline lives in the reusable workflow [`ShiplightAI/ci-triage`](https://github.com/ShiplightAI/ci-triage). On a failed run it reads the run logs **and** the uploaded report artifacts (screenshots, traces), classifies each failure, and for failures it classifies as fixable spec issues it applies the fix, re-runs the test, and opens a PR. It never auto-merges.

The reusable workflow does **not** post to Slack, Linear, or any other system of yours. It publishes its diagnosis as an artifact; a job you own decides what to notify or file, using your own credentials (Step 3).

::: tip GitHub Actions only
Auto-triage is built on the GitHub `workflow_run` trigger and reusable workflows, so it is available on GitHub Actions only.
:::

Set this up only once a test workflow (from the examples above) exists — triage triggers off that workflow's completion.

### Step 1 — upload the report artifact from the test workflow

Triage reads failure evidence from a GitHub artifact, which is separate from the cloud upload. Add this step to your `e2e.yml`, after the test step:

```yaml
- name: Upload test report (for triage)
  if: ${{ !cancelled() }}
  uses: ShiplightAI/ci-triage/upload-report@v1.1
  # sharded/matrix jobs: give each shard a unique name
  # with:
  #   name: test-report-shard-${{ matrix.shardIndex }}
  #   retention-days: "1"
```

The helper bakes in the `shiplight-report/` path and drops the heavy Playwright traces (`*.zip`) and videos (`*.webm`) the agent never reads (~80% of the artifact size). Keep your `npx shiplight report` step too — the cloud report still gets full traces and videos for humans.

### Step 2 — add the caller workflow

The `workflow_run` trigger and the per-repo credential mapping **must** live in your repo — a `workflow_run` trigger is illegal inside a reusable workflow, and secret names differ per repo. Everything else lives in the reusable workflow. Create `.github/workflows/ci-failure-triage.yml`:

```yaml
name: CI Failure Triage

on:
  workflow_run:
    workflows: [E2E Tests] # exact `name:` of each test workflow to watch
    types: [completed]

jobs:
  triage:
    uses: ShiplightAI/ci-triage/.github/workflows/triage.yml@v1.1 # pin to a release tag, not @main
    permissions:
      contents: write
      pull-requests: write
      actions: read
    with:
      triage-runner: ubuntu-latest # read-only diagnosis job
      autofix-runner: shiplight-medium # re-runs tests, so needs browsers/network
      node-version: "22"
      allowed-paths: "tests templates" # top-level dirs the autofix agent may edit (hard guard)
    secrets:
      claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
      openai_api_key: ${{ secrets.OPENAI_API_KEY }} # Codex fallback when Claude is unavailable
      autofix_github_token: ${{ secrets.AUTOFIX_GITHUB_TOKEN }} # PAT/App token to open the PR; falls back to GITHUB_TOKEN
      # Per-repo credential mapping → generic env for the autofix re-run.
      # One KEY=VALUE per line; values must be single-line.
      extra_env: |
        BASE_URL=${{ vars.BASE_URL || 'https://example.com' }}
        MY_TEST_USER_PASSWORD=${{ secrets.MY_TEST_USER_PASSWORD }}
        MY_TEST_USER_2FA_SECRET=${{ secrets.MY_TEST_USER_2FA_SECRET }}
```

Notes:

- `workflows:` must list the exact `name:` of each test workflow to watch. Never list the triage workflow itself there.
- `extra_env` maps your repo's secret names onto the generic env the autofix job uses, so `npx shiplight test` and the MCP browser can authenticate during the re-run. Mirror the `env:` block from your test workflow.
- Provide at least one model credential (`claude_code_oauth_token` or `anthropic_api_key`); `openai_api_key` enables the Codex fallback. `autofix_github_token` is optional.
- `autofix-runner` re-runs the failing test, so it needs browsers and network — use a Shiplight runner, or install Chromium on a stock runner the same way your test workflow does.
- These examples assume the Shiplight project lives at the repository root. If it lives in a subdirectory, adjust the report path and add `working-directory` to the relevant steps.

::: warning
The triage job runs privileged — `contents: write` plus live credentials for the autofix re-run. Pin `uses:` to a release tag (`@v1.1`), never a moving branch like `@main`. For an immutable supply-chain pin, use the full commit SHA of the tag with the tag as a comment.
:::

### Step 3 — notify your own system (optional)

Triage publishes its diagnosis as the `triage-context` artifact: `verdict.json` (machine-readable) and `triage.md` (the human-readable write-up). Add a normal job you own to read it and call Slack, Linear, Jira, or anything else — your provider credentials stay in that job and are never visible to the triage agent.

```yaml
publish-triage-incidents:
  needs: triage
  if: ${{ needs.triage.result != 'skipped' }}
  continue-on-error: true # an external system must never block triage or autofix
  runs-on: ubuntu-latest
  permissions:
    actions: read
    contents: read
  steps:
    - uses: actions/download-artifact@v4
      with:
        name: triage-context
        path: /tmp/triage-context
    - name: Publish to your system
      env:
        # Keep provider credentials here, never in `extra_env`.
        SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
        LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
      run: jq '.failures[]' /tmp/triage-context/verdict.json
```

Each entry in `verdict.json`'s `failures[]` has the test path, `classification`, `confidence`, `fixable`, a `fix_summary`, and a stable `dedup_key`. Pick your own policy — notify every failure, or file incidents only for `fixable == false`. For an issue tracker, key on `dedup_key` so a repeated failure updates the existing incident instead of opening a new one on every run.

Each autofix job additionally uploads an `autofix-result-<test>-<index>` artifact with the PR URL, verification status, and whether the fix was skipped. Read those in a later job if you want a separate post-autofix notification.

## Troubleshooting

### Job stays queued

- Confirm Shiplight Runners are enabled at [app.shiplight.ai/runners](https://app.shiplight.ai/runners), and check the **Dispatches** tab there for the failure reason. See [CI Runners](/cloud_v2/ci-runners#job-stays-queued).
- Verify the Shiplight GitHub App is installed on the repository with the required permissions.
- Check whether your organization's runner concurrency limit has been reached. Contact your Shiplight admin to raise the limit if needed.

### Test results not appearing in the dashboard

- Make sure the `shiplight report` step has `if: always()` so it runs after failures.
- Check the step logs for errors.

### Chromium is downloading during CI

The runner has Chromium pre-installed. Do not add `npx playwright install chromium` to your workflow. If Playwright still downloads a browser, check that your `package.json` does not override the Playwright version bundled with `shiplightai`.

## Related

- [CI/CD overview](/local/ci) — how the pieces fit together, and other providers
- [CLI Reference](/local/cli-reference) — full `shiplight test`, `shiplight report`, and other command documentation
- [Local Testing](/local/run-locally) — project structure, authentication, and configuration