Report to Cloud

View as Markdown

After a local test run, the Shiplight reporter can upload results to Shiplight Cloud — including per-step screenshots, video recordings, traces, and CI/git metadata. This lets you view and share results from local or CI runs alongside cloud-executed tests, without moving test execution to the cloud.

Setup

1. Get an API key

Get your API key from app.shiplight.ai/settings/api-tokens and add it to your .env file:

SHIPLIGHT_API_KEY=your-api-key-here

2. Enable in playwright.config.ts

Pass reportToCloud: true and your API key to shiplightConfig():

import { defineConfig, shiplightConfig } from "shiplightai";

export default defineConfig({
  ...shiplightConfig({
    reportToCloud: process.env.REPORT_TO_CLOUD === "true",
    apiKey: process.env.SHIPLIGHT_API_KEY,
  }),
});

Using an env var for reportToCloud lets you enable uploads selectively — for example, only in CI — without changing your config file.

After the run completes, the reporter logs the cloud report URL:

Shiplight cloud report: https://app.shiplight.ai/runs/12345

What Gets Uploaded

For each test:

Asset	Description
Per-step screenshots	One screenshot per YAML step, matched to step descriptions
Video	Full test recording (if enabled in Playwright config)
Trace	Playwright trace archive (if enabled)
Report JSON	Step-by-step results including status, duration, and AI messages

Alongside the results, the reporter collects CI and git metadata automatically — see CI Metadata below.

Configuration Options

All options are passed to shiplightConfig():

Option	Type	Default	Description
reportToCloud	boolean	false	Enable cloud upload after the run
apiKey	string	process.env.__SHIPLIGHT_API_KEY	Shiplight API key

To customize the output folder alongside cloud upload, override the reporter directly:

export default defineConfig({
  ...shiplightConfig({ reportToCloud: true, apiKey: process.env.SHIPLIGHT_API_KEY }),
  reporter: [
    ["list"],
    [
      "shiplightai/reporter",
      {
        outputFolder: "my-report",
        open: "on-failure",
        reportToCloud: true,
        apiKey: process.env.SHIPLIGHT_API_KEY,
      },
    ],
  ],
});

CI Metadata

The reporter automatically collects git and CI metadata and attaches it to the cloud run. No configuration is needed — it reads from standard CI environment variables.

Collected Fields

Field	Description
Commit SHA	The HEAD commit of the branch being tested
Branch	Source branch name
PR number	Pull request number (if triggered from a PR)
PR title	Pull request title
Commit message	Subject line of the HEAD commit
Author email	Commit author
CI build ID & URL	Link back to the CI run
Commit URL	Direct link to the commit on GitHub/GitLab

Environment Variable Overrides

Some CI setups — particularly repository_dispatch triggers where the workflow runs on the default branch rather than the PR branch — require explicit overrides. Set these environment variables to supply values the reporter cannot derive automatically:

Variable	Description
SHIPLIGHT_GIT_SHA	Override the commit SHA (e.g. the actual PR commit, not the merge commit)
SHIPLIGHT_PR_NUMBER	Override the PR number
SHIPLIGHT_PR_TITLE	Override the PR title
SHIPLIGHT_GIT_BRANCH	Override the branch name

These take precedence over any automatically detected values.

GitHub Actions

A minimal setup for GitHub Actions:

- name: Run E2E tests
  env:
    SHIPLIGHT_API_KEY: ${{ secrets.SHIPLIGHT_API_KEY }}
    REPORT_TO_CLOUD: "true"
  run: npx shiplight test

With Vercel Preview Deployments

When using repository_dispatch or deployment_status events triggered by Vercel, GITHUB_SHA points to the default branch — not the PR commit. Pass the correct metadata via the override env vars:

- name: Check for frontend changes
  id: check-changes
  uses: actions/github-script@v7
  with:
    script: |
      const sha = context.payload.client_payload?.git?.sha
                  ?? context.payload.deployment?.sha;
      const branch = context.payload.deployment?.ref;

      const { data: prs } = await github.rest.repos.listPullRequestsAssociatedWithCommit({
        owner: context.repo.owner,
        repo: context.repo.repo,
        commit_sha: sha,
      });

      core.setOutput('git_sha', sha ?? '');
      core.setOutput('git_branch', branch ?? prs[0]?.head?.ref ?? '');
      core.setOutput('pr_number', String(prs[0]?.number ?? ''));
      core.setOutput('pr_title', prs[0]?.title ?? '');
      // ... check changed files

- name: Run E2E tests
  env:
    SHIPLIGHT_API_KEY: ${{ secrets.SHIPLIGHT_API_KEY }}
    REPORT_TO_CLOUD: "true"
    PLAYWRIGHT_BASE_URL: ${{ github.event.client_payload.url || github.event.deployment_status.environment_url }}
    SHIPLIGHT_GIT_SHA: ${{ steps.check-changes.outputs.git_sha }}
    SHIPLIGHT_GIT_BRANCH: ${{ steps.check-changes.outputs.git_branch }}
    SHIPLIGHT_PR_NUMBER: ${{ steps.check-changes.outputs.pr_number }}
    SHIPLIGHT_PR_TITLE: ${{ steps.check-changes.outputs.pr_title }}
  run: npx shiplight test