Buildkite

View as Markdown

Run your Shiplight E2E tests in Buildkite and upload the results to Shiplight Cloud. Buildkite uses the same shiplight CLI as every other provider — only the pipeline syntax differs.

Before you start

• A scaffolded Shiplight test project (/shiplight init). See the CI/CD overview for how the pieces fit together.
• An org API token from app.shiplight.ai/api-tokens. Expose it to the job as SHIPLIGHT_API_TOKEN via your agent's secret mechanism (e.g. the secrets field or a secrets plugin) — never commit it to the pipeline file.

Pipeline

Create .buildkite/pipeline.yml at your repository root:

steps:
  - label: ":test_tube: Shiplight E2E"
    plugins:
      - docker#v5.11.0:
          image: "node:20"
    env:
      SHIPLIGHT_REPORT_TO_CLOUD: "1"
      # SHIPLIGHT_API_TOKEN is provided as a secret, not set here.
    command: |
      npm install
      npx playwright install --with-deps chromium
      set +e
      npx shiplight test; STATUS=$?
      npx shiplight report        # upload regardless of the test result
      exit $STATUS

Notes

• SHIPLIGHT_API_TOKEN and SHIPLIGHT_REPORT_TO_CLOUD=1 are required on every non-Shiplight runner — they are provided automatically only on Shiplight-hosted GitHub runners.
• Install Chromium yourself (npx playwright install --with-deps chromium); no stock image ships it.
• Buildkite stops a command list on the first failure, so the template captures the test exit code, always runs report, then re-exits with the original status — that's how the upload survives a red run.
• Not available on Buildkite: Shiplight-hosted runners, the self-healing action cache, automatic LLM credentials, and auto-triage — those are GitHub Actions only.
• Project in a subdirectory (e.g. tests/e2e)? cd tests/e2e as the first line of the command.

Related

• CI/CD overview · Custom setups · CLI Reference