Azure DevOps

View as Markdown

Run your Shiplight E2E tests in Azure Pipelines and upload the results to Shiplight Cloud. Azure DevOps uses the same shiplight CLI as every other provider — only the pipeline syntax differs.

Before you start

• A scaffolded Shiplight test project (/shiplight init). See the CI/CD overview for how the pieces fit together.
• An org API token from app.shiplight.ai/api-tokens, stored as a secret pipeline variable named SHIPLIGHT_API_TOKEN (Pipelines → Edit → Variables, "Keep this value secret"). Secret variables are not injected into the environment automatically — the template maps it explicitly via env:.

Pipeline

Create azure-pipelines.yml at your repository root:

trigger:
  - main

pool:
  vmImage: ubuntu-latest

variables:
  SHIPLIGHT_REPORT_TO_CLOUD: "1"

steps:
  - task: NodeTool@0
    inputs:
      versionSpec: "20.x"
    displayName: Use Node 20

  - script: npm install
    displayName: Install dependencies

  - script: npx playwright install --with-deps chromium
    displayName: Install Chromium

  - script: npx shiplight test
    displayName: Run E2E tests
    env:
      SHIPLIGHT_API_TOKEN: $(SHIPLIGHT_API_TOKEN) # map the secret variable into env

  - script: npx shiplight report
    displayName: Upload results to Shiplight
    condition: always() # upload even when tests fail
    env:
      SHIPLIGHT_API_TOKEN: $(SHIPLIGHT_API_TOKEN)

Notes

• SHIPLIGHT_API_TOKEN and SHIPLIGHT_REPORT_TO_CLOUD=1 are required on every non-Shiplight runner — they are provided automatically only on Shiplight-hosted GitHub runners.
• Secret variables must be mapped into env: per step (SHIPLIGHT_API_TOKEN: $(SHIPLIGHT_API_TOKEN)); non-secret variables like SHIPLIGHT_REPORT_TO_CLOUD are exposed automatically.
• Install Chromium yourself (npx playwright install --with-deps chromium); the hosted image doesn't ship it.
• condition: always() on the report step is what makes the upload survive a red run.
• Not available on Azure DevOps: Shiplight-hosted runners, the self-healing action cache, automatic LLM credentials, and auto-triage — those are GitHub Actions only.
• Project in a subdirectory (e.g. tests/e2e)? Add workingDirectory: tests/e2e to each script step.

Related

• CI/CD overview · Custom setups · CLI Reference