Where tests run

View as Markdown

A Shiplight suite is plain YAML plus a Playwright project in your repo, so it is not tied to any one machine. The same tests run in three places, and moving between them changes a workflow file, not your suite.

	Your machine	Shiplight-hosted runners	Your own CI
How	npx shiplight test	runs-on: shiplight-small in GitHub Actions	Any CI provider's runners
Best for	Authoring, debugging, verifying a change	Zero-setup CI on GitHub	Existing CI fleets, infra and compliance requirements
Browser install	You (or your agent) once	Prebaked (Chromium, Firefox, WebKit)	You, in the job
Credentials	.env in your project	Injected per run, revoked after	You store secrets yourself
Shiplight compute billed	No	Yes, per minute	No
Plan required	None for local authoring	Any plan with credit	Any plan, including Free

Your machine

Local runs are where suites are authored and repaired: /shiplight create-yaml-tests to write tests, npx shiplight test to run them, the visual debugger to hand-tune. You need Node, a browser, and an LLM key for AI-powered steps: your own provider key, or a Shiplight API token via the LLM proxy. See Run Locally.

Shiplight-hosted runners

Ephemeral VMs that stand in for GitHub's ubuntu-latest: browsers preinstalled, Shiplight and LLM credentials injected per run, result upload enabled by the platform, and the self-healing action cache persisted across runs. You switch a single runs-on line and pick a machine size. GitHub Actions only, and the repository must belong to a GitHub organization.

Compute is billed per minute against your plan's credit while a runner is up, plus LLM tokens for AI-powered steps (through the proxy by default, or your own key). Current rates are on the pricing page; a plan changes how much credit you start with, not what a minute costs. Setup, pools, limits, and debugging: CI Runners.

Your own CI

The shiplight CLI runs on any pipeline with Node: GitHub's own runners, GitLab, CircleCI, Jenkins, Buildkite, Azure DevOps, or your self-hosted machines. You install the browser in the job, store SHIPLIGHT_API_TOKEN as a secret to upload results, and supply LLM credentials (your own key, or a Shiplight token via the proxy). Neither Shiplight compute nor the proxy has to be in the path, and running this way is available on every plan, including Free.

Two things to know:

• Result uploads still work. shiplight report uploads from any machine; Shiplight Cloud dashboards do not care where the run happened.
• A few analytics need our side. Usage and cost views are built from what flows through Shiplight runners or the LLM proxy, so those specific views are limited when you bring your own everything. Test results, trends, and flakiness detection work regardless.

Provider templates: CI/CD.

How to choose

• Start local. Authoring and debugging happen on your machine either way.
• On GitHub and want CI working this hour: Shiplight runners. One runs-on line, no browser install, no secret plumbing.
• You already have a CI fleet, or infra rules say your code runs on your machines: your own CI. You keep full control and pay Shiplight nothing for compute.
• Cost-sensitive at scale: measure, then mix. Cached deterministic steps call no model, so a mature suite is cheap to run anywhere; right-sizing runners and bringing your own LLM key are the two biggest levers.

You can change your mind later, or use several at once (for example, hosted runners for PR checks and your own nightly machines): the YAML never changes.

Related

• Run Locally: project structure, configuration, the debugger
• CI/CD: wiring a provider, auto-triage, agent setup
• CI Runners: hosted-runner setup, sizes, pools, debugging
• LLM Providers: key options, including the Shiplight proxy