Create Tests

View as Markdown

Authoring tests is the main Shiplight workflow, all through /shiplight. Your agent writes YAML tests for most flows. For a whole feature or PR, a coverage pass decides what to test and drives it for you.

YAML tests

Run /shiplight create-yaml-tests for fast, replayable YAML E2E tests: the agent walks your app in a real browser and captures real locators.

/shiplight create-yaml-tests for the sign-up flow at http://localhost:3000 —
a new user should register, verify their email, and land on the dashboard

It assumes the project is already scaffolded. Run /shiplight init first on a new project, and see Authentication for the authenticated flow. It doesn't invent YAML: it explores the UI, captures locators, writes focused tests (one journey each), and validates them before finishing. On an existing project it reads your specs and tests first. It also handles updates:

/shiplight create-yaml-tests add a test for the "forgot password" flow

See the YAML Test Format chapter for the format.

Need something more flexible than a YAML test?

For core journeys that must not ship broken, run /shiplight create-agent-verification — the most comprehensive format, driving the journey through the UI and then proving the backend state it should have changed (API, DB, logs). A coding agent executes the Markdown case against a live environment and returns an auditable status report. See Agent Verification.

Comprehensive coverage

For a whole feature or PR, /shiplight cover decides what must be tested and how, then drives the producers to write it — a YAML test, an agent verification, or your project's own unit / contract / integration test — spending the expensive test budget (e2e, agent) where priority is highest and using the cheapest modality that can actually prove each check.

/shiplight cover the billing feature on this PR

It records the result as facts, not a graded verdict:

Artifact	What it holds
specs/<feature>/test-spec.md	The behaviors and invariants to verify, each with its priority
specs/<feature>/test-report.md	What was tested, by what test type, and what passed / failed / was blocked
TESTING.md (optional)	Project-wide testing posture (a baked-in default applies when absent)