Fix a failing test
Use /shiplight fix when a test fails after a deployment, a UI change, or without an obvious cause. Your coding agent reproduces the failure, identifies the cause, and repairs the test. Stale locators usually self-heal during the run; fix handles other failures and records permanent locator changes in the YAML.
Describe the failure
Provide whichever starting point you have:
- A failing YAML test or test suite.
- A local test report, such as
shiplight-report/index.html. - A link to a failed CI run or test.
- A description of the failure.
For example:
/shiplight fix — investigate the failures in shiplight-report/index.htmlWhen you provide a local report, the agent reads its failure details. When you provide a CI link, it retrieves the available logs and artifacts. It then identifies the failing test, reruns the smallest relevant target to reproduce the failure, and retries once if the failure appears transient. The agent also reviews the YAML and related specification, and inspects the application in a browser when the logs do not provide enough information.
Review the diagnosis
You receive the repaired YAML and a report that lists the commands and results, changed files, repaired or skipped tests, remaining failures, and any differences between the application and its specification.
Before editing anything, the agent identifies why the update is needed, because the reason determines the fix:
| Diagnosis | What the agent does |
|---|---|
| Locator drift — the UI changed, intended behavior didn't | Updates only implementation details such as locators, waits, and setup. It does not delete assertions or skip steps |
| Product change — intended behavior changed on purpose | Updates the specification first, then updates the YAML to match |
| Test bug — the implementation was wrong | Corrects the test logic |
| Coverage gap — new assertions or journeys are needed | Extends the test |
If the application is broken, the agent reports the issue instead of changing the test to hide it. The test remains failing until the application is fixed.
Before completing the workflow, the agent validates the edited YAML with npx shiplight transpile --strict and reruns the smallest changed target.
Choose another workflow
- The product is being intentionally redesigned — Plan and rewrite the tests for the new workflow instead of repairing the old ones. Start with Decide what to test.
- No test is failing and you want to check a UI change — Use Verify a change.
- The same locators change repeatedly — Review the application's testability with
review designinstead of continuing to repair the tests.
Repair failures in CI
The same diagnostic workflow can run automatically after a CI failure. See Auto-triage failures in the CI/CD overview.