Action cache in CI
Shiplight saves self-healed actions from passing tests so later runs can replay the repaired locators. To manage this cache in your own CI, set SHIPLIGHT_ACTION_CACHE_BACKEND=local and persist .shiplight/action-cache/ with either your CI provider's cache storage or Git.
Both approaches avoid the Shiplight action-cache API. AI calls and report uploads have separate credentials and routing: using local action cache does not disable the Shiplight LLM proxy or cloud reporting. To run AI steps without the proxy, supply a provider API key.
Choose a backend
Set SHIPLIGHT_ACTION_CACHE_BACKEND in the workflow environment or your project's .env:
| Value | Behavior |
|---|---|
auto (default) | Cloud when CI is set and SHIPLIGHT_API_TOKEN is available; local otherwise. |
local | Always use .shiplight/action-cache/, even in CI with a Shiplight token. No action-cache API calls. |
cloud | Always select cloud storage, including outside CI. Requires SHIPLIGHT_API_TOKEN. |
Unknown values are configuration errors. .env values override shell and workflow environment values; remove a conflicting setting from .env when selecting a backend in CI.
Use a CLI build that supports this setting. The examples assume your committed package.json and lockfile install that build. When testing an unreleased candidate, install the candidate tarball after npm ci; a version number alone does not distinguish a candidate from a published build with the same version.
Choose how to persist local cache
| CI cache storage | Git commits | |
|---|---|---|
| Restore | Restore the cache archive before tests | Normal repository checkout |
| Save | Save a new archive after tests | Commit and push changed cache JSON |
| Repository history | No cache commits | Cache changes appear in history |
| Permissions | Repository read access for checkout | Repository write access to the target branch |
| Isolation | Workflow and branch in the cache key | Branch; workflows on that branch share cache files |
| Concurrent updates | Separate shard keys or serialize writers | Rebase on latest remote; favor this run on conflicts |
Use one approach for a given workflow. Both examples run on GitHub's ubuntu-latest runner, assume the test project is at the repository root, and use a SHIPLIGHT_API_TOKEN repository secret for AI calls. You can replace that secret with supported provider credentials. Add your tests' account secrets as needed.
Option 1: GitHub Actions cache
Keep .shiplight/ ignored by Git. Save this as .github/workflows/e2e-cache.yml:
name: E2E with action cache
on:
workflow_dispatch:
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: false
jobs:
e2e:
runs-on: ubuntu-latest
env:
SHIPLIGHT_ACTION_CACHE_BACKEND: local
SHIPLIGHT_API_TOKEN: ${{ secrets.SHIPLIGHT_API_TOKEN }}
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v4
with:
node-version: "22"
- run: npm ci
- run: npx playwright install --with-deps chromium
- name: Restore action cache
id: action-cache
uses: actions/cache/restore@v4
with:
path: .shiplight/action-cache
key: shiplight-action-v1-${{ github.workflow }}-${{ github.ref }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
shiplight-action-v1-${{ github.workflow }}-${{ github.ref }}-
- name: Run tests
id: tests
run: npx shiplight test
- name: Save action cache
if: ${{ !cancelled() && steps.tests.outcome != 'skipped' && hashFiles('.shiplight/action-cache/*.json') != '' }}
uses: actions/cache/save@v4
with:
path: .shiplight/action-cache
key: ${{ steps.action-cache.outputs.cache-primary-key }}The save step also runs when some tests fail: passing tests may have produced useful repairs. Shiplight writes new cache entries only for tests whose final status is passed.
Understand the cache key
The key contains:
shiplight-action-v1: a namespace and manually managed cache version. Bumpv1to start a fresh cache.github.workflow: the workflow name, isolating workflows with different names.github.ref: the branch or other Git ref.github.run_id: one triggered execution, not the workflow definition's ID.github.run_attempt: the attempt number; rerunning a run keeps its run ID and increments this value.
The key intentionally has no operating-system or CLI-package hash. Renaming a workflow changes its cache prefix. Each run/attempt saves a new key; restoration uses the workflow-and-ref prefix to find a previous archive. For the immutable-key and restore-prefix behavior, see the GitHub cache action.
For matrix jobs, include a project/shard identifier in both the save key and restore prefix. Each shard then owns its cache. A shared archive does not automatically merge concurrent shard results.
Option 2: Commit cache changes to Git
Checkout restores the cache with the rest of the repository. After tests, a script commits only the cache files changed by this run and pushes them to the same branch.
Track only action cache JSON
Replace an existing .shiplight/ ignore rule with these rules in .gitignore:
.shiplight/*
!.shiplight/action-cache/
.shiplight/action-cache/*
!.shiplight/action-cache/*.jsonThe parent directory must no longer be ignored as a whole, otherwise Git cannot re-include the cache files. Other .shiplight artifacts remain ignored. Existing local action cache JSON becomes eligible for Git tracking after this change.
Example workflow
This minimal example commits only the action cache directory, rebases the cache commit onto the latest branch, and pushes it. Adapt the test command and credentials to your project.
name: E2E with Git action cache
on:
workflow_dispatch:
permissions:
contents: write
concurrency:
group: shiplight-git-action-cache-${{ github.ref }}
cancel-in-progress: false
jobs:
e2e:
runs-on: ubuntu-latest
env:
SHIPLIGHT_ACTION_CACHE_BACKEND: local
SHIPLIGHT_API_TOKEN: ${{ secrets.SHIPLIGHT_API_TOKEN }}
steps:
- uses: actions/checkout@v5
with:
ref: ${{ github.ref }}
- uses: actions/setup-node@v4
with:
node-version: "22"
- run: npm ci
- run: npx playwright install --with-deps chromium
- name: Run tests
id: tests
run: npx shiplight test
- name: Commit action cache
if: ${{ !cancelled() && steps.tests.outcome != 'skipped' && github.ref_type == 'branch' }}
run: |
if [ ! -d .shiplight/action-cache ]; then
exit 0
fi
# Shared GitHub Actions bot identity; this is not your personal account.
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add -A -- .shiplight/action-cache/
if git diff --cached --quiet; then
echo "No action cache changes; no commit needed."
exit 0
fi
git commit -m "Update local action cache"
git pull --rebase -X theirs origin "$GITHUB_REF_NAME"
git push origin "HEAD:refs/heads/$GITHUB_REF_NAME"During this rebase, -X theirs favors the cache commit being replayed when its changes conflict with the remote version. Non-conflicting remote changes are retained. This is Git's text merge, not a merge of individual action entries; serialize writers to the same test cache and use a shared concurrency group across workflows that update it.
No cache changes means no commit. The example uses a normal push, not a force-push. If the branch advances again between rebase and push, the push can be rejected; rerun the job or add a bounded fetch/rebase/push retry for a busy repository. Branch protection rules still apply.
contents: write lets checkout's GitHub token push the cache commit, subject to repository policy. The Shiplight token authenticates AI calls; it is not used for Git pushes. This example is for branch runs triggered manually. If adapting it for pull requests, keep automatic write-back limited to branches your workflow can write; do not push to a synthetic pull-request merge ref.
Pushes made with the workflow's GITHUB_TOKEN do not trigger another push workflow. If you replace it with a different token, account for that token's trigger behavior. See GitHub's workflow trigger documentation.
Verify that cache was used
Check persistence, loading, and execution separately. A successful restore or checkout does not prove a test used a cached action.
| Evidence | Meaning |
|---|---|
Cache restored from key: ... | GitHub restored an archive in option 1. |
Cache: loaded 2 cached action entities for 1 test file | CLI loaded entries from the local cache directory. |
[Shiplight Cache] 2 served from cache of 4 executed statements | Two executed statements used cached actions. |
[Shiplight Cache] 2 auto-healed of 4 executed statements | Two statements needed runtime repair. |
Cache: saved 2 action entities for 1 test | CLI persisted updated local cache for the test. |
Cache saved with key: ... | GitHub stored the updated archive in option 1. |
Successful git push and an Update local action cache commit | Option 2 created and pushed a cache commit. |
No action cache changes; no commit needed. | Option 2 found nothing to commit. |
For option 1, cache-hit: false does not mean restore failed: a prefix match can restore an archive without an exact key match. Check the restored key or cache-matched-key instead. See the restore action outputs.
The CLI may also print Cache: downloaded 1 action store in local mode. That message is shared with the cloud path and is not evidence of an API request. The effective SHIPLIGHT_ACTION_CACHE_BACKEND=local setting selects the file backend.
To verify the full cycle, copy a small test, keep its action intents, and deliberately break one or two locators. First run: confirm self-healing, a passing test, and a persisted cache. Second run: leave the YAML unchanged and look for served from cache. In the Git variant, a run with no new repairs should produce no new commit.
Cache lifetime and project layout
Cache is written per test file, with entries keyed by statement UID. New entries replace the same UID and preserve other entries. Editing an inline action's name, arguments, locator, or XPath invalidates a fingerprinted cache entry; older entries without a fingerprint remain applicable until refreshed by a later heal. Local cache has no time-based expiry or automatic cleanup.
If the first run has no cache, a restore miss is normal. If no passing test generates a repair, there may be nothing to persist. To reset option 1, change the namespace version; for option 2, remove the cache files and commit their deletion.
For a project below the repository root, adjust test working-directory and archive paths. The Git example assumes .shiplight/action-cache/ at the repository root; adjust the ignore rules, directory check, and git add path for a nested project. Run the Git commands from the repository root. Other CI platforms can persist the same directory using their native cache mechanism, or adapt the Git example's branch and authentication setup.