Shiplight Fixture

View as Markdown

The shiplightai package provides a Playwright fixture that extends the standard test object with additional capabilities. These are configured via the use: block in your YAML E2E test or in playwright.config.ts.

The YAML language defines what your test does. This page covers the fixture: the runtime that executes the test, providing authentication, Chrome extension support, and AI agent integration.

Authentication (auth)

Automatically log in before the test runs. Point to a TypeScript module that exports a login() function returning a storage state file path. For the shared-account vs per-test decision and the full login() contract, see Authentication:

goal: Verify dashboard after login
base_url: https://app.example.com

use:
  auth: ./auth.login.ts
  args:
    username: "{{TEST_USER}}"
    password: "{{TEST_PASS}}"
statements:
  - URL: /dashboard
  - VERIFY: Dashboard is displayed

The login() function handles login and returns a storage state for the test to run.

Chrome Extension Testing (extensionDir)

Load an unpacked Chrome extension into the browser:

goal: Verify extension injects banner
base_url: https://example.com

use:
  extensionDir: ./my-extension
statements:
  - URL: /
  - VERIFY: Extension banner is visible at the top of the page

The fixture launches a persistent Chromium context with --load-extension in headed mode (headless Chrome cannot load extensions).

extensionDir must point to an unpacked extension directory containing manifest.json, not a .zip, .crx, or individual file. Relative paths resolve from the project root—the directory containing playwright.config.*—rather than from the YAML test file.

Chromium launch options

Use Playwright's standard launchOptions to pass test-only browser options to extension and profile contexts. For example, a recorder extension can automatically select a capture source during a test:

import { defineConfig, shiplightConfig } from "shiplightai";

export default defineConfig({
  ...shiplightConfig(),
  use: {
    extensionDir: "./dist",
    headless: false,
    launchOptions: {
      args: ["--auto-select-screen-capture-source"],
      slowMo: 25,
    },
  },
});

Shiplight merges these options with the arguments required to load the extension. The fixture owns the following Chromium switches, so do not supply them in launchOptions.args:

• --load-extension
• --disable-extensions-except
• --user-data-dir
• --remote-debugging-port
• --remote-debugging-pipe

Passing one of these switches causes the test to fail with a configuration error instead of silently loading the wrong extension, profile, or debugging transport.

Test the Chrome action popup (extensionActionPopup)

Playwright does not expose a Chrome action popup through context.pages(). Shiplight's extensionActionPopup fixture opens the real toolbar popup and exposes it as a standard Playwright Page.

Use it directly from a YAML code step:

goal: Start recording from the extension popup

use:
  extensionDir: ./dist
statements:
  - description: Start recording
    js: |
      await extensionActionPopup
        .getByRole('button', { name: 'Start recording' })
        .click();
      await expect(extensionActionPopup.getByText('Recording')).toBeVisible();

Persistent Chrome Profile (userDataDir)

Reuse a Chrome profile directory across test runs. Works with or without extensions: useful for Google OAuth, cached sessions, or any state that lives in the Chrome profile:

goal: Verify app with persistent login
base_url: https://app.example.com

use:
  userDataDir: ./chrome-profile
statements:
  - URL: /
  - VERIFY: User is already logged in

Can also be combined with extensionDir:

goal: Verify extension with cached state
base_url: https://example.com

use:
  extensionDir: ./my-extension
  userDataDir: ./chrome-profile
statements:
  - URL: /
  - VERIFY: Extension shows saved settings

Extension Storage State (extensionStorageState)

Inject cookies into a persistent extension context (since Playwright's storageState option doesn't work with persistent contexts):

goal: Test extension on authenticated page
base_url: https://app.example.com

use:
  extensionDir: ./my-extension
  extensionStorageState: ./auth/storage-state.json
statements:
  - URL: /
  - VERIFY: User is logged in and extension is active

Test Context (testContext)

The testContext fixture provides a shared variable store accessible from YAML steps, custom functions, and inline code. It supports property-style access:

// In a custom function
export async function setup_user(page, testContext) {
  testContext.userId = "user-123"; // write
  const email = testContext.userEmail; // read
}

Variables set on testContext are available in YAML as {{variableName}}, and vice versa. The agent's $variableName resolves from the same store.

Agent (agent)

The agent fixture provides the AI for actions like VERIFY, intent: resolution, and ai_extract. It shares the same variable store as testContext. Configured automatically from environment variables:

Variable	Description
SHIPLIGHT_API_TOKEN	Recommended. Org token from app.shiplight.ai/api-tokens: also used to upload reports to Shiplight Cloud
GOOGLE_API_KEY, ANTHROPIC_API_KEY, or OPENAI_API_KEY	Bring your own provider key instead. At least one required if not using SHIPLIGHT_API_TOKEN; model is auto-detected
WEB_AGENT_MODEL	Override model selection

AI features (DRAFT statements, VERIFY, ai_extract) require either SHIPLIGHT_API_TOKEN or a provider API key. ACTION statements with js: or action: run without one.