Code Steps

View as Markdown

The CODE: step lets you run arbitrary JavaScript directly inside a YAML test. It executes in the same Node.js context as Playwright, giving you full access to page, browser APIs, HTTP requests, and the test variable store.

Use it when a natural language step isn't precise enough — calling an API, intercepting network requests, seeding browser state, capturing values mid-test, or writing exact assertions.

Table of Contents

1. Syntax
2. Available Context
3. Failing a Step
4. Common Use Cases
   • Call APIs
   • Page Operations
   • Store Values for Later Steps
   • Custom Assertions
   • Verify a Downloaded File
5. CODE vs ACTION js: vs VERIFY js:
6. Limitations

Syntax

Single-line:

statements:
  - CODE: "await page.waitForLoadState('networkidle')"

Multi-line (block scalar):

statements:
  - CODE: |
      await page.route('**/api/users', (route) => route.fulfill({
        status: 200,
        body: JSON.stringify([{ name: 'Alice' }]),
      }));
      testContext.apiMocked = true;

Quoting rules

Use double quotes for single-line code that doesn't contain double quotes. Use the block scalar (|) for multi-line code or any code containing special YAML characters ({, }, :, #).

Available Context

The code runs in Node.js (not the browser). The following variables are available without any imports:

Variable	Type	Description
page	Playwright Page	The current browser page
context	Playwright BrowserContext	The browser context (cookies, permissions, etc.)
expect	Playwright assertions	For writing assertions
request	Playwright APIRequestContext	For making HTTP requests
testContext	Shiplight variable store	Read and write test variables. Aliases: $, ctx
agent	Shiplight WebAgent	AI-powered actions — assertions, natural language step execution
console	Captured console	Logs appear in test output

Async/await is fully supported. All code runs inside an async context, so you can freely use await.

To run code inside the browser (accessing window, document, localStorage, etc.), use page.evaluate():

- CODE: |
    await page.evaluate(() => {
      localStorage.setItem('featureFlag', 'true');
    });

Working with testContext

testContext (also $ and ctx) is a proxy to the test's variable store. Values set here are available in all subsequent steps, including natural language steps.

// Set a variable
testContext.orderId = "12345";
$.theme = "dark";

// Read a variable
const id = testContext.orderId;

// Set a sensitive value (masked in logs)
testContext.set("apiToken", "secret-value", true);

// Read explicitly
const token = testContext.get("apiToken");

Using agent

The agent object gives you access to Shiplight's AI capabilities from inside a code step.

agent.assert(page, statement) — AI-powered assertion. The agent looks at the current page and verifies whether the statement is true.

- CODE: |
    await agent.assert(page, 'The shopping cart shows 3 items');

agent.execute(page, statement) — Execute a natural language instruction. The agent reads the page and performs the described action, just like a natural language step.

- CODE: |
    await agent.execute(page, 'Fill out the registration form with test data');
    await agent.execute(page, 'Click the Submit button');

agent.extract(page, description, variableName) — Extract a value from the page using AI and store it in testContext.

- CODE: |
    await agent.extract(page, 'the order confirmation number', 'orderNumber');
    console.log('Order:', testContext.orderNumber);

agent.getRecentDownloadedFilePath() — Get the path of the most recently downloaded file.

- CODE: |
    await agent.waitForDownloadComplete(page, 10);
    const filePath = agent.getRecentDownloadedFilePath();
    console.log('Downloaded:', filePath);

Failing a Step

The return value of the code is ignored. To fail a step, either throw an error or use expect():

// Fail via expect (throws if assertion doesn't hold):
const title = await page.title();
expect(title).toContain("Dashboard");

// Fail via throw:
const count = await page.locator(".item").count();
if (count === 0) {
  throw new Error("Expected at least one item, found none");
}

Common Use Cases

Call APIs

Use request to call HTTP endpoints directly from a test step — useful for seeding test data, triggering backend actions, or checking state that isn't visible in the UI.

statements:
  - CODE: |
      const response = await request.post('https://api.example.com/orders', {
        headers: { Authorization: `Bearer ${testContext.apiToken}` },
        data: { productId: 'prod_123', quantity: 2 },
      });
      expect(response.ok()).toBeTruthy();
      const order = await response.json();
      testContext.orderId = order.id;
  - URL: "https://app.example.com/orders/{{orderId}}"
  - VERIFY: Order details page is displayed

request uses the same authentication cookies as the browser session, so it works for authenticated API calls without any extra setup.

Page Operations

Interact directly with the browser — mock network requests, seed storage, inject cookies, or wait for specific states.

Mock an API response:

statements:
  - CODE: |
      await page.route('**/api/products', (route) => route.fulfill({
        status: 200,
        contentType: 'application/json',
        body: JSON.stringify([{ id: 1, name: 'Widget', price: 9.99 }]),
      }));
  - URL: https://app.example.com/shop
  - VERIFY: Widget is listed with price $9.99

Seed browser storage before a page load:

statements:
  - URL: https://app.example.com
  - CODE: |
      await page.evaluate(() => {
        localStorage.setItem('onboarding_complete', 'true');
      });
  - URL: https://app.example.com/dashboard
  - VERIFY: Onboarding banner is not shown

Inject a session cookie:

- CODE: |
    await context.addCookies([{
      name: 'session',
      value: 'abc123',
      domain: 'app.example.com',
      path: '/',
    }]);

Wait for a condition:

- CODE: "await page.waitForLoadState('networkidle')"

- CODE: |
    await page.waitForFunction(() => {
      return document.querySelectorAll('.item').length >= 10;
    }, { timeout: 10000 });

Store Values for Later Steps

Read something from the page and save it to testContext so later steps can reference it.

statements:
  - intent: Submit the order form
  - CODE: |
      const orderNumber = await page.locator('[data-testid="order-number"]').textContent();
      testContext.orderNumber = orderNumber?.trim();
  - URL: https://app.example.com/order-history
  - VERIFY: "Order {{orderNumber}} appears in the history"

Values stored in testContext are available in all subsequent steps — both code steps and natural language steps — for the duration of the test run.

Custom Assertions

Write exact, programmatic checks for cases where VERIFY: isn't precise enough — counts, numeric ranges, string patterns, or multi-element checks.

- CODE: |
    const rows = await page.locator('table tbody tr').count();
    expect(rows).toBe(5);

- CODE: |
    const total = await page.locator('[data-testid="cart-total"]').textContent();
    const amount = parseFloat(total!.replace('$', ''));
    expect(amount).toBeGreaterThan(0);
    expect(amount).toBeLessThan(1000);

Use throw when expect() doesn't fit:

- CODE: |
    const status = await page.locator('.order-status').textContent();
    if (!['Shipped', 'Delivered'].includes(status?.trim() ?? '')) {
      throw new Error(`Unexpected order status: ${status}`);
    }

Verify a Downloaded File

Shiplight automatically saves files to disk when a download is triggered. Use agent methods to wait for the download to complete and verify the result.

statements:
  - intent: Click the Export CSV button
  - CODE: |
      // Wait up to 10 seconds for the download to finish
      await agent.waitForDownloadComplete(page, 10);

      // Get the path of the most recently downloaded file
      const filePath = agent.getRecentDownloadedFilePath();
      expect(filePath).toBeTruthy();
      expect(filePath).toContain('.csv');
      console.log('Downloaded file:', filePath);

Verifying file contents

The code step sandbox does not have access to the filesystem, so you cannot read file contents directly. To verify what's inside a downloaded file, use a custom function — custom functions run with full Node.js access, including fs.

CODE vs ACTION js: vs VERIFY js:

All three let you write Playwright code, but they serve different purposes:

	CODE:	intent: + js:	VERIFY: + js:
Self-healing	No	Yes — falls back to intent:	Yes — falls back to AI vision
Purpose	Setup, utilities, mocking, assertions	UI interactions with a fast-replay cache	Assertions with an AI fallback
Failure behavior	Throws → test fails immediately	Throws → agent retries using intent:	Throws → agent evaluates VERIFY: via AI
Best for	Network setup, storage seeding, computed checks	Clicks, typing, keyboard, hover	Checking visible outcomes

Use CODE: when the operation is deterministic and doesn't need recovery — setting up mocks, seeding data, computing derived values, or asserting precise conditions.

Use intent: + js: when the step interacts with the UI and you want AI-powered self-healing if the page changes.

Use VERIFY: + js: when you have a preferred assertion but want the AI to verify the outcome in natural language if the code path fails.

Limitations

• No dynamic imports. You cannot use import() inside a code step. For reusable logic, use custom functions instead.
• Node.js context only. Code runs on the Node.js side, not in the browser. Use page.evaluate() to access browser globals like window, document, or localStorage.
• No return value. The return value of the code block is discarded. Use testContext to pass values between steps.