---
title: "Webhook Integration"
description: "Automatically receive test results when test runs complete through webhook notifications"
---

# Webhook Integration Guide

<div class="view-markdown-wrapper">
<ViewMarkdown />
</div>

## Overview

Shiplight Webhooks allow you to automatically receive test results when test runs complete. You can use this feature to:

- Integrate with internal monitoring systems
- Trigger automated workflows
- Send custom notifications
- Log test history to databases
- Integrate into CI/CD pipelines

---

## Quick Start

### 1. Configure Webhook Endpoint (One-Time Setup)

First, you need to register your webhook endpoint in Shiplight's global settings:

1. Navigate to **Settings → Notifications → Webhooks**
2. Click **Add Webhook Endpoint**
3. Fill in the information:
   - **Name**: Give your webhook a descriptive name (e.g., "Production Dashboard", "Slack Notifier")
   - **URL**: Your server's webhook receiver URL (must be HTTPS, e.g., `https://api.yourcompany.com/webhooks/shiplight`)
   - **Secret**: Click "Generate Secret" to auto-generate (used for signature verification)
   - **Enabled**: Toggle on to activate this endpoint globally
4. Click **Save**
5. **Important**: Record the Secret somewhere safe (e.g., password manager) - you'll need it to verify webhook signatures on your server

**Note**: This is a one-time setup. Once configured, you can reuse this endpoint for multiple test plans and test suites.

### 2. Subscribe to Webhooks

After configuring endpoints, you can subscribe to receive notifications:

#### For Test Plans (Scheduled Runs)

1. Navigate to **Test Plans → Select a Test Plan → Schedule Tab → Notifications**
2. Click on the **Webhook** tab
3. Toggle on the webhook endpoint(s) you want to subscribe to
4. Configure **Send When** conditions:
   - **All**: Send for every test run (regardless of result)
   - **Failed**: Only send when there are failed test cases
   - **Pass→Fail**: Only send when tests regress (previously passed tests now fail)
   - **Fail→Pass**: Only send when tests are fixed (previously failed tests now pass)
5. Click **Save**

**Trigger**: Test plan webhooks trigger on **Scheduled** runs. The frontend automatically sets `trigger = "Scheduled"` when you configure the subscription.

#### For Test Suites (GitHub Action Runs)

1. Navigate to **Test Suites → Select a Test Suite → Notifications Sidebar**
2. Click on the **Webhook** tab in the notification modal
3. Toggle on the webhook endpoint(s) you want to subscribe to
4. Configure **Send When** conditions:
   - **All**: Send for every test run (regardless of result)
   - **Failed**: Only send when there are failed test cases
   - **Passed**: Only send when all tests pass
5. Click **Save**

**Trigger**: Test suite webhooks trigger on **GitHub Action** runs. The frontend automatically sets `trigger = "GITHUB_ACTION"` when you configure the subscription.

### 3. Implement Server-Side Receiver

Refer to the code examples below to implement your webhook endpoint that receives and processes test results.

---

## Webhook Request Format

### HTTP Headers

```
POST /your-webhook-endpoint HTTP/1.1
Host: your-server.com
Content-Type: application/json
User-Agent: Shiplight-Webhooks/1.0
X-Webhook-Signature: sha256=<HMAC-SHA256 signature>
X-Webhook-Timestamp: <Unix timestamp>
```

### Request Body

Complete `TestRunAnalysisResults` JSON object:

```json
{
  "testRun": {
    "id": 12345,
    "testPlanId": 678,
    "organizationId": "org-abc123",
    "status": "Finished",
    "result": "Failed",
    "trigger": "Scheduled",
    "totalTestCaseCount": 10,
    "passedTestCaseCount": 8,
    "failedTestCaseCount": 2,
    "skippedTestCaseCount": 0,
    "duration": 330,
    "startTime": "2025-12-07T10:24:30Z",
    "endTime": "2025-12-07T10:30:00Z",
    "target": "Production"
  },
  "testSuiteId": null,
  "passToFail": [
    {
      "id": 123,
      "testCaseId": 456,
      "title": "Login Flow",
      "result": "Failed",
      "url": "https://app.shiplight.ai/test-case-results/123"
    }
  ],
  "failToPass": [],
  "failedTestCases": [
    {
      "id": 123,
      "testCaseId": 456,
      "title": "Login Flow",
      "result": "Failed",
      "url": "https://app.shiplight.ai/test-case-results/123"
    },
    {
      "id": 124,
      "testCaseId": 457,
      "title": "Checkout Process",
      "result": "Failed",
      "url": "https://app.shiplight.ai/test-case-results/124"
    }
  ],
  "flakyTests": []
}
```

---

## Field Descriptions

### Root Level Fields

| Field             | Type           | Description                                                               |
| ----------------- | -------------- | ------------------------------------------------------------------------- |
| `testRun`         | object         | Test run information (see testRun Object below)                           |
| `testSuiteId`     | number \| null | Test suite ID (only present for test suite runs, null for test plan runs) |
| `passToFail`      | array          | Test cases that changed from passing to failing (regressions)             |
| `failToPass`      | array          | Test cases that changed from failing to passing (fixes)                   |
| `failedTestCases` | array          | All failed test cases                                                     |
| `flakyTests`      | array          | Flaky test cases                                                          |

### testRun Object

| Field                  | Type           | Description                                                                                          |
| ---------------------- | -------------- | ---------------------------------------------------------------------------------------------------- |
| `id`                   | number         | Test run ID                                                                                          |
| `testPlanId`           | number \| null | Test plan ID (only present for test plan runs, null for test suite runs)                             |
| `organizationId`       | string         | Organization ID                                                                                      |
| `status`               | string         | Run status (typically "Finished")                                                                    |
| `result`               | string         | Test result: "Passed" or "Failed"                                                                    |
| `trigger`              | string         | Trigger type: "Scheduled" (for test plans), "GITHUB_ACTION" (for test suites), "Manual", "API", etc. |
| `totalTestCaseCount`   | number         | Total test case count                                                                                |
| `passedTestCaseCount`  | number         | Number of passed test cases                                                                          |
| `failedTestCaseCount`  | number         | Number of failed test cases                                                                          |
| `skippedTestCaseCount` | number         | Number of skipped test cases                                                                         |
| `duration`             | number         | Run duration (seconds)                                                                               |
| `startTime`            | string         | Start time (ISO 8601)                                                                                |
| `endTime`              | string         | End time (ISO 8601)                                                                                  |
| `target`               | string         | Target environment (optional)                                                                        |

### Test Case Arrays

Each test case array (`passToFail`, `failToPass`, `failedTestCases`, `flakyTests`) contains objects with:

| Field        | Type   | Description                            |
| ------------ | ------ | -------------------------------------- |
| `id`         | number | Test case result ID                    |
| `testCaseId` | number | Test case ID                           |
| `title`      | string | Test case title                        |
| `result`     | string | Test result                            |
| `url`        | string | Test result detail page URL (optional) |

### Distinguishing Test Plan vs Test Suite Webhooks

To determine whether a webhook is from a test plan or test suite run:

```javascript
// Check if it's a test plan run
if (payload.testRun.testPlanId !== null && payload.testSuiteId === null) {
  console.log("This is a test plan run");
  console.log("Test Plan ID:", payload.testRun.testPlanId);
  console.log("Trigger:", payload.testRun.trigger); // Usually "Scheduled"
}

// Check if it's a test suite run
if (payload.testSuiteId !== null && payload.testRun.testPlanId === null) {
  console.log("This is a test suite run");
  console.log("Test Suite ID:", payload.testSuiteId);
  console.log("Trigger:", payload.testRun.trigger); // Usually "GITHUB_ACTION"
}
```

---

## Signature Verification

**Important**: You must verify the request signature to ensure the request actually comes from Shiplight.

### Signature Algorithm

```
1. Get X-Webhook-Timestamp header
2. Get request body (raw JSON string)
3. Concatenate: data = "{timestamp}.{body}"
4. Sign data with HMAC-SHA256 using your secret
5. Compare computed signature with X-Webhook-Signature header
```

### Node.js Example

```javascript
const crypto = require("crypto");
const express = require("express");
const app = express();

// Use express.text() instead of express.json() to get raw body
app.post("/webhooks/shiplight", express.text({ type: "application/json" }), (req, res) => {
  const signature = req.headers["x-webhook-signature"];
  const timestamp = req.headers["x-webhook-timestamp"];
  const secret = process.env.SHIPLIGHT_WEBHOOK_SECRET; // Read secret from environment variable

  // 1. Verify signature
  const data = `${timestamp}.${req.body}`;
  const hmac = crypto.createHmac("sha256", secret);
  hmac.update(data);
  const expectedSig = `sha256=${hmac.digest("hex")}`;

  if (signature !== expectedSig) {
    console.error("Invalid signature");
    return res.status(401).json({ error: "Invalid signature" });
  }

  // 2. Verify timestamp (prevent replay attacks)
  const now = Math.floor(Date.now() / 1000);
  const requestTime = parseInt(timestamp);
  if (Math.abs(now - requestTime) > 300) {
    // 5 minute tolerance
    console.error("Request too old");
    return res.status(401).json({ error: "Request too old" });
  }

  // 3. Parse and process data
  const payload = JSON.parse(req.body);
  const { testRun, failedTestCases, passToFail } = payload;

  console.log(`Test run ${testRun.id} finished with result: ${testRun.result}`);
  console.log(`Failed tests: ${failedTestCases.length}`);
  console.log(`Regressions: ${passToFail.length}`);

  // Your business logic
  // ...

  // 4. Return success response
  res.status(200).json({ received: true });
});

app.listen(3000, () => {
  console.log("Webhook server listening on port 3000");
});
```

### Python (Flask) Example

```python
import hmac
import hashlib
import time
import json
from flask import Flask, request, jsonify
import os

app = Flask(__name__)

@app.route('/webhooks/shiplight', methods=['POST'])
def handle_webhook():
    signature = request.headers.get('X-Webhook-Signature')
    timestamp = request.headers.get('X-Webhook-Timestamp')
    secret = os.environ['SHIPLIGHT_WEBHOOK_SECRET']

    # 1. Verify signature
    body = request.get_data(as_text=True)
    data = f"{timestamp}.{body}"
    expected_sig = 'sha256=' + hmac.new(
        secret.encode('utf-8'),
        data.encode('utf-8'),
        hashlib.sha256
    ).hexdigest()

    if not hmac.compare_digest(signature, expected_sig):
        return jsonify({'error': 'Invalid signature'}), 401

    # 2. Verify timestamp
    now = int(time.time())
    if abs(now - int(timestamp)) > 300:
        return jsonify({'error': 'Request too old'}), 401

    # 3. Parse and process data
    payload = json.loads(body)
    test_run = payload['testRun']
    failed_tests = payload['failedTestCases']
    regressions = payload['passToFail']

    print(f"Test run {test_run['id']} finished: {test_run['result']}")
    print(f"Failed tests: {len(failed_tests)}")
    print(f"Regressions: {len(regressions)}")

    # Your business logic
    # ...

    # 4. Return success response
    return jsonify({'received': True}), 200

if __name__ == '__main__':
    app.run(port=3000)
```

### Go Example

```go
package main

import (
    "crypto/hmac"
    "crypto/sha256"
    "encoding/hex"
    "encoding/json"
    "fmt"
    "io"
    "log"
    "math"
    "net/http"
    "os"
    "strconv"
    "time"
)

type WebhookPayload struct {
    TestRun         TestRun         `json:"testRun"`
    PassToFail      []TestCase      `json:"passToFail"`
    FailToPass      []TestCase      `json:"failToPass"`
    FailedTestCases []TestCase      `json:"failedTestCases"`
    FlakyTests      []TestCase      `json:"flakyTests"`
}

type TestRun struct {
    ID                   int    `json:"id"`
    TestPlanID           int    `json:"testPlanId"`
    OrganizationID       string `json:"organizationId"`
    Status               string `json:"status"`
    Result               string `json:"result"`
    Trigger              string `json:"trigger"`
    TotalTestCaseCount   int    `json:"totalTestCaseCount"`
    PassedTestCaseCount  int    `json:"passedTestCaseCount"`
    FailedTestCaseCount  int    `json:"failedTestCaseCount"`
    SkippedTestCaseCount int    `json:"skippedTestCaseCount"`
    Duration             int    `json:"duration"`
    StartTime            string `json:"startTime"`
    EndTime              string `json:"endTime"`
    Target               string `json:"target"`
}

type TestCase struct {
    ID         int    `json:"id"`
    TestCaseID int    `json:"testCaseId"`
    Title      string `json:"title"`
    Result     string `json:"result"`
    URL        string `json:"url"`
}

func handleWebhook(w http.ResponseWriter, r *http.Request) {
    signature := r.Header.Get("X-Webhook-Signature")
    timestamp := r.Header.Get("X-Webhook-Timestamp")
    secret := os.Getenv("SHIPLIGHT_WEBHOOK_SECRET")

    // 1. Read body
    body, err := io.ReadAll(r.Body)
    if err != nil {
        http.Error(w, "Failed to read body", http.StatusBadRequest)
        return
    }

    // 2. Verify signature
    data := timestamp + "." + string(body)
    h := hmac.New(sha256.New, []byte(secret))
    h.Write([]byte(data))
    expectedSig := "sha256=" + hex.EncodeToString(h.Sum(nil))

    if !hmac.Equal([]byte(signature), []byte(expectedSig)) {
        http.Error(w, "Invalid signature", http.StatusUnauthorized)
        return
    }

    // 3. Verify timestamp
    requestTime, err := strconv.ParseInt(timestamp, 10, 64)
    if err != nil {
        http.Error(w, "Invalid timestamp", http.StatusBadRequest)
        return
    }
    now := time.Now().Unix()
    if math.Abs(float64(now-requestTime)) > 300 {
        http.Error(w, "Request too old", http.StatusUnauthorized)
        return
    }

    // 4. Parse payload
    var payload WebhookPayload
    if err := json.Unmarshal(body, &payload); err != nil {
        http.Error(w, "Invalid JSON", http.StatusBadRequest)
        return
    }

    // 5. Process data
    log.Printf("Test run %d finished: %s\n", payload.TestRun.ID, payload.TestRun.Result)
    log.Printf("Failed tests: %d\n", len(payload.FailedTestCases))
    log.Printf("Regressions: %d\n", len(payload.PassToFail))

    // Your business logic
    // ...

    // 6. Return success response
    w.WriteHeader(http.StatusOK)
    json.NewEncoder(w).Encode(map[string]bool{"received": true})
}

func main() {
    http.HandleFunc("/webhooks/shiplight", handleWebhook)
    log.Println("Webhook server listening on :3000")
    log.Fatal(http.ListenAndServe(":3000", nil))
}
```

---

## Testing Webhooks

### Testing with webhook.site

1. Visit [webhook.site](https://webhook.site)
2. Copy the unique URL displayed on the page
3. Create a webhook endpoint in Shiplight using the copied URL
4. Run a test
5. View the received request on the webhook.site page

### Local Testing (using ngrok)

```bash
# 1. Start local server
node your-webhook-server.js

# 2. Expose local port with ngrok
ngrok http 3000

# 3. Configure webhook with the HTTPS URL provided by ngrok
```

---

## Best Practices

### 1. Security

- ✅ **Always verify signature**: Prevent forged requests
- ✅ **Verify timestamp**: Prevent replay attacks (recommend 5-minute tolerance)
- ✅ **Use HTTPS**: Protect data transmission
- ✅ **Protect Secret**: Store in environment variables, never hardcode

### 2. Performance

- ✅ **Quick response**: Return 200 response within 30 seconds
- ✅ **Async processing**: Put time-consuming operations in background queue
- ✅ **Idempotency**: Same webhook may be sent multiple times, ensure duplicate processing doesn't cause issues

### 3. Error Handling

- ✅ **Return correct status codes**:
  - `200`: Successfully received
  - `401`: Signature verification failed
  - `500`: Server error
- ✅ **Log errors**: Facilitate troubleshooting
- ❌ **No retries**: Shiplight does not automatically retry failed webhooks (Phase 1)

### 4. Monitoring

Recommended monitoring metrics:

- Webhook reception success rate
- Processing time
- Error rate
- Signature verification failure count

---

## FAQ

### Q: Which HTTP methods are supported?

A: Only POST is supported.

### Q: What is the timeout?

A: 30 seconds. Please ensure your server returns a response within 30 seconds.

### Q: Are failed webhooks retried?

A: The current version (Phase 1) does not support automatic retries.

### Q: Can I customize the payload format?

A: The current version uses a fixed format. Future versions may support custom templates.

### Q: Can I configure multiple webhook endpoints?

A: Yes. You can create multiple endpoints, and each test plan/test suite can subscribe to multiple endpoints.

### Q: Can I update the Secret?

A: Yes. Click "Edit" in Settings/Notifications/Webhooks to update the secret. Note: After updating, you need to synchronize the secret on your server.

### Q: How do I test during local development?

A: Use ngrok or similar tools to expose your local service as an HTTPS address.

### Q: Why does signature verification always fail?

A: Common reasons:

1. Using the wrong secret
2. Body was modified (make sure to use the raw body string, don't parse then stringify)
3. Timestamp exceeds 5-minute tolerance

---

## Support

If you have questions, please contact technical support or check the API Documentation.