Shiplight AI

Appearance

Test Editor - Debugging Features ​

View as Markdown

LEGACY — Shiplight Cloud v1

This page documents Shiplight Cloud v1 (app.shiplight.ai), which is no longer the current product. For the current cloud platform, see Shiplight Cloud.

The Test Editor provides comprehensive debugging capabilities for developing, troubleshooting, and refining your automated tests.

Table of Contents ​

  1. Debug Control Bar
  2. Debug Execution Modes
  3. Live View Panel
  4. Debug Status Indicators
  5. Debugging AI-Generated Tests
  6. Error Handling and Troubleshooting
  7. Debug Logging
  8. Best Practices for Debugging
  9. Debugging Tips
  10. Troubleshooting Guide
  11. Next Steps

Debug Control Bar ​

The debug control bar appears at the top of the test editor and provides the following controls:

Primary Controls ​

Start/Step Button

  • Start - Initializes the test session and sets up the browser
  • Step - Executes the next action (appears after starting)

Run Button

  • Executes all remaining steps from the current position
  • Continues until test completion or error

Reset Button

  • Terminates the current debug session
  • Closes the browser instance
  • Returns to pre-execution state

Test Management ​

Save Button

  • Saves the current test configuration
  • Creates a checkpoint for reverting

Revert Button

  • Returns test to last saved state
  • Discards unsaved changes
  • Useful for undoing experimental changes

Debug Execution Modes ​

Step-by-Step Debugging ​

Execute tests one action at a time for detailed analysis:

  1. Click Start to initialize the test
  2. Use Step to execute each action individually
  3. Inspect results after each step
  4. Make adjustments as needed

Benefits:

  • Observe each action's effect
  • Identify exact failure points
  • Verify element interactions
  • Debug timing issues

Continuous Execution ​

Run the entire test or continue from current position:

  1. Click Run to execute all steps
  2. Test runs until completion or error
  3. Execution pauses on failures
  4. Can resume with Step or Run

Partial Execution ​

Execute specific portions of your test:

Run Until Here:

  • Click the "Run until..." icon on any step
  • Or select "Run Until This Step" from the ⋯ dropdown menu
  • Execution runs from the beginning and stops at the selected step

Advanced Debug Controls ​

Skip Step:

  • Click the "Skip" button that appears next to a pending step during debugging
  • Marks the step as skipped without executing it
  • Useful for bypassing known issues or testing specific paths
  • The step will show a skipped indicator

Rollback:

  • Click the "Rollback" button after a step has executed
  • Reverts the test execution to before that step
  • Resets the browser state to the previous checkpoint
  • Allows you to re-execute steps with different parameters
  • Particularly useful when debugging flaky tests or timing issues

When to Use Skip:

  • Temporarily bypass steps that are blocking debugging
  • Test different execution paths without modifying the test
  • Skip optional steps during development
  • Isolate specific functionality for testing

When to Use Rollback:

  • Re-execute a step that failed due to timing
  • Try different input values without restarting
  • Undo an action to explore alternative paths
  • Debug intermittent failures by repeating steps

Live View Panel ​

The right panel provides real-time visibility during debugging:

Live Tab ​

Real-time Browser View:

  • Shows actual browser window
  • Updates as test executes
  • Full-size preview available

Preview Tab ​

Screenshot Gallery:

  • Captures at key points
  • Before/after action comparisons
  • Error state screenshots
  • Labeled for easy identification

Console Tab ​

Execution Logs:

  • Step-by-step execution details
  • JavaScript console output
  • Error messages and stack traces

Log Levels:

  • Info - General execution information
  • Warning - Potential issues
  • Error - Failures and exceptions
  • Debug - Detailed diagnostic data

Context Tab ​

Test Variables:

  • Current variable values
  • Test context data
  • Environment information
  • Session state

Debug Status Indicators ​

Statement Highlighting ​

During debugging, statements are highlighted to show execution status:

Current Statement (Blue border)

  • Statement being executed
  • Execution paused here
  • Ready for next step

Successful (Green)

  • Statement executed successfully
  • No errors encountered
  • Results as expected

Failed (Red)

  • Statement execution failed
  • Error details available
  • Execution stopped

Pending (Gray)

  • Not yet executed
  • Waiting for execution
  • Part of remaining flow

Execution Details ​

Hover over executed statements to see:

  • Execution time
  • Result status
  • Error messages (if any)
  • Generated code

Debugging AI-Generated Tests ​

Generation Monitoring ​

During test generation, observe:

  • Real-time Progress - See steps as they're created
  • AI Decisions - Understand AI reasoning
  • Browser State - Track page changes

Generation Status ​

The test shows "Generating" status with:

  • Progress indicators
  • Current action being generated
  • Completion estimates
  • Ability to stop generation

Error Handling and Troubleshooting ​

Common Debugging Scenarios ​

Element Not Found:

  • Pause before the failing step
  • Inspect element in live view
  • Verify selector accuracy
  • Check timing issues

Timeout Errors:

  • Add explicit waits
  • Check network delays
  • Verify page load completion
  • Review async operations

Assertion Failures:

  • Inspect actual vs expected values
  • Check timing of assertions
  • Verify data accuracy
  • Review test assumptions

Debug Information ​

Each failed step provides:

  • Error Message - Specific failure reason
  • Stack Trace - Code execution path
  • Screenshot - Visual state at failure
  • Suggested Fixes - AI-powered recommendations

Debug Logging ​

Add debug output to tests using Code blocks:

javascript
// In Code blocks
console.log("Debug:", variableName);
console.table(dataArray);
console.time("Operation");
// ... code ...
console.timeEnd("Operation");

Best Practices for Debugging ​

Systematic Approach ​

  1. Reproduce the issue consistently
  2. Isolate the problem to specific steps
  3. Gather information using debug tools
  4. Form hypothesis about the cause
  5. Test solution with minimal changes
  6. Verify fix across scenarios

Debug Session Management ​

  • Save working test state before debugging
  • Use meaningful checkpoint names
  • Clean up debug code after fixing
  • Document solutions for future reference

Debugging Tips ​

Quick Debugging ​

  • Keep console tab visible
  • Take screenshots at key points

Complex Issues ​

  • Break tests into smaller parts
  • Use Code blocks for detailed inspection
  • Compare working vs failing tests
  • Check environment differences

Performance Issues ​

  • Optimize wait strategies
  • Reduce unnecessary actions

Troubleshooting Guide ​

Test Won't Start ​

Possible Causes:

  • Environment not configured
  • Invalid test account
  • Network connectivity issues
  • Browser launch failure

Solutions:

  • Verify environment settings
  • Check test account credentials
  • Test network connection
  • Restart debug session

Steps Execute Too Quickly ​

Solutions:

  • Use Step mode instead of Run
  • Add explicit waits

Can't See Live View ​

Solutions:

  • Check panel layout
  • Ensure debugging is active
  • Verify browser connection
  • Refresh editor page

Next Steps ​

Released under the MIT License.

Copyright © 2025-2026 Shiplight AI, Inc.