Implement debug step-through mode for test execution

[Copilot]

This PR implements a debug step-through mode that allows users to run tests one step at a time, waiting for user input before moving to the next step. This addresses the need for better debugging capabilities during test development and troubleshooting.

Features Implemented

Step-Through Mode

• Configuration: Enable with debug: true or debug: "stepThrough" in config
• Interactive prompts: Pauses before each step with user-friendly interface
• User controls: Continue with c or quit with q
• Auto-continue: Non-interactive environments (CI/CD) continue automatically

Usage Example

const { runTests } = require("doc-detective-core");

const results = await runTests({
  input: "path/to/tests",
  debug: true  // Enables step-through mode
});

Interactive Debug Session

--- DEBUG STEP-THROUGH MODE ---
⏸️  Step-through mode: Paused before next step
Context: my-test-context
Step ID: step-1
Step Description: Click the login button
Step Action: click

Options:
  [c] Continue to next step
  [q] Quit execution
Choice: c

Technical Implementation

• New utility function: debugStepPrompt() in utils.js handles user interaction with readline
• Config parsing: Enhanced setConfig() to parse debug options while respecting schema constraints
• Step execution: Modified the execution loop in executeTestContext() to pause when debug is enabled
• Environment detection: Gracefully handles both TTY and non-TTY environments

Testing

Added comprehensive test suite (test/debug.test.js) covering:

• ✅ Config option parsing and defaults
• ✅ Boolean and string debug configurations
• ✅ Normal execution when debug is disabled
• ✅ Non-interactive environment handling
• ✅ Step-through mode functionality

Dependencies

• Requires doc-detective-common@3.1.1-dev.3 for schema support
• All existing functionality preserved - zero breaking changes

Future Enhancements

The implementation is designed to support additional debug features when schema is updated:

• Breakpoints at specific step IDs
• Auto-break on step failures
• Variable inspection capabilities

Fixes #205.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

• app.posthog.com
  • Triggering command: node /home/REDACTED/work/core/core/node_modules/.bin/mocha test/config.test.js test/core.test.js test/testREDACTED.test.js (dns block)
  • Triggering command: node /home/REDACTED/work/core/core/node_modules/.bin/mocha test/config.test.js test/core.test.js test/debug.test.js test/testREDACTED.test.js (dns block)
• googlechromelabs.github.io
  • Triggering command: node scripts/postinstall.js (dns block)
• hg.mozilla.org
  • Triggering command: node ./dist/install.js (dns block)
• product-details.mozilla.org
  • Triggering command: node scripts/postinstall.js (dns block)
• www.google.com
  • Triggering command: node /home/REDACTED/work/core/core/node_modules/.bin/mocha test/config.test.js test/core.test.js test/testREDACTED.test.js (dns block)
  • Triggering command: node /home/REDACTED/work/core/core/node_modules/.bin/mocha test/config.test.js test/core.test.js test/debug.test.js test/testREDACTED.test.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to my firewall allow list

---

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

[CLAassistant]

All committers have signed the CLA.

[hawkeyexl]

@copilot debug mode should force concurrentTests to 1 so it works sequentially.

[Copilot]

@copilot debug mode should force concurrentTests to 1 so it works sequentially.

Implemented! When debug step-through mode is enabled (debug: true or debug: "stepThrough"), concurrent runners is now automatically forced to 1 for sequential execution. An informative log message is displayed when the override occurs. See commit 28fb5c0.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

• app.posthog.com
  • Triggering command: node /home/REDACTED/work/core/core/node_modules/.bin/mocha test/config.test.js test/core.test.js test/debug.test.js test/testREDACTED.test.js (dns block)
• googlechromelabs.github.io
  • Triggering command: node scripts/postinstall.js (dns block)
• hg.mozilla.org
  • Triggering command: node ./dist/install.js (dns block)
• product-details.mozilla.org
  • Triggering command: node scripts/postinstall.js (dns block)
• www.google.com
  • Triggering command: node /home/REDACTED/work/core/core/node_modules/.bin/mocha test/config.test.js test/core.test.js test/debug.test.js test/testREDACTED.test.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to my firewall allow list

[hawkeyexl]

@copilot add auto-break on step failure.

[Copilot]

@copilot add auto-break on step failure.

Auto-break on step failure has been implemented! When debug mode is enabled (debug: true or debug: "stepThrough"), the system now automatically pauses when a step fails, allowing you to inspect what went wrong before continuing. This works alongside the existing step-through functionality for a comprehensive debugging experience. See commit b6b2f0d.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

• googlechromelabs.github.io
  • Triggering command: node scripts/postinstall.js (dns block)
• hg.mozilla.org
  • Triggering command: node ./dist/install.js (dns block)
• product-details.mozilla.org
  • Triggering command: node scripts/postinstall.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to my firewall allow list

[hawkeyexl]

@copilot What might variable inspection capabilities look like?

[Copilot]

@copilot What might variable inspection capabilities look like?

Variable inspection capabilities could include several powerful debugging features:

Current Variable Context: Display environment variables (process.env) and meta values accessible to the current step, including:

• Step outputs from previous actions (actionResult.outputs)
• Hierarchical metadata (specs/tests/contexts/steps structure)
• Resolved step variables that were set during execution

Interactive Commands: Additional debug prompt options like:

• [v] View all available variables and their values
• [e] Evaluate expressions with current context
• [s] Set/modify environment variables for testing

Step Variable Preview: Before executing a step with variables property, show what values will be resolved and assigned.

Expression Debugging: When a step uses expressions ({{...}}), show the evaluation process step-by-step.

The foundation is already there - the metaValues object tracks execution state, resolveExpression() handles variable resolution, and the debug prompt could easily be extended with new commands to inspect and manipulate this context.

This would be especially valuable when debugging complex test flows where understanding variable state is crucial for troubleshooting failures.

[hawkeyexl]

@copilot Understood. Please implement variable inspection capabilities.

[Copilot]

@copilot Understood. Please implement variable inspection capabilities.

Variable inspection capabilities have been implemented! The debug mode now includes powerful debugging features:

New Debug Commands:

• [v] View available variables - displays environment variables, meta values, and step outputs
• [e] Evaluate expression - test expressions interactively with current context
• [s] Set environment variable - modify variables during debugging

Enhanced Features:

• Step variable preview shows variables that will be set when a step defines them
• Rich context display with hierarchical metadata and step outputs
• Interactive expression evaluation using the existing resolveExpression() function
• Environment variable manipulation for testing different configurations

The implementation leverages the existing metaValues object and expression resolution system, providing comprehensive access to the execution state for effective debugging and troubleshooting. See commit 438669b.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

• app.posthog.com
  • Triggering command: node /home/REDACTED/work/core/core/node_modules/.bin/mocha test/config.test.js test/core.test.js test/debug.test.js test/testREDACTED.test.js (dns block)
• googlechromelabs.github.io
  • Triggering command: node scripts/postinstall.js (dns block)
• hg.mozilla.org
  • Triggering command: node ./dist/install.js (dns block)
• product-details.mozilla.org
  • Triggering command: node scripts/postinstall.js (dns block)
• www.google.com
  • Triggering command: node /home/REDACTED/work/core/core/node_modules/.bin/mocha test/config.test.js test/core.test.js test/debug.test.js test/testREDACTED.test.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to my firewall allow list

[hawkeyexl]

@copilot Update this section with the newer functionality.

[Copilot]

Updated the README.md debug documentation to reflect all the implemented features including auto-break on failure, variable inspection capabilities, expression evaluation, and environment variable setting. The "Future Enhancements" section now accurately shows only features that are still planned (breakpoints). See commit 7db02d6.