Add debug mode support to test and spec schemas#132
Conversation
|
|
Co-authored-by: hawkeyexl <5209367+hawkeyexl@users.noreply.github.com>
|
@copilot also add breakpoint support for steps per the PRD. |
Co-authored-by: hawkeyexl <5209367+hawkeyexl@users.noreply.github.com>
Added an example demonstrating step breakpoint support in combination with test-level debug mode. The step schema already includes the |
WalkthroughA new optional "debug" property has been added to several JSON schemas at both the specification and test levels. This property allows boolean values or the string "stepThrough" to control debugging behavior. Multiple example objects demonstrating the use of the new property have also been added across the schemas. Changes
Sequence Diagram(s)sequenceDiagram
participant Config
participant Spec
participant Test
Config->>Spec: Pass debug setting (optional)
Spec->>Test: Pass debug setting (optional, can override)
Test->>Test: Use own debug property if present, else inherit from Spec, else Config
Test-->>User: Pause for input if debug=true or debug="stepThrough"
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~15 minutes Assessment against linked issues
Assessment against linked issues: Out-of-scope changesNo out-of-scope changes found. Possibly related issues
Poem
Note 🔌 MCP (Model Context Protocol) integration is now available in Early Access!Pro users can now connect to remote MCP servers under the Integrations page to get reviews and chat conversations that understand additional development context. ✨ Finishing Touches🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
|
📝 Documentation updates detected! New suggestion: Document debug mode support for tests and specifications |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (10)
src/schemas/schemas.json (10)
36365-36378: Duplicate of earlier spec-level debug schema — apply same default/oneOf/$defs suggestionsSame changes as Lines 24892-24905.
36980-36993: Duplicate of earlier test-level debug schema — apply same default/oneOf/$defs suggestionsSame changes as Lines 25507-25520.
46752-46797: Duplicate examples — consider adding the override scenarios here tooReplicate any new override examples consistently.
46964-47010: Duplicate spec-level examples — consider adding the override scenario here tooReplicate the spec-true/test-false example for consistency.
48621-48634: Duplicate of earlier spec-level debug schema — apply same default/oneOf/$defs suggestionsSame changes as Lines 24892-24905.
49236-49249: Duplicate of earlier test-level debug schema — apply same default/oneOf/$defs suggestionsSame changes as Lines 25507-25520.
59008-59053: Duplicate examples — ensure override and breakpoint-behavior examples are mirroredReplicate any added override examples consistently.
59220-59266: Duplicate spec-level examples — mirror the override scenarioMaintain consistency across example blocks.
64441-64454: Duplicate of earlier test-level debug schema — apply same default/oneOf/$defs suggestionsSame changes as Lines 25507-25520.
74213-74258: Duplicate examples — mirror any added override and breakpoint-behavior examplesKeep examples consistent across sections.
🧹 Nitpick comments (15)
src/schemas/src_schemas/spec_v3.schema.json (1)
38-49: Clarify precedence and add an explicit default for debug
- Clarify that spec-level debug can be overridden by test-level debug.
- Set an explicit default to encode the “false by default” behavior.
"debug": { - "description": "Enable debugging mode for this specification. Overrides the debug setting from config level. `true` allows pausing on breakpoints, waiting for user input before continuing. `stepThrough` pauses at every step, waiting for user input before continuing. `false` disables all debugging for this specification.", + "description": "Enable debugging mode for this specification. Overrides the debug setting from config level and can be overridden by the test-level debug setting. `true` allows pausing on breakpoints, waiting for user input before continuing. `stepThrough` pauses at every step, waiting for user input before continuing. `false` disables all debugging for this specification.", + "default": false, "anyOf": [ { "type": "boolean" }, { "type": "string", "enum": ["stepThrough"] } ] },src/schemas/src_schemas/test_v3.schema.json (1)
46-57: Add explicit default and keep precedence wording tight
- Add a default to encode “false by default”.
- Wording tweak to mirror the hierarchy: test overrides spec and config.
"debug": { - "description": "Enable debugging mode for this test. Overrides the debug setting from config and spec levels. `true` allows pausing on breakpoints, waiting for user input before continuing. `stepThrough` pauses at every step, waiting for user input before continuing. `false` disables all debugging for this test.", + "description": "Enable debugging mode for this test. Overrides the debug setting from spec and config levels. `true` allows pausing on breakpoints, waiting for user input before continuing. `stepThrough` pauses at every step, waiting for user input before continuing. `false` disables all debugging for this test.", + "default": false, "anyOf": [ { "type": "boolean" }, { "type": "string", "enum": ["stepThrough"] } ] },Optional follow-up: To avoid duplication across schemas, consider defining a shared
#/components/schemas/debugin a common schema and referencing it here and inspec_v3.schema.json.src/schemas/output_schemas/test_v3.schema.json (2)
596-609: Optional: useoneOfinstead ofanyOffor mutually exclusive branches.Boolean and "stepThrough" are disjoint;
oneOfcan yield clearer validation errors. If you prefer to keep consistency with existinganyOfusage across schemas, feel free to skip.- "anyOf": [ + "oneOf": [ { "type": "boolean" }, { "type": "string", "enum": [ "stepThrough" ] } ]
10367-10413: Examples cover all modes; consider adding a "breakpoint ignored when debug=false" case.To make override behavior explicit for users, add an example where
debugisfalsebut a step hasbreakpoint: true(no pause should occur)."debug": "stepThrough", "steps": [ { "checkLink": "https://api.example.com", "breakpoint": true } ] }, + { + "testId": "debug-test-ignored-breakpoint", + "description": "Even with a step breakpoint, debug=false means no pause", + "debug": false, + "steps": [ + { + "goTo": "https://www.example.com", + "breakpoint": true + } + ] + }src/schemas/output_schemas/resolvedTests_v3.schema.json (3)
11423-11436: Add an explicit default and consider centralizing the type.
- Add "default": false to reflect the documented default behavior and help tooling that honors schema defaults.
- Optional: extract this union type into $defs (e.g., "#/$defs/DebugSetting") and $ref it here and at other occurrences to avoid divergence.
Apply within this hunk:
"debug": { "description": "Enable debugging mode for this specification. Overrides the debug setting from config level. `true` allows pausing on breakpoints, waiting for user input before continuing. `stepThrough` pauses at every step, waiting for user input before continuing. `false` disables all debugging for this specification.", "anyOf": [ { "type": "boolean" }, { "type": "string", "enum": [ "stepThrough" ] } - ] + ], + "default": false },If you centralize the type, define once:
"$defs": { "DebugSetting": { "anyOf": [ { "type": "boolean" }, { "type": "string", "enum": ["stepThrough"] } ], "default": false } }Then replace the inline definition with: { "$ref": "#/$defs/DebugSetting", "description": "..." }.
12038-12051: Mirror default and reuse the shared definition at test level.
- Add "default": false here as well for consistency with the stated default.
- If you extract a shared "#/$defs/DebugSetting", $ref it here to eliminate duplication.
Apply within this hunk:
"debug": { "description": "Enable debugging mode for this test. Overrides the debug setting from config and spec levels. `true` allows pausing on breakpoints, waiting for user input before continuing. `stepThrough` pauses at every step, waiting for user input before continuing. `false` disables all debugging for this test.", "anyOf": [ { "type": "boolean" }, { "type": "string", "enum": [ "stepThrough" ] } - ] + ], + "default": false },
22022-22068: Spec-level examples look good; consider adding an override interplay example.
- Optional: include one spec example where a nested test overrides the spec-level debug (e.g., spec debug: true, one test debug: false) to demonstrate precedence.
- If your convention is to include $schema in example objects for linting (as used elsewhere in this repo), consider adding it here too.
src/schemas/output_schemas/report_v3.schema.json (3)
614-627: Spec-level debug schema looks correct; consider default and examples for clarityThe anyOf(Boolean | "stepThrough") is right and matches the PR semantics. Two small improvements:
- Add a default false to document the implicit default.
- Add examples to aid tooling and docs.
Apply:
"debug": { - "description": "Enable debugging mode for this specification. Overrides the debug setting from config level. `true` allows pausing on breakpoints, waiting for user input before continuing. `stepThrough` pauses at every step, waiting for user input before continuing. `false` disables all debugging for this specification.", + "description": "Enable debugging mode for this specification. Overrides the debug setting from config level. `true` allows pausing on breakpoints, waiting for user input before continuing. `stepThrough` pauses at every step, waiting for user input before continuing. `false` disables all debugging for this specification.", + "default": false, "anyOf": [ { "type": "boolean" }, { "type": "string", "enum": [ "stepThrough" ] } - ] + ], + "examples": [false, true, "stepThrough"] },
1229-1242: Test-level debug schema is on point; mirror spec improvementsSame suggestion here to document the implicit default and provide examples for consistency with spec-level.
"debug": { - "description": "Enable debugging mode for this test. Overrides the debug setting from config and spec levels. `true` allows pausing on breakpoints, waiting for user input before continuing. `stepThrough` pauses at every step, waiting for user input before continuing. `false` disables all debugging for this test.", + "description": "Enable debugging mode for this test. Overrides the debug setting from config and spec levels. `true` allows pausing on breakpoints, waiting for user input before continuing. `stepThrough` pauses at every step, waiting for user input before continuing. `false` disables all debugging for this test.", + "default": false, "anyOf": [ { "type": "boolean" }, { "type": "string", "enum": [ "stepThrough" ] } - ] + ], + "examples": [false, true, "stepThrough"] },
11212-11226: Spec-level debug examples are solid; add one to show test-level override precedenceTo illustrate the hierarchy (spec → test), add an example where a spec sets debug true but a nested test overrides it to false. This will make the precedence explicit.
Add alongside the other spec examples:
{ "specId": "debug-spec-with-override", "description": "Spec debug enabled; nested test overrides to disable", "debug": true, "tests": [ { "debug": false, "steps": [ { "checkLink": "https://www.google.com" } ] } ] }Also applies to: 11228-11245, 11247-11259
src/schemas/output_schemas/spec_v3.schema.json (4)
596-609: Spec-level debug: add an explicit default and tighten the schemaLooks good. Two small tweaks to match the documented default and avoid ambiguity:
- Add default: false.
- Optionally switch anyOf → oneOf (no overlap anyway).
Apply:
"debug": { "description": "Enable debugging mode for this specification. Overrides the debug setting from config level. `true` allows pausing on breakpoints, waiting for user input before continuing. `stepThrough` pauses at every step, waiting for user input before continuing. `false` disables all debugging for this specification.", - "anyOf": [ + "default": false, + "oneOf": [ { "type": "boolean" }, { "type": "string", "enum": [ "stepThrough" ] } ] },
1211-1224: Test-level debug: mirror the default and (optionally) oneOfSame suggestion here to align with the described default and make the validation intent explicit.
"debug": { "description": "Enable debugging mode for this test. Overrides the debug setting from config and spec levels. `true` allows pausing on breakpoints, waiting for user input before continuing. `stepThrough` pauses at every step, waiting for user input before continuing. `false` disables all debugging for this test.", - "anyOf": [ + "default": false, + "oneOf": [ { "type": "boolean" }, { "type": "string", "enum": [ "stepThrough" ] } ] },
10822-11028: Examples (test-level debug): add one example highlighting breakpoint vs stepThroughThe new examples are clear. To illustrate breakpoint behavior when debug=true (contrasted with stepThrough), consider toggling a breakpoint in the basic case.
{ "testId": "debug-test-basic", "description": "Test with debug mode enabled", "debug": true, "steps": [ { - "checkLink": "https://www.google.com" + "checkLink": "https://www.google.com", + "breakpoint": true } ] },
11194-11241: Examples (spec-level debug): show precedence by overriding at the testGreat set. Add one case that demonstrates precedence (spec debug enabled, single test disables it).
{ "specId": "debug-spec-basic", "description": "Specification with debug mode enabled", "debug": true, "tests": [ { + "debug": false, "steps": [ { "checkLink": "https://www.google.com" } ] } ] },src/schemas/schemas.json (1)
35491-35537: Spec-level examples: add a case showing test-level override of spec debugConsider including a spec with "debug": true that contains a test with "debug": false to illustrate override clearly.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (4)
dist/schemas/report_v3.schema.jsonis excluded by!**/dist/**dist/schemas/resolvedTests_v3.schema.jsonis excluded by!**/dist/**dist/schemas/spec_v3.schema.jsonis excluded by!**/dist/**dist/schemas/test_v3.schema.jsonis excluded by!**/dist/**
📒 Files selected for processing (7)
src/schemas/output_schemas/report_v3.schema.json(4 hunks)src/schemas/output_schemas/resolvedTests_v3.schema.json(4 hunks)src/schemas/output_schemas/spec_v3.schema.json(4 hunks)src/schemas/output_schemas/test_v3.schema.json(2 hunks)src/schemas/schemas.json(14 hunks)src/schemas/src_schemas/spec_v3.schema.json(2 hunks)src/schemas/src_schemas/test_v3.schema.json(2 hunks)
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: hawkeyexl
PR: doc-detective/common#117
File: src/schemas/output_schemas/resolvedTests_v3.schema.json:24-30
Timestamp: 2025-05-29T16:54:47.192Z
Learning: In doc-detective/common schemas, `$schema` properties within object definitions are intentionally added to allow described JSON objects to be self-describing for linting purposes. This enables JSON linters and validators to automatically validate objects against the referenced schema. The version information is embedded in the filename using the pattern `<schema>_v3.schema.json`.
📚 Learning: 2025-05-29T16:54:47.192Z
Learnt from: hawkeyexl
PR: doc-detective/common#117
File: src/schemas/output_schemas/resolvedTests_v3.schema.json:24-30
Timestamp: 2025-05-29T16:54:47.192Z
Learning: In doc-detective/common schemas, `$schema` properties within object definitions are intentionally added to allow described JSON objects to be self-describing for linting purposes. This enables JSON linters and validators to automatically validate objects against the referenced schema. The version information is embedded in the filename using the pattern `<schema>_v3.schema.json`.
Applied to files:
src/schemas/src_schemas/spec_v3.schema.jsonsrc/schemas/output_schemas/test_v3.schema.jsonsrc/schemas/output_schemas/report_v3.schema.jsonsrc/schemas/src_schemas/test_v3.schema.jsonsrc/schemas/output_schemas/spec_v3.schema.jsonsrc/schemas/schemas.jsonsrc/schemas/output_schemas/resolvedTests_v3.schema.json
🔇 Additional comments (9)
src/schemas/src_schemas/spec_v3.schema.json (1)
222-269: No changes needed: shorthand string notation is supported
Thestep_v3.schema.jsonexplicitly allows both string shorthand and object forms forcheckLinkandgoTo(see its own examples at lines 316–326 and 339–345). The examples inspec_v3.schema.jsontherefore validate correctly—no update is required.Likely an incorrect or invalid review comment.
src/schemas/src_schemas/test_v3.schema.json (1)
279-324: Step-level breakpoints and shorthand examples are valid
Both thebreakpointproperty and string shorthand forcheckLink/goToare defined and demonstrated instep_v3.schema.json. No changes needed.src/schemas/output_schemas/test_v3.schema.json (2)
596-609: Test-leveldebugproperty is well-scoped and matches the precedence model.Accepts boolean or "stepThrough"; leaving default undefined is correct to avoid clobbering config/spec values. Reads cleanly and aligns with step-level
breakpoint.
596-609: debug property is consistent across all v3 schemasI’ve verified that in
config_v3.schema.json,spec_v3.schema.json,test_v3.schema.json,report_v3.schema.json, andresolvedTests_v3.schema.json, thedebugfield is defined as:"debug": { "description": "Enable debugging mode…", "anyOf": [ { "type": "boolean" }, { "type": "string", "enum": ["stepThrough"] } ] }All descriptions and allowed values match exactly—no further changes required.
src/schemas/output_schemas/resolvedTests_v3.schema.json (2)
21810-21855: Examples are clear and cover all debug modes.Good spread across true, "stepThrough", and false; this will help consumers understand expected shapes.
21846-21853: Breakpoint support confirmed
Verified that thebreakpointproperty is explicitly defined on step objects in all relevant output schemas (e.g.step_v3.schema.json,resolvedTests_v3.schema.json,spec_v3.schema.json, etc.), so the example will validate correctly.• Optional: since
"stepThrough"already pauses every step, you may simplify the demo by usingdebug: truewith a single step breakpoint to highlight both debugging and breakpoints without redundancy.src/schemas/output_schemas/report_v3.schema.json (2)
11000-11046: Good examples covering test-level debug; coverage looks balancedExamples demonstrate true, "stepThrough", and false, plus a breakpoint example elsewhere. Looks good.
614-627: Debug property semantics are consistent across v3 schemas
All checked files—spec_v3.schema.json,test_v3.schema.json,resolvedTests_v3.schema.json, andreport_v3.schema.json—define thedebugproperty as:
anyOf:
–{ "type": "boolean" }
–{ "type": "string", "enum": ["stepThrough"] }No additional values are accepted or used in examples. No further changes needed.
src/schemas/output_schemas/spec_v3.schema.json (1)
596-609: All v3 schemas use the samedebugunion type and semanticsA full grep across both input (
src_schemas) and output (output_schemas) versions ofspec_v3.schema.json,test_v3.schema.json,report_v3.schema.json, andresolvedTests_v3.schema.jsonshows everydebugdefinition uses:
- an
anyOfwith
•{ "type": "boolean" }
•{ "type": "string", "enum": ["stepThrough"] }- identical descriptions for pause and step-through behavior
No discrepancies were found.
| "debug": { | ||
| "description": "Enable debugging mode for this specification. Overrides the debug setting from config level. `true` allows pausing on breakpoints, waiting for user input before continuing. `stepThrough` pauses at every step, waiting for user input before continuing. `false` disables all debugging for this specification.", | ||
| "anyOf": [ | ||
| { | ||
| "type": "boolean" | ||
| }, | ||
| { | ||
| "type": "string", | ||
| "enum": [ | ||
| "stepThrough" | ||
| ] | ||
| } | ||
| ] | ||
| }, |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Set default=false and prefer oneOf over anyOf; minor wording nit
- Encode the default in the schema.
- Use oneOf (types are disjoint) to reflect exclusive alternatives.
- Minor copy edit for clarity.
Proposed diff:
- "debug": {
- "description": "Enable debugging mode for this specification. Overrides the debug setting from config level. `true` allows pausing on breakpoints, waiting for user input before continuing. `stepThrough` pauses at every step, waiting for user input before continuing. `false` disables all debugging for this specification.",
- "anyOf": [
+ "debug": {
+ "description": "Enable debugging mode for this specification. Overrides the debug setting from the config level. `true` allows pausing on breakpoints, waiting for user input before continuing. `stepThrough` pauses at every step, waiting for user input before continuing. `false` disables all debugging for this specification.",
+ "default": false,
+ "oneOf": [
{
"type": "boolean"
},
{
"type": "string",
"enum": [
"stepThrough"
]
}
- ]
+ ]
},Optional: Factor this definition into $defs and reference it everywhere to avoid duplication:
{
"$defs": {
"debugSetting": {
"description": "Enable debugging mode. `true` pauses on breakpoints; `stepThrough` pauses at every step; `false` disables debugging.",
"default": false,
"oneOf": [
{ "type": "boolean" },
{ "type": "string", "const": "stepThrough" }
]
}
}
}Then replace the property with:
- "debug": { ...existing schema... }
+ "debug": { "$ref": "#/$defs/debugSetting" }🤖 Prompt for AI Agents
In src/schemas/schemas.json around lines 24892 to 24905, the schema should
explicitly set "default": false and replace "anyOf" with "oneOf" to indicate
mutually exclusive types. Refactor the debug setting definition into a reusable
$defs entry with the described structure, then reference this $defs definition
wherever the debug setting is used to avoid duplication and improve clarity.
| "debug": { | ||
| "description": "Enable debugging mode for this test. Overrides the debug setting from config and spec levels. `true` allows pausing on breakpoints, waiting for user input before continuing. `stepThrough` pauses at every step, waiting for user input before continuing. `false` disables all debugging for this test.", | ||
| "anyOf": [ | ||
| { | ||
| "type": "boolean" | ||
| }, | ||
| { | ||
| "type": "string", | ||
| "enum": [ | ||
| "stepThrough" | ||
| ] | ||
| } | ||
| ] | ||
| }, |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Mirror spec changes: default=false, oneOf, and wording
- Add "default": false.
- Prefer oneOf instead of anyOf.
- Copy edit: “from the config and spec levels.”
- "debug": {
- "description": "Enable debugging mode for this test. Overrides the debug setting from config and spec levels. `true` allows pausing on breakpoints, waiting for user input before continuing. `stepThrough` pauses at every step, waiting for user input before continuing. `false` disables all debugging for this test.",
- "anyOf": [
+ "debug": {
+ "description": "Enable debugging mode for this test. Overrides the debug setting from the config and spec levels. `true` allows pausing on breakpoints, waiting for user input before continuing. `stepThrough` pauses at every step, waiting for user input before continuing. `false` disables all debugging for this test.",
+ "default": false,
+ "oneOf": [
{ "type": "boolean" },
{ "type": "string", "enum": ["stepThrough"] }
- ]
+ ]
},Optional (if adopting $defs): use
- "debug": { ... }
+ "debug": { "$ref": "#/$defs/debugSetting" }Committable suggestion skipped: line range outside the PR's diff.
🤖 Prompt for AI Agents
In src/schemas/schemas.json around lines 25507 to 25520, update the schema by
adding "default": false where applicable, replace anyOf with oneOf to improve
validation accuracy, and revise the wording to say “from the config and spec
levels.” Additionally, if using $defs for definitions, consider adopting it for
better schema organization.
| { | ||
| "testId": "debug-test-basic", | ||
| "description": "Test with debug mode enabled", | ||
| "debug": true, | ||
| "steps": [ | ||
| { | ||
| "checkLink": "https://www.google.com" | ||
| } | ||
| ] | ||
| }, | ||
| { | ||
| "testId": "debug-test-step-through", | ||
| "description": "Test with step-through debug mode", | ||
| "debug": "stepThrough", | ||
| "steps": [ | ||
| { | ||
| "goTo": "https://www.google.com" | ||
| }, | ||
| { | ||
| "find": { | ||
| "selector": "[name='q']" | ||
| } | ||
| } | ||
| ] | ||
| }, | ||
| { | ||
| "testId": "debug-test-disabled", | ||
| "description": "Test with debug mode explicitly disabled", | ||
| "debug": false, | ||
| "steps": [ | ||
| { | ||
| "checkLink": "https://www.google.com" | ||
| } | ||
| ] | ||
| }, | ||
| { | ||
| "testId": "critical-test", | ||
| "description": "Test with debug mode and step breakpoints", | ||
| "debug": "stepThrough", | ||
| "steps": [ | ||
| { | ||
| "checkLink": "https://api.example.com", | ||
| "breakpoint": true | ||
| } | ||
| ] | ||
| } |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Examples look good; add explicit override scenarios and verify step breakpoint support
- Add an example where the spec has "debug": true but a contained test sets "debug": false to demonstrate precedence.
- Add an example where test "debug": false and a step has
"breakpoint": true(should not pause), clarifying behavior.
To confirm the example with "breakpoint": true is valid across all step types, search for the breakpoint property in step schemas:
🏁 Script executed:
#!/bin/bash
# Verify 'breakpoint' is defined in step/action schemas
rg -n --no-heading -A 3 -B 3 '"breakpoint"\s*:\s*\{' src/schemas | sed -n '1,200p'
rg -n --no-heading -A 2 -B 2 '"breakpoint"\s*:\s*true' src/schemas | sed -n '1,200p'Length of output: 32688
Add explicit override and breakpoint examples
Breakpoint support has been confirmed across all step schemas (see step_v3.schema.json and related test/spec schemas). Please update your examples accordingly:
- Include a scenario where the spec has
"debug": truebut an individual test sets"debug": falseto demonstrate that the test-level setting takes precedence. - Include a scenario where a test sets
"debug": falseand a step within that test has"breakpoint": true. Clarify that execution will not pause in this case when debug mode is disabled.
🤖 Prompt for AI Agents
In src/schemas/schemas.json between lines 35279 and 35324, update the examples
to explicitly show override and breakpoint behavior for debug settings. Add one
example where the spec-level debug is true but an individual test sets debug to
false, demonstrating the test-level debug setting takes precedence. Also add an
example where a test has debug false but a step within it has breakpoint true,
clarifying that execution does not pause since debug mode is disabled. Ensure
these examples clearly illustrate the intended behavior for debugging and
breakpoints.
This PR extends Doc Detective's debug mode functionality by adding debug configuration support at the test and specification levels, completing the debug mode hierarchy.
What's New
Test-Level Debug Control
Tests can now override debug settings from config and spec levels:
{ "testId": "critical-test", "debug": "stepThrough", "steps": [ { "checkLink": "https://api.example.com", "breakpoint": true } ] }Specification-Level Debug Control
Specifications can override global config debug settings:
{ "specId": "api-tests", "debug": true, "tests": [...] }Debug Mode Hierarchy
The complete debug configuration hierarchy is now:
config.debug)spec.debug)test.debug)step.breakpoint)Supported Values
All debug properties support:
false- Disable debugging (default)true- Enable debugging with breakpoint support"stepThrough"- Pause at every step waiting for user inputValidation
This provides users with granular control over debugging behavior, allowing them to enable debug mode globally while selectively disabling it for specific specs or tests, or vice versa.
Fixes #131.
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.
Summary by CodeRabbit
true), step-through debugging ("stepThrough"), or disable debugging (false).