Skip to content

Implement status check violation detection#7

Closed
dimeloper wants to merge 13 commits intowarestack:mainfrom
dimeloper:feat/issue-4-status-check-violations
Closed

Implement status check violation detection#7
dimeloper wants to merge 13 commits intowarestack:mainfrom
dimeloper:feat/issue-4-status-check-violations

Conversation

@dimeloper
Copy link
Contributor

feat: Implement comprehensive status check violation detection

Overview

Resolves the issue #4 where pull requests with failing status checks (such as required CI jobs) were not surfacing violations in Watchflow, even when corresponding rules were present and enabled in .watchflow/rules.yaml.

Changes

Core Infrastructure

  • Added STATUS event type to EventType enum for handling GitHub status events
  • Implemented StatusProcessor with agentic rule evaluation for GitHub status events
  • Created StatusEventHandler for webhook routing with factory registration
  • Enhanced event processing pipeline to support status check workflows

GitHub API Integration

  • Enhanced GitHub API client with get_pr_checks() method supporting both:
    • Modern Check Runs API (GitHub Apps)
    • Legacy Status API (older integrations)
  • Unified data format that normalizes both check types for consistent rule evaluation
  • Upgraded PullRequestProcessor to fetch and include status check data in rule evaluation

Rule Validation System

  • Implemented RequiredChecksValidator with comprehensive support for:
    • Check run API data (newer GitHub Apps)
    • Legacy status API data (commit statuses)
    • Mixed environments with both check types
    • Missing check scenarios and error handling
    • Different check conclusion/status combinations

Documentation & Testing

  • Added comprehensive test suite with 6 test cases covering all validator scenarios
  • Updated getting-started documentation with status check rule examples
  • Enhanced development guide with pre-commit hooks setup instructions
  • Created working examples for common CI/CD scenarios

Key Features

  • Dual API Support: Works with both modern Check Runs API and legacy Status API
  • Flexible Rule Configuration: Supports exact check names and pattern matching
  • Comprehensive Error Handling: Graceful fallbacks when API calls fail
  • Agentic Evaluation: Uses RuleEngineAgent for intelligent rule processing
  • Backward Compatible: Doesn't break existing functionality

Usage Example

Users can now create rules like this in .watchflow/rules.yaml:

rules:
  - name: "Require CI Tests"
    description: "All CI tests must pass before merging"
    event_types: ["pull_request"]
    conditions:
      required_checks:
        - "ci/test"
        - "build"
        - "lint"

  - name: "Security Checks Required"
    description: "Security scans must pass"
    event_types: ["pull_request"]
    conditions:
      required_checks:
        - "security/scan"
        - "vulnerability-check"

  - name: "Code Quality Gates"
    description: "Code quality checks must pass"
    event_types: ["pull_request"]
    conditions:
      required_checks:
        - "codecov/patch"
        - "codeclimate"
        - "sonarqube"

Deployment Notes

This is a backward-compatible feature addition. Existing rules and functionality remain unchanged. New status check rules can be added to repositories without affecting existing workflows.

@watchflow
Copy link

watchflow bot commented Jul 27, 2025

Watchflow rules file not found

The file .watchflow/rules.yaml is missing from your repository.

How to fix:

  1. Create a file at .watchflow/rules.yaml in your repository root.
  2. Add your rules in the following format:
    rules:
      - id: example-rule
        description: Example rule description
        ...
  3. Read the documentation for more details.

After adding the file, push your changes to re-run validation.

@dkargatzis dkargatzis closed this Sep 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants