Skip to content

Comments

feat: make feasibility agent choose validators from catalog#30

Merged
dkargatzis merged 1 commit intowarestack:mainfrom
naaa760:feat/feasibility-validator-selection
Dec 12, 2025
Merged

feat: make feasibility agent choose validators from catalog#30
dkargatzis merged 1 commit intowarestack:mainfrom
naaa760:feat/feasibility-validator-selection

Conversation

@naaa760
Copy link
Contributor

@naaa760 naaa760 commented Dec 12, 2025

  • Feasibility prompt now lists the validator catalog and asks the LLM to pick applicable validators (no hardcoded rules).

  • Structured output/state adds chosen_validators; AgentResult returns it for transparency.

  • YAML generation uses only the chosen validators; if none fit, it returns empty instead of inventing params.

  • Pre-commit passes; scope limited to feasibility agent prompt/state/nodes/output.

@naaa760 naaa760 requested a review from dkargatzis as a code owner December 12, 2025 15:52
@gemini-code-assist
Copy link

Summary of Changes

Hello @naaa760, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the feasibility agent's intelligence by allowing it to dynamically select appropriate validators from a catalog when analyzing rule descriptions and generating YAML configurations. This change moves away from static, hardcoded rule logic, making the agent more adaptable and robust in handling diverse rule requirements. The update also improves the transparency of the agent's decision-making by explicitly tracking the chosen validators throughout the process.

Highlights

  • Dynamic Validator Selection: The feasibility agent's prompt now includes a comprehensive validator catalog, enabling the Large Language Model (LLM) to dynamically select applicable validators instead of relying on hardcoded rules.
  • Structured Output Enhancement: The FeasibilityAnalysis and FeasibilityState models, along with the AgentResult, now include a chosen_validators field for increased transparency and structured data flow.
  • Intelligent YAML Generation: The YAML generation process has been refined to exclusively utilize the chosen_validators. If no suitable validators are identified, the system will now return an empty configuration, preventing the generation of arbitrary parameters.
  • Prompt Refinement: Both the RULE_FEASIBILITY_PROMPT and YAML_GENERATION_PROMPT have been updated to incorporate the validator catalog and guide the LLM in making informed decisions based on available validators.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request is a solid improvement, refactoring the feasibility agent to dynamically select validators from a catalog rather than relying on hardcoded rules. This change significantly enhances the agent's extensibility and maintainability. The prompt engineering has been thoughtfully updated to guide the language model in this new, more flexible workflow. The modifications to data models and agent logic are consistent and well-executed. I have one minor suggestion to improve code conciseness and readability.

Comment on lines +28 to +36
validator_catalog = []
for v in get_validator_descriptions():
validator_catalog.append(
f"- name: {v.get('name')}\n"
f" event_types: {v.get('event_types')}\n"
f" parameter_patterns: {v.get('parameter_patterns')}\n"
f" description: {v.get('description')}"
)
validators_text = "\n".join(validator_catalog)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The loop for building validators_text can be made more concise and idiomatic by using a generator expression directly within "\n".join(). This improves readability by reducing boilerplate and expressing the transformation more directly.

        validators_text = "\n".join(
            f"- name: {v.get('name')}\n"
            f"  event_types: {v.get('event_types')}\n"
            f"  parameter_patterns: {v.get('parameter_patterns')}\n"
            f"  description: {v.get('description')}"
            for v in get_validator_descriptions()
        )

@codecov-commenter
Copy link

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 30.00000% with 7 lines in your changes missing coverage. Please review.

❌ Your patch status has failed because the patch coverage (30.0%) is below the target coverage (80.0%). You can increase the patch coverage or adjust the target coverage.
❌ Your project status has failed because the head coverage (32.0%) is below the target coverage (80.0%). You can increase the head coverage or adjust the target coverage.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

@@           Coverage Diff           @@
##            main     #30     +/-   ##
=======================================
+ Coverage   24.7%   32.0%   +7.2%     
=======================================
  Files         79      85      +6     
  Lines       4497    5001    +504     
=======================================
+ Hits        1115    1604    +489     
- Misses      3382    3397     +15     

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a518bcb...ddb4dce. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@dkargatzis dkargatzis merged commit 0b55233 into warestack:main Dec 12, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants