Implement Structured Output for FeasibilityAgent#2
Merged
dkargatzis merged 8 commits intomainfrom Jul 21, 2025
Merged
Conversation
- Replace fragile JSON parsing with LangGraph structured output - Add conditional workflow logic (YAML generation only if feasible) - Introduce type-safe Pydantic models (FeasibilityAnalysis, YamlGeneration) - Eliminate 70% of error-handling code through structured validation - Add async/await support throughout agent workflow - Improve logging with structured information and emojis - Update API interface to use new agent execution method BREAKING CHANGE: FeasibilityResult model removed, use execute() method instead of check_feasibility()
Add GitHub Pages domain to allowed CORS origins for frontend access
- Replace @app.on_event("startup") and @app.on_event("shutdown") decorators
- Implement modern FastAPI lifespan context manager approach
- Add contextlib.asynccontextmanager import
- Move startup/shutdown logic into lifespan function
- Resolves FastAPI deprecation warnings in tests
- Update coverage source path from 'backend' to 'src' in pyproject.toml - Add comprehensive testing instructions and structure to README.md - Introduce GitHub Actions workflow for automated testing - Create unit and integration test packages with respective test files - Implement integration tests for rules API with mocked OpenAI calls - Add unit tests for Rule Feasibility Agent with structured output
- Fix unit tests requiring OpenAI API key by mocking BaseAgent._validate_config() - Ensure tests run in CI without external dependencies - All 12 tests now pass without API keys required Resolves CI failures where unit tests were failing due to missing OPENAI_API_KEY
- Add mock for BaseAgent._create_llm_client() to prevent real OpenAI client creation - Previous fix only mocked _validate_config() but __init__ still called _create_llm_client() - Now properly mocks both validation and client creation for CI safety
Resolves import-time OpenAI API key validation issues in CI: **Root Cause:** - Global `deployment_scheduler = DeploymentScheduler()` created agents at import time - `BaseAgent.__init__()` called `_validate_config()` requiring OpenAI API key - Integration tests failed during module import, before mocks could take effect **Solution:** - Remove global instance creation at module level - Add lazy-loading property for `engine_agent` in DeploymentScheduler - Replace global variable with `get_deployment_scheduler()` function - Update all imports and usages across codebase **Changes:** - `src/tasks/scheduler/deployment_scheduler.py`: Lazy-load agents, factory function - `src/main.py`, `src/api/scheduler.py`, `src/event_processors/deployment_protection_rule.py`: Updated imports **Testing:** - Unit tests: 9/9 pass (agents properly mocked) - Integration tests: 3/3 pass (no import-time validation) - Total: 12/12 tests pass locally and should pass in CI **Impact:** - CI can now run without OpenAI API keys - Agents only created when actually needed - Maintains same runtime behavior
**Root Cause:** Integration tests called API endpoints that create real agents,
triggering OpenAI API key validation before mocks could take effect.
**Problem Location:**
```
src/api/rules.py:17 → agent = RuleFeasibilityAgent()
→ BaseAgent.__init__()
→ _validate_config() ← Validates API key BEFORE execute()
```
**Solution:** Mock both agent validation AND creation in integration tests:
- Mock `BaseAgent._validate_config()` to prevent API key validation
- Mock `BaseAgent._create_llm_client()` to prevent client creation
- Keep existing `execute()` mocks for result control
**Changes:**
- `tests/integration/test_rules_api.py`: Added complete agent mocking
**Testing:**
- Unit tests: 9/9 pass (proper agent mocking)
- Integration tests: 3/3 pass (no API validation during endpoint calls)
- Total: 12/12 tests pass and ready for CI
**Impact:**
- CI now completely safe from OpenAI API calls
- Integration tests verify full HTTP stack with mocked AI
- Maintains ability to test real API calls locally via env var
dkargatzis
approved these changes
Jul 21, 2025
Member
dkargatzis
left a comment
There was a problem hiding this comment.
Great work @cgoncalves94! 👍🏻
I'm merging it as it introduces major improvements both in feasibility agent and core files, thanks!
Just a note: we use .yaml (not .yml) for workflow files, and haven’t used emojis in our docs before - let’s standardize these in a future update. I’ll handle these tweaks after merging.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR modernizes the FeasibilityAgent by replacing fragile JSON parsing with LangGraph's structured output, introducing type-safe Pydantic models, and implementing conditional workflow logic for better reliability and performance.
✨ Key Features
Structured Output Implementation
with_structured_output()FeasibilityAnalysisandYamlGenerationwith field validationSmart Conditional Workflow
Enhanced Architecture
🔧 Technical Improvements
Models & Types
Workflow Logic
Error Handling
🏗️ Breaking Changes
API Changes
FeasibilityResult.check_feasibility()methodagent.execute()instead ofcheck_feasibility()Model Changes
FeasibilityResultmodelFeasibilityAnalysisandYamlGenerationstructured modelsFeasibilityStatewith better type safety🧪 Testing