Skip to content

Multiple changes for project improvements#144

Merged
asdek merged 44 commits intomasterfrom
feature/project_improvements
Feb 25, 2026
Merged

Multiple changes for project improvements#144
asdek merged 44 commits intomasterfrom
feature/project_improvements

Conversation

@asdek
Copy link
Contributor

@asdek asdek commented Feb 24, 2026

Problem

PentAGI needed several critical improvements across security, functionality, documentation, and code quality. Users reported issues with external network access configuration, multiple security vulnerabilities were identified in the codebase, and the platform lacked advanced exploit discovery capabilities.

Solution

This aggregating PR consolidates 30+ commits across multiple merged pull requests, addressing:

  • Security hardening: Fixed OAuth validation, resource leaks, TLS configuration, and session handling vulnerabilities
  • New features: Integrated Sploitus exploit aggregator and user preferences with favorite flows
  • Documentation: Added comprehensive external network access guide and fixed numerous typos
  • Code quality: Removed debug statements, fixed bugs, and improved error handling
  • Dependencies: Updated langchaingo to stable release version

Closes #91, addresses #101

Type of Change

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 📚 Documentation update
  • 🔧 Configuration change
  • 🧪 Test update
  • 🛡️ Security update

Areas Affected

  • Core Services (Frontend UI/Backend API)
  • AI Agents (Researcher/Developer/Executor)
  • Security Tools Integration
  • Analytics Platform (Langfuse)
  • External Integrations (LLM/Search APIs)
  • Documentation
  • Infrastructure/DevOps

Testing and Verification

Test Configuration

PentAGI Version: Latest development (feature/project_improvements)
Docker: 24.0.x+
Host OS: Linux/macOS/Windows
LLM Provider: OpenAI, Anthropic, Google Gemini, OpenRouter
Enabled Features: Core features + Langfuse + Sploitus

Test Steps

  1. Run all backend unit tests (go test ./...) - all pass ✅
  2. Test Sploitus integration with real exploit searches
  3. Verify OAuth authentication flow with state validation
  4. Test external network access with documented configuration
  5. Verify user preferences and favorite flows functionality
  6. Test Langfuse client with proper TLS configuration
  7. Validate terminal resource handling with large files
  8. Test password change functionality with strong validation

Test Results

  • ✅ All existing unit tests pass
  • ✅ New Sploitus tests added and passing
  • ✅ OAuth security vulnerabilities fixed and verified
  • ✅ Resource leaks eliminated (browser, terminal, tar operations)
  • ✅ External network access documented and verified
  • ✅ No breaking changes to existing functionality
  • ✅ Frontend linting passes
  • ✅ Backend go fmt and go vet clean

Security Considerations

Critical Fixes (Issue #101):

  1. OAuth Validation - Fixed CSRF bypass in GET callback (missing state parameter validation)
  2. OAuth State Parser - Added explicit field existence checks for 'exp' and 'provider'
  3. OAuth Callback - Added missing return statement preventing double response writes
  4. Resource Leaks - Fixed response body leak in browser scraper, added tar writer cleanup
  5. Buffer Overflow - Added 50MB file size validation in terminal tar operations to prevent OOM
  6. TLS Configuration - Langfuse client now respects EXTERNAL_SSL_INSECURE and EXTERNAL_SSL_CA_PATH
  7. Session Handling - Improved error messages and validation

New Features:

  • Sploitus Integration - No API key required, uses public endpoint, respects proxy configuration
  • User Preferences - New privilege model for favorites management

No New Attack Vectors:

  • All changes maintain or improve security posture
  • No new credentials or secrets required
  • Backward compatible with existing deployments

Performance Impact

Improvements:

  • Sploitus: Single HTTPS POST with 30s timeout, minimal overhead
  • TLS Optimization: Langfuse now uses system cert pool as base (more efficient)
  • Resource Management: Fixed leaks reduce memory pressure over time
  • Terminal Operations: File size validation prevents unbounded allocations

No Degradation:

  • OAuth validation adds negligible overhead (existing flow)
  • User preferences use standard database queries with proper indexes
  • Debug console.log removal improves browser console performance
  • All changes maintain existing performance characteristics

Documentation Updates

  • README.md updates - External network access section, Sploitus mention, Trendshift badge
  • Configuration documentation updates - SPLOITUS_ENABLED env var, OAuth examples fixed
  • GraphQL schema updates - User preferences types, favorite flows mutations
  • Code comments - Fixed typos ("Incoming", "application", "permissions")
  • Other: Fixed EULA company names, renamed misspelled files (screenshots.go, integration docs)

Deployment Notes

New Environment Variables (Optional):

# Sploitus exploit search (defaults to true)
SPLOITUS_ENABLED=true

# For external network access (see README)
PENTAGI_LISTEN_IP=0.0.0.0
PUBLIC_URL=https://your-server-ip:8443
CORS_ORIGINS=https://your-server-ip:8443

Database Migrations:

  • 20260223_120000_add_sploitus_search_type.sql - Adds 'sploitus' to SEARCHENGINE_TYPE enum
  • 20260223_220000_user_preferences.sql - User preferences table and privileges
  • Migrations run automatically on startup

Container Recreation:

  • Required if changing network binding configuration
  • Run: docker compose down && docker compose up -d --force-recreate

Compatibility:

  • ✅ Fully backward compatible
  • ✅ No breaking API changes
  • ✅ Existing deployments work without changes
  • ✅ New features are opt-in or auto-enabled safely

Checklist

Code Quality

  • My code follows the project's coding standards
  • I have added/updated necessary documentation
  • I have added tests to cover my changes
  • All new and existing tests pass
  • I have run go fmt and go vet (for Go code)
  • I have run npm run lint (for TypeScript/JavaScript code)

Security

  • I have considered security implications
  • Changes maintain or improve the security model
  • Sensitive information has been properly handled

Compatibility

  • Changes are backward compatible
  • Breaking changes are clearly marked and documented
  • Dependencies are properly updated

Documentation

  • Documentation is clear and complete
  • Comments are added for non-obvious code
  • API changes are documented

Additional Notes

Key Changes by Category

🛡️ Security Fixes (Issue #101)

OAuth Authentication Hardening (PRs #120, #125, #127):

  • Fixed CSRF bypass vulnerability in GET callback handler
  • Added comprehensive state parameter validation
  • Improved error handling in OAuth flow
  • Contributor: @mason5052

Resource Management (PR #129):

  • Fixed HTTP response body leak in browser scraper
  • Added file size validation (50MB limit) to prevent OOM attacks
  • Fixed tar writer resource leak
  • Contributors: @mason5052, @s-b-repo

TLS Configuration (PR #132):

  • Langfuse client now respects EXTERNAL_SSL_INSECURE config
  • Proper system certificate pool usage with custom CA support
  • Removed hardcoded InsecureSkipVerify: true
  • Contributor: @mason5052

🚀 New Features

Sploitus Exploit Aggregator (PR #133):

  • Integrated Sploitus for advanced exploit discovery
  • Aggregates data from Exploit-DB, Packet Storm, GitHub Advisories, VulHub, etc.
  • No API key required, uses public JSON API
  • Returns structured results with URLs, CVSS scores, CVE references
  • Contributor: @Priyanka-2725

User Preferences & Favorite Flows:

  • New GraphQL mutations for managing favorite flows
  • Privilege-based access control
  • Database migration with proper indexes
  • Contributor: @asdek

📚 Documentation

External Network Access Guide (Issue #91, PR #135):

  • Comprehensive configuration guide for remote access
  • PENTAGI_LISTEN_IP, PUBLIC_URL, CORS_ORIGINS explained
  • Firewall configuration for multiple Linux distributions
  • Troubleshooting steps and common issues
  • Contributor: @Vaibhavee89

Typo Fixes & Improvements (PR #121):

  • Fixed product name "PegtAGI" → "PentAGI"
  • Fixed company names in EULA
  • Corrected OAuth env var names in README
  • Renamed misspelled files (screenshots.go, integration docs)
  • Contributor: @mason5052

🐛 Bug Fixes

Terminal Command Handling (PR #124):

  • Fixed command execution logic in terminal tool
  • Contributor: @Priyanka-2725

Password Handling:

  • Fixed database column mapping for UserPassword
  • Improved error handling in development mode
  • Enhanced form validation with visibility toggle
  • Contributor: @asdek

Frontend Cleanup (PR #141):

  • Removed 7 debug console.log statements
  • Contributor: @mason5052

Minor Fixes:

📦 Dependencies

langchaingo Update:

  • Updated to stable release v0.1.14-update.1
  • Contributor: @asdek

🎨 Miscellaneous

  • Added Trendshift badge to README
  • Updated Gemini testing report with new model versions
  • Streamlined TLS configuration in Langfuse client

Contributors

This release includes contributions from:

  • @asdek (Dmitry Ng) - Project maintainer, user preferences, password fixes, integrations
  • @mason5052 - Security fixes, OAuth hardening, resource management, TLS config, typo fixes
  • @Priyanka-2725 (Priyanka Singh) - Sploitus integration, terminal bug fix
  • @Vaibhavee89 (Vaibhavee Singh) - External network access documentation
  • @SkyFlyingMouse - Docker constant name fix
  • @s-b-repo (S.B) - Security fixes, typo corrections

Special thanks to all contributors for making PentAGI more secure, feature-rich, and user-friendly! 🎉

Merged Pull Requests

mason5052 and others added 30 commits February 21, 2026 23:43
The AuthLoginGetCallback handler accepted requests with an empty state
query parameter, bypassing CSRF validation. When state was empty, the
condition `queryState != "" && queryState != state.Value` short-circuited
to false, skipping the state mismatch check entirely.

Split the validation into two explicit checks: first reject missing
state parameter, then verify it matches the stored cookie value. This
aligns the GET callback with the POST callback handler which already
validates strictly via `data.State != state.Value`.

Ref: #101
- Fix product name typo "PegtAGI" -> "PentAGI" in frontend/README.md
- Fix inconsistent capitalization "PentAGi" -> "PentAGI" in config guide
  (10 occurrences in backend/docs/config.md)
- Fix misspelled company names in EULA.md:
  "Depp Infra" -> "Deep Infra", "Traversal" -> "Traversaal"
- Fix misleading OAuth env var names in README.md:
  GITHUB_CLIENT_ID -> OAUTH_GITHUB_CLIENT_ID,
  GOOGLE_CLIENT_ID -> OAUTH_GOOGLE_CLIENT_ID
  (matching .env.example and docker-compose.yml)
- Fix typo "applacation" -> "application" in auth.go comments (3 occurrences)
- Fix typo "pemissions" -> "permissions" in auth_middleware.go error message
- Rename misspelled files:
  sreenshots.go -> screenshots.go,
  processor-wizard-integation.md -> processor-wizard-integration.md,
  terminal-wizard-integation.md -> terminal-wizard-integration.md
…cy metrics

- Changed model versions from gemini-2.0-flash-lite to gemini-2.5-flash-lite for simple and simple_json agents.
docs: fix typos across documentation and code comments
…alidation

fix: require state parameter in OAuth GET callback
Fixed command injection in terminal.go — added path escaping for shell command logging

Fixed buffer overflow/DoS in terminal.go — added file size validation (100 MB limit) before memory allocation

Fixed resource leak in terminal.go — properly close tarWriter

Signed-off-by: S.B <30941141+s-b-repo@users.noreply.github.com>
(cherry picked from commit f84c435)
Removed dead code in executor.go — unused  json.MarshalIndent call

Fixed typo in executor.go — "Incomming" → "Incoming"

Signed-off-by: S.B <30941141+s-b-repo@users.noreply.github.com>
(cherry picked from commit 2422d41)
Fix: correct terminal command handling logic
Add explicit existence checks for 'exp' and 'provider' fields in
parseState() before accessing them. Previously, missing fields would
produce misleading error messages (e.g., strconv.ParseInt on empty
string gives "invalid syntax" rather than indicating the field is
missing). An empty provider string causes a confusing "not initialized"
error downstream in authLoginCallback().

This provides defense-in-depth validation with clear error messages
for each missing required field.

Ref: #101 (item 8)

Signed-off-by: mason5052 <ehehwnwjs5052@gmail.com>
In authLoginCallback(), when url.Parse fails on the return_uri,
response.Success writes a 200 OK but execution continues to
http.Redirect, causing a superfluous response.WriteHeader call
(double response write). Add the missing return statement.

Ref: #101

Signed-off-by: mason5052 <ehehwnwjs5052@gmail.com>
NewLangfuseClient receives *config.Config which provides two fields
for controlling external TLS behavior:
  - ExternalSSLInsecure (EXTERNAL_SSL_INSECURE, default false)
  - ExternalSSLCAPath   (EXTERNAL_SSL_CA_PATH, default empty)

The previous implementation hardcoded InsecureSkipVerify: true,
ignoring both config fields. This means:
  - TLS certificate verification was always disabled, even when
    the user had not set EXTERNAL_SSL_INSECURE=true
  - Custom CA certificates (EXTERNAL_SSL_CA_PATH) were never loaded

The fix mirrors the pattern in backend/pkg/system/utils.go:
  - Set InsecureSkipVerify from cfg.ExternalSSLInsecure (default false)
  - Load and apply RootCAs from cfg.ExternalSSLCAPath when set

Users who set EXTERNAL_SSL_INSECURE=true retain the previous behavior.
The default is now secure (certificate verification enabled).

Ref: #101
Signed-off-by: mason5052 <ehehwnwjs5052@gmail.com>
x509.NewCertPool() creates an empty pool, which discards all
OS-trusted CAs when a custom EXTERNAL_SSL_CA_PATH is set. This
prevents verification of any certificate not signed by the custom CA,
breaking connections to Langfuse instances using public CAs.

Use x509.SystemCertPool() as the starting pool (falling back to an
empty pool if the system pool is unavailable) and append the custom
CA to it, matching the pattern in backend/pkg/system/utils.go.

Fixes review comment on PR #132.

Signed-off-by: mason5052 <ehehwnwjs5052@gmail.com>
- Introduced a new SQL migration for user preferences, including a table for storing user-specific settings and a set of privileges.
- Implemented GraphQL mutations for adding and deleting favorite flows.
- Added resolvers for managing user preferences, including fetching and updating favorite flows.
- Updated the GraphQL schema to include user preferences types and related queries/mutations.
- Enhanced the flow provider to support title management and added necessary database queries for user preferences.
The @failure 403 swagger comment for GetUser was missing
the closing double-quote on its description string, which
causes malformed Swagger/OpenAPI documentation output.
Remove 7 leftover console.log calls from success paths in
SettingsPrompt. These were debug statements with no value
in production and clutter the browser console.
fix: respect ExternalSSLInsecure config in Langfuse client TLS
Use system.GetSystemCertPool() instead of duplicating certificate pool
initialization logic. Removes unsafe fallback to empty cert pool.
Addresses #91

Added comprehensive documentation for configuring PentAGI to be accessible
from external networks. Many users encounter issues when trying to access
PentAGI from other machines because the default configuration binds to
localhost (127.0.0.1) only.

Changes:
- Added "Accessing PentAGI from External Networks" section to README
- Documented proper configuration of PENTAGI_LISTEN_IP, PUBLIC_URL, and
  CORS_ORIGINS
- Included step-by-step instructions with examples
- Added firewall configuration guidance
- Provided troubleshooting steps for common issues
- Clarified that 0.0.0.0 should only be used for binding, not in URLs

Key points covered:
1. Environment variable configuration with actual examples
2. Container recreation requirement
3. Port binding verification
4. Firewall configuration for different Linux distributions
5. Common troubleshooting scenarios
6. Security considerations

This documentation helps prevent the common "unable to access externally"
issue by providing clear, actionable steps during initial setup.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
fix: validate required fields in OAuth state parser
fix: add missing return after error in OAuth callback redirect
chore(backend): fix docker client constant name typo
fix: add missing closing quote in GetUser swagger annotation
fix: remove debug console.log statements from settings-prompt
asdek and others added 3 commits February 24, 2026 02:53
Three resource management fixes:

1. browser.go callScraper(): Move defer resp.Body.Close() before the
   status code check. Previously, returning on non-200 status leaked
   the response body (connection held open until GC).

2. terminal.go ReadFile(): Add tar header size validation before
   allocating fileContent buffer. A malicious or corrupted tar with
   a multi-GB Size field could cause OOM. Rejects files over 50MB
   and negative sizes.

3. terminal.go WriteFile(): Add missing tarWriter.Close() call after
   writing content. Without it, the tar footer is not flushed,
   producing an incomplete archive that may fail on extraction.

Ref: #101

Signed-off-by: mason5052 <ehehwnwjs5052@gmail.com>
- Updated configuration files to include SPLOITUS_ENABLED environment variable.
- Enhanced documentation to reflect the addition of Sploitus in search engine options.
- Implemented functionality for handling Sploitus search requests and responses.
- Added tests to ensure proper operation of the Sploitus search tool.

This integration enhances the toolset available for penetration testing and security research.
@vxcontrol vxcontrol deleted a comment from cursor bot Feb 24, 2026
- Added configuration details for using PentAGI with Podman in rootless mode in .env.example.
- Updated README to include a new section on running PentAGI with Podman, detailing necessary changes for scraper service configuration.
- Clarified the use of non-privileged ports and HTTP for Podman rootless setups.

These changes enhance the documentation for users transitioning from Docker to Podman.
asdek added 10 commits February 25, 2026 06:09
- Updated logging in various tool handlers to use enriched log fields, improving traceability.
- Added error logging for unmarshal failures and required parameter checks in search actions.
- Implemented a new function to enrich log fields with flow, task, and subtask IDs for better context in logs.

These changes improve the observability and debuggability of the toolset, aiding in troubleshooting and monitoring.
- Updated max_tokens for primary_agent, assistant, generator, refiner, coder, installer, and pentester to enhance processing capabilities.
- Adjusted values to improve performance and accommodate larger input sizes.

These changes optimize the configuration for better stability.
- Added CORS support for Google OAuth by including its origin in allowed origins.
- Updated cookie handling to set SameSite attribute based on the OAuth provider, ensuring compatibility with Google’s POST callback.
- Improved user preferences fetching logic in the FavoritesProvider to only retrieve preferences for authenticated users.
- Refactored user authentication flow to handle popup checks and timeouts more effectively.

These changes improve the integration of Google OAuth and enhance user experience by managing preferences more efficiently.
…ation

- Introduced a new `user_preferences` table to store user-specific preferences.
- Implemented transaction handling in the user creation process to ensure user and preferences are created atomically.
- Added tests to verify the creation of user preferences alongside user accounts and to handle rollback scenarios on errors.

These changes enhance user management by allowing personalized preferences and ensuring data integrity during user creation.
- Introduced a new function `wrapMetadataWithStopReason` to enrich metadata with the stop reason from the response.
- Updated multiple instances in `WrapGenerateFromSinglePrompt` and `WrapGenerateContent` to utilize the new function, ensuring that stop reasons are captured and logged appropriately.
- These changes improve the observability of generation processes by providing additional context in the metadata.
…owID

- Modified the FlowUpdated method in flowPublisher to publish updates using userID, enhancing user-specific flow management.
- Updated the FlowUpdated method in flowSubscriber to subscribe using userID, ensuring consistent user context in subscriptions.

These changes improve the accuracy of flow updates and subscriptions by aligning them with user-specific identifiers.
- Added build arguments for version information in the Dockerfile, allowing the embedding of package version and revision during the build.
- Updated the build commands for backend utilities to include versioning flags, improving traceability of builds.
- Introduced a new entrypoint script to manage SSL certificate generation, enhancing security setup for the service.
- Updated README with instructions for using the new versioning scripts for Docker builds, improving user guidance.
- Added versioning scripts for Linux/macOS and Windows to streamline the build process and ensure consistent versioning across environments.

These changes improve the build process by integrating version control and enhancing the security setup for the application.
…iption support

- Added new fields `msgchain_id`, `tool_call_id_template`, `trace_id`, and `use_agents` to the Assistant and Flow models to improve data handling.
- Updated the TokenService and AssistantService to include a SubscriptionsController for better flow management and event publishing.
- Introduced new conversion functions to facilitate the transformation of models to database structures, ensuring consistency in data representation.
- Enhanced the FlowService to handle flow renaming and improved flow deletion logic by publishing updates to subscribers.

These changes improve the overall functionality and observability of the application by integrating new features and enhancing existing services.
@asdek asdek merged commit e97bbe5 into master Feb 25, 2026
6 checks passed
@asdek asdek deleted the feature/project_improvements branch February 25, 2026 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unable to access externally

5 participants