Feature Request: LLM Profile Management for OpenHands CLI

What problem or use case are you trying to solve?

Currently, the OpenHands CLI (openhands-cli) requires users to go through the configuration/settings pipeline every time they want to switch between different LLM models or providers. This workflow is cumbersome for users who:

1. Work with multiple LLM providers: Users may have API keys for multiple providers (OpenAI, Anthropic, Mistral, etc.) and want to switch between them based on the task at hand.

2. Experiment with different models: Power users who want to test how different models perform on similar tasks need to reconfigure settings repeatedly.

3. Use different models for different projects: Users working on multiple projects may prefer different models based on project requirements (e.g., GPT-4 for complex reasoning, Claude for code generation, GPT-4o-mini for quick iterations).

4. Share configurations across teams: Teams may want to share standardized LLM configurations without manual setup.

Current Pain Point: Every model switch requires navigating through the entire settings flow, which interrupts the workflow and makes quick experimentation difficult.

Describe the UX or technical implementation you have in mind

Proposed Feature: LLM Profiles

Introduce a profile-based system that allows users to:

1. Pre-define multiple LLM configurations (profiles) with different credentials
2. Quickly select which profile to use at the beginning of each conversation
3. Manage (create, edit, delete, list) profiles easily

User Experience

1. Profile Management Commands

# List available profiles
openhands profiles list

# Create a new profile interactively
openhands profiles create [profile-name]

# Edit an existing profile
openhands profiles edit [profile-name]

# Delete a profile
openhands profiles delete [profile-name]

# Set default profile
openhands profiles set-default [profile-name]

2. Profile Selection at Conversation Start

When starting a new conversation, users can:

Option A: Command-line flag

# Start with a specific profile
openhands --profile gpt4
openhands --profile claude-sonnet

Option B: Interactive selection (if no --profile flag)

Welcome to OpenHands! Select an LLM profile:
  1. gpt4-turbo (OpenAI GPT-4 Turbo)
  2. claude-sonnet (Anthropic Claude 3.5 Sonnet)
  3. gpt4o-mini (OpenAI GPT-4o-mini)
  4. local-ollama (Ollama - llama3)
  5. Create new profile...
  6. Manage profiles...

[Use arrow keys to select, Enter to confirm]

3. Profile Storage Format

File Location: ~/.openhands/profiles/

• default.json - The default profile (backward compatible with current agent_settings.json)
• gpt4-turbo.json
• claude-sonnet.json
• config.json - Profile metadata (default profile, profile list, etc.)

Profile Structure (same as current agent_settings.json):

{
  "name": "gpt4-turbo",
  "description": "OpenAI GPT-4 Turbo for complex tasks",
  "llm": {
    "model": "gpt-4-turbo",
    "api_key": "sk-...",
    "base_url": null
  },
  "condenser": { ... },
  "mcp_config": { ... },
  "security_analyzer": { ... }
}

config.json:

{
  "default_profile": "gpt4-turbo",
  "profiles": [
    {
      "name": "gpt4-turbo",
      "file": "gpt4-turbo.json",
      "description": "OpenAI GPT-4 Turbo for complex tasks"
    },
    {
      "name": "claude-sonnet",
      "file": "claude-sonnet.json", 
      "description": "Anthropic Claude 3.5 Sonnet"
    }
  ]
}

Technical Implementation

Key Files to Modify

1. openhands-cli/openhands_cli/locations.py

   • Add PROFILES_DIR constant
   • Add PROFILE_CONFIG_PATH constant

2. openhands-cli/openhands_cli/tui/settings/store.py

   • Refactor AgentStore to support profiles
   • Add ProfileManager class with methods:
     • list_profiles()
     • load_profile(name)
     • save_profile(name, agent)
     • delete_profile(name)
     • get_default_profile()
     • set_default_profile(name)

3. openhands-cli/openhands_cli/argparsers/main_parser.py

   • Add --profile argument to CLI parser
   • Add new subcommand profiles with sub-actions (list, create, edit, delete, set-default)

4. openhands-cli/openhands_cli/setup.py

   • Update setup_conversation() to accept profile name
   • Integrate profile selection logic

5. openhands-cli/openhands_cli/simple_main.py

   • Add profile management command handling
   • Pass profile argument to conversation setup

6. openhands-cli/openhands_cli/tui/profile_selector.py (new file)

   • Interactive profile selection UI
   • Display profile list with descriptions

7. openhands-cli/openhands_cli/user_actions/profile_actions.py (new file)

   • Profile creation wizard
   • Profile editing interface
   • Profile deletion confirmation

Migration Strategy

For backward compatibility:

1. On first launch after upgrade, migrate existing agent_settings.json to profiles/default.json
2. Set "default" as the default profile
3. If no --profile is specified and no default is set, use the "default" profile
4. Maintain the same settings flow for users who don't use profiles

Example Implementation Workflow

# openhands-cli/openhands_cli/tui/settings/profile_manager.py

class ProfileManager:
    def __init__(self, profiles_dir: Path):
        self.profiles_dir = profiles_dir
        self.config_path = profiles_dir / "config.json"
        self._ensure_directories()
    
    def list_profiles(self) -> list[ProfileInfo]:
        """List all available profiles"""
        config = self._load_config()
        return config.get("profiles", [])
    
    def load_profile(self, name: str) -> Agent:
        """Load a specific profile by name"""
        profiles = self.list_profiles()
        profile_info = next((p for p in profiles if p["name"] == name), None)
        if not profile_info:
            raise ProfileNotFoundError(f"Profile '{name}' not found")
        
        profile_path = self.profiles_dir / profile_info["file"]
        agent = Agent.model_validate_json(profile_path.read_text())
        return agent
    
    def save_profile(self, name: str, agent: Agent, description: str = "") -> None:
        """Save an agent configuration as a profile"""
        # Implementation...
    
    def delete_profile(self, name: str) -> None:
        """Delete a profile"""
        # Implementation...
    
    def get_default_profile(self) -> str:
        """Get the name of the default profile"""
        config = self._load_config()
        return config.get("default_profile", "default")
    
    def set_default_profile(self, name: str) -> None:
        """Set the default profile"""
        # Implementation...

Testing Considerations

Add tests for:

• Profile creation, loading, editing, deletion
• Default profile management
• Profile selection during conversation start
• Migration from single config to profiles
• Profile validation and error handling
• Profile switching between conversations

Test files to create:

• tests/test_profile_manager.py
• tests/test_profile_selection.py
• tests/test_profile_migration.py

Additional context

Benefits

1. Improved UX: Eliminates repetitive configuration steps
2. Faster experimentation: Quick switching between models for A/B testing
3. Better organization: Named profiles with descriptions make it clear which configuration to use
4. Team collaboration: Profiles can be exported/imported or version controlled
5. Backward compatible: Existing users won't be affected

Related Code Locations

• Current agent storage: openhands-cli/openhands_cli/tui/settings/store.py
• Settings configuration: openhands-cli/openhands_cli/tui/settings/settings_screen.py
• CLI entry point: openhands-cli/openhands_cli/simple_main.py
• Argument parsing: openhands-cli/openhands_cli/argparsers/main_parser.py

Future Enhancements

• Profile import/export functionality
• Cloud-synced profiles
• Team workspace profiles
• Profile templates for common configurations
• Profile usage analytics

If you find this feature request useful, make sure to add a 👍 to the issue!