Skip to content

Improve OpenEvolve launcher setup, run performance eval for speedup, and standardize task result output#20

Open
sharareh-y wants to merge 1 commit intoAMD-AGI:mainfrom
sharareh-y:sharareh/openevolve-fix
Open

Improve OpenEvolve launcher setup, run performance eval for speedup, and standardize task result output#20
sharareh-y wants to merge 1 commit intoAMD-AGI:mainfrom
sharareh-y:sharareh/openevolve-fix

Conversation

@sharareh-y
Copy link
Contributor

Improve OpenEvolve launcher setup, run performance eval for speedup, and standardize task result output

What this PR changes

This PR updates the openevolve agent integration to make setup and evaluation behavior consistent and to emit richer task results.

  1. Adds ensure_openevolve_setup() in agents/openevolve/launch_agent.py

    • Detects whether local geak-openevolve package + GEAK-eval-OE exist.
    • If missing, automatically runs agents/openevolve/agent_setup.sh.
    • Switches OpenEvolve imports to dynamic import after setup (importlib) to avoid import-time failures.
  2. Fixes OpenEvolve setup script in agents/openevolve/agent_setup.sh

    • Uses HTTPS clone URLs.
    • Checks out required branches.
    • Installs both repos in editable mode to make modules importable in launcher execution.
  3. Executes performance_command during evaluator runs

    • Previously performance command existed in task config but was not used by the OpenEvolve evaluator path.
    • Evaluator now runs performance commands after compile/correctness and captures outputs.
    • Parses perf timing from perf/*.json (ms fields) or falls back to stdout/stderr time parsing.
  4. Adds baseline/optimized timing + speedup computation

    • Records:
      • base_execution_time
      • best_optimized_execution_time
      • speedup
    • Uses baseline vs current best timing and updates combined score accordingly.
  5. Updates task_result.yaml writing logic

    • Writes standard fields expected by downstream flow, including:
      • best_optimized_source_file_path
      • best_optimized_kernel_functions
      • pass_compilation, compilation_error_message
      • pass_correctness, correctness_error_message
      • timing fields + speedup_ratio
    • Improves failure-path result writing with explicit error messages.
  6. Tunes OpenEvolve config defaults (agent_config_amd_claude.yaml)

    • Adjusts iteration/population/timeouts and adds max_code_length.
    • Uses agent_config_amd_claude.yaml as launcher config source.

@sharareh-y sharareh-y requested a review from irvineoy February 24, 2026 17:06
@sharareh-y
Copy link
Contributor Author

WIP: need to add some fix commits

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant