Human-in-the-Loop Implementation

[mjschock]

Resolves #636.

See #636 (comment).

OPENAI_API_KEY="your_api_key_here" uv run python examples/agent_patterns/human_in_the_loop.py

================================================================================
Run interrupted - tool approval required
================================================================================
State saved to result.json
Loading state from result.json

Tool call details:
  Agent: Weather Assistant
  Tool: get_temperature
  Arguments: {"city":"Oakland"}

Do you approve this tool call? (y/n): y
✓ Approved: get_temperature

Resuming agent execution...

================================================================================
Final Output:
================================================================================
The weather in Oakland is sunny, and the temperature is 20°C.

OPENAI_API_KEY="your_api_key_here" uv run python examples/agent_patterns/human_in_the_loop_stream.py 

================================================================================
Human-in-the-loop: approval required for the following tool calls:
================================================================================

Tool call details:
  Agent: Weather Assistant
  Tool: get_temperature
  Arguments: {"city":"Oakland"}

Do you approve this tool call? (y/n): y
✓ Approved: get_temperature

Resuming agent execution...

================================================================================
Final Output:
================================================================================
The current weather in Oakland is sunny, with a temperature of 20°C.

Done!

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Prime server tracker when resuming streaming runs

When Runner.run_streamed resumes from a RunState with conversation_id or previous_response_id, _start_streaming constructs a _ServerConversationTracker but never seeds it with the prior model_responses that were already sent. Unlike the synchronous path, no call to track_server_items is made, so prepare_input treats every previously generated item as unsent and resubmits them to the server. This duplicates earlier messages and breaks server-side conversation threading when a run is resumed.

Useful? React with 👍 / 👎.

[mjschock]

Fixed in a56ce0a. Added server conversation tracker priming at lines 1076-1079 to match the non-streaming implementation and prevent message duplication when resuming from RunState.

[chatgpt-codex-connector]

Streaming resume ignores existing turn count

The streaming execution path always initializes current_turn = 0 when _start_streaming is called, even if a RunState with an existing _current_turn is supplied. The loop then increments from zero, so any turns completed before the interruption are ignored and the max_turns guard is reset. After each interruption, a resumed streaming run can exceed the user’s turn limit and misreport the current turn number.

Useful? React with 👍 / 👎.

[mjschock]

This was already fixed in 74c50fd at line 914: current_turn=run_state._current_turn if run_state else 0. The turn counter is properly restored from the RunState.

[seratch]

Thanks for sending this patch!

I currently don't have the bandwidth to check this in depth, but one thing I wanted to mention is that, while implementing the sessions feature in openai-agents-js project, I found that the internals of runner need to take various HITL patterns into consideration. There might not be necessary to make those changes in this Python SDK, but sufficient testing for the sessions scenarios is worth doing.

[mjschock]

Thanks for sending this patch!

I currently don't have the bandwidth to check this in depth, but one thing I wanted to mention is that, while implementing the sessions feature in openai-agents-js project, I found that the internals of runner need to take various HITL patterns into consideration. There might not be necessary to make those changes in this Python SDK, but sufficient testing for the sessions scenarios is worth doing.

Happy to contribute! I added a couple examples using SQLiteSession and OpenAIConversationsSession and made sure they work:

OPENAI_API_KEY="your_api_key_here" uv run python examples/memory/memory_session_hitl_example.py 
=== Memory Session + HITL Example ===
Session id: :memory:
Enter a message to chat with the agent. Submit an empty line to exit.
The agent will ask for approval before using tools.

You: What cities does the Bay Bridge connect?
Assistant: The Bay Bridge connects San Francisco and Oakland in California.

You: What's the weather in those cities?

Agent HITL Assistant wants to call 'get_weather' with {"location":"San Francisco, CA"}. Approve? (y/n): y
Approved tool call.

Agent HITL Assistant wants to call 'get_weather' with {"location":"Oakland, CA"}. Approve? (y/n): y
Approved tool call.
Assistant: San Francisco is currently foggy with a temperature of 58°F. Oakland is sunny with a temperature of 72°F.

You: 

OPENAI_API_KEY="your_api_key_here" uv run python examples/memory/openai_session_hitl_example.py 
=== OpenAI Session + HITL Example ===
Enter a message to chat with the agent. Submit an empty line to exit.
The agent will ask for approval before using tools.

You: What cities does the Bay Bridge connect?
Assistant: The Bay Bridge, officially known as the San Francisco–Oakland Bay Bridge, connects the cities of **San Francisco** and **Oakland** in California.

You: What's the weather in those cities?

Agent HITL Assistant wants to call 'get_weather' with {"location":"San Francisco, CA"}. Approve? (y/n): y
Approved tool call.

Agent HITL Assistant wants to call 'get_weather' with {"location":"Oakland, CA"}. Approve? (y/n): y
Approved tool call.
Assistant: San Francisco is currently foggy and 58°F, while Oakland is sunny and 72°F.

You: 

I'm hoping that just about covers everything but lemme know if there are other areas I should make sure address.

[mjschock]

Thanks @seratch for adding it to the 0.6.x milestone! I'll make sure to address any feedback.

[mjschock]

Just a heads up. I noticed there are some discrepancies with the serialized RunState when interrupting in Python and resuming in TypeScript and visa versa. I'm addressing those this weekend and will push the update.

[seratch]

@codex Can you do reviews on the code changes in this PR with fresh eyes? Please focus on the potential issues for existing apps and edge case scenarios rather than general feedback.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Avoid duplicating session history when resuming streaming runs

When Runner.run_streamed is resumed from a RunState, _start_streaming still calls _prepare_input_with_session and _save_result_to_session unconditionally (streamed_result.input is recomputed and the original prompt is saved again). In the non‑streaming path these calls are skipped when a RunState is supplied so that the session history isn’t duplicated. Re‑preparing and saving here means a resumed streaming run writes the same user message back into the session and the next turn will see duplicate user prompts, breaking conversations for apps that rely on Session or session_input_callback. Consider gating these calls with run_state is None as the synchronous path does.

Useful? React with 👍 / 👎.

[mjschock]

👍 Fixed in 466c306

[seratch]

Thanks for working on this. I ran codex reviews many times and it detected these issues:

• [P1] Resumed HITL flow never executes approved non-function tools — src/agents/_run_impl.py:626-716
  When resuming an interrupted turn after a tool approval, resolve_interrupted_turn only re-runs function tools and computer actions; it never executes queued shell, local
  shell, or apply_patch calls stored in processed_response (execute_shell_calls/execute_apply_patch_calls are not invoked). Any ShellTool/ApplyPatchTool configured with
  needs_approval will therefore return a ToolApprovalItem, but after the user approves and re-runs the agent the tool output is never produced, leaving the workflow stuck
  without the requested side effect or tool output. This affects any HITL approval scenario for these tools.

• [P1] Resuming pending MCP approvals raises TypeError — src/agents/_run_impl.py:733-784
  When resuming a turn that contains hosted MCP approval requests, resolve_interrupted_turn stores the pending ToolApprovalItem objects in a set (pending_hosted_mcp_approvals).
  ToolApprovalItem is not hashable, so the first add() at this point raises TypeError: unhashable type: 'ToolApprovalItem', aborting the resume flow for any run interrupted on
  an MCP approval. This breaks HITL resumption whenever a hosted MCP approval is still pending.

• [P1] Route local shell calls to remote shell tool — src/agents/_run_impl.py:1044-1052
  When processing model output, LocalShellCall items are dispatched to shell_tool if one exists (lines 1044–1050) before falling back to local_shell_tool. This means that if
  an agent registers both ShellTool and LocalShellTool, or if the model emits a local_shell_call despite only intending local execution, the call will be run through the remote
  shell tool instead of the local one, bypassing the local tool’s approval/on_approval hooks and executing in the wrong environment. Local shell calls should be handled by
  LocalShellTool; prioritizing shell_tool here misroutes the execution and can break HITL approval or shell behavior for any agent that enables both tools.

• [P1] Streaming drops prior turn items — src/agents/run.py:3618-3624
  In _run_single_turn_streamed the pre_step_items list was changed to always be empty, so SingleStepResult.generated_items now contains only the current turn’s items instead
  of the accumulated history (streamed_result.new_items). On any streaming run with multiple turns, previous tool calls/outputs are dropped from both the next model request and
  the items persisted/emitted after the first turn, breaking conversation continuity and session persistence. This regression occurs whenever a streamed run progresses beyond
  the first turn.

• [P1] Preserve max_turns when resuming from RunResult state — src/agents/result.py:213-223
  RunResult.to_state hardcodes max_turns=10 when creating a RunState, assuming the runner will override it. However Runner.run does not pull max_turns from the state—
  it uses the value passed by the caller or defaults to 10. If a run configured with a higher max_turns (e.g., 20) is interrupted for tool approval and resumed via state =
  result.to_state(); Runner.run(agent, state) without re‑passing max_turns, the resume path will treat current_turn from the state against a max_turns of 10 and will raise
  MaxTurnsExceeded on the next turn, even though budget remains. The max_turns used for the run needs to be persisted in the RunState instead of being reset to 10.

• [P1] Deserialize only function approvals, breaks HITL for other tools — src/agents/run_state.py:983-999
  When restoring a run from JSON, the interruption reconstruction assumes every approval is a ResponseFunctionToolCall and blindly instantiates that type for each rawItem. Any
  HITL approval for shell, apply_patch, MCP, etc. will either fail validation or raise because their rawItem payloads do not match ResponseFunctionToolCall, making resuming
  from saved state impossible for those tool types. The deserializer needs to branch on the tool call type (or keep the original dict) instead of always coercing to a function
  call.

• [P1] Deserializing interruptions assumes function tool calls — src/agents/run_state.py:1168-1177
  Resuming a saved RunState fails for approvals on non-function tools because from_json blindly instantiates ResponseFunctionToolCall for every interruption. If the pending
  approval was for a shell/apply_patch/hosted tool call (all supported by the new HITL flow), this constructor raises validation errors and the run cannot be resumed. Please
  deserialize rawItem without forcing it to a function call or branch by type so non-function approvals survive round‑tripping.

• [P2] Avoid error-level logging on tracker creation — src/agents/run.py:161-168
  _ServerConversationTracker.post_init logs an error with a full stack trace every time a tracker is instantiated (lines 161‑168). This runs for all server-managed
  conversations, so normal runs now emit error-level messages even when nothing is wrong, polluting logs and alerting systems. Consider removing or downgrading this log, or
  guarding it behind a debug flag.