Question; Handling of Errors during APO rollouts

[nariaki3551]

Thank you to the developers for the amazing work on Agent Lightning !

During APO training, many my rollouts terminate due to a LangGraph recursion limit:

langgraph.errors.GraphRecursionError: Recursion limit of 25 reached without hitting a stop condition.

So, I’m wondering:

Are these rollout failures and exceptions (e.g, recursion limit) taken into account when generating critiques for prompt updates during APO?

[KellySit]

@nariaki3551 I think there are several possible reasons behind this, do you mind sharing your prompt? As much as I know, failed rollout are Included in critique generation. You can also check apo algorithms if that helps.

[nariaki3551]

Thanks for the reply! @KellySit

To clarify, my concern is not about fixing errors in my actual workflow.
Rather, I’m wondering whether including runtime errors (e.g., Timeout, GraphRecursionError) directly in the critique input could help APO generate better critiques and improved prompts.

For example, if a large number of rollouts hit a GraphRecursionError, I would expect the critique model to propose prompt changes that reduce unnecessary loops in the agent workflow.

Looking at the critique prompt used in APO:
https://github.com/microsoft/agent-lightning/blob/main/agentlightning/algorithm/apo/prompts/text_gradient_variant01.poml

It seems that only the rollout experiment.status is passed to the critique model, and detailed error information is not provided as part of the input.

If this approach seems useful, I’d be happy to open a PR adding error information to the critique prompt.
( @ultmaster )

[ultmaster]

I think you can contribute another text_gradient_variant. We might find a way to tag each variant and let users select variants by tag in future.