[issue-4270] [P SDK] Improve GEval compatibility with DashScope Qwen judge model

[Susan9001]

Details

This PR is a follow-up to #4229 and makes DashScope Qwen more robust as a GEval judge model when used via LiteLLMChatModel.

Currently, when a model advertises logprobs and top_logprobs support, GEval enables the logprobs-aware scoring path. For DashScope Qwen this can occasionally lead to MetricComputationError("Failed to calculate g-eval score") because the returned logprobs do not always match the OpenAI-style format expected by the parser.

This PR treats DashScope Qwen as not logprobs-supported in this context, so GEval falls back to the standard text/JSON-based parsing path instead of relying on logprobs.

Change checklist

• User facing
• Documentation update

Issues

• Resolves: [Bug]: GEval LiteLLMChatModel with DashScope Qwen sometimes fails to calculate g-eval score #4270
• Related: [FR]: Improve DashScope Qwen support in LiteLLMChatModel for GEval #4229

Testing

Locally:

• pytest tests/unit/evaluation/models/test_litellm_chat_model.py
• Ran more examples with dashscope/qwen-flash as the judge model with code snippets:

        self.judge_model = models.LiteLLMChatModel(
            model_name=judge_model_name,
            api_base="https://dashscope.aliyuncs.com/compatible-mode/v1",
            api_key=os.getenv("DASHSCOPE_API_KEY"),
        )

  All samples now score successfully without Failed to calculate g-eval score.

Documentation

[yaricom]

Hi @Susan9001 ! Thank you for a contribution! Please fix merge conflicts with current branch.

Cheers,
Iaroslav

[Susan9001]

Hi @yaricom ,
I have just resolved the merge conflicts. Sorry it took me a bit to get back to this. Please let me know if anything else I need to adjust.

[yaricom]

Hi @Susan9001 ! Thank you for the contribution!

Happy coding!
Iaroslav