feat(ollama): logprobs support in Ollama

[keenborder786]

Issue: #34207
Description: Log probabilities have been enabled. The _chat method already returns a logprobs field (see source), which is inserted into generation_info and metadata is updated accordingly. Only the parameters need to be added to activate this feature.

[codspeed-hq]

CodSpeed Performance Report

Merging #34218 will not alter performance

_{Comparing keenborder786:feat/ollama_log_probs (7ee250a) with master (dff229d)¹}

Summary

✅ 1 untouched
⏩ 33 skipped²

Footnotes

1. No successful run was found on master (2aa0555) during the generation of this report, so dff229d was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

2. 33 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[paascorb]

Hi there, KeenBorder.

It's my first time doing a pull request in a Github public repository and maybe you can help me. I've checked your modification in order to make my code work but it is harder than just add the params "logprobs" and "top_logprobs" to the constructor. I've modify the file with and it is working with the UQLM library. I needed to modify the functions "_chat_stream_with_aggregation" and "_iterate_over_stream" and its async counterparts in order to make it work.

Here is the modified version:
chat_models(1).py

It is not finished yet because the top_logprobs does not work, but my plan is to finish it. How can I create the pull request when I have finished the job?

Regards,
Pablo

[keenborder786]

@paascorb I saw your modification, you need to pass logprobs and top_logprobs NOT in options but direct in params. Pleasee see

[paascorb]

Hi there @keenborder786 Thanks for your replies.

With your solution I get:

With mine I get:

Regards,
Pablo

[paascorb]

I've modify the last part to make the top_logprobs work if you want to check it out.

chat_models.zip

Thanks in advance,

Pablo

[keenborder786]

@paascorb, I liked your approach better since it is synced with how we are getting other params. I have modified.

[paascorb]

Hi @keenborder786 , the key is that it is not only necessary to add the parameters from the constructor, but also to receive the logprobs in each chunk. Please see the methods _chat_stream_with_aggregation and _iterate_over_stream. Especially lines 1089, 1090 and 1091.

[keenborder786]

It will be passed through _chat_params