Misc. bug: OpenAI API v1/responses llama-server

[foxbg]

Name and Version

.\llama-server --version
...
version: 5902 (4a4f426)
built with clang version 19.1.5 for x86_64-pc-windows-msvc

Operating systems

Windows

Which llama.cpp modules do you know to be affected?

llama-server

Command line

.\llama-server -m Llama-3.2-3B-Instruct-Q6_K_L.gguf -ngl 33 --port 8081 --host 0.0.0.0

Problem description & steps to reproduce

When OpenAI compatible API is used and client uses v1/responses I get 404
Possibly not yet supported?
ref:
https://platform.openai.com/docs/api-reference/responses

First Bad Commit

Not sure

Relevant log output

Client
`POST "http://192.168.x.x:8081/v1/responses": 404 Not Found {"code":404,"message":"File Not Found","type":"not_found_error"`
Server

main: server is listening on http://0.0.0.0:8081 - starting the main loop
srv  update_slots: all slots are idle
srv  log_server_r: request: POST /v1/responses 192.168.x.x 404

[SomeOddCodeGuy]

I don't believe that endpoint is available. If you go to server.cpp line 4834, you can see the registered endpoints. v1/responses is not one of them.

[foxbg]

Is there a plan to add this endpoint to improve OpenAI API compatibility?

[ggerganov]

There is no plan atm, but we can create one. I haven't seen yet what are the changes introduced in /v1/responses, but unless there are some major blockers we should consider supporting it.

[foxbg]

As it is in API specification tools will start to use it. In my case it was fabric(danielmiessler/Fabric#1559). I've opened a discussion there too danielmiessler/Fabric#1625.

[ksylvan]

Hi. I'm one of the maintainers of Fabric. For OpenAI compatible vendors I have a way to mark the ones that have /responses API implemented.

I'll get clarity with @foxbg on his use case and provide a solution for users with similar issues.

[morozover]

@ggerganov

Here the description of /v1/responses
https://platform.openai.com/docs/api-reference/responses/create

[iansmathew]

+1 👍 to support the implementation of this endpoint.

My current use case is using fabric as well, and although there is a workaround, I would love if llama-server was more OpenAI compatible.

[dariomanda]

The additon of the OpenAIs new responses API would be great.

+1 for this

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.