WIP: Update outlines (and outlines prompts) and transformers versions#256
WIP: Update outlines (and outlines prompts) and transformers versions#256
Conversation
bf92614 to
542d191
Compare
|
Alright so functionally this all seems to work, but when running the integration tests, there are several differences (mostly that makes sense as enough pieces have changed here that I wouldn't expect to get the exact same output). However, it seems like there's some non-determinism even with I need to spend some more time with this to understand what's really going on before feeling confident in merging. Anecdotally it does seem a bit faster now, but if it's not deterministic and doesn't enforce the entire schema what's the point. |
Copy in old outlines.prompt decorator for compatibility
6ffb3cf to
e76e5a5
Compare
@eveenhuis @jadie1 @barry-ravichandran FYSA
This is a WIP, but wanted to draw attention to some of the changes that need to be made for the latest outlines version. Specifically:
Theoutlines.pomptdecorate has been removed, now the guidance is to use theoutlines.Template.from_stringfunction.outlines.Template.from_stringwe also need to change how these templates are invoked wherever they're used, and that's scattered around several algorithms; not worth) future templates should strongly considering using this new approach thoughsamplerhas been removed from outlines, instead inference control parameters are just passed through to whatever backend is loading / serving the model (so for determinstic sampling using transformers as the model provider we would setdo_sample=Falsein thegeneration_kwargs)