Skip to content

Commit 9d1b419

Browse files
committed
fix stream generation kwargs
1 parent a00fc66 commit 9d1b419

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

src/inferencesh/models/llm.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -606,8 +606,9 @@ def _generate_worker():
606606
"temperature": temperature,
607607
"top_p": top_p,
608608
"stop": stop,
609-
**kwargs
610609
}
610+
if kwargs:
611+
completion_kwargs.update(kwargs)
611612
if tools is not None:
612613
completion_kwargs["tools"] = tools
613614
if tool_choice is not None:

0 commit comments

Comments
 (0)