File tree Expand file tree Collapse file tree 1 file changed +3
-0
lines changed
Expand file tree Collapse file tree 1 file changed +3
-0
lines changed Original file line number Diff line number Diff line change @@ -164,6 +164,7 @@ To install the server package and get started:
164164pip install llama-cpp-python[server]
165165python3 -m llama_cpp.server --model models/7B/llama-model.gguf
166166```
167+
167168Similar to Hardware Acceleration section above, you can also install with GPU (cuBLAS) support like this:
168169
169170``` bash
@@ -173,6 +174,8 @@ python3 -m llama_cpp.server --model models/7B/llama-model.gguf --n_gpu_layers 35
173174
174175Navigate to [ http://localhost:8000/docs ] ( http://localhost:8000/docs ) to see the OpenAPI documentation.
175176
177+ To bind to ` 0.0.0.0 ` to enable remote connections, use ` python3 -m llama_cpp.server --host 0.0.0.0 ` .
178+ Similarly, to change the port (default is 8000), use ` --port ` .
176179
177180## Docker image
178181
You can’t perform that action at this time.
0 commit comments