make it clear you need to make a chat template jinja file
Browse files
README.md
CHANGED
|
@@ -359,9 +359,16 @@ numactl -N "$SOCKET" -m "$SOCKET" \
|
|
| 359 |
--jinja
|
| 360 |
```
|
| 361 |
|
| 362 |
-
|
| 363 |
-
|
| 364 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 365 |
|
| 366 |
Another option for mainline tool calling users is to check out [pwilkin's autoparser branch](https://github.com/ggml-org/llama.cpp/pull/18675).
|
| 367 |
|
|
|
|
| 359 |
--jinja
|
| 360 |
```
|
| 361 |
|
| 362 |
+
The chat template baked into these GGUFs from the [original
|
| 363 |
+
one](https://huggingface.co/stepfun-ai/Step-3.5-Flash/blob/main/chat_template.jinja).
|
| 364 |
+
Because of [this note and later updated official gguf chat
|
| 365 |
+
template](https://huggingface.co/stepfun-ai/Step-3.5-Flash-GGUF-Q4_K_S/discussions/16)
|
| 366 |
+
you will probably want to copy paste the template
|
| 367 |
+
from the [official gguf chat
|
| 368 |
+
template](https://huggingface.co/stepfun-ai/Step-3.5-Flash-GGUF-Q4_K_S/blob/main/step3p5_flash_Q4_K_S-00001-of-00012.gguf)
|
| 369 |
+
and use `--chat-template-file myTemplate.jinja`.
|
| 370 |
+
|
| 371 |
+
Also Check [Discussion 1 for a tested working chat template for tool use thanks to mindkrypted](https://huggingface.co/ubergarm/Step-3.5-Flash-GGUF/discussions/1#69878ca7ae66ac235fc2ca95)!
|
| 372 |
|
| 373 |
Another option for mainline tool calling users is to check out [pwilkin's autoparser branch](https://github.com/ggml-org/llama.cpp/pull/18675).
|
| 374 |
|