ubergarm commited on
Commit
dcb412d
·
1 Parent(s): c1aefbd

make it clear you need to make a chat template jinja file

Browse files
Files changed (1) hide show
  1. README.md +10 -3
README.md CHANGED
@@ -359,9 +359,16 @@ numactl -N "$SOCKET" -m "$SOCKET" \
359
  --jinja
360
  ```
361
 
362
- For tool use you can always bring your own template with `--chat-template-file myTemplate.jinja` and might need `--special` etc. The chat template baked into these GGUFs from the [original one](https://huggingface.co/stepfun-ai/Step-3.5-Flash/blob/main/chat_template.jinja).
363
-
364
- Check [Discussion 1 for a tested working chat template for tool use thanks to mindkrypted](https://huggingface.co/ubergarm/Step-3.5-Flash-GGUF/discussions/1#69878ca7ae66ac235fc2ca95)!
 
 
 
 
 
 
 
365
 
366
  Another option for mainline tool calling users is to check out [pwilkin's autoparser branch](https://github.com/ggml-org/llama.cpp/pull/18675).
367
 
 
359
  --jinja
360
  ```
361
 
362
+ The chat template baked into these GGUFs from the [original
363
+ one](https://huggingface.co/stepfun-ai/Step-3.5-Flash/blob/main/chat_template.jinja).
364
+ Because of [this note and later updated official gguf chat
365
+ template](https://huggingface.co/stepfun-ai/Step-3.5-Flash-GGUF-Q4_K_S/discussions/16)
366
+ you will probably want to copy paste the template
367
+ from the [official gguf chat
368
+ template](https://huggingface.co/stepfun-ai/Step-3.5-Flash-GGUF-Q4_K_S/blob/main/step3p5_flash_Q4_K_S-00001-of-00012.gguf)
369
+ and use `--chat-template-file myTemplate.jinja`.
370
+
371
+ Also Check [Discussion 1 for a tested working chat template for tool use thanks to mindkrypted](https://huggingface.co/ubergarm/Step-3.5-Flash-GGUF/discussions/1#69878ca7ae66ac235fc2ca95)!
372
 
373
  Another option for mainline tool calling users is to check out [pwilkin's autoparser branch](https://github.com/ggml-org/llama.cpp/pull/18675).
374