Instructions to use meetkai/functionary-small-v2.5 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use meetkai/functionary-small-v2.5 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="meetkai/functionary-small-v2.5", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("meetkai/functionary-small-v2.5", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("meetkai/functionary-small-v2.5", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use meetkai/functionary-small-v2.5 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "meetkai/functionary-small-v2.5"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "meetkai/functionary-small-v2.5",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/meetkai/functionary-small-v2.5

SGLang

How to use meetkai/functionary-small-v2.5 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "meetkai/functionary-small-v2.5" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "meetkai/functionary-small-v2.5",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "meetkai/functionary-small-v2.5" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "meetkai/functionary-small-v2.5",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use meetkai/functionary-small-v2.5 with Docker Model Runner:
```
docker model run hf.co/meetkai/functionary-small-v2.5
```

functionary-small-v2.5

Commit History

Update config.json

c5c46b5
verified

jeffreymeetkai commited on Oct 16, 2024

Delete training_args.bin

d881c3f
verified

khaimai commited on Sep 25, 2024

Update tokenizer_config.json

eb56caf
verified

jeffreymeetkai commited on Sep 17, 2024

Update tokenizer_config.json

963847a
verified

jeffreymeetkai commited on Aug 14, 2024

Delete tokenization_functionary.py

25693df
verified

jeffreymeetkai commited on Aug 14, 2024

Update README.md

9d78f40
verified

jeffreymeetkai commited on Aug 14, 2024

Update config.json

89e6716
verified

jeffreymeetkai commited on Jul 3, 2024

Fixes

5df5630

jeffrey-fong commited on May 30, 2024

Update config.json

2b6b788
verified

jeffreymeetkai commited on May 30, 2024

Update README.md

3f9c742
verified

jeffreymeetkai commited on May 24, 2024

fixes

6fdf956

jeffreymeetkai commited on May 24, 2024

update README

f9b715e

jeffreymeetkai commited on May 24, 2024

add respone parsing remove code, update README

2e54059

jeffreymeetkai commited on May 24, 2024

add tokenizer remote code

7fae8ba

Jeffrey Fong commited on May 23, 2024

Update README.md

6f7ed43
verified

khaimai commited on May 17, 2024

Update tokenizer_config.json

1753889
verified

khaimai commited on May 9, 2024

Upload folder using huggingface_hub

486ee32
verified

khaimai commited on May 6, 2024

Upload folder using huggingface_hub

01ce2c1
verified

khaimai commited on May 6, 2024

initial commit

50d442d
verified

khaimai commited on May 6, 2024

Commit History

Update config.json c5c46b5 verified

Delete training_args.bin d881c3f verified

Update tokenizer_config.json eb56caf verified

Update tokenizer_config.json 963847a verified

Delete tokenization_functionary.py 25693df verified

Update README.md 9d78f40 verified

Update config.json 89e6716 verified

Fixes 5df5630

Update config.json 2b6b788 verified

Update README.md 3f9c742 verified

fixes 6fdf956

update README f9b715e

add respone parsing remove code, update README 2e54059

add tokenizer remote code 7fae8ba

Update README.md 6f7ed43 verified

Update tokenizer_config.json 1753889 verified

Upload folder using huggingface_hub 486ee32 verified

Upload folder using huggingface_hub 01ce2c1 verified

initial commit 50d442d verified

Update config.json

c5c46b5
verified

Delete training_args.bin

d881c3f
verified

Update tokenizer_config.json

eb56caf
verified

Update tokenizer_config.json

963847a
verified

Delete tokenization_functionary.py

25693df
verified

Update README.md

9d78f40
verified

Update config.json

89e6716
verified

Fixes

5df5630

Update config.json

2b6b788
verified

Update README.md

3f9c742
verified

fixes

6fdf956

update README

f9b715e

add respone parsing remove code, update README

2e54059

add tokenizer remote code

7fae8ba

Update README.md

6f7ed43
verified

Update tokenizer_config.json

1753889
verified

Upload folder using huggingface_hub

486ee32
verified

Upload folder using huggingface_hub

01ce2c1
verified

initial commit

50d442d
verified