Instructions to use DaertML/TinyGauss-1.1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DaertML/TinyGauss-1.1B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DaertML/TinyGauss-1.1B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("DaertML/TinyGauss-1.1B")
model = AutoModelForCausalLM.from_pretrained("DaertML/TinyGauss-1.1B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DaertML/TinyGauss-1.1B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DaertML/TinyGauss-1.1B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DaertML/TinyGauss-1.1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/DaertML/TinyGauss-1.1B

SGLang

How to use DaertML/TinyGauss-1.1B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DaertML/TinyGauss-1.1B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DaertML/TinyGauss-1.1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DaertML/TinyGauss-1.1B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DaertML/TinyGauss-1.1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use DaertML/TinyGauss-1.1B with Docker Model Runner:
```
docker model run hf.co/DaertML/TinyGauss-1.1B
```

YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

TinyGauss 1.1B

Adding math capabilities to models in the 1B parameter count regime, with the main objective of providing Math and logic reasoning capabilities to LLMs meant for robotics.

MathInstruct Finetune over TinyLlama/TinyLlama-1.1B-Chat-v1.0. The model is capable of following corrections from the user when it makes mistakes; opening the door to models that can correct mistakes following user feedback in a multi-turn interaction.

Finetuned using LLaMAFactory.

Set the "system prompt" for llama2 to have the problem, do not communicate with the model over "input" field, only talk to it giving the whole thing as a system prompt, even further interactions.

Examples:

Q: Calculate the length of the circle with radius 3 A: Given the radius (r) of the circle: r = 3

Now, we can use the formula for the diameter (D) of the circle: D = 2 * r D = 2 * 3 D = 6

Now, we can use the formula for the area (A) of the circle: A = π * r^2 A = π * (3)^2 A = π * 9 A = 18

So, the area of the circle with radius 3 is: A = π * 9

The diameter (D) of the circle is also 9 units, so the formula for the diameter of the circle is: D = 2 * 9 D = 18 So, the diameter of the circle is 18 units.

Q: Calculate the area of a square of side 4 A: Area of a square = 4 * side^2

So, the area of the square is 4 * 4 = 16 square units.

datasets:

TIGER-Lab/MathInstruct

language:

metrics: - bleu-4 15.1603 - rouge-1 34.3819 - rouge-2 9.9343 - rouge-l 17.0017

Downloads last month: 5

Safetensors

Model size

1B params

Tensor type

F32

F16

Model tree for DaertML/TinyGauss-1.1B

Merges

1 model