Instructions to use tcclaviger/Tess-27B-RFI with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tcclaviger/Tess-27B-RFI with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="tcclaviger/Tess-27B-RFI")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("tcclaviger/Tess-27B-RFI")
model = AutoModelForMultimodalLM.from_pretrained("tcclaviger/Tess-27B-RFI")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tcclaviger/Tess-27B-RFI with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tcclaviger/Tess-27B-RFI"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tcclaviger/Tess-27B-RFI",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/tcclaviger/Tess-27B-RFI

SGLang

How to use tcclaviger/Tess-27B-RFI with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tcclaviger/Tess-27B-RFI" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tcclaviger/Tess-27B-RFI",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tcclaviger/Tess-27B-RFI" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tcclaviger/Tess-27B-RFI",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use tcclaviger/Tess-27B-RFI with Docker Model Runner:
```
docker model run hf.co/tcclaviger/Tess-27B-RFI
```

RFI/RFA hybrid quant of migtissera/Tess-4-27B

Runtime: requires tcclaviger/vllm:latest — an RDNA 4 (gfx12xx) vLLM image and the only build with the RFI/RFA kernels; no other vLLM build loads these weights. Not validated on any other hardware at this time.

Tess-27B-RFI

Mixed-precision (RFI/RFA hybrid) quantization of Tess-4-27B by Migel Tissera — an agentic, thinking-native finetune of Qwen3.6-27B. All credit for the model to its author; this repo only changes the numerics.

Quantization by component

Attention and all other linear layers — 8-bit integer weights (RFI): group size 32, symmetric, Hadamard-32 rotation, block-float scales stored as int8 mantissa + int8 exponent, int8 activation compute path.
MTP speculative-decode head — also 8-bit RFI: its fc, MLP, and attention projections are all int8-packed like the main attention; only its norms stay bf16.
MLP layers — 4-bit float weights (RFA): IQ4_NL non-linear grid, group size 16, asymmetric, Hadamard-16 rotation, the same block-float int8 scale encoding.
Kept in bf16 (not quantized) — vision encoder, linear-attention (GDN) blocks, embeddings, norms, and the lm_head.

Serving context — 512K via YaRN

All evaluation below was run while serving at --max-model-len 524288 (512K tokens), extended from the native 256K window with YaRN via --hf-overrides:

{"text_config": {"rope_parameters": {"rope_type": "yarn", "factor": 2.0,
 "original_max_position_embeddings": 262144, "mrope_interleaved": true,
 "mrope_section": [11, 11, 10], "partial_rotary_factor": 0.25,
 "rope_theta": 10000000}}}

Evaluation results

Six builds measured with identical methodology, each against its own live vLLM endpoint (July 2026): Qwen3.6-27B (bf16 base) → Tess-4-27B (the tune, bf16) → Tess-27B-RFI (this quant) → Tess-27B-RFA (all-attention 4-bit sibling) → Tess-FP8 (W8A8 block-128 FP8 sibling), with Qwen3.6-35B-A3B (MoE, bf16) as a comparative reference point. Bold marks the best score in each row (ties all bolded).

Tune impact — Qwen3.6-27B → Tess-4-27B

Metric	Change
WikiText-2 perplexity	7.056 → 6.669 (−5.5%)
Codeneedle overall recall	97.8% → 97.7% (≈ flat)
MC accuracy (4 tasks)	≈ flat (−0.1 to +1.6 pp)
Tool-eval (full 69, TC-61 excl)	86 → 85 (−1)
GSM8K / MMLU (50q each)	98→94% / 74→76%
Decode @ conc 1 (ISL 128)	57.0 → 59.9 tok/s (+5%)
Decode @ conc 50 (ISL 128)	556 → 528 tok/s (−5%)

Quantization cost (RFI/RFA hybrid) — Tess-4-27B → Tess-27B-RFI

Metric	Change
Checkpoint size	55.6 → 28.9 GB (−48%)
WikiText-2 perplexity	6.669 → 6.663 (≈ flat)
Codeneedle overall recall	97.7% → 98.0% (≈ flat)
MC accuracy (4 tasks)	≈ flat (−0.4 to +0.1 pp)
Tool-eval (full 69, TC-61 excl)	85 → 87 (+2)
GSM8K / MMLU (50q each)	94→98% / 76→82% (both up)
Decode @ conc 1 (ISL 128)	59.9 → 75.3 tok/s (+26%)
Decode @ conc 50 (ISL 128)	528 → 453 tok/s (−14%)

Quality

Metric	Qwen3.6-27B (base)	Tess-4-27B	Tess-27B-RFI	Tess-27B-RFA	Tess-FP8	Qwen3.6-35B-A3B
WikiText-2 PPL (n_ctx 2048, lower is better)	7.0559	6.6691	6.6632	6.6292	6.6627	6.5092
ARC-Challenge (acc_norm)	59.30%	60.84%	60.41%	60.32%	60.49%	55.20%
ARC-Easy (acc_norm)	75.93%	77.53%	77.40%	78.87%	77.82%	71.13%
Winogrande (acc)	77.51%	77.43%	77.51%	76.80%	77.66%	73.40%
HellaSwag (acc_norm)	84.12%	84.21%	84.27%	84.05%	84.13%	82.95%

Multiple-choice accuracy is lm-eval loglikelihood scoring, 0-shot.

Long-context positional recall (codeneedle)

Verbatim function recall under 10K–80K-token contexts.

Corpus	Qwen3.6-27B (base)	Tess-4-27B	Tess-27B-RFI	Tess-27B-RFA	Tess-FP8	Qwen3.6-35B-A3B
Python	100%	100%	100%	100%	99.55%	99.09%
C++	98.12%	98.12%	98.44%	98.75%	98.75%	98.44%
Rust	99.69%	99.69%	99.69%	99.69%	99.69%	99.38%
JS (~80K tokens)	93.44%	93.13%	93.75%	93.75%	93.44%	92.19%
Tools	98.26%	99.57%	99.57%	99.57%	99.57%	93.48%
Overall recall	97.81%	97.73%	97.97%	98.05%	97.86%	97.28%

Tool calling & accuracy benches

Bench	Qwen3.6-27B (base)	Tess-4-27B	Tess-27B-RFI	Tess-27B-RFA	Tess-FP8	Qwen3.6-35B-A3B
tool-eval final (full 69, TC-61 excl)	86	85	87	86	87	90
GSM8K (50q)	98.0%	94.0%	98.0%	98.0%	98.0%	96.0%
MMLU (50q)	74.0%	76.0%	82.0%	80.0%	76.0%	64.0%
IFEval (20 prompts, prompt-level)	90.0%	90.0%	90.0%	95.0%	90.0%	90.0%

Decode throughput — tok/s output (ISL 128 / ISL 512)

vllm bench serve, random dataset, OSL 128, saturation, 4× R9700 (gfx1201), TP 4.

Concurrency	Qwen3.6-27B (base)	Tess-4-27B	Tess-27B-RFI	Tess-27B-RFA	Tess-FP8	Qwen3.6-35B-A3B
1	57.0 / 61.4	59.9 / 63.8	75.3 / 69.3	67.9 / 58.2	86.0 / 85.1	91.9 / 114.4
10	304.5 / 233.6	289.6 / 227.8	281.5 / 219.2	292.2 / 194.8	280.6 / 269.2	434.9 / 440.3
25	424.4 / 321.7	492.0 / 341.1	429.1 / 284.0	369.0 / 260.3	533.2 / 355.4	688.9 / 563.7
50	556.0 / 349.8	527.9 / 345.6	452.8 / 295.0	422.6 / 278.4	560.3 / 394.1	889.8 / 702.9

MTP draft acceptance by work category

Measured from live serving logs, k=5 draft tokens, drafted-token-weighted aggregation.

Work category	Overall acceptance	Pos 1	Pos 2	Pos 3	Pos 4	Pos 5
JSON generation	79.0%	92.5%	85.9%	79.3%	71.6%	65.6%
Math	78.5%	94.4%	87.4%	78.2%	70.7%	62.0%
Code	64.2%	88.7%	74.5%	63.2%	51.2%	43.6%
Creative English	60.0%	87.0%	72.6%	57.5%	44.9%	38.1%

Notes

All builds serve on the tcclaviger/vllm:latest image, which has kernel tunes baked in. TunableOp is untuned — GEMMs run on default heuristic-determined values. Base Qwen3.6-27B figures are the 2026-07-12 re-measurement on the same tcclaviger/vllm:latest image and thinking-OFF methodology as every other build, replacing an earlier non-comparable run.

Credits

Tess-4-27B by Migel Tissera (migtissera/Tess-4-27B) — the model quantized here:

@misc{tissera2026tess4,
  title        = {Tess-4-27B},
  author       = {Migel Tissera},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/migtissera/Tess-4-27B}}
}

codeneedle (positional recall) originally by Alexander Ziskind, expanded test suite by tcclaviger (Rob Smith).
Tool-calling scenarios (incl. TC-61) run on tool-eval-bench by SeraphimSerapis (Tim Messerschmidt), scenario methodology adapted from ToolCall-15 by stevibe.
GSM8K, MMLU, and IFEval run via tool-eval-bench's built-in accuracy benchmarks at their defaults: GSM8K 8-shot CoT, MMLU 5-shot, IFEval zero-shot.

Downloads last month: 53

Safetensors

Model size

20B params

Tensor type

BF16

Model tree for tcclaviger/Tess-27B-RFI

Base model

Qwen/Qwen3.6-27B

Finetuned

migtissera/Tess-4-27B

Quantized

(30)

this model