Abiray/Qwen3.6-35B-A3B-Q4_K_S-GGUF

Qwen3.6-35B-A3B - Q4_K_S GGUF Quantization

This repository contains the Q4_K_S GGUF format of the Qwen3.6-35B-A3B model.

These files were quantized by Abiray using llama.cpp to make the model accessible for consumer hardware and CPU-heavy environments.

📦 Other Available Formats

I have processed this model into several different quantization formats. You can find them in my other repositories:

• Abiray/Qwen3.6-35B-A3B-Q8_0-GGUF (High quality, largest file)
• Abiray/Qwen3.6-35B-A3B-Q6_K-GGUF (Very high quality)
• Abiray/Qwen3.6-35B-A3B-Q5_K_M-GGUF (Great balance of quality and size)
• Abiray/Qwen3.6-35B-A3B-Q4_K_M-GGUF (Good quality, recommended for most users)
• Abiray/Qwen3.6-35B-A3B-Q3_K_M-GGUF (Low quality, smallest file)

💻 How to run with llama.cpp

You can run this model locally using llama-cli from the llama.cpp project.

# Example command (adjust threads and context size to your machine)
./llama-cli -m Qwen3.6-35B-A3B-Q4_K_S.gguf -p "Your prompt here" -n 512 -t 8 -c 4096