This is an experimental quantization of CohereLab's BLS-Mini-Code-1.0, a 30B A3B MoE model, and is still in development.
I am not affiliated with CohereLabs in any way and this is my own independent work.
To use these GGUFs with llama.cpp, you must install the llama.cpp PR #24260 for the Cohere2-MoE :

git fetch origin pull/24260/head:cohere2-moe
git checkout cohere2-moe

OR

gh pr checkout 24260
Downloads last month
436
GGUF
Model size
30B params
Architecture
cohere2moe
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for michaelw9999/BLS-Mini-Code-1.0

Quantized
(5)
this model