Will nvfp4 be coming?
#11
by Ozzkozz - opened
Would you please add the nvfp4 model?
Hello! I’d like to suggest adding a quantized version of the model to the project using paroquant (https://github.com/z-lab/paroquant). This would greatly reduce model size and make it easier to run on CPU / small GPUs. Thanks!
Hello! I’d like to suggest adding a quantized version of the model to the project using paroquant (https://github.com/z-lab/paroquant). This would greatly reduce model size and make it easier to run on CPU / small GPUs. Thanks!
Unfortunately, I don’t have enough knowledge of software or computers to do it myself