| --- |
| '[object Object]': null |
| license: mit |
| tags: |
| - audio |
| - deep-learning |
| - pytorch |
| - generative-adversarial-network |
| - codec |
| - gans |
| - compression-algorithm |
| - audio-compression |
| - RVQ |
| --- |
| |
|
|
| # Descript Audio Codec |
|
|
| 👉 With Descript Audio Codec, you can compress **44.1 KHz audio** into discrete codes at a **low 8 kbps bitrate**. <br> |
| 🤌 That's approximately **90x compression** while maintaining exceptional fidelity and minimizing artifacts. <br> |
| 💪 Our universal model works on all domains (speech, environment, music, etc.), making it widely applicable to generative modeling of all audio. <br> |
| 👌 It can be used as a drop-in replacement for EnCodec for all audio language modeling applications (such as AudioLMs, MusicLMs, MusicGen, etc.) <br> |
|
|
| ## Model Details |
|
|
| ### Model Description |
|
|
| - **License:** MIT |
|
|
| ### Model Sources |
|
|
| - **Repository:** [Github Repo](https://github.com/descriptinc/descript-audio-codec) |
| - **Paper:** [arXiv Paper: High-Fidelity Audio Compression with Improved RVQGAN |
| ](http://arxiv.org/abs/2306.06546) |
| - **Demo:** [Demo Site](https://descript.notion.site/Descript-Audio-Codec-11389fce0ce2419891d6591a68f814d5) |
|
|
| ## Uses |
|
|
| The model is intended for compressing audio files containing speech, music and environmental sounds. |
|
|
| ### Out-of-Scope Use |
|
|
| It is not intended to be used for compressing other file formats such as text, images, etc. |
|
|
| ## Bias, Risks, and Limitations |
| Our model has difficulty reconstructing some challenging audio. It |
| performs best for speech and has more issues with environmental sounds. It |
| does not model some musical instruments perfectly, such as glockenspeil, or synthesizer sounds. |
|
|
|
|
| ## How to Get Started with the Model |
| This model is meant to be used with our official repo linked above. We release the model here for redundancy purposes. |
| Our code is able to pull the weights from their |
| [original location on Github](https://github.com/descriptinc/descript-audio-codec/releases/download/0.0.1/weights.pth). |
| Please refer to the official [README](https://github.com/descriptinc/descript-audio-codec#readme) for usage instructions. |
|
|
| ## Citation |
|
|
| **BibTeX:** |
|
|
| ``` |
| @misc{kumar2023highfidelity, |
| title={High-Fidelity Audio Compression with Improved RVQGAN}, |
| author={Rithesh Kumar and Prem Seetharaman and Alejandro Luebs and Ishaan Kumar and Kundan Kumar}, |
| year={2023}, |
| eprint={2306.06546}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.SD} |
| } |
| ``` |
