Sequence to sequence pretraining for a less-resourced Slovenian language
Paper • 2207.13988 • Published
t5-sl-large
t5-sl-large model is a Slovene T5 model. It has 24 encoder and 24 decoder layers, in total about 750 million parameters. It was trained for 3 epochs on the following corpora:
Corpora
The following corpora were used for training the model:
- Gigafida 2.0
- Kas 1.0
- Janes 1.0 (only Janes-news, Janes-forum, Janes-blog, Janes-wiki subcorpora)
- Slovenian parliamentary corpus siParl 2.0
- slWaC
Evaluation
The model is described in detail and evaluated in our paper "Sequence to sequence pretraining for a less-resourced Slovenian language"
- Downloads last month
- 4
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support