KeyError: "bloom"
#4
by yongzx - opened
Currently, I think the transformers library hasn't support the bloom model. Our multilingual modeling group is starting to finetune it on new languages, so I wonder if there's any resource/pointer to adding model support into transformers.
I did a quick run through the HF's documentation but couldn't find any, and before I start looking at GitHub commits, I decide to post the question here.
>>> model = transformers.AutoModel.from_pretrained("bigscience/bloom-1b3", cache_dir="/users/zyong2/data/zyong2/huggingface/", use_auth_token=True)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/gpfs/data/sbach/zyong2/bigscience/env_lang_adapter/lib/python3.7/site-packages/transformers/models/auto/auto_factory.py", line 425, in from_pretrained
pretrained_model_name_or_path, return_unused_kwargs=True, trust_remote_code=trust_remote_code, **kwargs
File "/gpfs/data/sbach/zyong2/bigscience/env_lang_adapter/lib/python3.7/site-packages/transformers/models/auto/configuration_auto.py", line 657, in from_pretrained
config_class = CONFIG_MAPPING[config_dict["model_type"]]
File "/gpfs/data/sbach/zyong2/bigscience/env_lang_adapter/lib/python3.7/site-packages/transformers/models/auto/configuration_auto.py", line 372, in __getitem__
raise KeyError(key)
KeyError: 'bloom'
The BLOOM class is not yet merged into main. You'll have to checkout this PR: https://github.com/huggingface/transformers/pull/17202
Sorry, this PR: https://github.com/huggingface/transformers/pull/17474 (The previous one was moved into this one)
Gotcha! I am reading through the PR right now.