error when loading the model on text generation web-ui
#1
by aminedjeghri - opened
AttributeError: 'LlamaCppModel' object has no attribute 'model'
llama.cpp: loading model from models\starcoder\starcoder.ggmlv3.q4_1.bin
error loading model: missing tok_embeddings.weight
llama_init_from_file: failed to load model
Traceback (most recent call last):
File "C:\Users\AmineDjeghri\Downloads\oobabooga_windows\oobabooga_windows\text-generation-webui\server.py", line 1079, in <module>
shared.model, shared.tokenizer = load_model(shared.model_name)
File "C:\Users\AmineDjeghri\Downloads\oobabooga_windows\oobabooga_windows\text-generation-webui\modules\models.py", line 94, in load_model
output = load_func(model_name)
File "C:\Users\AmineDjeghri\Downloads\oobabooga_windows\oobabooga_windows\text-generation-webui\modules\models.py", line 271, in llamacpp_loader
model, tokenizer = LlamaCppModel.from_pretrained(model_file)
File "C:\Users\AmineDjeghri\Downloads\oobabooga_windows\oobabooga_windows\text-generation-webui\modules\llamacpp_model.py", line 49, in from_pretrained
self.model = Llama(**params)
File "C:\Users\AmineDjeghri\Downloads\oobabooga_windows\oobabooga_windows\installer_files\env\lib\site-packages\llama_cpp\llama.py", line 197, in __init__
assert self.ctx is not None
AssertionError
Exception ignored in: <function LlamaCppModel.__del__ at 0x0000018E453C2A70>
Traceback (most recent call last):
File "C:\Users\AmineDjeghri\Downloads\oobabooga_windows\oobabooga_windows\text-generation-webui\modules\llamacpp_model.py", line 23, in __del__
self.model.__del__()
AttributeError: 'LlamaCppModel' object has no attribute 'model'
Yes. Please see the README for supported clients/libraries. text-generation-ui can not load it at this time.
Thanks !
from the readme :
These files are not compatible with llama.cpp.
Currently they can be used with:
- KoboldCpp, a powerful inference engine based on llama.cpp, with good UI: KoboldCpp
- The ctransformers Python library, which includes LangChain support: ctransformers
- The GPT4All-UI which uses ctransformers: GPT4All-UI
- rustformers' llm
aminedjeghri changed discussion status to closed
You say it works with KoboldCpp but it does not.. gives an error which someone pointed out and you then say you need to use https://github.com/ggerganov/llama.cpp.git which is not even compiled for Windows.