Trying to deploy this model with vllm in Sagemaker
#2
by aastha6 - opened
I have a custom container built like so -
FROM deepjavalibrary/djl-serving:deepspeed-nightly
RUN apt-get update -y && \
DEBIAN_FRONTEND=noninteractive apt-get install -y git && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
RUN pip3 install --no-cache-dir \
"git+https://github.com/huggingface/transformers"
Pushed the image to ECR and referring the image_uri while creating the endpoint in Sagemaker.
Getting this error:
[WARN ] PyProcess - W-133-model-stderr: KeyError: 'transformer.h.0.mlp.dense_4h_to_h.bias'
How should I solve this issue ?