Instructions to use OpenNLPLab/TransNormerLLM3-15B-Intermediate-Checkpoints with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OpenNLPLab/TransNormerLLM3-15B-Intermediate-Checkpoints with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="OpenNLPLab/TransNormerLLM3-15B-Intermediate-Checkpoints", trust_remote_code=True)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("OpenNLPLab/TransNormerLLM3-15B-Intermediate-Checkpoints", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use OpenNLPLab/TransNormerLLM3-15B-Intermediate-Checkpoints with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OpenNLPLab/TransNormerLLM3-15B-Intermediate-Checkpoints"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenNLPLab/TransNormerLLM3-15B-Intermediate-Checkpoints",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/OpenNLPLab/TransNormerLLM3-15B-Intermediate-Checkpoints

SGLang

How to use OpenNLPLab/TransNormerLLM3-15B-Intermediate-Checkpoints with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OpenNLPLab/TransNormerLLM3-15B-Intermediate-Checkpoints" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenNLPLab/TransNormerLLM3-15B-Intermediate-Checkpoints",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OpenNLPLab/TransNormerLLM3-15B-Intermediate-Checkpoints" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenNLPLab/TransNormerLLM3-15B-Intermediate-Checkpoints",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use OpenNLPLab/TransNormerLLM3-15B-Intermediate-Checkpoints with Docker Model Runner:
```
docker model run hf.co/OpenNLPLab/TransNormerLLM3-15B-Intermediate-Checkpoints
```

TransNormerLLM3-15B-Intermediate-Checkpoints

Commit History

Update README.md

4dd5be2
verified

OpenNLPLab commited on Apr 7, 2024

Update README.md

39c0658
verified

OpenNLPLab commited on Apr 7, 2024

Update README.md

f3bfc9d
verified

OpenNLPLab commited on Apr 7, 2024

Update README.md

49235ee
verified

OpenNLPLab commited on Apr 7, 2024

Update README.md

0e06ad0
verified

OpenNLPLab commited on Apr 7, 2024

Update README.md

9600986
verified

OpenNLPLab commited on Mar 8, 2024

Update README.md

60b0f0a
verified

OpenNLPLab commited on Feb 19, 2024

Update README.md

60c48a0
verified

OpenNLPLab commited on Feb 19, 2024

Update modeling_transnormer.py

b9d0948

Xuyang Shen commited on Feb 18, 2024

Update README.md

201dd8a
verified

OpenNLPLab commited on Feb 2, 2024

Upload 2 files

b71c38e
verified

OpenNLPLab commited on Jan 23, 2024

Delete images/images_lightning3-leopard.jpg

b56bd2d
verified

OpenNLPLab commited on Jan 23, 2024

Delete images/images_TransNormer3.jpg

81df6ee
verified

OpenNLPLab commited on Jan 23, 2024

Resume images

0e62da8
verified

OpenNLPLab commited on Jan 23, 2024

Delete images

4986740
verified

OpenNLPLab commited on Jan 23, 2024

Update discord link

9d4a4d9
verified

OpenNLPLab commited on Jan 23, 2024

Update README.md

7af1936
verified

OpenNLPLab commited on Jan 23, 2024

Publish step26000-100Btokens

8fd5614
verified

OpenNLPLab commited on Jan 12, 2024

Publish 100B ckpt

4a8c07f
verified

OpenNLPLab commited on Jan 12, 2024

Update README.md

8499a6e
verified

OpenNLPLab commited on Jan 11, 2024

Update README.md

590b4b9
verified

OpenNLPLab commited on Jan 11, 2024

Upload lightning3-leopard.jpg

9e5c19e
verified

OpenNLPLab commited on Jan 11, 2024

Delete images/lightning3-leopard.png

9d04b06
verified

OpenNLPLab commited on Jan 11, 2024

Upload lightning3-leopard.png

0fad114
verified

OpenNLPLab commited on Jan 11, 2024