Instructions to use PygmalionAI/pygmalion-6b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use PygmalionAI/pygmalion-6b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="PygmalionAI/pygmalion-6b") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("PygmalionAI/pygmalion-6b") model = AutoModelForCausalLM.from_pretrained("PygmalionAI/pygmalion-6b") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use PygmalionAI/pygmalion-6b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "PygmalionAI/pygmalion-6b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "PygmalionAI/pygmalion-6b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/PygmalionAI/pygmalion-6b
- SGLang
How to use PygmalionAI/pygmalion-6b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "PygmalionAI/pygmalion-6b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "PygmalionAI/pygmalion-6b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "PygmalionAI/pygmalion-6b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "PygmalionAI/pygmalion-6b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use PygmalionAI/pygmalion-6b with Docker Model Runner:
docker model run hf.co/PygmalionAI/pygmalion-6b
Can character be a male?
I am trying to create male characters. But in output still points that "she" with the character.
She is a she. Maybe re-code? Re-train and remove identity?
Yes, it is possible. There are too many potential causes for this kind of issue to really go into detail... Just some simple suggestions: be careful about how you describe {{user}} as the AI could get confused about who you are talking about. It also helps to reinforce traits if they're not sticking (though generally you shouldn't have to do this for the sex of the bot), but for example, you could describe {{char}} as a man, and later describe him as masculine, and then also talk about physical features that define him as a man. Finally, make sure you're following one of the recommended formats for describing bots.
If you're using Ali:Chat format to describe your bot, make sure your dialogue examples make it clear that {{char}} is male. Definitely read your bot definition again and carefully consider if anything in there could potentially be confusing the AI. Also avoid using different ways to talk about the same character, in other words, avoid interchangeably using 'they', 'he', 'their', 'sam' (or whatever his name is), decide on a convention and stick to it. Also, start small, don't write an essay, less is often more when it comes to AI.
Also be aware of how the example dialog section works, you should have lots of small examples separated by START tokens as they disappear one example block at a time as text is added to the context. So if you just have one giant block of example text, the entire block will disappear and then you'll have no example dialog.