Instructions to use zss01/BiPS with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use zss01/BiPS with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="zss01/BiPS")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("zss01/BiPS")
model = AutoModelForMultimodalLM.from_pretrained("zss01/BiPS")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use zss01/BiPS with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "zss01/BiPS"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zss01/BiPS",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/zss01/BiPS

SGLang

How to use zss01/BiPS with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "zss01/BiPS" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zss01/BiPS",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "zss01/BiPS" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zss01/BiPS",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use zss01/BiPS with Docker Model Runner:
```
docker model run hf.co/zss01/BiPS
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

BiPS — Bi-directional Perceptual Shaping for Multimodal Reasoning

This model card describes BiPS (Bi-directional Perceptual Shaping), a training-time framework proposed in “See Less, See Right: Bi-directional Perceptual Shaping For Multimodal Reasoning” [CVPR 2026].

Paper: https://arxiv.org/abs/2512.22120
Code: https://github.com/zss02/BiPS

What is BiPS?

Many VLMs fail on multimodal reasoning because they look at the wrong visual evidence (especially for charts, thin lines, intersections, and small regions). BiPS transforms visual cues into training-time policy constraints by constructing evidence-preserving and evidence-ablated views, enabling the model to internalize fine-grained perception without inference-time overhead. This formulation can also be interpreted from the perspective of visual on-policy distillation (OPD) / on-policy self-distillation, where information-asymmetric visual views provide dense policy-shaping signals for GRPO training.

Key idea

BiPS trains a VLM with two complementary view transformations:

Evidence-Preserving View: keep only the visual evidence needed to answer, reduce distractions.
→ enforce consistency between predictions from the original image and the preserved view.
Evidence-Ablated View: remove the key evidence so the image no longer supports the answer.
→ enforce separation so the model cannot rely on shortcuts.

These constraints are typically implemented with KL-based objectives and can be integrated into GRPO training.

Why it matters

Better fine-grained evidence alignment
Less “guessing” from language priors
No additional inference overhead (views are used only during training)

How to use

BiPS is mainly a training recipe. To apply it:

Follow the official repo to set up dependencies and scripts.
Train your base VLM with BiPS-generated preserve/ablate views.
Use the resulting checkpoint as a standard VLM at inference time (no extra steps).

Citation

@article{zhang2025bips,
  title={See Less, See Right: Bi-directional Perceptual Shaping For Multimodal Reasoning},
  author={Zhang, Shuoshuo and Zhang, Yizhen and Fu, Jingjing and Song, Lei and Bian, Jiang and Yang, Yujiu and Wang, Rui},
  journal={arXiv preprint arXiv:2512.22120},
  year={2025}
}