qwen2-Alarm-json

Authors

• Lucía Herraiz Cano
• Álvaro Pérez Ortega
• Beatriz Otero Casanovas
• Alberto Prieto González

Universidad Pontificia Comillas — ICAI

License

MIT

Fine-tuned version of Qwen/Qwen2.5-7B-Instruct for news alert classification.

Given a news article with its image caption, named entities (NER) and sentiment analysis, the model determines whether an alert should be triggered and classifies it by level and category. Output is always a structured JSON object.

Model details

Base model	Qwen/Qwen2.5-7B-Instruct
Fine-tuning method	QLoRA (4-bit NF4 quantization, r=16, alpha=32)
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Training data	977 news articles labelled by Gemini 2.0 Flash
Train / Val / Test split	80% / 10% / 10%
Epochs	2
Batch size	4
Learning rate	5e-5
Hardware	NVIDIA H200 (141 GB VRAM)

Output format

The model always responds with a single JSON object:

{
  "is_alert": true,
  "alert_level": "info",
  "category": "geopolitical",
  "reason": "One-sentence explanation of why the alert was triggered."
}

alert_level — "critical" · "info" · null

category — "brand_safety" · "legal_compliance" · "operational_risk" · "financial_market" · "geopolitical" · "esg_controversy" · "executive_risk" · "reputation_risk" · null

Benchmark results

Evaluated on 146 held-out test examples (same split used for training).

Binary classification (alert vs no_alert)

Model	Accuracy
Fine-tuned (this model)	0.8219
Base (Qwen2.5-7B-Instruct)	0.7945

	Precision	Recall	F1
alert	0.926	0.694	0.794
no_alert	0.761	0.946	0.843

Field classification (on alert examples only, n=72)

Field	Fine-tuned	Base
alert_level accuracy	0.375	0.444
category accuracy	0.611	0.583

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "beaotero05/qwen2-Alarm-json"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)

messages = [
    {
        "role": "system",
        "content": (
            "You are a news alert classifier. "
            "Given a news article with its image caption, named entities, and sentiment, "
            "determine if an alert should be triggered. "
            "Always respond with a single valid JSON object and nothing else."
        )
    },
    {
        "role": "user",
        "content": """Article: <article text>

Image caption: <caption>

Named entities (NER): <entities>

Sentiment: <sentiment>

Return ONLY a JSON object with this exact structure:
{
  "is_alert": <true | false>,
  "alert_level": <"critical" | "info" | null>,
  "category": <"brand_safety" | "legal_compliance" | "operational_risk" | "financial_market" | "geopolitical" | "esg_controversy" | "executive_risk" | "reputation_risk" | null>,
  "reason": <"one-sentence explanation" | null>
}"""
    }
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=350, do_sample=False)

response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)