qwen3.6-35b-a3b-unslop-good-lora-v1

• Developed by: Oysiyl
• Base model: Qwen/Qwen3.6-35B-A3B
• Dataset: N8Programs/unslop-good (train split, 1000 rows)
• Training stack: TRL SFTTrainer + PEFT LoRA + 4-bit QLoRA base loading (nf4)
• Output type: LoRA adapter weights (not merged full model)

Training configuration (run: 69e379baac288e522d8efce2)

• max_seq_length: 4096
• epochs: 2
• max_steps: 500
• batch_size: 1
• grad_accumulation: 4
• effective_batch_size: 4
• learning_rate: 1e-4
• scheduler: cosine
• warmup_steps: 10
• weight_decay: 0.01
• precision: bf16 (COMPUTE_DTYPE=bf16)
• optimizer: paged_adamw_8bit
• LoRA: r=8, alpha=20, dropout=0.0
• LoRA target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• thinking mode in chat template: disabled (enable_thinking=false)

Training loss vs training progress

• observed points: 500
• first loss: 1.9088 (step 1)
• last loss: 1.7068 (step 500)
• min logged loss: 0.8096
• final train_loss (trainer summary): 1.6124

Live evaluation setup (Modal endpoint)

Evaluation was run against the deployed live endpoint right before this README update:

• endpoint: https://dmitriy-kisil--qwen3-6-35b-a3b-unslop-api.modal.run/rewrite
• model_id returned by endpoint: Oysiyl/qwen3.6-35b-a3b-unslop-good-lora-v1
• decoding params (Qwen3.6 instruct / non-thinking preset):
  • max_new_tokens=700
  • temperature=0.7
  • top_p=0.8
  • top_k=20
  • min_p=0.0
  • presence_penalty=1.5
  • repetition_penalty=1.0
  • do_sample=true
  • chat_template_kwargs={"enable_thinking": false}

Recommended evaluation sample

Use this held-out fiction passage prompt:

Rewrite this passage in cleaner, more natural prose while preserving meaning and tone. Return only the rewritten passage.

The wind rose all at once, as if the mountain itself had inhaled. She stood at the edge of the pass, cloak snapping like a torn banner behind her, staring down at the valley where the last lights were going out one by one. Each flame extinguished felt deliberate—an answer to a question she had been too afraid to ask. Somewhere below, bells began to ring, not in warning, but in mourning. They already knew. “Don’t,” he said, breathless, reaching for her arm. His fingers brushed fabric, then air. “If you cross that line, there’s no coming back.” She didn’t turn. The cold burned her lungs, sharpened her thoughts. “There was never a way back,” she replied softly. “Only a way forward you were willing to pretend didn’t exist.” Thunder rolled, low and ancient. For a moment, the sky fractured into white fire, and in that instant he saw her face—resolved, terrified, incandescent with purpose. Not the girl he had sworn to protect, but the woman the world had demanded she become. “You’ll die,” he said. She smiled then, finally facing him, eyes bright with unshed tears. “So will everyone,” she answered. “The difference is what we leave burning when we’re gone.” She stepped forward. The mountain exhaled.

Full observed output on that sample (new preset)

The wind surged suddenly, as though the mountain itself had drawn a sharp breath. She stood at the edge of the pass, her cloak whipping behind her like a tattered banner, staring down into the valley where the last lights were flickering out one by one. Each extinguished flame felt deliberate—a response to a question she’d been too afraid to voice. Below, bells began to toll, not in warning, but in mourning. They already knew.

"Don't," he gasped, lunging for her arm. His fingers grazed fabric, then caught nothing but air. "If you cross that line, there’s no coming back."

She didn’t turn. The cold bit into her lungs, sharpening her thoughts. "There was never a way back," she said softly. "Only a path forward you refused to see."

Thunder rumbled, deep and ancient. For a heartbeat, the sky split with white fire, and in that flash, he saw her face—resolved, terrified, blazing with purpose. This wasn’t the girl he had sworn to protect, but the woman the world had forced her to become.

"You’ll die," he whispered.

She smiled, finally turning to face him, her eyes bright with unshed tears. "So will everyone else. The difference is what we leave burning when we’re gone."

She stepped forward.

The mountain exhaled.