AACL-IJCNLP 2026 / Long paper
PAIRSAFE
How many matched pairs can one refusal threshold separate?
Wenzhang Du · 150 XSTest pairs · 2 models
Explore one shared threshold
Move the boundary across the archived scores. A pair counts when its safe member falls at or below the threshold and its unsafe member falls above it.
Loading the verified threshold counts…
- Both members correct
- Safe members accepted
- Unsafe members refused
The slider uses one threshold on all 150 pairs. The published held-out metric fits five thresholds on separate training folds.
Published results
| Model | Ordering | Best threshold | Held-out |
|---|---|---|---|
| Qwen2.5-3B | 148 / 150 | 133 / 150 | 132 / 150 |
| SmolLM2-1.7B | 146 / 150 | 120 / 150 | 114 / 150 |
In the separate Qwen generation check, 105 of 150 pairs received the expected labels on both members.