AACL-IJCNLP 2026   /   Long paper

PAIRSAFE

How many matched pairs can one refusal threshold separate?

Wenzhang Du  ·  150 XSTest pairs  ·  2 models

Explore one shared threshold

Move the boundary across the archived scores. A pair counts when its safe member falls at or below the threshold and its unsafe member falls above it.

Loading the verified threshold counts…

Published results

Primary 2+2 candidate margin, 150 pairs per model
ModelOrderingBest thresholdHeld-out
Qwen2.5-3B148 / 150133 / 150132 / 150
SmolLM2-1.7B146 / 150120 / 150114 / 150

In the separate Qwen generation check, 105 of 150 pairs received the expected labels on both members.