Absolute Rating Format

An annotation task format asking annotators to score each response independently on a numeric scale (e.g., 1-10) or with thumbs-up/down signals.

Responsibility. Elicits per-response scalar quality scores.

Also known as: Explicit scalar reward, Likert rating

Variant of Annotation Task Format abstract

When to choose. Used in early RLHF work; generally avoid for preference learning because annotators interpret scales inconsistently across people and over time.