Build a verifier that never ties: continuous scoring from LLM logprobs

TL;DR: Standard LLM judges collapse scoring to a single token, producing ties 27% of the time on Terminal-Bench. A verifier that reads the full logprob distribution and computes a continuous score eliminates ties entirely, improves accuracy by 3-5 points, and gives you three tuning knobs. Granularity, repeated evaluation, and criteria decomposition. No training required. Works with any logprob-exposing model including local ones.

Key takeaways:

• A judge picks one token. A verifier reads all token probabilities and computes a weighted average. The difference is zero ties
• You already have access to logprobs. OpenAI, DeepSeek, Gemini, and every local model through vLLM or llama.cpp expose them
• For models that block logprobs (Claude, GPT-5.5), a two-stage pipeline recovers most of the gain: use the closed model for reasoning, an open model for scoring
• Three scaling axes. Score granularity, repeated evaluations, criteria decomposition. Each improves accuracy independently
• Terminal-Bench V2: 86.5% SOTA. SWE-Bench Verified: 78.2%. Zero ties across all comparisons

I shipped an agent evaluator that tied 88 times out of 100.

It was a simple pairwise comparison. I asked GPT-4o-mini to judge two code-editing trajectories. Same task, two different approaches. One verified its output against the database. The other didn’t. Any human engineer could spot the difference in seconds.

The judge gave both a 5 out of 5. Tie. I ran it 100 times. 88 ties. The judge could tell there was a difference. It just could not express one.

Here is how it works and why you should build a verifier instead.

What makes a judge tie so often?

A standard LLM judge works like this: you prompt the model to output a score in a fixed range.

Rate this trajectory from 1 to 5.
Trajectory: [agent run]
Score: 4

The model picks one token. In this case the token 4. That is your score. The problem is that two different trajectories often produce the same token. Both get a 4. Both get a 5. You have no way to tell them apart.

The authors of LLM-as-a-Verifier ran the numbers. On Terminal-Bench V2, the discrete judge produced ties in 26.7% of comparisons at a single evaluation. Some tasks were worse. In the query-optimize case study I described above, two trajectories that differed in a critical verification step both got the same 5/5 from the judge. Tie rate: 88%.

This isn’t a model quality problem. It is a measurement resolution problem. Discrete scoring collapses the model’s internal belief into a single bucket. Two beliefs that sit in different parts of the same bucket look identical.

How does a verifier avoid ties?

A verifier doesn’t pick one token. It reads the probability distribution over all score tokens and computes a weighted average.

Here is the difference. When you ask the model to score a trajectory on a 1-20 scale, the model’s forward pass computes a probability for every token in the vocabulary. The probability for each score token reflects how strongly the model believes that score applies.

A judge reads only the highest-probability token:

score = argmax P(token)
score = 14  (just the single token)

A verifier reads all of them and computes the expectation:

score = Σ P(token_i) × value(token_i)
score = (0.45 × 14) + (0.30 × 15) + (0.15 × 13) + ...
score = 14.37  (a continuous number)

Two trajectories that both map to token 14 will almost certainly have different distributions around it. One trajectory gets 14.37. The other gets 13.92. No tie. The verifier always picks a winner.

The catch: you need access to token logprobs. Not every API exposes them. But enough do.

Which models expose output logprobs?

Model family	Logprobs available?	How
OpenAI (GPT-4o, GPT-4)	Yes	logprobs=True parameter
Google Gemini	Yes	Default in API
DeepSeek	Yes	logprobs=True
Any open model via vLLM	Yes	--logprobs N flag
Any local model via llama.cpp	Yes	Default in output
Together AI, Fireworks	Yes	Supported
Anthropic Claude	No	Doesn’t expose
GPT-5.5	No	Blocks logprobs

If you are using a model that blocks logprobs, the paper has a two-stage workaround. First, prompt the closed model to produce a detailed reasoning trace and a discrete score. Then forward that reasoning to a logprob-exposing model. Gemini 2.5 Flash or any open model. And read the continuous scores from there. The cheap model works because scoring is easier than generating.

Results on Terminal-Bench V2 with this workaround:

Method	Accuracy	Tie rate
GPT-5.5 discrete scores only	74.9%	10.9%
GPT-5.5 reasoning → Gemini verifier	80.1%	0.0%

Why it works: the closed model contributes domain-specific reasoning quality. The open model provides the calibrated probability distribution. You get the best of both.

How do I actually build the verifier?

The core implementation is about 20 lines. Here is the pattern.

Step 1: Get the logprobs

For OpenAI-compatible APIs, include logprobs=True and top_logprobs=N in your request:

from openai import OpenAI

client = OpenAI()

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "Rate this agent trajectory from 1 to 20. ..."},
        {"role": "user", "content": trajectory_text}
    ],
    logprobs=True,
    top_logprobs=20
)

For local models via vLLM or llama.cpp, the same parameters work with any OpenAI-compatible endpoint.

Step 2: Extract the score distribution

The logprobs for each output token include the probability the model assigned to that token. You filter for the score tokens (1 through 20) and collect their probabilities:

def extract_score_distribution(response, score_range=range(1, 21)):
    content = response.choices[0].message.content
    logprobs = response.choices[0].logprobs.content
    
    distribution = {}
    for score in score_range:
        score_str = str(score)
        for token_logprob in logprobs:
            if token_logprob.token.strip() == score_str:
                distribution[score] = math.exp(token_logprob.logprob)
                break
    
    return distribution

Step 3: Compute the continuous score

def continuous_score(distribution):
    total_prob = sum(distribution.values())
    if total_prob == 0:
        return 0.0
    weighted = sum(score * prob for score, prob in distribution.items())
    return weighted / total_prob

That is the verifier. Two trajectories, two continuous scores, always a winner.

Step 4: Use it for selection

When you have N candidate trajectories from parallel sampling, score all of them and pick the highest:

candidates = [traj1, traj2, traj3, traj4, traj5]
scores = [continuous_score(extract_score_distribution(verify(t))) for t in candidates]
best_trajectory = candidates[argmax(scores)]

This is the concurrent inference pattern from the diagram earlier. Run N attempts in parallel with temperature > 0, then let the verifier pick the best one. The paper calls this the Probabilistic Pivot Tournament and it scales efficiently to large candidate pools.

What are the three scaling knobs?

The verifier’s accuracy improves along three independent dimensions. You tune them based on your latency budget.

1. Score granularity (k)

The number of score tokens you read. Granularity=1 is a judge (just the top token). Granularity=20 reads all score token probabilities. The signal-to-noise ratio improves from 0.775 at k=1 to 0.799 at k=20. Accuracy on Terminal-Bench V2 rises from 73.1% to 77.5%.

2. Repeated evaluations (g)

Average the verifier’s score across multiple independent evaluations. This reduces variance. Accuracy improves from 74.7% at g=1 to 77.5% at g=16. The catch: gains diminish past g=8.

3. Criteria decomposition (c)

Instead of one monolithic correctness rubric, decompose into sub-criteria and average. For code tasks, the paper uses three criteria: Specification (meets all requirements), Output (correct format), and Errors (no failure signals in logs). The ensemble (78.3%) outperforms any single criterion (75.2%-76.4%).

Combine all three and you get the headline results: 86.5% on Terminal-Bench V2, 78.2% on SWE-Bench Verified, zero ties.

Where do I plug this into my agent system?

Three places. Each uses the same verifier function but serves a different purpose.

Test-time selection

Generate N candidate trajectories per task. Score each. Submit the best. This is the paper’s primary use case and delivers the largest accuracy gain.

Production gating

After an agent finishes a task, run the verifier. If the score falls below a threshold, reject the output and trigger a retry or a human review. No second-guessing by hand.

Progress monitoring

Run the verifier periodically during a long agent run. The score correlates with task progress. It rises naturally as the agent gets closer to completion. A flat or dropping score means the agent is off track. Stop early, log the failure mode, retry.

The paper ships a proxy called TurboAgent that sits between your agent and the LLM provider and does this automatically. It is early stage but the pattern is clear: verification as a continuous background process, not a one-time judgment call.

What about reinforcement learning?

The verifier’s continuous score works as a dense reward signal for RL. Instead of waiting for a sparse pass/fail at the end of a rollout, you feed the verifier score at every step.

The paper tested this in two regimes:

• Off-policy (DSRL-SAC on LIBERO robotics): the verifier reward achieves the same success rate with 1.8x fewer environment steps and a higher final success rate (0.76 vs 0.69)
• On-policy (GRPO on MATH): the verifier reward improves sample efficiency by 1.1x

The mechanism is straightforward: during early training, when all sampled responses produce incorrect final answers, group-relative advantage collapses to zero. The verifier score differentiates responses based on the quality of their reasoning traces instead, generating a useful gradient where there was none.

FAQ

Do I need to fine-tune a model for the verifier? No. The verifier is training-free. It works with any LLM that exposes output logprobs. The same framework applies across coding, robotics, and medical domains without per-domain fine-tuning.

Does the verifier need a strong model? No. The paper uses Gemini 2.5 Flash as the verifier. A mid-range model. It consistently scores trajectories from GPT-5.5 and Opus 4.5. Scoring is easier than generating. A 7B-12B model running locally through Ollama is likely sufficient.

What if my model doesn’t expose logprobs? Use the two-stage workaround: let your primary model produce free-form reasoning about the trajectory, then forward that reasoning to a logprob-exposing model for scoring. The paper recovers a +5.2-point accuracy gain with this approach.

How do I choose the granularity, repeats, and criteria for my use case? Start with granularity=20, repeats=8, and a single criterion. That gives you most of the benefit. Add criteria decomposition if you need more separation. Reduce repeats if latency is a concern. The single-pass verifier (g=1) already beats the discrete judge baseline.

Is this compatible with best-of-N sampling? Yes. Best-of-N with a continuous verifier is exactly what the paper evaluates. Generate N candidates, score with the verifier, pick the best. The Probabilistic Pivot Tournament algorithm makes this efficient for large N.

Related Posts

• Your LLM judge flips a coin 13.6% of the time. The problem this verifier solves: LLM judges tie 13.6% of the time even between identical runs.
• When a ‘worse’ model beats a frontier model for agent work. Why evaluation signal matters more than benchmark scores for agent selection.
• Your agent passed the test. It still used skills wrong.. How SkillCoach uses multi-dimensional rubrics to catch process failures that final scores miss.

---

This article was published on Agentic Up (https://agenticup.dev): practical guides for developers and founders building with AI agents. Reach me at [email protected]