Claude Haiku 5.5 Benchmarks: Can a Tiny AI Model Really Beat GPT-6 Luna?

Anthropic has released Claude Haiku 5.5, and its benchmark results challenge the idea that smaller AI models must deliver significantly weaker performance.

Announced on October 7, 2026, Haiku 5.5 combines a one-million-token context window, adaptive reasoning, and API pricing starting at just $0.10 per million input tokens.

The biggest surprise comes from its performance against competing models. In Anthropic's published benchmarks, Haiku 5.5 outperforms GPT-6 Luna across several computer-use, coding, and professional-work evaluations. It also narrows the performance gap with the more expensive Claude Sonnet 5.5.

However, higher benchmark scores do not automatically mean better real-world performance or lower operating costs. Haiku 5.5 introduces a new tokenizer, adaptive thinking, and a pricing threshold that can dramatically change the cost of long-context requests.

This guide breaks down the official benchmark results, explains what they mean, and examines whether Haiku 5.5 is the right choice for coding, AI agents, automation, and production applications.

Key Takeaways

• 72.4% on OSWorld 2.1: Haiku 5.5 dramatically improves computer-use performance compared with Haiku 4.5's 15.7%.
• 1,620 Elo on GDPval-AA v2.1: Anthropic reports a higher professional-work score than GPT-6 Luna's 1,437 Elo.
• 39.2% on Terminal-Bench 4.0: Agentic coding performance improves substantially, although Sonnet 5.5 remains stronger at 70.6%.
• Starting at $0.10 input and $0.50 output per million tokens: The lowest pricing applies to prompts of 100,000 tokens or fewer.
• 1M-token context window: The model supports large documents and extended conversations, but long prompts trigger higher prices.
• Adaptive thinking: Developers can adjust reasoning effort to balance quality, speed, and cost.

The main takeaway: Haiku 5.5 is a promising choice for high-volume AI workloads, structured extraction, browser automation, and specialized subagents. Larger models still offer important advantages for complex autonomous coding and demanding reasoning tasks.

What Is Claude Haiku 5.5?

Claude Haiku 5.5 is Anthropic's latest small AI model, positioned below Sonnet and Opus in the Claude family.

Its primary focus is delivering useful intelligence with lower latency and operating costs. Rather than targeting only the most difficult reasoning problems, Haiku 5.5 is optimized for applications that need to process large numbers of requests efficiently.

Typical use cases include customer support, information extraction, classification, document processing, tool calling, and lightweight AI agents.

According to Anthropic's official announcement, the model offers substantial performance improvements over Haiku 4.5 while reducing average operating costs by approximately 75% under Anthropic's workload assumptions.

Claude Haiku 5.5 Specifications

Haiku 5.5 is available through the Claude API and supported cloud platforms, including Amazon Bedrock, Google Cloud, and Microsoft Foundry.

Its combination of large context capacity and low starting prices makes it particularly interesting for developers building AI-powered applications at scale.

Claude Haiku 5.5 Benchmark Results: Full Comparison

Anthropic published benchmark comparisons against Haiku 4.5, GPT-6 Luna, and Sonnet 5.5 at launch.

The following table summarizes the official results.

Important: These are Anthropic-published benchmark results, not independently reproduced measurements. OSWorld uses an offline subset, Chartography is evaluated without tools, and Sonnet 5.5's FrontierCode score uses an Xhigh configuration. Differences in evaluation settings can affect direct comparisons.

The results reveal three important trends: major improvements in computer operation, stronger professional knowledge work, and significantly better agentic coding.

1. OSWorld 2.1: The Biggest Performance Breakthrough

The most impressive improvement appears in OSWorld 2.1, where Haiku 5.5 reaches 72.4%, compared with just 15.7% for Haiku 4.5.

OSWorld evaluates whether AI agents can operate computers to complete real tasks. Rather than answering questions directly, models must interact with graphical interfaces, navigate applications, and perform sequences of actions.

OSWorld 2.1 Results

Haiku 5.5 improves by 56.7 percentage points over Haiku 4.5 and leads GPT-6 Luna by 23.5 percentage points in the reported evaluation.

This is approximately a 4.6-fold increase over the previous Haiku model's score.

Why Does This Matter?

Computer-use agents have to do more than understand instructions. They must recognize interface elements, choose the correct actions, recover from mistakes, and verify the final result.

A small error early in a multi-step task can cause the entire workflow to fail.

For example, an AI assistant might need to open a business dashboard, locate a customer record, inspect recent transactions, and prepare a summary.

Such workflows require reliable interaction with an interface rather than simply generating text.

Haiku 5.5's stronger results suggest that smaller models are becoming more practical for browser automation, administrative tasks, and GUI-based AI agents.

However, a 72.4% benchmark result should not be interpreted as a guaranteed success rate on arbitrary websites. Actual performance depends on the interface, task complexity, tool implementation, and recovery mechanisms.

Practical recommendation: Evaluate Haiku 5.5 for browser automation and repetitive interface tasks, but require explicit authorization for sensitive actions such as purchases, account modifications, or data deletion.

2. GDPval-AA: Stronger Performance on Professional Work

Haiku 5.5 also makes a substantial improvement on GDPval-AA v2.1, which evaluates AI-generated professional work.

The model achieves 1,620 Elo, compared with 735 for Haiku 4.5 and 1,437 for GPT-6 Luna.

Sonnet 5.5 remains ahead with 1,840 Elo.

Unlike a simple accuracy percentage, an Elo-style rating represents relative performance within an evaluation system. It should not be interpreted as the percentage of professional tasks completed successfully.

A higher score indicates that the model performs more competitively on the evaluated work products.

What Does This Mean for Businesses?

Many business applications involve tasks that are more complicated than answering a short question but less demanding than building an entire software system.

Examples include:

• Summarizing long business reports.
• Extracting information from contracts.
• Comparing product specifications.
• Organizing research findings.
• Preparing structured operational summaries.
• Analyzing customer feedback.

Haiku 5.5's improvement makes it a stronger candidate for handling these tasks at scale.

The AA-Briefcase v1.1 benchmark supports this conclusion. Haiku 5.5 scores 1,578 Elo, compared with 614 for Haiku 4.5 and 1,336 for GPT-6 Luna.

The important insight is that small models are becoming increasingly capable of structured knowledge work, potentially reducing the need to use expensive frontier models for every routine business operation.

3. Coding Benchmarks: Can Haiku 5.5 Replace Sonnet?

Coding performance provides a clearer picture of Haiku 5.5's strengths and limitations.

On Terminal-Bench 4.0, Haiku 5.5 achieves 39.2%, compared with 16.4% for GPT-6 Luna and 70.6% for Sonnet 5.5.

Terminal-Bench evaluates AI systems on multi-step command-line tasks that can involve inspecting files, running commands, identifying errors, and verifying solutions.

The improvement over Haiku 4.5's reported 0.0% baseline is significant. However, that result is specific to the benchmark setup and does not mean the earlier model was incapable of writing useful code.

Terminal-Bench 4.0 Results

Haiku 5.5 trails Sonnet 5.5 by 31.4 percentage points.

That difference is important for developers considering which model to use in autonomous coding agents.

FrontierCode Shows a Smaller Gap

On FrontierCode 1.1, Haiku 5.5 scores 46.4%, while GPT-6 Luna scores 42.4% and Sonnet 5.5 scores 52.1%.

This suggests that the relative advantage of larger models depends heavily on the coding task and benchmark methodology.

Haiku 5.5 is worth evaluating for narrowly defined programming tasks such as generating unit tests, explaining code, extracting repository information, and making small changes from detailed specifications.

Sonnet 5.5 is a better starting point for complicated debugging sessions, repository-wide refactoring, difficult architectural decisions, and long autonomous workflows.

A Better Strategy: Use Multiple Models

A practical AI coding system does not need to assign every task to the same model.

A stronger model can handle planning, difficult reasoning, and final verification, while Haiku 5.5 processes simpler subtasks.

For example, a coding agent could use Sonnet to design a feature and identify affected files, then delegate test generation or code summarization to Haiku.

This approach can reduce costs while preserving stronger reasoning where it matters most.

4. Reasoning and Visual Understanding

Haiku 5.5 also improves considerably on Humanity's Last Exam and Chartography.

On Humanity's Last Exam, it scores 45.9% without tools and 57.4% with tools. Haiku 4.5 scores 10.2% and 18.7% under the corresponding conditions.

These results suggest stronger performance on difficult multidisciplinary questions, particularly when the model can use external tools.

On Chartography, Haiku 5.5 reaches 46.4%, compared with 6.4% for Haiku 4.5 and 29.1% for GPT-6 Luna.

Chartography evaluates visual reasoning involving charts and graphical information.

This matters for applications that analyze reports, presentations, dashboards, and data visualizations.

However, developers should continue validating extracted values against original documents. Better visual reasoning does not eliminate errors involving chart axes, labels, legends, or missing context.

Claude Haiku 5.5 vs GPT-6 Luna: Which Is Better?

The comparison between Haiku 5.5 and GPT-6 Luna is especially interesting because both compete in the low-cost AI market.

Anthropic's published results show Haiku 5.5 leading GPT-6 Luna on six shared benchmark categories, including OSWorld, GDPval-AA, AA-Briefcase, Terminal-Bench, FrontierCode, and Chartography.

But benchmark leadership does not establish that Haiku is the better choice for every application.

The models also have different long-context pricing structures.

API Pricing Comparison

These are standard API rates, excluding caching discounts and special processing tiers. OpenAI's rates are documented in its official pricing reference.

For short requests, both models offer the same published input and output token rates.

However, Haiku 5.5 becomes more expensive when a prompt exceeds 100,000 tokens. GPT-6 Luna uses a higher long-context threshold.

Example: A 150K-Token Request

Consider an application sending 150,000 input tokens and receiving 5,000 output tokens.

For Haiku 5.5:

• Input cost: $0.075.
• Output cost: $0.0125.
• Total: $0.0875.

For GPT-6 Luna, using its standard short-context rates:

• Input cost: $0.015.
• Output cost: $0.0025.
• Total: $0.0175.

For the same token counts, Haiku 5.5 costs five times as much in this example.

Actual application costs can differ because the models use different tokenizers and may generate different amounts of reasoning and output.

Recommendation: Haiku 5.5 is particularly attractive for computer-use and professional-work tasks reflected in the published evaluations. GPT-6 Luna deserves close consideration for long-context, cost-sensitive applications.

The final decision should depend on measured accuracy, latency, tool compatibility, and cost per successful result.

Claude Haiku 5.5 vs Sonnet 5.5: Is the Upgrade Worth It?

Sonnet 5.5 remains stronger than Haiku 5.5 across every benchmark included in Anthropic's launch comparison.

The difference is particularly significant on Terminal-Bench, where Sonnet scores 70.6% against Haiku's 39.2%.

But Sonnet also costs considerably more.

At the lowest pricing tier, Haiku 5.5 charges $0.10 per million input tokens and $0.50 per million output tokens. Sonnet 5.5 charges $2 and $10 respectively.

That represents a 20-fold difference in published base token rates.

Which Model Should You Use?

These recommendations are starting points rather than guarantees.

For many production applications, the most cost-effective design is a hybrid system that assigns simple tasks to Haiku and escalates difficult work to Sonnet.

Claude Haiku 5.5 API Pricing Explained

Haiku 5.5's advertised starting price is attractive, but understanding its pricing tiers is essential.

The model uses one set of rates for prompts up to 100,000 tokens and a higher set for longer prompts.

Official Pricing Table

All prices are in US dollars per million tokens.

The Batch API also provides a 50% discount on eligible input and output usage.

Current rates are available in the Claude Platform documentation.

Example: A Short Document Processing Request

Suppose an application processes 50,000 input tokens and generates 5,000 output tokens.

At the lower tier:

• Input cost: 50,000 × $0.10 / 1,000,000 = $0.005.
• Output cost: 5,000 × $0.50 / 1,000,000 = $0.0025.
• Total estimated cost: $0.0075.

For the same token counts, Haiku 4.5 would cost $0.075 using its standard $1 input and $5 output rates.

That is a 90% reduction in listed cost for identical token counts.

However, it does not guarantee a 90% reduction for identical text or workflows.

The Tokenizer Difference

Haiku 5.5 uses a newer tokenizer that can produce approximately 30% more tokens for the same text compared with Haiku 4.5.

This matters for both budgeting and context management.

A prompt that previously consumed 80,000 tokens might require around 104,000 tokens after migration if it experiences that approximate increase.

Such a change could push a request into Haiku 5.5's higher pricing tier.

Developers should recount prompts using the new model rather than relying on historical token usage.

Three Ways to Reduce Haiku 5.5 Costs

1. Use shorter prompts. Remove repeated instructions, irrelevant documents, and unnecessary conversation history.
2. Enable prompt caching. Reuse stable instructions and document prefixes when the workload supports it.
3. Use Batch processing for offline tasks. Large-scale tagging, extraction, and summarization jobs can benefit from discounted asynchronous processing.

For agentic workflows, also measure retries, tool calls, and failed attempts. Token prices alone do not reveal the total operating cost.

Adaptive Thinking: Choosing the Right Effort Level

Haiku 5.5 introduces adaptive thinking with configurable reasoning effort.

Instead of allocating a fixed manual thinking budget, developers can select an effort level that influences how much reasoning the model performs.

The default setting is medium.

Suggested Effort Levels

Higher effort can improve performance on difficult tasks, but it may also increase latency and token consumption.

For simple operations, lower effort may provide better economics without materially reducing quality.

The best effort setting should be selected through application-specific testing, not by automatically choosing the highest level.

How to Use Claude Haiku 5.5 With the API

Developers can access the model using the Anthropic Messages API and the model identifier claude-haiku-5-5.

The following Python example demonstrates a simple classification request using low reasoning effort.

Install a recent version of the official Anthropic SDK and configure the ANTHROPIC_API_KEY environment variable before running the code.

from anthropic import Anthropic

client = Anthropic()

response = client.messages.create(
    model='claude-haiku-5-5',
    max_tokens=2048,
    output_config={'effort': 'low'},
    messages=[
        {
            'role': 'user',
            'content': 'Classify this review as positive, negative, or mixed: The app is useful, but it crashes frequently.'
        }
    ]
)

for block in response.content:
    if block.type == 'text':
        print(block.text)

The output_config.effort parameter controls reasoning effort. For production classification systems, developers should also consider structured outputs and schema validation.

Migrating From Claude Haiku 4.5: Important Changes

Haiku 5.5 is not always a drop-in replacement for existing Haiku 4.5 applications.

Anthropic's official migration guide identifies several changes that developers should address.

1. Replace Manual Thinking Budgets

Older requests using thinking with type: enabled and budget_tokens are not supported by Haiku 5.5.

Developers should migrate to adaptive thinking and configure the effort level through output_config.effort.

2. Remove Unsupported Sampling Parameters

Haiku 5.5 restricts temperature, top_p, and top_k.

Non-default values can cause HTTP 400 errors. The simplest approach is to omit these parameters unless the current API documentation explicitly supports the intended configuration.

3. Handle Thinking Blocks Correctly

Adaptive thinking is enabled by default.

Responses may begin with a thinking block rather than a text block. Applications should inspect response content by block type instead of assuming the first block is the final answer.

4. Recalculate Token Usage

The newer tokenizer changes token counts for existing prompts.

Reevaluate prompt lengths, max_tokens settings, pricing estimates, and context usage after migration.

5. Replace Assistant Prefill Patterns

Haiku 5.5 does not support the older assistant-prefill approach for constraining output formats.

Developers should use supported structured-output features or tool schemas instead.

6. Update Computer-Use Integrations

Applications using older computer-use tools may need to update their tool versions and integration logic.

Testing these changes before a full production migration can prevent avoidable failures.

What Do Early Real-World Evaluations Show?

Anthropic's launch materials include feedback from organizations that evaluated Haiku 5.5 before its public release.

HubSpot reported an average score of 92.8% across three runs of a simulated CRM-task evaluation.

AlphaSense evaluated 400 document-question-answering queries and reported a score of 0.84 for Haiku 5.5, compared with 0.76 for Haiku 4.5.

Asana reported more than a 30% reduction in task-completion latency relative to the model it was previously using.

These early results suggest that Haiku 5.5's improvements extend beyond public benchmark datasets.

However, these are customer evaluations featured by Anthropic. They should not be interpreted as independent universal benchmarks, and their results may not generalize to unrelated applications.

Best Use Cases for Claude Haiku 5.5

1. High-Volume Data Extraction

Haiku 5.5 is a promising candidate for converting unstructured information into structured records.

Examples include extracting product specifications, organizing directory listings, and categorizing documents.

Applications should validate structured output and verify important facts against source material.

2. Browser Automation

The OSWorld improvement makes Haiku 5.5 particularly interesting for browser-based agents.

Potential applications include navigating dashboards, collecting information, and carrying out repetitive interface operations.

Production systems still require permission controls, logging, and reliable recovery mechanisms.

3. AI Customer Support

Low latency and inexpensive short requests make Haiku attractive for support assistants that answer routine questions and organize incoming tickets.

Answers should be grounded in approved knowledge bases to reduce unsupported claims.

4. Coding Subagents

Haiku 5.5 can support stronger coding agents by performing narrow tasks such as code summarization, test generation, and repository inspection.

More complex implementation and verification tasks can be handled by larger models.

5. Document Analysis

Applications processing reports, articles, or contracts can evaluate Haiku 5.5 for summarization, classification, and information extraction.

Long-context pricing should be considered carefully when entire documents or large conversation histories are included in each request.

How to Benchmark Haiku 5.5 Fairly

Public benchmark scores are useful, but developers should also test models on representative production tasks.

A reliable evaluation process should measure more than answer quality.

Step 1: Prepare realistic test cases. Collect examples from actual workloads, including difficult inputs and edge cases.

Step 2: Define success criteria. Measure factual accuracy, task completion, structured-output validity, or passing tests depending on the application.

Step 3: Control testing conditions. Use comparable prompts, tool permissions, timeout policies, and reasoning settings where possible.

Step 4: Measure total cost. Include input tokens, output tokens, retries, tool calls, and caching.

Step 5: Compare cost per successful task. Divide total expenditure by the number of acceptable completed tasks.

This approach can reveal important differences that raw benchmark leaderboards miss.

For example, a model that is inexpensive per request but frequently fails may require more retries and human intervention than a more capable alternative.

Likewise, a larger model may complete a difficult task with fewer attempts and lower total cost despite its higher token rates.

Claude Haiku 5.5 Benchmark Limitations

Several factors should be considered before treating Haiku 5.5 as a universal replacement for larger AI models.

• Official results are not guarantees. Benchmark averages do not predict success on every real application.
• Testing configurations matter. Reasoning effort, tools, execution limits, and evaluation conditions affect outcomes.
• Different benchmarks measure different capabilities. Strong computer-use performance does not automatically imply equally strong software engineering performance.
• Tokenizers differ. Identical source text can result in different token counts across models.
• Long-context costs vary. Haiku's low starting prices do not apply to every prompt length.
• Production environments are unpredictable. Interface changes, authentication issues, unavailable tools, and unusual user requests can reduce reliability.

These limitations do not undermine the reported improvements. They help explain why real-world evaluation remains necessary.

Frequently Asked Questions

Is Claude Haiku 5.5 better than GPT-6 Luna?

Haiku 5.5 scores higher on six shared benchmark categories published by Anthropic, including OSWorld 2.1 and Terminal-Bench 4.0. However, GPT-6 Luna can offer a pricing advantage for longer prompts, and actual performance depends on the application and testing conditions.

What is Claude Haiku 5.5's OSWorld score?

Anthropic reports 72.4% on the OSWorld 2.1 offline subset, compared with 15.7% for Haiku 4.5, 48.9% for GPT-6 Luna, and 83.9% for Sonnet 5.5.

Is Claude Haiku 5.5 good for coding?

Haiku 5.5 performs substantially better than Haiku 4.5 on Anthropic's published agentic coding evaluations. It is useful to evaluate for focused coding tasks, although Sonnet 5.5 remains stronger on demanding Terminal-Bench workloads.

How much does Claude Haiku 5.5 cost?

For prompts up to 100,000 tokens, standard input pricing starts at $0.10 per million tokens and output pricing starts at $0.50. For longer prompts, the respective rates rise to $0.50 and $2.50.

Does Claude Haiku 5.5 have a 1M-token context window?

Yes. It supports a one-million-token context window and up to 128,000 output tokens under its standard output limit. A larger beta output limit is available under specific Batch API conditions.

Is Haiku 5.5 cheaper than Haiku 4.5?

Yes, based on published standard API rates. Its lowest input and output prices are 90% below Haiku 4.5's rates. Anthropic estimates an average operating-cost reduction of approximately 75% after accounting for workload characteristics and tokenizer differences.

Can Haiku 5.5 replace Sonnet 5.5?

For many simple and repetitive tasks, Haiku 5.5 may provide a better cost-performance balance. However, Sonnet 5.5 remains substantially stronger on some complex coding and agentic evaluations. A hybrid approach may be more effective than relying exclusively on either model.

Conclusion

Claude Haiku 5.5 represents an important step forward for affordable AI models.

Its 72.4% OSWorld score, 1,620 GDPval-AA Elo rating, and 39.2% Terminal-Bench result demonstrate substantial improvements over Haiku 4.5. In Anthropic's published evaluations, it also outperforms GPT-6 Luna across several demanding benchmarks.

More importantly, these results suggest that a growing range of useful AI tasks can be handled by smaller and less expensive models.

For developers building browser automation, document-processing systems, AI agents, or high-volume applications, Haiku 5.5 deserves serious consideration.

However, the model's 100K-token pricing threshold, adaptive reasoning costs, and tokenizer differences make careful evaluation essential.

The best next step is to compare Haiku 5.5 with GPT-6 Luna and Sonnet 5.5 on real workloads. Measure accuracy, latency, retries, and cost per successful result—not just benchmark scores or advertised API prices.

That is how developers can determine whether Haiku 5.5 delivers the best value for their applications.