# MobileCLIP2 S0 vs S2: a reproducible CPU test

On 24 synthetic images, S0 and S2 classified the same 21 correctly. S0 had lower warm latency on this machine. This small diagnostic set did not reveal an accuracy benefit from S2.

Canonical URL: https://aiidelist.com/blog/mobileclip2-s0-vs-s2

Language: en

Published: 2026-09-24

Updated: 2026-09-24

MOBILECLIP2 · SITE MEASUREMENTS

On 24 synthetic images, S0 and S2 classified the same 21 correctly. S0 had lower warm latency on this machine. This small diagnostic set did not reveal an accuracy benefit from S2.

Measurement date: September 12, 2026 · one machine · six classes · no tuning on these examples

## Measured by FastVLM.net

| Metric | S0 | S2 |
| --- | --- | --- |
| Correct top-ranked classes | 21/24 | 21/24 |
| Median warm classification | 109.8 ms | 273.2 ms |
| 95th percentile warm classification | 119.5 ms | 357.3 ms |
| Peak process RSS | 998.6 MiB | 1210.4 MiB |
| Parameter tensors (float32) | 285.3 MiB | 378.2 MiB |

## Test scope and limits

Apple M5 Pro host with 48 GiB unified memory, CPU only, float32, 4 PyTorch threads, batch size 1. Each model ran in its own process: 3 warmups, then 5 fixed-order repeats of all 24 images. The six candidate text embeddings were computed once. Timing includes image file decode, preprocessing, image encoding and scoring; excludes download, model loading and text encoding.

Peak RSS includes Python, library imports, checkpoint loading and initialization. Parameter-tensor bytes exclude buffers and runtime overhead. Neither figure is a minimum RAM requirement or GPU-memory measurement.

Six synthetic template families with four variants each, using six fixed English candidate labels. All three errors were small-text documents classified as interfaces. A different candidate-label set, photographs or real documents can change the result. Equal performance here does not mean equal general accuracy.

## Apple-published results: a separate benchmark

Apple reports ImageNet zero-shot top-1 of 71.5% for S0 and 77.2% for S2. Those values are from a different dataset and measurement procedure; do not combine them with this 24-image result or compare our CPU latency to Apple’s device benchmarks. [Apple S0](https://huggingface.co/apple/MobileCLIP2-S0) · [Apple S2](https://huggingface.co/apple/MobileCLIP2-S2)

## How to choose

S0 is a reasonable starting point when these document-category distinctions and lower CPU latency fit the task. Choose S2 after testing whether it improves your own labels and images enough to justify the extra resources. Neither model generates a written answer.

## Reproduce and inspect

[Download fixtures, results and scripts ↓](/fastvlm/evaluations/2026-09-12/evaluation-kit.zip) · [Classification setup guide →](/blog/mobileclip2)Run each variant in a separate process

```text
python -m pip install "open_clip_torch==3.3.0" "timm==1.0.29" torch pillow
python scripts/evaluate-mobileclip.py --model S0
python scripts/evaluate-mobileclip.py --model S2
```

[S0 JSON](/fastvlm/evaluations/2026-09-12/mobileclip2-s0.json) · [S2 JSON](/fastvlm/evaluations/2026-09-12/mobileclip2-s2.json) · [Manifest](/fastvlm/evaluations/2026-09-12/manifest.json) · [Provenance](/fastvlm/evaluations/2026-09-12/provenance.json)

## Inspect every case

Category Only missing-answer / misclassified cases 24 cases shown[![receipt-1](/fastvlm/evaluations/2026-09-12/images/receipt-1.png)](/fastvlm/evaluations/2026-09-12/images/receipt-1.png)

### receipt-1

**Expected class:** a receipt

**S0 Prediction:** a receipt · 108.7 ms

**S2 Prediction:** a receipt · 270.8 ms

[![receipt-2](/fastvlm/evaluations/2026-09-12/images/receipt-2.png)](/fastvlm/evaluations/2026-09-12/images/receipt-2.png)

### receipt-2

**Expected class:** a receipt

**S0 Prediction:** a receipt · 110.7 ms

**S2 Prediction:** a receipt · 267.6 ms

[![receipt-3](/fastvlm/evaluations/2026-09-12/images/receipt-3.png)](/fastvlm/evaluations/2026-09-12/images/receipt-3.png)

### receipt-3

**Expected class:** a receipt

**S0 Prediction:** a receipt · 110.7 ms

**S2 Prediction:** a receipt · 263.7 ms

[![receipt-4](/fastvlm/evaluations/2026-09-12/images/receipt-4.png)](/fastvlm/evaluations/2026-09-12/images/receipt-4.png)

### receipt-4

**Expected class:** a receipt

**S0 Prediction:** a receipt · 110.4 ms

**S2 Prediction:** a receipt · 268.9 ms

[![chart-1](/fastvlm/evaluations/2026-09-12/images/chart-1.png)](/fastvlm/evaluations/2026-09-12/images/chart-1.png)

### chart-1

**Expected class:** a bar chart

**S0 Prediction:** a bar chart · 109.9 ms

**S2 Prediction:** a bar chart · 264.2 ms

[![chart-2](/fastvlm/evaluations/2026-09-12/images/chart-2.png)](/fastvlm/evaluations/2026-09-12/images/chart-2.png)

### chart-2

**Expected class:** a bar chart

**S0 Prediction:** a bar chart · 114.8 ms

**S2 Prediction:** a bar chart · 285.3 ms

[![chart-3](/fastvlm/evaluations/2026-09-12/images/chart-3.png)](/fastvlm/evaluations/2026-09-12/images/chart-3.png)

### chart-3

**Expected class:** a bar chart

**S0 Prediction:** a bar chart · 112.7 ms

**S2 Prediction:** a bar chart · 280.6 ms

[![chart-4](/fastvlm/evaluations/2026-09-12/images/chart-4.png)](/fastvlm/evaluations/2026-09-12/images/chart-4.png)

### chart-4

**Expected class:** a bar chart

**S0 Prediction:** a bar chart · 109.5 ms

**S2 Prediction:** a bar chart · 268.2 ms

[![interface-1](/fastvlm/evaluations/2026-09-12/images/interface-1.png)](/fastvlm/evaluations/2026-09-12/images/interface-1.png)

### interface-1

**Expected class:** a user interface screenshot

**S0 Prediction:** a user interface screenshot · 110.3 ms

**S2 Prediction:** a user interface screenshot · 271.0 ms

[![interface-2](/fastvlm/evaluations/2026-09-12/images/interface-2.png)](/fastvlm/evaluations/2026-09-12/images/interface-2.png)

### interface-2

**Expected class:** a user interface screenshot

**S0 Prediction:** a user interface screenshot · 110.5 ms

**S2 Prediction:** a user interface screenshot · 271.5 ms

[![interface-3](/fastvlm/evaluations/2026-09-12/images/interface-3.png)](/fastvlm/evaluations/2026-09-12/images/interface-3.png)

### interface-3

**Expected class:** a user interface screenshot

**S0 Prediction:** a user interface screenshot · 110.6 ms

**S2 Prediction:** a user interface screenshot · 284.5 ms

[![interface-4](/fastvlm/evaluations/2026-09-12/images/interface-4.png)](/fastvlm/evaluations/2026-09-12/images/interface-4.png)

### interface-4

**Expected class:** a user interface screenshot

**S0 Prediction:** a user interface screenshot · 111.0 ms

**S2 Prediction:** a user interface screenshot · 267.8 ms

[![small-text-1](/fastvlm/evaluations/2026-09-12/images/small-text-1.png)](/fastvlm/evaluations/2026-09-12/images/small-text-1.png)

### small-text-1

**Expected class:** an English text document

**S0 Prediction:** an English text document · 111.0 ms

**S2 Prediction:** an English text document · 270.9 ms

[![small-text-2](/fastvlm/evaluations/2026-09-12/images/small-text-2.png)](/fastvlm/evaluations/2026-09-12/images/small-text-2.png)

### small-text-2

**Expected class:** an English text document

**S0 Prediction:** a user interface screenshot · 108.1 ms

**S2 Prediction:** a user interface screenshot · 284.5 ms

[![small-text-3](/fastvlm/evaluations/2026-09-12/images/small-text-3.png)](/fastvlm/evaluations/2026-09-12/images/small-text-3.png)

### small-text-3

**Expected class:** an English text document

**S0 Prediction:** a user interface screenshot · 109.0 ms

**S2 Prediction:** a user interface screenshot · 282.1 ms

[![small-text-4](/fastvlm/evaluations/2026-09-12/images/small-text-4.png)](/fastvlm/evaluations/2026-09-12/images/small-text-4.png)

### small-text-4

**Expected class:** an English text document

**S0 Prediction:** a user interface screenshot · 109.4 ms

**S2 Prediction:** a user interface screenshot · 277.1 ms

[![chinese-1](/fastvlm/evaluations/2026-09-12/images/chinese-1.png)](/fastvlm/evaluations/2026-09-12/images/chinese-1.png)

### chinese-1

**Expected class:** a Chinese text document

**S0 Prediction:** a Chinese text document · 108.7 ms

**S2 Prediction:** a Chinese text document · 272.5 ms

[![chinese-2](/fastvlm/evaluations/2026-09-12/images/chinese-2.png)](/fastvlm/evaluations/2026-09-12/images/chinese-2.png)

### chinese-2

**Expected class:** a Chinese text document

**S0 Prediction:** a Chinese text document · 108.1 ms

**S2 Prediction:** a Chinese text document · 271.8 ms

[![chinese-3](/fastvlm/evaluations/2026-09-12/images/chinese-3.png)](/fastvlm/evaluations/2026-09-12/images/chinese-3.png)

### chinese-3

**Expected class:** a Chinese text document

**S0 Prediction:** a Chinese text document · 109.6 ms

**S2 Prediction:** a Chinese text document · 287.6 ms

[![chinese-4](/fastvlm/evaluations/2026-09-12/images/chinese-4.png)](/fastvlm/evaluations/2026-09-12/images/chinese-4.png)

### chinese-4

**Expected class:** a Chinese text document

**S0 Prediction:** a Chinese text document · 109.0 ms

**S2 Prediction:** a Chinese text document · 274.4 ms

[![shapes-1](/fastvlm/evaluations/2026-09-12/images/shapes-1.png)](/fastvlm/evaluations/2026-09-12/images/shapes-1.png)

### shapes-1

**Expected class:** an illustration of geometric shapes

**S0 Prediction:** an illustration of geometric shapes · 108.4 ms

**S2 Prediction:** an illustration of geometric shapes · 274.3 ms

[![shapes-2](/fastvlm/evaluations/2026-09-12/images/shapes-2.png)](/fastvlm/evaluations/2026-09-12/images/shapes-2.png)

### shapes-2

**Expected class:** an illustration of geometric shapes

**S0 Prediction:** an illustration of geometric shapes · 110.7 ms

**S2 Prediction:** an illustration of geometric shapes · 264.1 ms

[![shapes-3](/fastvlm/evaluations/2026-09-12/images/shapes-3.png)](/fastvlm/evaluations/2026-09-12/images/shapes-3.png)

### shapes-3

**Expected class:** an illustration of geometric shapes

**S0 Prediction:** an illustration of geometric shapes · 109.7 ms

**S2 Prediction:** an illustration of geometric shapes · 267.7 ms

[![shapes-4](/fastvlm/evaluations/2026-09-12/images/shapes-4.png)](/fastvlm/evaluations/2026-09-12/images/shapes-4.png)

### shapes-4

**Expected class:** an illustration of geometric shapes

**S0 Prediction:** an illustration of geometric shapes · 108.1 ms

**S2 Prediction:** an illustration of geometric shapes · 265.2 ms