Testing Small Language Models on a Phone: 4K/8K Context, Memory, and Routing Accuracy

We have been interested in a simple question: are compact language models already good enough to run useful agent-style tasks locally on a phone?

To make the answer less subjective, we tested five popular GGUF text models with two context sizes, 4K and 8K. The resource check ran on an iPhone 15 Pro Max. The capability check used the same 96 deterministic agent-routing cases for every model, covering skill selection, action extraction, mode classification, missing-slot handling, and simple planning behavior.

This is not a broad academic benchmark. It is a phone-first engineering check: can the model load, how much memory does 8K cost, how quickly does it produce the first token, and does it produce stable structured decisions for a small local agent runtime?

Models tested

Why test Qwen3.5 when Qwen3.8 exists?

Qwen's newest mainline has moved beyond 3.5. As of this update, the public Qwen3.8 family is led by much larger models such as Qwen3.8-Max, Qwen3.8-2.4T-A95B, and Qwen3.8-27B. They are important models, but they are not in the same deployment class as a 1B-4B phone-local routing model.

The point of this test is not to benchmark the newest cloud flagship. It is to find a model that can actually live inside a phone runtime with a 4K default context, an optional 8K mode, predictable memory use, and stable structured decisions. Under that constraint, Qwen3.5-2B is still one of the most relevant Qwen-family candidates because the newer Qwen3.6, Qwen3.7, and Qwen3.8 lines do not currently provide an official 1B/2B-class open model that fits this exact role.

Coverage and missing candidates

This table is not a claim that every phone-suitable model has already been tested. It covers the local GGUF text models we had available and could run through the same phone resource check plus the same 96-case routing protocol.

It also is not a China-only comparison. The current table includes Qwen and MiniCPM from Chinese model families, plus Gemma from Google and Phi from Microsoft. The gap is that several newer or still-interesting international mobile candidates were not yet in the local benchmark set.

The next iteration should add those candidates only when the same conditions are met: a compatible local artifact, the same 4K/8K phone resource test, and the same structured routing or multimodal task protocol.

The phone-side environment was an iPhone 15 Pro Max with A17 Pro, 8 GB physical memory, iOS 26.6.1, SwiftLlama with llama.cpp b8668, and Metal full-layer offload. Each model/context pair used three independent launches.

Measurement flow

flowchart LR
    A[Select GGUF model] --> B[Run phone 4K and 8K resource check]
    B --> C[Record load time, TTFT, process memory, Metal memory]
    C --> D[Run the same 96 routing cases]
    D --> E[Score parse, skill, action, mode, slots, planning]
    E --> F[Apply a phone-first deployment gate]

The phone resource check used a short Chinese prompt and measures model loading, context allocation, short-prompt TTFT, basic generation, and memory footprint. It is not a full 4K/8K prefill stress test.

The 96-case capability check used the same model files and deterministic settings, but ran on a desktop Metal evaluation runner so that all model comparisons were repeatable. This separates phone resource feasibility from structured routing quality.

iPhone 4K and 8K resource results

All five models loaded successfully at both context sizes.

The most important resource result is the 4K-to-8K delta. Qwen3.5-2B only added about 52 MiB of Metal memory in this setup, while Phi-4-mini added about 504 MiB. That makes Qwen3.5-2B a much cleaner fit for a phone runtime that normally stays at 4K but occasionally needs 8K for harder tasks.

96-case routing capability results

The capability set contains 96 Chinese and English agent-style requests. It checks whether the model can output a valid structured decision, choose the correct skill, choose the correct action, classify retrieve/act/clarify/answer behavior, preserve required slots, and identify planning cases.

The 1B-class models are attractive on resource cost, but they failed too often on structured action and mode decisions. For a local assistant that can touch reminders, email drafts, memory updates, or other write paths, that is a real product risk, not just a benchmark number.

Qwen3.5-2B was the only model in this batch that cleared the proposed deployment gate: high parse stability, above-90% skill and action accuracy, above-95% mode accuracy, and phone-side 8K Metal memory below 2 GiB.

Practical ranking

What this suggests

For phone-side local agents, the best model is not simply the smallest model. The smallest models won on TTFT and memory, but their structured decision quality dropped sharply. The heavier models did not automatically win either: Qwen2.5-3B and Phi-4-mini both consumed more memory than Qwen3.5-2B while scoring lower on this routing task.

The useful middle ground in this test was Qwen3.5-2B. It kept 8K memory within a practical range and produced stable JSON-like routing decisions across Chinese and English cases. The default context should still be 4K for battery and thermal reasons, with 8K used only when task complexity justifies it.

Limits and next tests

These numbers should not be overgeneralized.

The phone resource test used a short prompt, so the next step is a long-prefill test at about 3.5K and 7.5K prompt tokens, plus a sustained 20-turn session that records tokens per second, thermal state, battery drop, and jetsam risk.

The capability test focuses on text routing. Multimodal candidates such as Gemma 3n and MiniCPM-V should be evaluated with a separate vision-language harness that includes screen understanding, OCR, photo questions, and permission-bound privacy cases.

There are also new on-device candidates worth testing next, including LFM2.5-1.2B-Instruct, LFM2.5-1.2B-Thinking, Llama 3.2 1B/3B, SmolLM-family small instruct models, Qwen3.5-4B, Gemma 3n E2B, and MiniCPM-V-4.6. They should enter the table only after the same phone-side resource check and the same capability protocol are available.