benchmarks.site

Platforms

Platforms: errors, reasoning billing and prices

Error codes, how 'thinking' is billed, and price lists of model hosting services.

Key finding

On DeepInfra, check the `model` field and reasoning-token counts (a "Thinking" variant was served by Instruct), prefer dated model names (undated DeepSeek-V4-Flash was often overloaded), and do not trust price-list cost estimates for Gemini (approx. 13× too low); 402/404/429 codes and error text distinguish account limits, missing models and pool overload.

How to read the numbers

Prices and observations as of the stated date.

Hover a column name for its meaning, or see the glossary.

How we tested & caveats · Markdown · CSV

Observations about platforms (DeepInfra, OpenAI, Google), collected during model measurements (September–October 2026). This is the state on the stated dates; platforms change, and load affects latency. Caveats: overview.

DeepInfra

10 rows · 3 columnsCSV
The Qwen/Qwen3-VL-235B-A22B-Thinking variant was served by the Instruct model: 0 reasoning tokens in the response, at a rate approx. 2× higher than for Qwen3-VL-235B-A22B-Instruct2026-09-03worth checking the model field and the number of reasoning tokens in the response
With missing balance, every call (including partner models, e.g. Gemini, and a request for 1 token) ends with HTTP 402 You need positive balance to do inference. Please add balance manually or setup top-up2026-10-13 … 2026-10-16the state lasted at least 3 days; the balance ran out during a series of approx. 140 test calls (cause unconfirmed)
A model name that does not exist gives HTTP 404 model_not_found even with zero balance, while an existing name gives 402; so the response code tells you whether the model exists2026-10-163 names checked
/v1/openai/models returned 180 models; google/gemma-4-E4B-it was not on the list, but the name was recognized (402, not 404), and in September the model responded normally2026-10-16the list is not a complete inventory of callable models
deepseek-ai/DeepSeek-V4-Flash (undated name): approx. every second request HTTP 429 engine_overloaded "Model busy, retry later", a retry after a second usually went through; deepseek-ai/DeepSeek-V4-Flash-0731 (dated) responded every time in 1.2–2.3 s2026-10-1312 + 6 calls; details in LLM text, section 1
429 with engine_overloaded / "Model busy" in the body means overload of the model pool, not an account limit; 429 with a body about the request rate limit, and 402, are account limits2026-10-13distinguishable only by the error body
Gemini (google/gemini-3.1-pro) bills reasoning tokens that are not in completion_tokens; the cost from the token price list came out approx. 13× too low compared with estimated_cost2026-09-17reading pages from photos, 5 photos
Automatic prompt caching: with a repeated prompt of approx. 5.7 thousand tokens, DeepSeek-V4-Flash-0731 returned cached_tokens 5632 and an approx. 3.6× lower cost2026-10-13without any parameter in the request
Gemini and Claude (anthropic/*) models are available through the same DeepInfra key (forwarded to Google / Anthropic)2026-09-13, 2026-10-16
The anthropic/claude-opus-5-5 model page does not document support for temperature, reasoning_effort or response_format in the OpenAI-compatible API (the example contains only max_tokens)2026-10-16live behavior not checked (402)

Prices from /v1/openai/models (2026-10-16)

USD per million tokens. "cache" = cache read, if given. Tags from the same list.

14 rows · 5 columnsCSV
anthropic/claude-fable-510.0050.00–chat, vlm, vision, reasoning_effort, reasoning
anthropic/claude-opus-5-54.0020.000.20chat, vlm, vision, prompt_cache, reasoning_effort, reasoning; context 1000000
anthropic/claude-sonnet-5-52.0010.00–chat, vlm, vision, reasoning_effort, reasoning
moonshotai/Kimi-K32.8514.250.285chat, vlm, vision, prompt_cache, reasoning
google/gemini-3.1-pro2.0012.00–chat, vlm, vision, reasoning_effort, reasoning; context 1000000
deepseek-ai/DeepSeek-V4-Pro-08131.302.600.10chat, prompt_cache
google/gemini-3.5-flash1.509.00–context 1000000 (tags not read)
google/gemini-3.7-flash0.753.75–chat, vlm, vision, reasoning_effort, reasoning
deepseek-ai/DeepSeek-V4-Flash-Vision-Exp0.441.320.014chat, vlm, vision, prompt_cache, reasoning
Qwen/Qwen3-VL-235B-A22B-Instruct0.200.880.11chat, vlm, vision, prompt_cache, reasoning_effort
google/gemma-4-31B-it0.200.40–chat, vlm, vision, reasoning
Qwen/Qwen3.6-35B-A3B0.100.950.10chat, vlm, vision, prompt_cache, reasoning_effort, reasoning
google/gemma-4-26B-A4B-it0.070.34–chat, vlm, vision, reasoning; context 262144
deepseek-ai/DeepSeek-V4-Flash-07310.060.180.015chat, prompt_cache, reasoning

Note: list prices do not cover reasoning tokens that are not visible in completion_tokens (see above), so the real cost can be higher.

OpenAI

  • gpt-5: reasoning_effort: "minimal" is accepted but gives 0 reasoning tokens (2026-09-03).
  • gpt-5.6 (terra, sol): the minimal value is rejected; the lowest accepted level is low (2026-09-03).
  • gpt-5.6-terra / gpt-5.6-sol with an output limit of 2000 tokens and the lowest reasoning level returned empty content; a limit of approx. 6000–8000 was needed (2026-09-03).
  • See Reasoning, section 1.

Google Cloud Document AI (Enterprise Document OCR)

  • No accuracy or timing measurements in the sources. Billing per processed page (documentation, not measured).
  • Qualitative observation: the OCR returns all the text in the area, including the printed text of the exam sheet, so the reading of the area around a student's answer can be several times longer than the answer itself (see OCR, section 7).