Browser agent benchmarks
Our extraction benchmark task, 10 runs, 2 runtimes, 5 models. A careers page listing 14 vacancies whose listings live inside an embedded job board on another domain, so a naive read of the page returns nothing at all. Every run had the same schema, the same spending cap, and had to come back with all 14.
| Runtime | Model | Reasoning | Steps | Time | Tokens | LLM cost | Records |
|---|---|---|---|---|---|---|---|
| OpenBrowse | gpt-5.6-luna | max | 36 | 17m 03s | 1.08M | $0.22 | 14/14 |
| OpenBrowse | gpt-5.6-terra | none | 11 | 1m 47s | 202k | $0.24 | 14/14 |
| OpenBrowse | claude-sonnet-5 | high | 10 | 4m 02s | 242k | $0.40 | 14/14 |
| OpenBrowse | gpt-5.6-sol | none | 8 | 2m 03s | 136k | $0.41 | 14/14 |
| OpenBrowse | claude-sonnet-5 | none | 9 | 5m 18s | 237k | $0.51 | 14/14 |
| OpenBrowse | gpt-5.6-terra | high | 17 | 5m 05s | 434k | $0.66 | 14/14 |
| Browser Use Cloud | claude-sonnet-5 | high | 10 | 2m 36s | 859k | $0.78 | 14/14invented fields |
| OpenBrowse | gpt-5.6-sol | medium | 16 | 5m 24s | 339k | $1.12 | 14/14 |
| OpenBrowse | claude-opus-5 | medium | 15 | 3m 56s | 398k | $1.32 | 14/14 |
| OpenBrowse | claude-opus-5 | none | 17 | 4m 53s | 480k | $1.62 | 14/14 |
Showing 10 of 10 runs.
Every OpenBrowse run recovered all 14 records without inventing a field. Browser Use Cloud recovered all 14 too, and populated values the page never displayed, job seniority among them.
Costs are LLM token spend. OpenBrowse charges nothing on top, and Browser Use Cloud's own platform fee is not in its figure, so the real difference in what you pay is wider than this.
What we learned running it
- The saving is token efficiency, not a cheaper model
- Hold the model steady and the gap is still there. claude-sonnet-5 at reasoning high costs $0.78 on Browser Use Cloud and $0.40 here, because the run burns 242k tokens instead of 859k. What changes is how many tokens the runtime spends getting the agent to the answer.
- Reading a whole listing at once is where it goes
- Open 14 pages one at a time and you pay for 14 rounds of page context. read_pages resolves the same 14 in three waves of real tabs, so the agent reasons over the set instead of rebuilding context per page.
- We do not recommend our own cheapest run
- gpt-5.6-luna at reasoning max was the cheapest complete extraction at $0.22, and it took 17m 03s to get there. gpt-5.6-terra at none cost two cents more and finished in 1m 47s. That is why the cheapest row is not the recommended one.
- Every reasoning level got the same answer
- Every OpenBrowse run recovered 14 of 14 records at every reasoning level. What moved was time, steps and cost, sometimes threefold. Treat reasoningEffort as a cost and latency control and set it deliberately per model family.
Reasoning cuts both ways
On browser tasks the two families fail in opposite directions. OpenAI models do better with less reasoning: they spend less time planning ahead and more time reacting to the page actually in front of them. Anthropic's 5-series Claude models lean towards rabbit holes and need reasoning time to refocus on the goal.
OpenAI
Turn it downgpt-5.6-terra costs $0.24 at reasoning none and $0.66 at high. At high it also burns 2.15x the tokens, takes 17 steps instead of 11 and runs 3m 18s longer.
Anthropic
Turn it upclaude-sonnet-5 costs $0.40 at reasoning high and $0.51 at none. At none it also runs 1m 16s longer.
More reasoning is not simply better, and the right answer moves in opposite directions depending on who made the model. This is why a session that sends no reasoningEffort runs at the level measured here rather than at whatever the provider would have chosen unprompted.
Which model to reach for
Three starting points, depending on what you are optimising for.
- gpt-5.6-terranone
- gpt-5.6-solnone
- claude-sonnet-5high
The best balance of reliability, accuracy and cost.
- claude-opus-5medium
- gpt-5.6-solnone
Both are strong here, but watch token burn.
- gpt-5.6-lunamax
Pair it with a tightly focused prompt. It takes a while and it is more prone to hallucination on broad prompts, but the extractions themselves are still good quality.
OpenAI
supported- gpt-5.6-sol
- gpt-5.6-terra
- gpt-5.6-luna
Anthropic
supported- claude-mythos-5
- claude-fable-5
- claude-opus-5
- claude-sonnet-5
- claude-opus-4.8
- claude-opus-4.8[1m]
- claude-opus-4.7
- claude-opus-4.7[1m]
- claude-opus-4.6
- claude-opus-4.6[1m]
- claude-sonnet-4.6
- claude-sonnet-4.6[1m]
Not available yet.
Run it yourself
OpenBrowse runs on any Debian or Ubuntu machine, from a Raspberry Pi to a VPS, and installs in about ten minutes. Point an existing browser-use-sdk client at it, send it the same task file these runs used, and compare what comes back against the rows above.