On a public board that tests web search for coding agents, the cheapest provider is not the one that finishes the most work, and it is not the fastest. The sample is 100 tasks. Completion is the mean of three runs, not a large trial. TinyFish finishes 59.3% at a median cost of $0.035, with a 28-second median run. Perplexity, in a low-context search setting, finishes 77.3% at $0.059 and 18.3 seconds. Firecrawl lands at 70.3% for that same $0.059, and the same 28-second median as TinyFish. Cheap, complete, and fast are three different products.
Those figures come from the search-only view of OpenBenchmarks’ web-search board for coding agents. Reopen that page before you treat any write-up as the spec. Boards move.
TinyFish sells web infrastructure for AI agents. In a Marktechpost write-up the company sponsored, it argues that a demo is the wrong exam. A search agent can look finished in a walkthrough and still fall apart when the evidence is thin, a retry repeats a move that already failed, or the run blows past time and money. Read that as a design argument from the vendor, not as an independent audit. The checkable numbers live on a different page, and they do not show one identical mode under different logos.
Named modes, not the same product
The page’s own headline matches the split above. Perplexity completes the most tasks, at 77.3%. It is also fastest, at 18.3 seconds. TinyFish costs least, at $0.035. Cost is search plus model, per task. Tokens are context consumed per task. Median time, cost, and tokens are pooled across the three runs. Completion is the mean, plus or minus the standard deviation.
The board has more rows than a four-name comparison. These are the ones the headline depends on.
- TinyFish, GET api.search.tinyfish.ai: 59.3% ± 1.5, 28-second median, $0.035, 7, 469 median tokens. The visible search-cost component is $0.000, and the rest of the cost split is cut off. You cannot tell whether the $0.035 median is a lighter workload or a pricing structure.
- Perplexity, search_context_size=low, not an unspecified maximum: 77.3% ± 2.1, 18.3-second median, $0.059 ($0.025 search plus $0.034 in tokens), 8, 765 tokens. Average search call: 957 milliseconds.
- Firecrawl, POST /v2/search: 70.3% ± 1.5, 28 seconds, $0.059, 7, 456 tokens. Average search call: 2.87 seconds, much slower than Perplexity’s, at a similar median task cost.
- Exa fast, type=fast, which is not Exa deep: 66.3% ± 1.5, 20 seconds, $0.103, 22, 344 tokens.
- Parallel fast: 66.7% ± 1.5, 22 seconds, $0.054, 12, 460 tokens. More search-only tasks finished than TinyFish, at a higher median cost.
- Nimble lite: 46.7% ± 3.8, 33 seconds, $0.043. The next-lowest median cost cited against TinyFish, and well behind on completion.
The gap versus Perplexity is wide next to the reported spreads, 77.3% ± 2.1 against 59.3% ± 1.5. No significance test is published, so the point estimates have to stand on their own. The board also does not publish cost per successful task. Lowest median cost and highest completion are different rankings.
A sponsored summary called TinyFish the lowest token footprint. On this search-only board, Firecrawl’s median is 7, 456 tokens and TinyFish’s is 7, 469. For practical purposes, that is a tie.
A sponsored explainer attached to the Marktechpost piece shows a different job, search-and-fetch. It is not on the public search-only board. The two setups measure different retrieval work, so do not read them as one ranking.
- TinyFish: $0.050 median task cost, 79.0% completion. This row, and the three beside it, come from that sponsored explainer, not from the public search-only page.
- Exa deep: $0.127, 83.0%.
- Perplexity: $0.083, 77.7%.
- Firecrawl: $0.082, 76.0%.
Even inside the explainer, TinyFish is the cheap option. It does not lead on completion. The jump from 59.3% on search-only to 79.0% on search-and-fetch is unexplained. The public board does not establish that only the provider varied, or which model sat underneath. Named modes already differ. Search-and-fetch token counts were not published with the 79.0% figure.
Bring a forklift to a typo and you still have a typo
A heavier tool does not fix a bad query. That is the buying rule inside TinyFish’s design notes, as summarized in the sponsored write-up. The scoreboard tells you not to buy a logo. The notes, read as a vendor argument, tell you what else to ask for.
The useful idea is old engineering. The model reasons. The surrounding system owns state, tools, evidence, and the decision to stop. A February 2026 survey on arXiv, “From Prompt, Response to Goal-Directed Systems: The Evolution of Agentic AI Software Architecture, ” makes that broader point without describing TinyFish. It puts the model inside a governed loop, with tool interfaces, traces, retries, and cost and rate limits. State and control flow do work the weights cannot do alone. Planning, in that paper, is a spectrum: implicit in the model’s output, or an explicit planner that emits a task graph, or iterative reflection. The survey is not a TinyFish architecture, and it does not validate anyone’s benchmark.
TinyFish’s guides, as relayed in the sponsored write-up, describe a control loop and four web primitives: TinySearch, TinyFetch, TinyBrowser, and TinyAgent. You do not need the seven-phase tour. Five demands cover what a buyer can actually use.
An evidence contract comes first. Required fields, acceptable sources, how fresh the pages must be, and the rule for calling the job done. A pricing brief might require a number, a date, an allowed set of domains, and an incomplete state if two of those are missing. Without that, the agent cannot tell a partial from a finish, so it keeps talking.
Tool choice follows the information gap. Do not climb a ladder of heavier tools. The sponsored write-up is plain about it. Weak results need a better query. Conflicting sources need stronger evidence. A heavier tool fixes neither. No URL yet means search first. Paying to open pages before you know which pages exist is how this work gets expensive without getting smarter. A known URL means fetch, and skip search. A scripted click path stays in your own code. A path that keeps changing is the case for a web agent that holds the goal and the run state while the route changes. Bad evidence gets a rewritten query.
Retries have to match the failure. A transient timeout can justify a repeat. Incomplete or conflicting evidence needs a different approach, not the same call with hope attached. Browser retries also have to remember what already happened. Submit the same form twice and you get a duplicate order, a duplicate lead, or a locked account, depending on whose site you just hit. That is not persistence.
The allowed endings include incomplete and human review required, not only completed. TinyFish’s explainer puts the production virtue in one line: at the boundary, the agent stops and says so, rather than filling the gap with a fluent guess.
Budget the whole trajectory, not just the last model call. Time, tokens, tool calls, retries, and cost. Latency is planning plus search plus retrieval plus evaluation plus retries plus interactive steps plus queueing. Cost is tokens plus tool calls plus page or browser use plus retries plus infrastructure. A dashboard that only prices the last completion will hide the run that looped itself into a loss.
What broke when they built deep research
TinyFish also published lessons from a deep-research agent that searches, browses, and writes cited reports. None of the before-and-after claims come with measured deltas. Treat them as engineering testimony from a sponsor, not as a result.
A standalone planner, called fresh each time and handed only structured state, drifted. A follow-up like “focus only on pricing risks in Southeast Asia” wandered back into broad market research. Planning moved into one reasoning loop so the follow-up stayed in the same conversation as the evidence. That sits in tension with the February survey, which treats an explicit planner that emits a task graph as a legitimate option, not as a mistake. Both can be true in different systems. Neither side has published a controlled comparison. In a pilot, ask which of those you are buying, and how a narrow follow-up is supposed to survive the handoff.
Ask the same question of the handoffs underneath. TinyFish says one subagent per search query raised switching cost, and that ranking, classification, and extraction were turned into ordinary tool calls so the main loop saw a distilled result. Again, no measured delta. The demand is still fair. Can the orchestrator see the contract and the evidence, or only another agent’s scratchpad?
The product lesson matters more. The system was first tuned for source coverage. Users cared whether the evidence supported the question. The same sources can be ranked differently for a pricing-risk brief than for a market map. TinyFish says it added controls to prioritize, exclude, or restrict domains, and an internal judge for relevance, evidence strength, citation trust, and source diversity. No scores, judge model, or agreement figures have been published. If you cannot inspect the harness, all you have is a claim.
Questions worth asking before the pilot
- Is the cheapest web search the best one for coding agents?
Not on the search-only board. TinyFish is cheapest at $0.035 and completes 59.3% in a 28-second median run. Perplexity’s low-context mode completes 77.3% at $0.059 and 18.3 seconds, and Parallel fast finishes 66.7% at $0.054. Lowest median cost, highest completion, and fastest run are different rankings. The board does not publish cost per successful task.
- Does TinyFish use the fewest tokens?
No. On search-only, Firecrawl’s median is 7, 456 tokens and TinyFish’s is 7, 469. Search-and-fetch token counts were not published with the 79.0% figure in the sponsored explainer.
- What should a buyer demand besides a completion rate?
The exact mode, not the brand. A task-level budget across time, tokens, tool calls, retries, and cost. A terminal state that can say incomplete. Domain controls tied to the question, not a coverage score. And a clear line between a sponsored explainer and the public board, which you should reopen rather than archive a write-up.
- What would you compute before a pilot, and what would make you walk away?
Cost per successful task, in the named mode you will actually call, plus the cost of human review on the misses. The board does not publish that figure, so do not treat a vendor’s median cost as the price of a finished answer. Walk away if the miss rate means your team will redo most of the work, if the system cannot return incomplete when the budget hits the wall, or if the vendor will not name the mode behind the headline rate.
Buy the stop condition, not the demo
When a company buys web search for its agents, the mistake is treating it as a commodity API under a logo. A sales team does not need the run that visited the most domains. It needs the run that can say the Southeast Asia pricing sources conflict, here is what is solid, and a person should look before this number hits a quote. Coverage is a vanity metric if the ranking ignores the question.
Start with the contract. Name the fields, the domains you trust, the freshness you require, and what done means for that workflow. Then price the whole trajectory, including the retries you hope not to see. Compare providers in named modes, on completion, median run time, median task cost, and tokens, and keep search-only scores in a different column from search-and-fetch. Ask what the system returns when the budget hits the wall. If the answer is another polished paragraph, you do not have automation yet. You have a confident intern with a corporate card.