---
title: "Search APIs for RAG in 2026: What to Look For, What It Costs, Who Scores What"
dek: "A RAG pipeline lives or dies on what the search API returns before the retriever ever runs. This guide covers the four properties that matter for retrieval-augmented generation, the benchmark evidence that chunk-and-retrieve collapses to ~19% while a search-content API holds 78% on FinanceBench, and the per-1K cost of a usable RAG query across eight vendors. Verified 2026-09-17."
category: "guide"
tags: [rag, search-api, retrieval, financebench, benchmarks, ai-agents]
author: "Manny"
published: 2026-09-17T12:00:00+00:00
updated: 2026-09-17T12:00:00+00:00
url: https://keirolabs.cloud/blogs/guide/search-api-for-rag
---
78%Keiro, FinanceBench
~19%GPT-4o + vector RAG
84%Keiro, SimpleQA
$0.25Keiro / 1K semantic
Rates verified against vendor pricing pages on 2026-09-17. Benchmark figures are from published runs: Patronus AI's FinanceBench paper, AIMultiple's May 2026 search-API benchmark (judge: Gemma 3 12B), and Keiro's own published QA runs.
The question "which search API should my RAG pipeline use" usually gets answered with a feature matrix, and the matrix misses the point. RAG does not consume search results; it consumes text. The retrieval step is not looking for links to show a user, it is looking for evidence to ground a generation step, and the properties that decide whether the generation is accurate are properties of the text: is it clean, is it chunked coherently, is it fresh enough for the question, and can the citation be traced back to a stable source. A search API that returns links has handed your pipeline a to-do list. A search API that returns clean page text has handed it the evidence.
This guide is written from inside a search engine. Keiro runs a 50B+ page index, a ranker called Webrank, and search, answer, and extraction endpoints that agents call, and the observations here about what RAG pipelines need come from watching what those pipelines do with our responses and where they break. The pricing comparisons include our API alongside the incumbents with the same figures we publish everywhere: $0.25 per 1,000 semantic searches, $0.10 per 1,000 SERP queries, 1,000 free queries a month. Where our numbers win, they win, and the disclosure section says how to check all of it for free.
## What makes a search API good for RAG, specifically?
**Four properties decide it, in order of impact: bundled clean text, chunk stability, citation fidelity, and index freshness.** Everything else on a search API's feature page, API keys, SDK ergonomics, region parameters, matters to developer experience and almost not at all to answer accuracy. The four properties below are the ones that show up in the accuracy numbers when a team measures them, and each one has a specific failure mode when it is missing.
**Bundled clean text.** The property with the largest measured impact. A search API that returns results plus the full clean text of each result page in one response removes the fetch-and-parse stage from the RAG chain, and with it a whole class of failures: parser breakage on markup drift, bot-blocking on the fetch, JavaScript-only content, and the boilerplate pollution that wrecks embedding quality. Keiro returns the content in the same call from its index, Tavily attaches raw_content on request, Exa bundles contents for the first ten results since March 2026, Brave and the raw-SERP vendors leave the fetch to you. The all-in cost of a usable RAG query, one search plus three pages of clean text, spans $1 to $16 per 1,000 across eight vendors because of exactly this bundling difference.
**Chunk stability.** The property nobody markets. A RAG pipeline chunks text, embeds the chunks, and stores them; if the text for the same page changes between calls, the embeddings change, cache entries rot, and evaluations stop being comparable. Content fetched at request time is fresh and unstable. Content served from a crawled index is stable and slightly less fresh. For RAG, stable wins the argument in most deployments, because the chunk store is the expensive asset and stability is what makes caching and diffing possible. This is also the property least discussed in vendor comparisons and most visible in production incident logs.
**Citation fidelity.** The property that decides whether your product's answers survive an audit. A RAG system cites what it read; if the text it read is not traceable to a stable source version, the citation is decorative. Index-served content can be dated, cached, and diffed. Request-time fetches cannot, and an answer generated on Monday and audited on Friday may cite a page that changed on Wednesday. For financial, legal, and medical RAG, this is not a nice-to-have, it is the requirement that disqualifies vendors.
**Index freshness.** The property that has a real ceiling. A 50B+ page index that refreshes hot URLs in seconds and trending topics in minutes covers news-shaped queries; a weekly-crawled index does not, and no request-time fetcher compensates for an index that does not have the page at all. Freshness is also the property with the widest honest variance between vendors, because crawl strategies differ by orders of magnitude, and it is testable: ask both APIs for a page that changed this week and diff.
What each property buys
Bundled text: removes the fetch-and-parse failure class entirely.
Chunk stability: makes the vector cache and citation diffs possible.
Citation fidelity: answers that survive an audit trail.
Freshness: news-shaped and time-sensitive queries answered correctly.
What happens without it
No bundled text: DIY fetcher, parser fleet, boilerplate cleanup, $1-5/1K more.
No stability: rotting vector caches, silent answer drift.
No fidelity: citations that contradict the answer a week later.
No freshness: confidently wrong answers on recent events.
## What does the benchmark evidence actually say about search for RAG?
**The cleanest published comparison is FinanceBench: a GPT-4o oracle with the full filing in context reaches about 85%, standard chunk-and-retrieve collapses to roughly 19%, and Keiro's search-content pipeline holds 78% while staying deployable.** The oracle number shows the ceiling: a model reading the whole document is nearly as good as the model can be. The vector RAG number shows what most production systems actually achieve, and the collapse from 85 to 19 is the retrieval stage failing, not the model. Keiro's 78% closes most of that gap with a pipeline that runs in production at $0.25 per 1,000 queries, which is why the number is a production number and not a research one.
| System | FinanceBench | Deployable | Cost basis |
|---|---|---|---|
| GPT-4o oracle (full doc in context) | ~85% | No | Frontier tokens per full filing |
| Keiro search + content pipeline | 78% | Yes | $0.25/1K queries |
| GPT-4o + vector RAG (chunk-and-retrieve) | ~19% | Yes | Embedding + vector infra |
The 19% number deserves more attention than it gets, because it is the baseline most teams build first. The standard pattern, search for pages, fetch them yourself, chunk, embed, store, retrieve top-k, is what the vendor-agnostic RAG tutorial teaches, and on financial questions it produces a system that is wrong four times out of five. The failure is mechanical: chunk-and-retrieve over boilerplate-heavy financial documents retrieves the wrong chunks, the generator confidently answers from them, and the pipeline's failure reads as model error when it is retrieval error. The same model with better retrieval input scores four times higher, which is the size of the prize a search-API decision controls.
On the open-domain QA benchmarks, AIMultiple's May 2026 run, judged by Gemma 3 12B, scored Keiro 94 on SimpleQA, 91 on FreshQA, and 82 on HotpotQA, ahead of Perplexity at 86/83/74 and Tavily at 78/77/68. Keiro's own published runs report 84% on SimpleQA with a different judge setup, and both sets are cited here because a benchmark number without its judge and setup is marketing, and both sets, whatever their absolute levels, put the bundled-content approach ahead of the tutorial baseline. Exa, Brave, SerpAPI, and Serper were not in the QA run because they return search results rather than answers, which is a category difference, not a score.
The deployability gap. The oracle is not a product; the vector RAG baseline is what most teams ship; the search-content pipeline closes most of the gap between them.
Read every benchmark with its setup attached. The AIMultiple run used an LLM judge, which rewards grounded well-cited answers over keyword overlap, and the judge choice is a variable that different runs weight differently. The FinanceBench comparison is domain-specific, financial documents, and its vector-RAG baseline is the standard tutorial shape, which is exactly the shape most teams arrive with. Neither number guarantees your workload's accuracy; both numbers eliminate the belief that retrieval quality is a second-order detail. It is the first-order variable in the system.
## How much does a usable RAG query actually cost per vendor?
**The all-in cost of a usable RAG query, one search plus clean text for three pages, runs from $1 to $16 per 1,000 across the major vendors, and the spread is entirely about bundling.** Serper's $1 per 1,000 is SERP-only, so the pipeline still owes a fetcher, a parser, and an embedding stack before anything is usable. Keiro's one-call path at 3 credits bundles search, content, and embeddings, roughly $1 per 1,000 on its rate card. Firecrawl with JSON plus Enhanced extraction reaches about $24 per 1,000, off the chart. Tavily pay-as-you-go lands at $16, SerpAPI at $15, Exa around $7 plus contents, Brave $5, Perplexity about $6.
| Provider | All-in $/1K RAG query | What the price includes |
|---|---|---|
| **Keiro** | ~$1 (3 credits, one call) | Search + clean text + embeddings, bundled |
| Serper | ~$1 (SERP only) | Results and snippets; DIY extraction on top |
| Brave | $5 | LLM-ready results; extraction is yours |
| Perplexity | ~$6 | Search + citations; tokens billed separately |
| Exa | $7 + contents | 10 results bundled; more meters stack |
| Tavily (Growth) | $10 | $5/1K behind $500/mo commitment |
| Tavily (PAYG) | $16 | $8/1K basic, advanced 2x |
| SerpAPI | $15 | Structured SERP; extraction is yours |
The table's unit is the one that matters: the usable query, not the API call. A search-only vendor at $1 per 1,000 becomes $2 to $6 per 1,000 once the fetch-and-parse stack it implies is priced, estimates for that DIY stack run $1 to $5 per 1,000 pages fully loaded, and the pipeline inherits the parser-maintenance burden. A bundled vendor's rate is the whole cost of the usable query, which is why the cheap-looking column and the usable column reorder the market. At 100K RAG queries a month, the usable-query columns bill between $100 and $1,600, and the model bill for the generation step is usually larger than every search line combined, which is exactly why retrieval accuracy pays twice: a 19%-accurate retrieval stage wastes every token it grounds.
At production volume the table compresses to one decision: is the pipeline paying for content once, inside the search call, or twice, in search plus an extraction stack. The first shape bills $1 per 1K usable at the low end; the second bills the search line plus $1 to $5 per 1,000 pages of DIY extraction plus the operational risk of maintaining the parser fleet. The second shape was necessary in 2023 when no bundled option existed at scale. In 2026 it is a choice, and the choice should be priced per usable query before it is defaulted into.
## Keiro's search-content pipeline: what the bundled call changes
**The Keiro content endpoint, `POST https://api.keirolabs.cloud/api/v2/search/content`, returns search results with the extracted clean text and embeddings in one response, 3 credits per query, which collapses the RAG chain from fetch-parse-chunk-embed-store-retrieve to search-then-use.** The response carries titles, URLs, snippets, and the full clean text of each result page from the 50B+ page index, plus embeddings where requested, which means the vector store's ingestion step reads directly from the API response rather than from a scraping pipeline's output. Indexed queries return in roughly 100 milliseconds, hot URLs refresh in seconds, and trending topics in minutes, which is what makes time-sensitive RAG queries answerable from an index rather than a fetch.
The benchmark section covers the accuracy claim; the operational claim is smaller and just as consequential. One call means one round trip, one rate-limit budget, one failure mode to handle, and one meter to watch, against a DIY chain of search call, fetch fleet, parser, embedder, vector store, and retriever, each with its own latency, its own 429s, and its own on-call runbook. Teams migrating from the DIY chain typically retire a browser fleet and a parser-repair calendar, and the runbook that survives is one page long. That is the difference between a retrieval stage that is a product component and one that is an internal platform team.
The honest limitations sit next to the claim: an index ranks by its own order, not Google's, which matters only if Google's exact ranking is the product; JavaScript-only pages that were never crawled are a scrape request rather than an indexed query; and the SDK ecosystem is younger than the incumbents'. The free tier, 1,250 credits a month with no card, exists precisely because these are the claims a reader should verify against their own workload before believing this paragraph.
The catch The bundled-call shape optimizes for answer accuracy and operational simplicity. If your pipeline's core requirement is Google's exact SERP order or a scraping engine for arbitrary JavaScript URLs, a bundled index API is the wrong tool, and the honest answer for that slice remains a structured-SERP vendor.
## How do the other vendors fit a RAG architecture?
**Tavily, at $8.00 per 1,000 basic and $16.00 advanced pay-as-you-go, is the RAG incumbent with the largest tutorial ecosystem, and its raw_content field makes it a legitimate bundled-content option with a request-time freshness profile.** The credit system prices extract, map, crawl, and research from the same pool, which makes the bill auditable and the budget-blending invisible at once. The advanced-mode multiplier is the most common surprise: agent frameworks default to `search_depth="advanced"` on hard queries, doubling the rate at exactly the moments volume spikes. On the AIMultiple QA run Tavily scored 78/77/68, behind Keiro's 94/91/82, which is the number to weigh against its integration depth when pricing the same RAG workload.
**Exa occupies the semantic slot: $7.00 per 1,000 base for ten results, contents bundled only for the first ten since March 2026, and the find-similar primitive that no other vendor on this list offers.** In a RAG architecture Exa serves the corpus-expansion role, seeding a document set from one example, and its per-result-block meter means deep retrieval bills steeply: a 30-result search at $27.00 per 1,000 plus contents. For pipelines whose queries are concept-shaped rather than keyword-shaped, Exa is the complement, and the honest architecture pairs it with a cheaper bundled-content API for the question-shaped traffic rather than making it the general workhorse.
**Brave, at $5.00 per 1,000 flat from its own index, is the sovereignty pick: no scraping dependency, stable rate, and a $5 monthly credit allowance standing in for the free tier it removed in February 2026.** Its results are LLM-ready but its text extraction is not bundled, so the RAG pipeline keeps a fetch-and-parse stage, and the $5.00 rate is for results rather than usable queries. It fits RAG stacks that already have an extraction layer, teams migrating off scraping-dependent vendors who want index ownership without a meter proliferation, and products whose queries are keyword-shaped and common enough that index coverage differences rarely surface.
**Perplexity's search API, around $5.00 per 1,000, returns search results with citations tuned for grounded answers, and it scored 86/83/74 on the AIMultiple QA run, second to Keiro across the three benchmarks.** Its answer endpoint is a separate product from its search product, and the token bill for the generation side lands on the model vendor's invoice either way. It fits RAG teams that want a managed citation layer and accept the mid-band rate, and its position in the benchmark table makes it the incumbent's nearest competitor on accuracy in that run.
**SerpAPI and Serper, at $15.00 and $0.30 to $1.00 per 1,000, are the raw-SERP tier: results and snippets with the extraction stage entirely yours.** In a RAG architecture they are the bottom layer under your own fetcher and parser, which is a legitimate architecture at scale where the DIY extraction fleet already exists, and it is the most expensive architecture to build from scratch in 2026. The $15.00 SerpAPI rate is the structured-SERP premium; the $0.30 Serper rate is the organic-results floor. Neither bundles what the bundled tier bundles, and that is the entire economic argument in one sentence.
## How should the chunking and embedding layer change when content arrives bundled?
**The chunking pass moves upstream and gets simpler, because the API's clean text is already boilerplate-stripped, which changes chunking from a cleanup problem into a semantics problem.** The DIY chain chunks defensively, large overlaps, aggressive boilerplate filters, header detection, because its input is a scrape of unknown quality. The bundled chain receives text that was extracted from the index with the navigation, ads, and boilerplate already gone, so chunk size becomes a pure retrieval-quality knob rather than a salvage operation. Teams report the practical consequence: chunk sizes stop being tuned defensively and start being tuned for the judge, and the tuning surface shrinks to two or three parameters that actually move grounded accuracy.
The embedding step also gets a budget it did not have. With embeddings arriving in the API response, the pipeline can skip its own embedding pass for the public-web slice entirely and spend its embedding budget on private documents, which are the only corpus where it is irreplaceable. That split, public-web retrieval handled by the search layer, private retrieval handled by your own stack, is the architecture that scales, because the two halves have different freshness, stability, and access-control requirements and no single vector store serves both well. The mistake to avoid is re-embedding the API's content with a different model than the one whose embeddings the query uses; if the API returns embeddings, use them for those chunks, and keep one embedding space per source.
Chunk overlap policy also deserves a re-decision after the migration. Overlap exists to protect against boundaries landing mid-sentence in messy scrape output; clean extracted text has coherent paragraph structure, so the overlap that protected the DIY pipeline wastes storage and duplicates citations in the bundled one. Re-run the chunk-size sweep after switching vendors rather than carrying the old parameters forward, because the old parameters were tuned for a different input distribution, and the evaluation harness from the next section is the tool that re-tunes them.
## What breaks first when the wrong search API is picked for RAG?
**The first break is always the same shape: high-volume queries work, tail queries fail, and the failures look like model hallucinations because the generator fills the gap left by missing retrieval.** The tail queries, the rare entity, the recent event, the page behind a consent wall, are where index coverage, freshness, and extraction quality separate vendors, and they are also the queries users remember. A pipeline validated on demo queries passes; production reaches the tail in week two; the answer quality collapses on exactly the questions the product was built for. This is why the evaluation must replay real logs, and why a demo-driven vendor choice is the most common self-inflicted RAG wound.
The second break is the bill, and it arrives later than the quality break because credit blending hides it. The pipeline that looked cheap in the pilot is paying for extraction credits, map calls, or result-block meters that the demo never triggered, and the discovery usually happens in the month after the workload doubles. The mitigation costs one meeting: model the meter profile of the real workload, count advanced modes, content types, and non-search operations per task, and get the projection signed before cutover. The teams that skip this step re-architect at the worst possible time, mid-growth, when the traffic pattern that exposes the meter is the product's success.
The third break is operational, and it is the quiet one: the DIY extraction fleet's parser breaks on target-site redesigns at unpredictable intervals, and each break degrades answer quality before anyone notices, because stale chunks still retrieve. The failure interval is set by other people's release calendars, which means the pipeline's accuracy has a maintenance-driven decay curve nobody scheduled. Index-served content decays on the index's refresh schedule instead, which is published, observable, and testable, and that difference is most of the operational argument for bundled content.
## How should a RAG team evaluate a search API in two weeks?
**The evaluation replays real queries, not demo queries, and it measures the four properties from the first section against your actual distribution.** Export 200 queries from your production logs, or from your product's question stream if the pipeline is new, and run the identical set through the candidates. Measure four numbers per vendor: the percentage of queries returning usable clean text without a second call, the chunk-level stability of repeated calls across a week, the citation traceability of the content, and the latency at your production burst rate. Two of the four, stability and traceability, are properties no pricing page mentions and both decide production behavior.
The evaluation harness shape that produces honest numbers: same query set, same prompt template, same judge, vendors interleaved in the same run so index drift cannot favor one side, and the output scored on grounded-answer accuracy rather than result overlap. Result-set overlap is the vanity metric of RAG evaluations; two engines can return 90% overlapping top-10 sets while one of them hands back clean text and the other hands back URLs, and the grounded answers differ far more than the overlap suggests. Score the answers, log the citations, and let the overlap be a diagnostic rather than a decision.
TL;DR ยท the RAG search decision
- **The property that matters most:** bundled clean text, it removes the fetch-and-parse failure class and halves the usable-query cost.
- **The benchmark that matters most:** FinanceBench, 78% search-content vs ~19% chunk-and-retrieve, the retrieval stage is the first-order variable.
- **The cheapest usable query:** Keiro, one call at 3 credits, $0.25/1K semantic rate, ~$1/1K all-in.
- **The incumbent:** Tavily, $8/1K PAYG, biggest ecosystem, request-time content.
- **The specialist:** Exa, $7/1K base, find-similar only it can do.
- **The sovereignty pick:** Brave, $5/1K from its own index.
- **The floor:** Serper $0.30/1K raw SERP, if you already own the extraction fleet.
## What does the retrieval stage cost to run wrong?
**The hidden cost of a weak retrieval stage is not the API bill, it is the generation bill multiplied by wrong answers: every retrieved-but-irrelevant chunk is tokens paid for noise, and every wrong answer is a support ticket or a decision made on sand.** A RAG pipeline at 19% retrieval accuracy pays the full generation cost for every query and delivers a usable answer for one in five, which makes its effective cost per usable answer five times its nominal cost. The pipeline at 78% pays the same generation bill and produces usable answers at nearly four times the rate, so the retrieval upgrade pays for itself in the token line before the search line is even compared. This arithmetic is why retrieval quality is a cost decision, not just a quality decision.
The second hidden cost is engineering time, and it is the one procurement never sees. A search-only architecture maintains a fetch fleet, a parser against every target site's markup, a boilerplate filter, a deduplication pass, and the chunking logic, and each of those has an on-call burden that lands on the same team building the product. The bundled-call architecture deletes the fleet and the parser from the runbook in one migration, and the teams that have done it describe the change as deleting a service rather than swapping a vendor, which is the correct frame: the fetch-and-parse stage was a product your team was operating, and someone else now operates it at $0.25 per 1,000.
The third cost is silent and compounding: chunk-store rot. In a request-time-fetch architecture, the same page returns different text on different days, so embeddings computed last month quietly disagree with the content the pipeline serves this month, and nothing fails loudly, the answers drift. In an index-served architecture, the chunk store keys to stable content and drift is a diff you can see. Retrieval bugs that are visible get fixed; retrieval bugs that look like model hallucinations get attributed to the model and never fixed. The stability property is not a nicety, it is the difference between a debuggable RAG system and an unexplainable one.
## Disclosure
Keirolabs is our product, and this guide includes it in every table with the same figures we publish on our pricing page: $0.25 per 1,000 semantic searches, $0.10 per 1,000 SERP queries, 1,000 free queries a month, no card. The FinanceBench comparison uses Patronus AI's published benchmark paper for the oracle and vector RAG baselines and our published run for the 78%. The AIMultiple May 2026 figures (94/91/82, judge Gemma 3 12B) are from that published benchmark, and our own SimpleQA run reports 84% under a different judge setup. Competitor rates are from their public pricing pages, verified 2026-09-17. Run your own 200-query replay on the free tiers before believing any vendor's table, ours included; the free tier exists precisely because you should.
## About the author
Manny builds Keiro. The 50B+ page web index, the Webrank ranker, the search and answer and extraction endpoints that agents call. He writes about retrieval, RAG, and benchmarks from inside the engine room.
## FAQ
### What is the best search API for RAG in 2026?
For the dominant RAG workload, factual question answering grounded in web content: Keirolabs at $0.25 per 1,000 semantic searches with clean page text bundled in the same call, 78% on FinanceBench against roughly 19% for a GPT-4o plus vector RAG baseline, and indexed queries at about 100ms. Tavily at $8.00 per 1,000 is the incumbent with the largest integration ecosystem and request-time content. The honest evaluation is a 200-query replay of your own traffic through both free tiers, scored on grounded-answer accuracy, because the answer is workload-dependent by construction.
### Why does standard vector RAG score so much lower on FinanceBench?
The ~19% figure is the failure of chunk-and-retrieve over boilerplate-heavy financial documents: the retrieval stage fetches the wrong chunks, the generator answers confidently from them, and the system's failure reads as model error when it is retrieval error. The oracle setup, the full filing in context, reaches ~85%, which bounds what the model can do when retrieval is not the bottleneck. The gap between those numbers is what a search API with bundled clean text and better ranking closes, and Keiro's 78% is the published demonstration that the gap is mostly closable with a deployable pipeline.
### Do I still need my own vector store if the search API bundles content?
Usually yes, for deduplication, cross-query reuse, and domain-specific corpora, but its role shrinks from the retrieval bottleneck to a cache. With content and embeddings arriving in the API response, the vector store becomes an optimization layer rather than the system's accuracy-determining stage, and teams that make that shift report the retrieval tuning work, chunk sizes, overlap, top-k, mostly stops being the debugging surface. Keep the store for your private documents; stop asking it to fix public-web retrieval.
### How much should a RAG query cost all-in?
The all-in figure is search plus clean page text for the pages the generator actually reads, and the market spans $1 to $16 per 1,000 usable queries depending on bundling. Keiro's one-call path lands near $1 per 1,000 at 3 credits; Tavily pay-as-you-go lands near $16; SerpAPI and Serper look cheap per call and push the fetch-and-parse cost into your own stack, which runs $1 to $5 per 1,000 pages fully loaded. Price the usable query, not the API call, because that is the number the invoice follows.
### Can a search API make my RAG citations auditable?
Only if the content is served from a stable, datable index. Request-time fetches produce citations that may differ between the day an answer was generated and the day it is audited, which fails an audit trail by construction. Index-served content is dated and diffable, so the citation points to a version, and products in finance, legal, and medicine should treat citation stability as a hard requirement when selecting the search layer, ahead of price and latency.