--- title: "The Search API Crackdown: SerpAPI's Lawsuit, Bing's Shutdown & SearchGuard" dek: "SearchGuard, the Bing Search API shutdown, and Google v. SerpAPI ended the era of scraping Google at scale. Independent indexes are the only safe bet for agents in 2026." category: "guide" tags: [search-api, serpapi, bing, google, ai-agents, scraping] author: "Dave" published: 2026-08-17T12:00:00+00:00 updated: 2026-08-17T12:00:00+00:00 url: https://keirolabs.cloud/blogs/guide/search-api-crackdown-2026-serpapi-lawsuit-bing-shutdown --- > **TL;DR** : Google's SearchGuard, the Bing Search API shutdown, and the Google v. SerpAPI lawsuit closed the era of scraping Google at scale; independent indexes like Keirolabs are the only search APIs insulated from both legal and supply risk. ## The search API market just split in two **The search API market split in two in January 2025, and almost nobody noticed at the time.** I have watched this market for two years. I benchmark these APIs from the bills, not the landing pages. And I can tell you the exact day the old world ended: January 2025, when Google deployed SearchGuard and broke nearly every SERP scraper overnight. Before that day, there was one search API market. You rented access to Google's index, or Bing's, and you resold it. The product was a proxy. The moat was a bypass. After that day, there are two markets. One rents someone else's index and hopes it stays available. The other owns the index and sells access to its own data. The first market is dying. The second is where the money went. Everyone is comparing the wrong numbers. The headline price per 1,000 queries is a floor, not a bill. The real cost is the loop: the extraction call, the token fees, the rate-limit upgrade, the cache you build yourself. And in 2026, that loop just got permanently more expensive for anyone who rents Google's results. Here is the claim I am willing to be wrong about in public: in 24 months, the scraper-based search API will be a footnote. Not a category. A footnote. The companies that survive will be the ones that own their index, and the ones that do not will be acquired for their customer lists or shut down. That is not a prediction. That is the trajectory of the last 18 months, extended. The people who run the scraper APIs know this. Watch what they do, not what they say. They are not buying crawlers. They are not building indexes. They are selling their customer lists and pivoting to whatever Google will tolerate next. The smart ones already moved. The rest are waiting for the next SearchGuard update to make the decision for them. **This changes everything about how you build an agent.** The decision you make today, which search API you build on, determines whether your application breaks in a year. I am going to show you the five events that ended the scraper era, the $500M that flowed to the winners, and the math that makes the choice obvious. ## What happened to the search API market in 2025 and 2026? **The search API market flipped from scraping Google to running your own index in 18 months.** Three events did it: Google deployed SearchGuard in January 2025, Microsoft decommissioned the Bing Search API in August 2025, and Google sued SerpAPI under the DMCA in December 2025. Each one raised the cost of scraping and lowered the value of reselling someone else's index.

| Date | Event | What changed | |---|---|---| | Jan 2025 | Google deploys SearchGuard | JavaScript-based bot protection breaks nearly every SERP scraper overnight | | Aug 2025 | Bing Search API decommissioned | Microsoft replaces it with Grounding with Bing Search, 40-483% more expensive | | Sep 2025 | Google removes the &num=100 parameter | Scrapers must make 10x more requests for the same result count | | Dec 2025 | Google v. SerpAPI filed | DMCA anti-circumvention suit over SearchGuard bypass | | Jul 2026 | SerpAPI wins motion to dismiss | Case not fully closed, but the legal ground shifted |

The five events that ended the scraper era, January 2025 to July 2026. Source: Google, Microsoft lifecycle docs, court docket.

The five events are not five separate stories. They are one story told five times. Google hardened the target, Microsoft pulled the supply, Google removed the cheap parameter, Google filed the lawsuit, and the court handed the scraper ecosystem a partial win that changed nothing about the underlying risk. **Each event hit the same business model: rent someone else's index, resell it, hope nothing changes.** The money followed the shift. More than $500M flowed into the space in 12 months. Tavily was acquired by Nebius for $275M, up to $400M with earnouts. Exa raised $250M at a $2.2B valuation with revenue up 1,010% year over year. You.com raised $100M at $1.5B. Parallel AI raised $100M at $740M. Investors are betting on indexes, not scrapers. We watched this from the inside. Keirolabs runs its own index, so the crackdown did not touch our supply. But we saw the scramble in our own user base: teams that had built on scraper-based APIs started asking about migration within weeks of each event. The pattern was always the same. First a latency spike, then a support ticket, then a migration. Two business models now compete in this market. A scraper-based API rents access to someone else's index, usually Google's, and resells it. An independent index owns the crawl, the storage, and the ranking, and sells access to its own data. **The crackdown made the first model structurally worse and the second one structurally better.** That is the whole story of the last 18 months. ## What is Google SearchGuard and why did it break scraping? **SearchGuard is Google's JavaScript-based anti-bot system, deployed in January 2025, that requires a real browser to run and broke nearly every SERP scraper overnight.** It is the internal name for Google's Web Application Attestation system applied to Search. It runs inside a bytecode virtual machine, tracks mouse movement, keyboard rhythm, scroll behavior, and timing jitter, and rotates cryptographic tokens so bypasses expire within minutes. The visible change was simple: Google began requiring JavaScript to use Search. A Google spokesperson told [TechCrunch](https://techcrunch.com/2025/01/17/google-begins-requiring-javascript-for-google-search/) the change protects "our services and users from bots and evolving forms of abuse and spam." Fewer than 0.1% of searches come from users who disable JavaScript, so the cost to real users was near zero. The cost to scrapers was not. For anyone scraping Google at scale, SearchGuard raised latency and cost by an order of magnitude. Tools that relied on cheap HTTP requests had to run headless browsers, solve challenges, and rotate fingerprints. Several SEO tools went dark for hours or days. The ones that recovered did so by running real browser engines, which is expensive and fragile. The details matter if you are evaluating a search API. SearchGuard is not a CAPTCHA wall. It is behavioral: it watches how a mouse moves, how fast keys are pressed, how a page scrolls. It fingerprints 100+ DOM elements and detects WebDriver, so Puppeteer and Selenium are visible. And it rotates its cryptographic tokens, so a bypass that works today is dead tomorrow. **That is why the disruption was not a one-time event. It is a permanent cost increase.** Here is the part the landing pages do not show you. Every bypass has a maintenance cost. Every SearchGuard update resets the work. The scraper API's cost structure is a treadmill: the harder Google makes it, the more the API has to spend, and the more it has to charge you. That is not a pricing model. That is a tax. I have watched teams burn a month of engineering time on this. They build a scraper, it works for a week, Google rotates the tokens, and they are back to zero. The ones who switched to an index stopped thinking about it entirely. **The difference is not effort. The difference is who controls the target.** SearchGuard also changed the latency math. A scraper that used to answer in 200 milliseconds now has to spin up a browser, run the attestation, solve whatever challenge appears, and parse the rendered DOM. That is not a 200 millisecond call anymore. It is a 2 second call, and for an agent that searches eight times per task, that latency compounds into a visible slowdown. The agent feels it. The user feels it. The bill feels it. ## Why did Microsoft shut down the Bing Search API? **Microsoft decommissioned the Bing Search API on August 11, 2025, and replaced it with Grounding with Bing Search inside Azure AI Agents, at 40-483% higher cost.** The old API stopped accepting new signups and existing instances were decommissioned completely. Microsoft's [lifecycle announcement](https://learn.microsoft.com/en-us/lifecycle/announcements/bing-search-api-retirement) directs customers to the new tool. The replacement is not a drop-in swap. Grounding with Bing Search only works inside Azure AI Agents, now part of Microsoft Foundry Agent Service. It costs more per 1,000 transactions, and it pulls data outside the Azure compliance boundary, which means the Microsoft Data Protection Addendum does not apply. For a developer who just wanted a search API, the migration is a rewrite, not a config change. The 40-483% price range is not a typo. The low end is the cheapest transaction class, the high end is the most expensive. Either way, the direction is the same: Microsoft used the migration to raise prices, and it locked the product into Azure so you cannot take the data anywhere else. **The API was a product. The replacement is a funnel.** The shutdown matters beyond the developers who used Bing directly. A lot of "multi-engine" search APIs quietly proxied Bing. When Microsoft pulled the plug, those APIs lost a supply source overnight. If your application sat on top of one, you found out the hard way. This is the supply risk in its purest form. Microsoft did not break a bypass. It did not sue anyone. It just stopped selling the product. Every API that depended on Bing had to find a new source, renegotiate, or shut down. **That is the risk of building on someone else's index, and it applies to Google scrapers just as much as it applied to Bing proxies.** Read Microsoft's move the way a strategist would. Microsoft did not kill the Bing Search API because it was losing money. It killed it because search results are now the input to an AI agent, and Microsoft wants that input to flow through Azure. The message to every API reseller was unambiguous: you do not get to be the middleman anymore. The compliance detail is the one most people miss. The old Bing Search API was covered by the Microsoft Data Protection Addendum. The replacement pulls data outside the Azure compliance boundary, so that addendum does not apply. For a regulated customer, that is not a pricing problem. That is a dealbreaker. Microsoft did not just raise the price. It changed the legal terms of the data. ## What is the Google v. SerpAPI lawsuit about? **Google sued SerpAPI in December 2025 under the DMCA's anti-circumvention provisions, claiming SerpAPI bypassed SearchGuard to scrape hundreds of millions of queries a day.** The case is Google LLC v. SerpApi, LLC, No. 25-cv-10826, in the Northern District of California. Google alleges SerpAPI circumvented a technological measure and trafficked in circumvention technology, reselling scraped results to paying customers. Hundreds of millions of queries a day. Let that number sit. That is not a hobbyist scraping Google for a side project. That is industrial-scale resale of Google's index, and it is exactly the kind of operation the DMCA was written to reach. The DMCA angle is the new part. Earlier scraping fights, like hiQ v. LinkedIn, were about contract and trespass. The DMCA's Section 1201 makes it illegal to circumvent an access control, and it carries statutory damages. Google argued that SearchGuard is an access control protecting copyrighted content, and that SerpAPI's rotating IPs, browser emulation, and CAPTCHA solving are circumvention. **This case is different from every previous scraping case because it attacks the tool, not the act.** hiQ v. LinkedIn was about whether you could scrape public data. This case is about whether the tool you use to scrape is itself illegal. That is a different question, and it is a much more dangerous one for the scraper industry. Contract law is a dispute between two parties. The DMCA is a statute with statutory damages attached. SerpAPI's answer was blunt. Its [motion to dismiss](https://storage.courtlistener.com/recap/gov.uscourts.cand.461513/gov.uscourts.cand.461513.22.0.pdf) argued Google is not a copyright owner, SearchGuard is a website control protecting ad revenue rather than a copyright control, and the content is publicly available anyway. The company's [CEO published a post](https://serpapi.com/blog/google-v-serpapi-motion-to-dismiss-why-were-in-the-right/) arguing Google is "the largest scraper on the planet" and is trying to weaponize the DMCA to protect its ad business. The case is worth reading even if you do not use SerpAPI. It is the first serious test of whether the DMCA can be used to stop search scraping. If Google wins on appeal, every scraper-based API faces the same theory. If SerpAPI wins, the theory is weakened. **Either way, the uncertainty is the point: it is now a legal risk to build on a Google scraper, not just an operational one.** The timing tells you something. Google filed in December 2025, three months after it removed the &num=100 parameter and eleven months after SearchGuard launched. Google did not file because scraping was new. It filed because scraping had not stopped. The technical controls slowed the scrapers down, but the scrapers kept going. The lawsuit is Google's admission that the technical arms race alone was not enough. ## Did SerpAPI win the case? **SerpAPI won its motion to dismiss in July 2026, but the case is not fully closed.** The court agreed with enough of SerpAPI's arguments to dismiss the claims, but the ruling is not a final judgment. Google can amend or appeal. The practical effect is that the DMCA anti-circumvention theory against Google scraping is on shaky ground, at least for now.

SerpAPI's motion to dismiss put a number on Google's DMCA theory: $7.06 trillion in statutory damages, more than US GDP. Source: SerpAPI motion to dismiss, No. 25-cv-10826 (N.D. Cal.).

The win matters for the whole scraper ecosystem, not just SerpAPI. If the DMCA theory had held, any API that bypasses SearchGuard would face the same statutory damages math. SerpAPI's filing put a number on it: Google's interpretation could yield $7.06 trillion in statutory damages, more than US GDP. **That number is why the case got attention.** But a motion to dismiss is not a license. The case is still alive in some form, and Google has resources to keep litigating. The legal risk around scraping Google did not disappear. It just moved from certain to unresolved. For a developer choosing an API in 2026, "probably fine" is not the same as "no exposure." Do not read the dismissal as a green light. A motion to dismiss tests whether the complaint states a claim, not whether the claim is true. The court said SerpAPI's arguments were strong enough to survive the first gate. Google can amend the complaint, and Google does not lose lawsuits by giving up. Here is what I know: the legal theory is now wounded, but the operational reality is unchanged. SearchGuard is still there. The bypasses still expire. The browser farms still cost money. **Even a total SerpAPI victory would not make scraping Google cheap again.** The lawsuit was never the only problem. It was the exclamation point. ## What is the risk to scraper-based search APIs? **Scraper-based APIs carry two kinds of risk: supply risk and legal risk, and both are now live.** Supply risk is what the Bing shutdown demonstrated: when the upstream source disappears, your API breaks with no warning. Legal risk is what the SerpAPI case demonstrated: even if you win, you spend years and millions defending a lawsuit. SerpAPI and Serper are the two big Google-scraped APIs. Both depend on Google's search results, which means both depend on Google's tolerance. SearchGuard already raised their cost structure. The lawsuit raised their legal exposure. And Google controls the timeline: it can change SearchGuard, remove parameters, or file more cases whenever it wants. The fragility is structural. A scraper-based API is a middleman with no asset of its own. It does not own an index, a crawler, or a ranking system. It owns a bypass. **When the bypass stops working, the product stops working.** That is not a business model you want to build an application on. The cost math compounds. SearchGuard forced scrapers into headless browsers, which multiplied their infrastructure spend. The &num=100 removal forced them to make 10x more requests for the same result count. Both changes hit the same APIs at the same time. The result is a category that is more expensive to run and more exposed to legal action, while charging customers more to cover it. The &num=100 removal is the quietest event on the timeline and the most expensive one in practice. For years, scrapers asked Google for 100 results in a single request. Google removed the parameter, so the same 100 results now take 10 requests. Ten times the requests, ten times the rate-limit surface, ten times the chance of a block. Nobody wrote a press release about it. Every scraper felt it. | Property | Scraper-based API (SerpAPI, Serper) | Independent index (Exa, Brave, Parallel, Linkup, Keirolabs) | |---|---|---| | Owns the index | No | Yes | | Legal exposure | Google v. SerpAPI theory applies | None from Google scraping | | Supply risk | Bing shutdown, SearchGuard changes | None, they are the supply | | Cost structure | Bypasses, browser farms, CAPTCHAs | Crawl once, amortize across every query | | Typical price | Several dollars per 1K | $0.10-$0.25 per 1K (Keirolabs) | Ask any scraper API what happens if Google changes SearchGuard tomorrow. The answer is always some version of "we adapt." **That is the sentence that tells you who carries the risk.** When a provider says "we adapt," they mean you wait, your application breaks, and they fix it on their schedule. You are the one with the broken product. ## Why do independent indexes win? **Independent indexes own their crawl, their index, and their ranking, so they are insulated from both the legal risk of the Google-SerpAPI case and the supply risk of the Bing shutdown.** Exa, Brave, Parallel, Linkup, and Keirolabs all run their own indexes. Nobody can sue them for scraping Google, because they do not scrape Google. Nobody can shut off their supply, because they are the supply. The funding numbers show where the market is going. Exa raised $250M at a $2.2B valuation with revenue up 1,010% year over year, per its [Series C announcement](https://exa.ai/blog/announcing-series-c). Parallel AI raised $100M at $740M to build its own index. You.com raised $100M at $1.5B. Tavily, which [Nebius agreed to buy for $275M](https://www.bloomberg.com/news/articles/2026-02-10/nebius-agrees-to-buy-ai-agent-search-company-tavily-for-275-million), positioned itself as an AI-native search layer. The pattern is consistent: capital is flowing to companies that control their own data. | Company | Round | Amount | Valuation | What they build | |---|---|---|---|---| | Tavily | Acquired by Nebius | $275M (up to $400M) | n/a | AI-native search API | | Exa | Series C | $250M | $2.2B | Independent web index | | You.com | Series C | $100M | $1.5B | AI search + API | | Parallel AI | Series A | $100M | $740M | Independent web index |

Deals in the search API market over 12 months: Tavily acquired by Nebius for $275M (up to $400M with earnouts), Exa raised $250M at a $2.2B valuation, You.com raised $100M at $1.5B, Parallel AI raised $100M at $740M. Source: company announcements.

Exa's revenue is up 1,010% year over year. Read that number again. Ten times the revenue in twelve months, at a $2.2B valuation. **That is not a company surviving a market shift. That is a company being the market shift.** The scraper APIs are not printing numbers like that, because the market is not paying them to exist anymore. An independent index also has a different cost curve. A scraper pays for bypasses, browser farms, and CAPTCHA solving, and those costs go up as the target hardens. An index pays for crawling and storage once, then amortizes it across every query. That is why an indexed API can charge $0.25 per 1,000 semantic searches while a scraper-based API charges several dollars and still has thin margins. The unit economics are the whole story. A scraper pays per query: every query costs a bypass, a browser, a CAPTCHA. An index pays per crawl: one crawl of the web, amortized across every query that ever hits it. The first model gets more expensive as you scale. The second gets cheaper. **That is not a small difference. That is the difference between a business and a tax.** There is a quality angle too. A scraper returns whatever the upstream returns, formatted for a human browser. An index can rank for the query, not for the page. That is the difference between a SERP screenshot and a search result. For an AI agent that needs an answer, not a list of blue links, the index is the better input. None of this is hypothetical. The 18-month timeline is the evidence. Every event in the table above hit scraper-based APIs directly and left index-based APIs untouched. **That asymmetry is not luck. It is the difference between renting a dependency and owning one.** ## What does this mean for developers? **If you are building an AI agent or RAG pipeline in 2026, pick an indexed search API and do not build on a scraper.** The decision you make today determines whether your application breaks in a year. A scraper-based API can be disrupted by a Google change, a Microsoft shutdown, or a court ruling. An indexed API can be disrupted by none of those. The practical checklist is short. Ask whether the provider owns its index. Ask what happens if Google changes SearchGuard tomorrow. Ask what happens if the provider loses a lawsuit. **If the answer to any of those is "we adapt," you are carrying the risk, not the provider.** Concretely, the difference shows up in failure modes. A scraper-based API fails when the upstream changes its bot detection, which happens without notice and without a changelog. An indexed API fails when its own infrastructure has a problem, which is visible, measurable, and fixable. One failure mode is a dependency you cannot debug. The other is a dependency you own. Do the due diligence like an investor, because you are investing your application's uptime. Read the provider's pricing page for the word "Google." Read the docs for the word "proxy." If the product is a proxy, the product is a liability. If the product is an index, the product is an asset. There is a second-order effect that most teams miss. When you build on a scraper, you are not just renting results. You are renting a rate limit, a retry policy, and a support queue that sits between you and the actual problem. When something breaks, you cannot debug it, because you cannot see the upstream. You file a ticket and wait. An index gives you a failure mode you can actually investigate, which is the difference between a dependency and a black box. The market context makes the choice easier. More than $500M flowed into the space in 12 months, and the winners are the index owners. Tavily went to Nebius for $275M. Exa is worth $2.2B. You.com raised $100M at $1.5B. The scraper-based APIs are not getting those rounds, because investors can read the same risk you can. The migration cost is real, and I am not going to pretend it is zero. You have to change the base URL, re-test the response shape, re-tune the retry logic. That is a day of work, maybe two. Compare that to the cost of a scraper API going dark on a Tuesday with no changelog. **The migration is the cheapest insurance you will ever buy.** We made this call early. Keirolabs built an index instead of a scraper, and the last 18 months validated it. When SearchGuard landed, our latency did not change. When Bing shut down, our supply did not change. When the lawsuit was filed, our legal exposure did not change. **That is the property you want in a dependency.** ## What should you build on in 2026? **Build on an independent index. Keirolabs runs its own index, charges $0.25 per 1,000 semantic searches and $0.10 per 1,000 SERP queries, and offers a free tier of 1,000 queries a month.** It is not a Google scraper and it is not a Bing proxy. It crawls, indexes, and ranks the web itself, so there is no upstream to shut off and no bypass to defend in court.

List price per 1,000 queries, 2026. Keirolabs: $0.10/1K SERP, $0.25/1K semantic. Scraper-based APIs charge several dollars per 1K because they pay for bypasses, browser farms, and CAPTCHAs. Source: vendor pricing pages.

The quality numbers back the price. Keirolabs scores 78% on FinanceBench and 84% on SimpleQA. Those are not marketing numbers. They are the retrieval layer's contribution to an agent that has to answer questions with evidence. A scraper cannot publish a FinanceBench score, because a scraper does not rank anything. It forwards whatever Google sends. The API is a single POST to `https://api.keirolabs.cloud/api/v2/search/content`. You get semantic search over the index, SERP results, and content extraction in one call. ```bash curl -X POST https://api.keirolabs.cloud/api/v2/search/content \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"query": "latest AI search API news", "maxResults": 10}' ``` The free tier is 1,000 queries a month, which is enough to build and test a real application before you pay anything. The search API market just went through an 18-month consolidation. The scraper era is closing. If you are starting a new project, start on an index. Sign up at [keirolabs.cloud](https://keirolabs.cloud) and run your first 1,000 queries free. **The next 12 months will make this look obvious.** The scraper era is closing, and the index era is opening. The teams that move now will be the ones that do not have to explain to their users why their search broke. The teams that wait will be the ones writing the post-mortem. I know which one I am building on. *Last updated: August 2026.* ## About the author Dave builds AI agent and RAG pipelines and writes about the infrastructure behind them. He benchmarks search APIs from the bills, not the landing pages. You can follow his work on [GitHub](https://github.com/Manasbh). ## FAQ ### Is SerpAPI legal? SerpAPI won its motion to dismiss in July 2026, but the case is not fully closed. Google sued under DMCA anti-circumvention in December 2025, arguing SerpAPI bypassed SearchGuard to scrape results. The ruling weakens the theory, but the legal risk around Google scraping is not zero. ### Why did Microsoft shut down the Bing Search API? Microsoft decommissioned the Bing Search API on August 11, 2025, and replaced it with Grounding with Bing Search inside Azure AI Agents. The replacement costs 40-483% more per 1,000 transactions and only works inside the Azure ecosystem. ### What is Google SearchGuard? SearchGuard is Google's JavaScript-based anti-bot system, deployed in January 2025. It requires JavaScript, tracks mouse movement, keyboard rhythm, and scroll behavior, and rotates cryptographic tokens. It broke nearly every SERP scraper overnight and raised scraping latency and cost by an order of magnitude. ### What is the safest search API to build on in 2026? An independent index that does not scrape Google. Exa, Brave, Parallel, Linkup, and Keirolabs all run their own indexes, so they carry neither the legal risk of the Google-SerpAPI case nor the supply risk of the Bing shutdown. Keirolabs charges $0.25/1K semantic search and $0.10/1K SERP with a free tier of 1,000 queries a month.