PulseAugur
EN
LIVE 11:48:19

Web search APIs use own indexes or scrape Google, with Bing API retired

Web search APIs employ various methods for sourcing results, with some operating their own independent indexes and others relying on scraping public search engine pages. Brave, Exa, and Perplexity claim to maintain their own indexes, while services like SerpApi and Serper access results by scraping Google's live pages. Microsoft has retired its Bing Search API, directing users to Azure AI Agents, which provides model-generated answers with citations rather than raw search results. Google's Custom Search JSON API is also being phased out, leaving scraping as a primary method for accessing Google results, though this carries legal risks. AI

IMPACT Clarifies how LLM-integrated search tools source information, impacting agent reliability and data provenance.

RANK_REASON The article discusses the operational methods and sourcing strategies of various web search APIs, including their use of independent indexes versus scraping public search engine results.

Read on dev.to — LLM tag →

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →

Web search APIs use own indexes or scrape Google, with Bing API retired

How we ranked this

Signal score
45 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
The article discusses the operational methods and sourcing strategies of various web search APIs, including their use of independent indexes versus scraping public search engine results.
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
product, infra
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
Breaking (< 6h)
Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

Full methodology in our editorial standards.

COVERAGE [1]

  1. dev.to — LLM tag TIER_1 English(EN) · Ricardo Batista ·

    Do web search APIs have their own index, or do they resell Google and Bing results?

    <p>Both kinds exist. Brave, Exa, Parallel and Perplexity say in their own docs that they crawl and run their own index, while SerpApi, Serper and the other Google SERP APIs return Google's live results page. Almost nobody resells Bing any more, because Microsoft retired the Bing …