Search & URL Markdown Extractor for LLMs
Turn a search query or a list of URLs into clean Markdown, plain text, or HTML pages, with filters for domain, keyword, and minimum content length. No login or cookies needed.
Open in Apify Store$0.99 / 1K on Free
Short answer
Search & URL Markdown Extractor for LLMs is an Apify Store Actor for Google Search and Bing data that returns 13 published output fields, including url, title, canonicalUrl, description. It lists $0.99 per 1,000 results on the Free plan (pay per event), $0.49 on the Business plan. It takes 25 input settings, none required. It covers 2 of the 5 core Google Search fields this site tracks. Last updated Oct 17, 2026.
From the public Store listing and published build schema, synced 2026-11-12.What it returns
13 fields from the published dataset schema. Export as JSON, CSV or Excel, or read the dataset through the Apify API.
urltitledescriptionwordCounthttpStatusCodeloadedTimesearchRankcanonicalUrlauthorpublishedDatecontentLengthenginecrawlDepth
Not in the schema among the core Google Search fields: snippet, position, serp_features. The field matrix shows which other tools return them.
How it differs from the other Google Search Actors
Among the 4 catalogue Actors for Google Search with a published schema, only this one returns description, wordCount, httpStatusCode, loadedTime, searchRank, canonicalUrl, author, publishedDate.
Pick another Actor if you need
| Actor | Fields it adds | Price / 1K |
|---|---|---|
| Google Image Search Results & Metadata Scraper | retrievalMethod, isFallback, fallbackReason, imageId | $0.99 |
| Google Autocomplete Keyword Expansion Scraper | seed, locale, relevance | $0.99 |
| Google Search Results Scraper: Organic & SERP Boxes | resultsTotal, hasNextPage, organicResults, searchQuery.term | $0.99 |
Input
| Setting | Type | Required | Default | What it does |
|---|---|---|---|---|
| query | string | No | what is retrieval augmented generation | What to look up, or a single web page URL to read. Examples: `vector database comparison`, `https://en.wikipedia.org/wiki/Retrieval-augmented_generation`. When `startUrls` or `queries` are also set, everything is processed in one run. |
| queries | array | No | [] | Additional queries for batch research runs. Each query is searched and its top results are fetched. A query that is itself a URL is fetched directly. |
| startUrls | array | No | [] | Pages to read directly. With `maxCrawlDepth` above 0, the actor walks links found on these pages within the same website instead. |
| searchEngine | string | No | auto | Which search engine to use. `Automatic` tries Google first and silently falls back to Bing, then DuckDuckGo when an engine is blocked — recommended. |
| searchLanguage | string | No | auto | Language for search results. `Automatic` lets each engine choose based on the query. |
| searchCountry | string | No | auto | Country bias for search results. `Automatic` means no country bias. |
| maxResults | integer | No | 3 | How many top organic results to fetch and emit for each search query. Values above 10 need multiple search-result pages (slower). With `Emit results for pages that could not be fetched` on, each of the requested results yields one record — a page record when the fetch succeeds, or a metadata record |
| maxItems | integer | No | 0 | Overall cap on how many pages the run may emit across all queries, URLs, and crawled pages. 0 means no global cap. |
| outputFormats | array | No | ["markdown"] | Which representations to include per page, in any combination. `Links` adds every link found on the page as structured `{url, text}` pairs. |
| removeElementsCssSelector | string | No | nav, footer, aside, script, style, .ad, .adve… | Comma-separated CSS selectors removed before extraction (navigation, footers, ads, cookie bars). Set a non-matching selector such as `dummy_keep_everything` to keep everything. |
Show the other 15 settings
| Setting | Type | Required | Default | What it does |
|---|---|---|---|---|
| keepElementsCssSelector | array | No | [] | When set, only content inside these selectors is kept — everything else is dropped before extraction. |
| htmlTransformer | string | No | readable-text | `Readable content` finds the article body and drops navigation and boilerplate. `None` keeps the full cleaned page. |
| maxContentLength | integer | No | 0 | Truncate the emitted text and Markdown of each page to this many characters. 0 returns the full content. Useful for controlling embedding cost. |
| includeMetadata | boolean | No | true | Add canonical URL, author, published date, keywords, social image, and site name (each only when the page provides it). |
| chunkMarkdown | boolean | No | false | Emit a `chunks` array per page — heading-aware Markdown chunks ready for embedding, never split inside code blocks or tables. |
| chunkMaxChars | integer | No | 2000 | Target size of each chunk when `chunkMarkdown` is on. Code blocks and tables are kept whole even when larger. |
| chunkOverlapChars | integer | No | 200 | Approximate overlap between consecutive chunks when `chunkMarkdown` is on, so context is not lost at chunk boundaries. |
| maxCrawlDepth | integer | No | 0 | 0 reads only the start URLs. 1+ follows links found on those pages, staying on the same website, up to this depth. |
| maxCrawlPages | integer | No | 25 | Safety cap on pages visited in crawl mode. |
| domainAllowlist | array | No | [] | Keep only pages whose host contains one of these substrings (case-insensitive). |
| domainBlocklist | array | No | [] | Drop pages whose host contains one of these substrings. |
| minTextLength | integer | No | – | Drop pages with fewer than this many characters of extracted text. Useful for skipping thin or placeholder pages. |
| excludeContentSelectors | array | No | [] | Additional CSS selectors removed before extraction, on top of `Remove elements`. |
| keywordsAnyOf | array | No | [] | Keep only pages whose title or content contains at least one of these keywords (case-insensitive). |
| emitFailedResults | boolean | No | true | When a search result page cannot be fetched after the full retry ladder, still emit one record for it with `url`, `title`, `description`, `searchResult`, `searchRank` and a `fetchError` note (no fabricated content). Turn off to skip failed pages entirely. |
Example input
Built from the input form's published example values and defaults.
{
"query": "what is retrieval augmented generation",
"queries": [],
"startUrls": [],
"searchEngine": "auto",
"searchLanguage": "auto",
"searchCountry": "auto",
"maxResults": 3,
"maxItems": 0,
"outputFormats": [
"markdown"
],
"removeElementsCssSelector": "nav, footer, aside, script, style, .ad, .advertisement, [aria-hidden='true']",
"keepElementsCssSelector": [],
"htmlTransformer": "readable-text",
"maxContentLength": 0,
"includeMetadata": true,
"chunkMarkdown": false,
"chunkMaxChars": 2000,
"chunkOverlapChars": 200,
"maxCrawlDepth": 0,
"maxCrawlPages": 25,
"domainAllowlist": [],
"domainBlocklist": [],
"excludeContentSelectors": [],
"keywordsAnyOf": [],
"emitFailedResults": false
}Run it from code or an AI agent
The same run through the Apify API, with the example input. You need an Apify API token; the run is billed to your account at the Store price.
Python
from apify_client import ApifyClient
client = ApifyClient("<YOUR_APIFY_TOKEN>")
run_input = {
"query": "what is retrieval augmented generation",
"queries": [],
"startUrls": [],
"searchEngine": "auto",
"searchLanguage": "auto",
"searchCountry": "auto",
"maxResults": 3,
"maxItems": 0,
"outputFormats": [
"markdown"
],
"removeElementsCssSelector": "nav, footer, aside, script, style, .ad, .advertisement, [aria-hidden='True']",
"keepElementsCssSelector": [],
"htmlTransformer": "readable-text",
"maxContentLength": 0,
"includeMetadata": True,
"chunkMarkdown": False,
"chunkMaxChars": 2000,
"chunkOverlapChars": 200,
"maxCrawlDepth": 0,
"maxCrawlPages": 25,
"domainAllowlist": [],
"domainBlocklist": [],
"excludeContentSelectors": [],
"keywordsAnyOf": [],
"emitFailedResults": False
}
run = client.actor("vexaloom-labs/rag-web-browser").call(run_input=run_input)
if run is None:
raise RuntimeError("The Actor run was not found.")
for item in client.dataset(run.default_dataset_id).iterate_items():
print(item)pip install apify-client
JavaScript
import { ApifyClient } from "apify-client";
const client = new ApifyClient({ token: "<YOUR_APIFY_TOKEN>" });
const input = {
"query": "what is retrieval augmented generation",
"queries": [],
"startUrls": [],
"searchEngine": "auto",
"searchLanguage": "auto",
"searchCountry": "auto",
"maxResults": 3,
"maxItems": 0,
"outputFormats": [
"markdown"
],
"removeElementsCssSelector": "nav, footer, aside, script, style, .ad, .advertisement, [aria-hidden='true']",
"keepElementsCssSelector": [],
"htmlTransformer": "readable-text",
"maxContentLength": 0,
"includeMetadata": true,
"chunkMarkdown": false,
"chunkMaxChars": 2000,
"chunkOverlapChars": 200,
"maxCrawlDepth": 0,
"maxCrawlPages": 25,
"domainAllowlist": [],
"domainBlocklist": [],
"excludeContentSelectors": [],
"keywordsAnyOf": [],
"emitFailedResults": false
};
const run = await client.actor("vexaloom-labs/rag-web-browser").call(input);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);npm install apify-client
cURL
curl -X POST "https://api.apify.com/v2/actors/vexaloom-labs~rag-web-browser/run-sync-get-dataset-items" -H "Authorization: Bearer $APIFY_TOKEN" -H "Content-Type: application/json" -d '{"query":"what is retrieval augmented generation","queries":[],"startUrls":[],"searchEngine":"auto","searchLanguage":"auto","searchCountry":"auto","maxResults":3,"maxItems":0,"outputFormats":["markdown"],"removeElementsCssSelector":"nav, footer, aside, script, style, .ad, .advertisement, [aria-hidden='''true''']","keepElementsCssSelector":[],"htmlTransformer":"readable-text","maxContentLength":0,"includeMetadata":true,"chunkMarkdown":false,"chunkMaxChars":2000,"chunkOverlapChars":200,"maxCrawlDepth":0,"maxCrawlPages":25,"domainAllowlist":[],"domainBlocklist":[],"excludeContentSelectors":[],"keywordsAnyOf":[],"emitFailedResults":false}'Waits for the run and returns the dataset items (runs over 300 seconds return HTTP 408; start an async run instead).
MCP (AI agents)
{
"mcpServers": {
"apify": {
"url": "https://mcp.apify.com?tools=vexaloom-labs/rag-web-browser",
"headers": {
"Authorization": "Bearer <YOUR_APIFY_TOKEN>"
}
}
}
}Adds this Actor as a tool in an MCP client that takes a remote server URL (Cursor, VS Code, Claude Code) through Apify's hosted MCP server. OAuth sign-in also works without the header. Clients that only start local servers can wrap the URL with npx mcp-remote.
What a run costs
The result rate falls on higher Apify plans. Each figure is the published per-result rate times the number of results.
| Apify plan | Per 1,000 | 10,000 results | 100,000 results |
|---|---|---|---|
| Free plan | $0.99 | $9.90 | $99 |
| Starter plan | $0.82 | $8.20 | $82 |
| Scale plan | $0.66 | $6.60 | $66 |
| Business plan | $0.49 | $4.90 | $49 |
- Every run also has a start event, charged per GB of memory (minimum one): $0.10 on Free, $0.03 on Business. At its default memory of 4 GB a run starts at $0.40 on Free. Fewer, larger runs keep this small.
- The Free plan includes $5 of monthly usage (Apify pricing, checked 2026-11-11); at the Free rate that covers at most about 5,050 results, less start events and platform usage.
- Worked example: 10,000 results in one run on the Free plan cost $10.30 ($9.90 for results + $0.40 start), plus Apify platform usage (compute), billed separately by Apify.
- Pay per event. Platform usage (compute) is billed by Apify on top of the event prices. Compare other tools in the cost estimator.
Where it fits among Google Search routes
It ranks 1 of 4 catalogue Actors for Google Search on the published formula (fields, price, reliability, freshness). Its line is highlighted on the map.
- 1 Search & URL Markdown Extractor for LLMs 13 output fields, pay per event 2 of 5 core fields
$0.99 / 1K - 2 Google Image Search Results & Metadata Scraper 29 output fields, pay per event 3 of 5 core fields
$0.99 / 1K - 3 Google Autocomplete Keyword Expansion Scraper 7 output fields, pay per event 1 of 5 core fields
$0.99 / 1K - 4 Google Search Results Scraper: Organic & SERP Boxes 18 output fields, pay per event 0 of 5 core fields
$0.99 / 1K - + 7 other Store tools, combined Reference only 4 of 5 core fields
from $0.35 / 1K - API Programmable Search Engine JSON API Limited access fields not compared
Stops are fields. A ring means the field is in that route's published output schema; no ring means the line passes without it. Bold labels are the core fields for this website, the rest are fields that differ between routes. A white capsule means every data line stops there. The red line is the official API; its field coverage is not compared. A crossed red line means there is no open official API.
Official route: Programmable Search Engine JSON API
Limited access. Google's programmable search API returns results for a configured search engine. It is not a general replacement for the live Google results page.
Free allowance plus per-query pricing is published; check the official overview for current rates and caps.
Other Store tools for Google Search
Reference only, from their public listings.
| Tool | Fields | Price / 1K |
|---|---|---|
| Cheapest Google Search Results (Fast and cheap SERP API) | 11 | $0.50 |
| Free Google Search Results Scraper | 11 | $0.35 |
| Google Search Results Scraper (SERP) | 5 | $2.00 |
| Google Search Results Scraper | 27 | $0.60 |
| Google Search Results | – | $0.50 |
What people use it for
- Collect web content
- Prepare retrieval datasets
- Extract citation metadata
Other catalogue Actors for Google Search
Google Image Search Results & Metadata Scraper
Run a Google Images search and get back full-resolution image URLs with dimensions, thumbnails, source page titles, and the web page each image…
Google Autocomplete Keyword Expansion Scraper
Expand seed keywords into long-tail suggestions using Google's autocomplete, with question-word, preposition, comparison, and A-to-Z expansion…
Google Search Results Scraper: Organic & SERP Boxes
Search Google and get organic results plus AI Overview, knowledge panel, People Also Ask, related searches, local pack, top stories and ads for…
Similar Actors
Domain-Scoped Site Crawler for Markdown Datasets
Crawl a website from one or more start URLs, staying on the same domain, and get back clean Markdown, text, or HTML per page with title,…
Amazon Brand Store Page & Merchandising Crawler
Crawl an Amazon Brand Store's navigation, creative modules (images, video, CTAs), and every product link found on its pages. Resolves a brand's…
Wayback Machine Snapshot History & Archived Text Extractor
Pull the Wayback Machine's capture history for any URL, host, or domain with timestamps, status codes, MIME types, and optional archived page text.
Facebook Page & Profile Finder by Keyword or URL
Find Facebook Pages or public profiles by keyword, or scrape a list of Facebook URLs directly, and get back contact info, category, ratings,…
Ahrefs Domain Rating, Backlinks & Traffic Lookup
Look up a domain's public Ahrefs Domain Rating, linking-website count, global rank, and estimated organic traffic, with a mode to fetch just…
Complete Page Capture - Screenshot & PDF
Capture complete full-page screenshots or PDFs of any URL list. Scrolls lazy content, waits for images and fonts, removes fixed headers and…
Questions people ask
How much does Search & URL Markdown Extractor for LLMs cost?
The published result rate depends on your Apify plan: $0.99 per 1,000 on Free, $0.82 per 1,000 on Starter, $0.66 per 1,000 on Scale, $0.49 per 1,000 on Business. So 10,000 results cost about $9.90 on the Free plan and $4.90 on Business, plus a start fee of $0.10 per run per GB of memory on Free. Platform usage is billed separately.
What data does Search & URL Markdown Extractor for LLMs return?
Its published output schema lists 13 fields: url, title, description, wordCount, httpStatusCode, loadedTime, searchRank, canonicalUrl, author, publishedDate, contentLength, engine, and 1 more. A listed field is not guaranteed to be filled in every row.
What input does Search & URL Markdown Extractor for LLMs need?
No input is strictly required; the defaults run as is. Common settings: query, queries, startUrls, searchEngine, searchLanguage, searchCountry.
What are the alternatives to Search & URL Markdown Extractor for LLMs for Google Search?
4 catalogue Actors cover Google Search. Close alternatives: Google Image Search Results & Metadata Scraper; Google Autocomplete Keyword Expansion Scraper; Google Search Results Scraper: Organic & SERP Boxes. The Google Search page compares all of them field by field.
How do I run Search & URL Markdown Extractor for LLMs from Python or the API?
Use the Apify API client: in Python, ApifyClient(token).actor("vexaloom-labs/rag-web-browser").call(run_input=...), then read the run's default dataset (run.default_dataset_id); in JavaScript the same call is client.actor("vexaloom-labs/rag-web-browser").call(input). Without a client, POST the input JSON to https://api.apify.com/v2/actors/vexaloom-labs~rag-web-browser/run-sync-get-dataset-items with your API token. Ready-to-paste code with the example input is on this page.
Sources
- Search & URL Markdown Extractor for LLMs on Apify Store
- Published build with input and output schema (JSON)
- Programmable Search Engine JSON API documentation
Cite this page
Scraper Finder. "Search & URL Markdown Extractor for LLMs." https://scraperfinder.com/catalogue/rag-web-browser/ (data checked 2026-11-12).
Run Search & URL Markdown Extractor for LLMs
$0.99 per 1,000 results. Paste the example input above, set your Google Search query, then export the dataset. Still choosing? Compare every Google Search route.