News, research and reference data: sources and access
News aggregators, scholarly indexes, patent databases, encyclopaedias and archives each publish data differently. Some offer open dumps or public APIs, others none; article text from publishers carries copyright that metadata does not.
The short answer
For news and research data, start from the record you need: an article with its date and publisher, a paper with its citations, a patent with its assignee, or a reference page with its revision. Several of these sources publish open datasets or APIs; use them first where they fit.
13 websites, 13 catalogue Actors, 13 of them with a published output schema. Data checked 2026-11-12. How the routes are scored.Websites in this group
Each website link opens the full comparison, field matrix, official API route and legal notes. The table shows how many catalogue Actors cover each website, how many publish an output schema, the cheapest published rate per 1,000 and the official access status.
| Website | Category | Catalogue tools | Schemas | Cheapest / 1K | Official API |
|---|---|---|---|---|---|
| Google News news.google.com | News and publishing | 1 | 1 | $0.99 / 1K | No open API |
| Yahoo News news.yahoo.com | News and publishing | 1 | 1 | $0.99 / 1K | No open API |
| Google Scholar scholar.google.com | Research and reference | 1 | 1 | $0.99 / 1K | No open API |
| Google Patents patents.google.com | Research and reference | 1 | 1 | $0.99 / 1K | Public API |
| Wikipedia wikipedia.org | Research and reference | 1 | 1 | $0.99 / 1K | Public API |
| Wayback Machine web.archive.org | Research and reference | 1 | 1 | $0.99 / 1K | Public API |
| Hacker News news.ycombinator.com | News and publishing | 1 | 1 | $0.99 / 1K | Public API |
| Medium medium.com | News and publishing | 1 | 1 | $0.99 / 1K | Limited access |
| Substack substack.com | News and publishing | 1 | 1 | $0.99 / 1K | Limited access |
| Crunchbase crunchbase.com | Startups and products | 1 | 1 | $0.99 / 1K | Limited access |
| Brazil CNPJ registry gov.br | Business registries | 1 | 1 | $0.99 / 1K | Public API |
| Florida Sunbiz sunbiz.org | Business registries | 1 | 1 | $0.99 / 1K | Public API |
| Udemy udemy.com | Education | 1 | 1 | $0.99 / 1K | Retired |
Counts are catalogue Actors mapped to the website, not every tool on the public Store. A schema count is the number of tools with at least three published output fields. Prices exclude Actor start events and plan differences.
Ranked lists, API pricing and error fixes in this group
- Official API pricingGoogle Patents API pricing, limits and access
- Error fix403 Forbidden with cf-mitigated: challenge: Cloudflare blocks
- Retired toolUdemy API retired: what still works for Udemy data
Catalogue starting points
These are the highest-ranked catalogue routes across the vertical, one per website where possible. They are a starting point for the use case, not a claim that another catalogue Actor cannot fit a narrower job.
Google News Keyword & Full-Text Article Scraper
Search Google News by keyword with real resolved article links, headlines, sources, and dates, plus optional full article text and image…
Yahoo News Trending, Category & Keyword Feed by Region
Get Yahoo News trending top stories, category feeds, or keyword-filtered headlines across six regional editions, with source, publish date, image,…
Google Scholar Paper & Citation Search Scraper
Search Google Scholar by keyword and get structured records for papers and citations, with author, venue, year, citation counts, and links to PDFs…
Google Patents Inventor, Assignee & Citation Lookup
Search Google's aggregated patent index by keyword, inventor, assignee, classification code, or citation graph, and get abstract, claims, dates,…
Wikipedia Article Lookup by URL or Keyword
Fetch structured data for Wikipedia articles by URL or keyword search, in any language edition - summary, description, categories, images, and…
Wayback Machine Snapshot History & Archived Text Extractor
Pull the Wayback Machine's capture history for any URL, host, or domain with timestamps, status codes, MIME types, and optional archived page text.
What teams use this data for
Media monitoring
Track headlines and articles for keywords, companies or topics with publisher, date and URL. Store the publish date as published, not the date you collected it.
Literature and patent research
Collect papers or patents with titles, authors or inventors, dates, citations and identifiers for review or analysis.
Reference and change history
Read reference pages or archived snapshots to see what a page said on a given date, with the snapshot or revision ID as the citation.
What to check before you build
- Prefer official dumps and APIs where they exist (Wikipedia, Google Patents public datasets, Hacker News) before page collection.
- Article text belongs to the publisher. Headline, date, publisher and URL are usually enough for monitoring.
- Search-engine news results change by location and time; record both with every result.
Use the field finder for a specific field, the cost estimator for a row count, and the complete Actor catalogue for the full shelf.
Questions people ask
Is there an official API for Google News or Google Scholar?
Google does not offer a public API for Google News results or Google Scholar. Google Patents data is available through public BigQuery datasets. Each website page here states the current official route with its source.
What is the cleanest source for reference data?
Wikipedia publishes official APIs and database dumps, and the Wayback Machine has its own availability and CDX endpoints. Use those before collecting rendered pages.
Can I republish collected news articles?
Collecting headlines, dates and links for monitoring is different from republishing article text, which is the publisher's copyright. Check the publisher's terms. Not legal advice.
Sources and data notes
The website rows are generated from the catalogue. Official API status comes from each website's documented route and is dated below.
- Google News API documentation
- Yahoo News API documentation
- Google Scholar API documentation
- Google Patents API documentation
- Wikipedia API documentation
- Data sources and check policy
- Ranking method and limits
Get Google News data with a catalogue Actor
Google News Keyword & Full-Text Article Scraper returns 6 of 6 core Google News fields at $0.99 per 1,000 results. Compare all 9 Google News routes, or see Yahoo News, Google Scholar, Google Patents and Wikipedia.