CC-News
CC-News is a dedicated news stream from the Common Crawl Foundation. Alongside its monthly web snapshots, Common Crawl continuously crawls news sites and packages the results as a separate dataset, so you get a steady flow of articles rather than a single point-in-time dump.
The data is published as WARC files (Web ARChive, the standard format that stores each crawled page's full HTTP response), the same format as the main Common Crawl archives. Because it is hosted on AWS as a public dataset, the cheapest way to work with it is from inside AWS, where there is no transfer cost. A ready-cleaned version is also mirrored on Hugging Face for quick loading.
For RAG, CC-News is a convenient source of news text for building current-affairs knowledge bases or fine-tuning models on journalistic writing. It has been used in several pretraining pipelines, including the corpus behind RoBERTa.
Related sources
GDELT
GDELT, the Global Database of Events, Language, and Tone, monitors print, broadcast, and web news in more than 100 languages from every country. It records events back to 1979, builds a Global Knowledge Graph, and adds sentiment and emotion analysis, all queryable on Google BigQuery.
Internet Archive
A nonprofit digital library giving free access to millions of books, films, audio recordings, software, archived websites, and television news. Its Wayback Machine has saved more than 800 billion web pages, making it a deep well of historical and current text.