News Ingestion API
Serverless news aggregator & RSS pipeline deployed on Cloudflare Workers.
01 / Overview
A high-throughput, edge-native news ingestion service running on Cloudflare Workers. Consolidates heterogeneous news streams—including RSS feeds, developer networks, and public Telegram channels—into clean, unified RESTful JSON endpoints with sub-second response times and automated scheduled polling.
02 / Architecture & Engineering
Implemented in TypeScript on Cloudflare Workers to deliver high concurrency with minimal operational overhead. Custom ingestion adapters normalize heterogeneous source protocols (XML/RSS parsers, REST webhooks, and Telegram channel scrapers) into a standardized article schema. Normalized feeds are cached and exposed via clean REST endpoints such as /api/hacker-news, /api/shega, /api/dev, and /api/telegram/:channel.
03 / Technical Highlights
Multi-Source Aggregation: Continuously ingests articles from RSS feeds (Hacker News, Shega), developer communities (DEV Community), and Telegram channels (ByteTikvah, Addis Standard, Meseret Media).
Normalized Data Contract: Harmonizes disparate payload structures into a unified schema containing external IDs, summaries, author metadata, tags, and media URLs.
Edge-Native Execution: Runs globally on Cloudflare Workers edge runtime, minimizing latency worldwide with sub-second cold starts and minimal operational overhead.
Automated Ingestion Engine: Scheduled background pollers fetch upstream updates, deduplicate incoming items, and serve pre-cached feeds through specialized endpoints.
Technologies Used
Deliverables
- ▸ Production Edge API on Cloudflare Workers
- ▸ Multi-Source Feeds (RSS, Dev.to, Telegram)
- ▸ Unified JSON Article Normalization Engine
- ▸ Automated Ingestion & Edge Cache Pipeline