xtractr

Back in 2023 I built a Cloudflare Workers page scraper. xtractr is the proper, grown-up version of that idea: extract clean, structured content from web pages with automatic short-link expansion and lightweight page-type detection.

It follows redirect chains, expands shortened links, and converts page content to clean Markdown with normalised metadata. It uses defuddle to pull out the readable content, and detects content types via Open Graph tags, JSON-LD or domain-based heuristics. It’s built for Cloudflare Workers but works in any runtime with fetch. This now powers the scraping in Otter.