Scraping & crawling

45 repos

  1. Agent-Reach by Panniantong, 112k stars. CLI giving agents read and search access to Twitter, Reddit, YouTube and more
  2. Scrapling by D4Vinci, 103k stars. Python scraping framework covering single requests through full crawls
  3. crawl4ai by unclecode, 87k stars. Apache-2.0 web crawler that outputs markdown for LLM pipelines
  4. huginn by huginn, 51k stars. Build agents that watch the web and act on events for you
  5. browser by lightpanda-io, 42k stars. Headless browser in Zig that speaks the Chrome DevTools protocol
  6. changedetection.io by dgtlmoon, 36k stars. Self-hosted website change monitor with alerts for price and content changes
  7. obscura by h4ckf0r0day, 35k stars. Headless browser built for AI agents and scraping with CDP support
  8. CloakBrowser by CloakHQ, 33k stars. Chromium build with fingerprint patches that works as a Playwright drop-in
  9. Scrapegraph-ai by ScrapeGraphAI, 33k stars. Python library that uses an LLM to extract data from web pages
  10. crawlee by apify, 26k stars. Node.js library for building reliable web crawlers and browser automation
  11. ai-goofish-monitor by Usagi-org, 20k stars. Playwright and AI monitor for Xianyu listings with scheduled scans and a UI
  12. maxun by getmaxun, 18k stars. No-code tool that turns web pages into structured data feeds and APIs
  13. monolith by Y2Z, 16k stars. Saves a full web page into one HTML file, as GUI, CLI or library
  14. browserless by browserless, 14k stars. Runs headless Chrome and Firefox in Docker for automation and scraping
  15. camofox-browser by jo-inc, 12k stars. Headless browser that evades bot detection, usable from Puppeteer or Playwright
  16. pinchtab by pinchtab, 11k stars. Browser automation bridge that runs multiple Chrome instances with a dashboard
  17. google-maps-scraper by gosom, 8.1k stars. Scraper that pulls place details, ratings and reviews from Google Maps
  18. cli by brightdata, 6.2k stars. Terminal CLI for scraping, searching, and extracting structured web data
  19. Website-downloader by AhmadIbrahiim, 6.1k stars. Node.js script that saves a site's HTML, scripts, styles and images locally
  20. patchright by Kaliiiiiiiiii-Vinyzu, 5.2k stars. Playwright fork that avoids common bot detection checks
  21. pipet by bjesus, 4.8k stars. Command line tool for scraping and pulling data out of web pages
  22. kage by tamnd, 3.5k stars. Downloads a website for offline viewing with scripts removed
  23. AnyCrawl by any4ai, 3.5k stars. Node.js crawler that turns sites and search results into structured data
  24. invisible_playwright by feder-cr, 2.6k stars. Python Playwright API on a stealth Firefox build that evades bot detection
  25. google-maps-scraper-kit by Mahanaicoach, 2.5k stars. Python scraper that exports Google Maps business listings and contact links to CSV
  26. HeadlessX by saifyxpro, 2.4k stars. Self-hosted browser automation platform built on Camoufox Firefox
  27. notte by nottelabs, 2.1k stars. Hosted cloud browsers that AI agents can drive for web automation
  28. ff-ruyijs by cutetailbud, 1.2k stars. Runs site JavaScript from Python on a real Firefox engine for automation
  29. x-tweet-fetcher by ythx-101, 1.0k stars. Fetches tweets, replies and timelines from X without an API key or login
  30. invisible_playwright_mcp by feder-cr, 865 stars. MCP server that gives agents a stealth Firefox browser through Playwright
  31. Scout by kiryano, 807 stars. CLI that scrapes social profiles for leads, pulls emails and exports CSV
  32. fortress by tiliondev, 802 stars. Stealth Chromium build that avoids bot detection for Playwright and Puppeteer
  33. video-scraping-apis by cporter202, 767 stars. Collection of APIs for pulling video metadata and engagement stats across platforms
  34. cfsolve by chris124567, 762 stars. Rust solver for Cloudflare Turnstile and challenge pages, built from request reversing
  35. oc by only-cli, 695 stars. Wraps a website as a compact CLI so agents can browse with few tokens
  36. goscrapy by tech-engine, 373 stars. Scraping framework in Go modeled on Python's Scrapy spider design
  37. PyScrappy by mldsveda, 265 stars. Python web scraping toolkit with self-healing selectors and an MCP server
  38. jev-social by socai-io, 212 stars. Local research agent that reads Instagram, TikTok and LinkedIn with cited captures
  39. foxprofile by chienbm98, 202 stars. Browser profile manager with per-profile fingerprints, cookies and proxies
  40. crawlie by spronta, 127 stars. Fast open source technical SEO and GEO crawler with an MCP interface

New repos every two days, stars refreshed daily. Visits are counted without cookies, and the numbers are public. How it works Data RSS Newsletter

The best new open source, once a weekOne short email: the repo everyone is suddenly starring, a handful worth trying, and one hidden gem, each with a plain note on what it does. Free, with no tracking pixels.

Read past issues

↑ ↓ to move   Enter to choose   Esc to close

Follow starwire by RSSCopy an address into your feed reader.
The day's bestOne item a day: the repos rising fastest and the biggest new picks.starwire.foo/top.xml
Every new pickAll of them as they are added, up to 60 a day.starwire.foo/feed.xml
The weekly emailEach issue of the newsletter, once a week.list.starwire.foo/archive.xml