- PaddleOCR by PaddlePaddle, 93k stars. OCR toolkit that turns PDFs and images into structured data for LLMs
- MinerU by opendatalab, 84k stars. Converts PDFs and Office files into markdown or JSON for LLM pipelines
- pathway by pathwaycom, 62k stars. Python stream processing framework with connectors, real-time analytics and RAG
- airflow by apache, 47k stars. Platform to write, schedule, and monitor data pipelines as Python code
- opendataloader-pdf by opendataloader-project, 30k stars. Java PDF parser that outputs JSON, markdown and HTML with accessibility tagging
- rocketmq by apache, 23k stars. Cloud native messaging and streaming platform for event-driven systems
- unstructured by Unstructured-IO, 16k stars. Open source ETL that converts documents like PDFs into structured data
- rabbitmq-server by rabbitmq, 14k stars. Message broker server supporting AMQP, MQTT and built-in plugins
- cocoindex by cocoindex-io, 12k stars. Incremental data processing engine for indexing and transforming data
- clearml by clearml, 6.9k stars. Experiment tracking, pipelines and serving for ML workloads
- river by riverqueue, 5.9k stars. Background job queue for Go, Java, JS, Ruby and Rust on Postgres or SQLite
- Hyper-Extract by yifanfeng97, 4.2k stars. Uses LLMs to turn unstructured text into graphs, hypergraphs and temporal records
- ingestr by bruin-data, 4.0k stars. Command line tool that copies data between databases in one command
- etl by supabase, 2.4k stars. Rust library and binary for streaming Postgres logical replication to other targets
- bruin by bruin-data, 1.8k stars. Builds data pipelines in SQL and Python with ingestion and quality checks
- clickhouse-etl by glassflow, 505 stars. Open source ingestion and transformation pipelines into ClickHouse
- filament by galaxy-io, 319 stars. Pluggable data replication with checkpoints, batching and integrity events
- rocky by rocky-data, 311 stars. SQL transformation engine that type-checks pipelines and tracks column lineage
- hflow by Hebbian-Robotics, 310 stars. SDK for checking robotics data quality before AI model training
- streambed by viggy28, 310 stars. Streams Postgres changes to Iceberg tables on S3 via logical replication
- datamimic by rapiddweller, 117 stars. Generates deterministic synthetic test data from models, with an MCP integration
- usenetarchive by wolfpld, 74 stars. Toolkit for archiving and reading Usenet newsgroup data
New repos every two days, stars refreshed daily. Visits are counted without cookies, and the numbers are public. How it works Data RSS Newsletter