A
Airbyte
Open-source ELT data integration and connector platform
← Home
Data Processing
This category collects batch processing, streaming, scheduling, integration, and compute frameworks for stable data pipelines.
Apache Airflow
Workflow orchestration system
Azkaban
Distributed workflow manager
Apache Beam
Unified batch and stream processing model
D
Dagster
Software-defined orchestration platform centered on data assets
d
dbt Core
SQL transformation and modeling framework for analytics engineering
D
Debezium
Open-source platform for database change data capture
A
Apache DolphinScheduler
Distributed visual workflow scheduler for big data platforms
D
Dremio
SQL query and semantic layer platform for Apache Iceberg lakehouses
F
Flink CDC
Apache Flink-based framework for real-time CDC data integration
Apache Flink
Unified stream and batch processing engine
AWS Glue
Serverless ETL service
Apache Kafka
Distributed streaming platform
K
Kestra
Event-driven open-source orchestration and automation platform
A
Apache NiFi
Visual dataflow automation and integration platform
Apache Oozie
Hadoop workflow scheduler
P
Prefect
Orchestration platform for Python dataflows and automation tasks
Presto
Distributed SQL query engine
A
Apache SeaTunnel
Open-source platform for large-scale data synchronization and integration
Apache Spark
Unified data processing engine
T
Trino
Distributed SQL query engine for lakehouse and multi-source analytics