Data Processing

Apache Airflow

Workflow orchestration system

Azkaban

Distributed workflow manager

Apache Beam

Unified batch and stream processing model

Apache Flink

Unified stream and batch processing engine

AWS Glue

Serverless ETL service

Apache Kafka

Distributed streaming platform

Apache Oozie

Hadoop workflow scheduler

Presto

Distributed SQL query engine

Apache Spark

Unified data processing engine