Partager le stage
ETL pipelines are crucial in the retail industry as they help retailers manage and leverage their data effectively. They’re used to extract batches of data from different sources, transform it into a consistent format, and load it into a database for analysis and reporting.
Quality checks are also often implemented to ensure data accuracy and to help identify and address data issues early in the process.
This project is to automate ETL batch pipelines by:
Scala, Spark, Cassandra, ADLS, Airflow, Jenkins, Graphana/Prometheus
Date d’expiration: 05 décembre, 2023