Description
Data Engineering and Python - PyCon Italia 2022
Data Engineering is the backbone of all analytics that happens in any organization. This marks data engineering as the central role in any data-driven organization. This talk aims to introduce the fundamentals of data engineering with python apps driving the core concepts. The aim of this talk is to introduce the audience to the world of big data analytics with pythonic tools and libraries at its core. One can expect the talk to cover the basics of Extract, Transform and Load(ETL) pipelines using python scripts and then Airflow with PySpark. These ETL pipelines are core to any data infrastructure. There will be a discourse on how we can use cloud providers and design an entire system that is responsible for analytics and ML in an organization. Finally a short outro into how one can start their journey to become a data engineer.
Speaker: Prakhar Srivastava