Contribute Media
A thank you to everyone who has made this possible: Read More

Build a data pipeline with Luigi

Description

Luigi is a Python library for building pipelines of batch processes. It "handles dependency resolution, workflow management, visualization, handling failures, command line integration, and much more"

In this tutorial, we will use Luigi to build a data pipeline that runs a series of interdependent jobs. We will also discuss some real-world use cases for Luigi, and show how it can make running a data pipeline much more robust and reliable.

Details

Improve this page