Contribute Media
A thank you to everyone who makes this possible: Read More

Build a data pipeline with Luigi


Luigi is a Python library for building pipelines of batch processes. It "handles dependency resolution, workflow management, visualization, handling failures, command line integration, and much more"

In this tutorial, we will use Luigi to build a data pipeline that runs a series of interdependent jobs. We will also discuss some real-world use cases for Luigi, and show how it can make running a data pipeline much more robust and reliable.


Improve this page