Contribute Media
A thank you to everyone who makes this possible: Read More

How I learned to time travel, or, data pipelining and scheduling with Airflow

Description

PyData DC 2016

Slides: http://www.slideshare.net/PyData/how-i-learned-to-time-travel-or-data-pipelining-and-scheduling-with-airflow-67650418

Data warehousing and analytics projects can, like ours, start out small - and fragile. With an organically growing mess of scripts glued together and triggered by cron jobs hiding on different servers, we needed better plumbing. After perusing the data pipelining landscape, we landed on Airflow, an Apache incubating batch processing pipelining and scheduler tool from Airbnb.

Details

Improve this page