Contribute Media
A thank you to everyone who makes this possible: Read More

Data Pipeline Modernization at Scale

Description

Healthcare organizations aggregate petabytes of data which help drive life-changing health and business decisions. In this presentation, we’ll show you how Indiana’s largest healthcare institution went from using proprietary tools that were hard to maintain and nearly impossible to scale — to implementing a new world of cloud native and open source tooling, including Airflow, Spark, Delta Lakes (delta.io) and Terraform. We’ll showcase how making a big change to a data pipeline infrastructure can: * simplify and automate workflow processes; * enable organizations to import existing and new data sources quickly, efficiently and at scale; * keep costs down in relation to the number and volume of data sources; * hire and onboard new developers — who have experience working with the open source tools — quickly, eliminating the tech learning curve that existed with the proprietary tools.

#PWC2022 attracted nearly 375 attendees from 36 countries and 21 time zones making it the biggest and best year yet. The highly engaging format featured 90 speakers, 6 tracks (including 80 talks and 4 tutorials) and took place virtually on March 21-25, 2022 on LoudSwarm by Six Feet Up.

More information about the conference can be found at: https://2022.pythonwebconf.com

Details

Improve this page