Contribute Media
A thank you to everyone who has made this possible: Read More

Real Time Change Detection on Streaming Data

Description

Sponsor Tutorial- Netflix This tutorial offers an introduction to how we perform change detection on data streams at Netflix using Python. We will develop a framework for running and evaluating change detection algorithms and then experiment with various techniques on real-world data gathered at Netflix.

This tutorial offers an introduction to how we perform change detection on data streams at Netflix using Python. We will introduce the problem and a framework for solving it, as well as a method for evaluating the effectiveness of different techniques. Once we've established the problem and a framework we will dive into some real-world data collected at Netflix about device call volume in our call centre, and attempt to detect a meaningful increase in call volume related to a device.

The associated notebook will scaffold out the framework, then introduce simple techniques, slowly building in complexity. Time permitting we will discuss these techniques and potentially the usefulness of an ensemble of techniques in a production environment to provide robustness against different behavior patterns of data.

Topics Covered: - Change deteciton problem definition and framework - Simple change detection techniques - Synthetic data generation - Real-world data evaluation

Slides available here: https://github.com/codyrioux/pydata2015seattle

Details

Improve this page