Description
In this talk, we will discuss the design of Quora's asynq framework, which provides an asynchronous API to a global scheduler for data requests. We will explore in depth the common problem that motivated it, the design of the framework, and how it has been used in practice to make both the product and development faster at Quora.
Abstract
In order to provide a fast distributed web application to millions of Quora users, we need to be smart about batching data requests to minimize the time spent blocked on network I/O. Moreover, it's important to accomplish this batching in a general way that doesn't require repetitive work every time we make a change or require new data. In this talk, we will discuss the design of Quora's asynq framework, which provides an asynchronous API to a global scheduler for data requests. We will explore in depth the common problem that motivated it, the design of the framework, and how it has been used in practice to make both the product and development faster at Quora.
Bio
Riley Patterson is a software engineering manager on the Platform Frameworks Team at Quora. Our core web application platform is built in Python on top of a web framework that we built on the core of Pylons. As such, the Platform Team uses and builds a wide variety of Python tools and abstractions to enable faster, more effective, and more enjoyable development across the entire team at Quora.