Description
Twisted up in a Distributed Tornado - a beginners guide to async frameworks in python
Filmed at PyData 2017
Description By coming to this tutorial you will learn:
What event-driven Python Frameworks are and why they are useful How to overcome any fears you may have about event-driven Python programming Enough examples to be able to consider projects like Dask/Distributed and Tornado for your next project Abstract At some point in a career in data science, software development or data engineering you will be looking to develop a piece of code that runs alongside another piece of code. For example, you might want to:
Use all of your processors when running a script Consume social media data as it is created Create a super simple task scheduler without an ugly while True loop Understand how Jupyter notebooks work Write to a database and move on to the next task without waiting for confirmation Call a function and throw an exception if it takes too long There are numerous ways of achieving such concurrency. If creating a project to run on one or a few machines which needs good exception handling then I find tools like Twisted, Tornado and Dask/Distributed ideal for the above tasks.
As a python developer I have always tended to work with Celery and felt intimidated by the different flow control when using async tools like Twisted and Tornado. In this tutorial I will walk through the examples above and talk about how I went from Async novice to Dask/Distributed contributor in 6 months. The tutorial will concentrate mainly on practical use cases. Along the way we will encounter some bumps in the road which will hopefully aid attendees' understanding of the basic dos and don'ts around these tools.
By coming to this tutorial you will learn:
What event-driven Python Frameworks are and why they are useful How to overcome any fears you may have about event-driven Python programming Enough examples to be able to consider projects like Dask/Distributed and Tornado for your next project.