Description
David Dao (@dwddao)
David is a PhD student at ETH Zurich, working on Deep Reinforcement Learning. Before joining ETH Zurich, he was an autonomous driving researcher at Mercedes-Benz Research in Silicon Valley and a graduate student at the Broad Institute of MIT and Harvard.
David is a firm believer in open source and is organising Germany's largest deep learning meetup series, and Silicon Valley's self-driving AI series. He is a contributor to popular machine intelligence frameworks such as TensorFlow and PyTorch and speaks chinese with swabian accent.
Abstract
Modern neural networks have hundreds of layers! How can we train such deep networks? Simply stacking layers on top doesn't work! This talk introduces the deep learning library PyTorch by explaining the exciting math, cool ideas and simple code behind what makes really deep neural networks work.
Description
Modern neural networks consist of hundreds of computation layers! These very deep architectures consistently outperform shallower networks in a variety of tasks. However just simply stacking layers on top of each other won't work because the gradients are either vanishing or exploding during optimisation procedure. This talk explains the exciting math, cool ideas and elegant code that modern neural network architectures such as ResNets, HighwayNets and DenseNets are applying to circumvent the problem using PyTorch. PyTorch is a relatively new deep learning framework that is deeply integrated into Python. Unlike other frameworks such as TensorFlow and Theano, it uses tape-based automatic differentiation to run computation immediately, supports dynamic neural networks and provides a powerful GPU-accelerated Tensor library. The talk concludes with some real-world use-cases for very deep neural networks in chemical-genetic profiling and autonomous driving.
Recorded at PyCon.DE 2017 Karlsruhe: https://de.pycon.org/
Video editing: Sebastian Neubauer & Andrei Dan
Tools: Blender, Avidemux & Sonic Pi