Description
As GPUs continue to become faster, PyTorch, one of the most widely used frameworks in AI/ML has faced challenges keeping up with performance demands. To mitigate this, parts of PyTorch have been moved into C++. This approach goes against the original intent of PyTorch as a Python-based framework and complicates contributions from the community. The PyTorch development team recognized the need to address these challenges while maintaining PyTorch's Python roots and set ambitious goals to improve performance, decrease memory usage, enable state-of-the-art distributed capabilities, and ensure more PyTorch code is written in Python. To achieve these goals, they developed a Python compiler. Attendees of this talk will get an inside look at how the PyTorch development team approached these challenges and implemented their innovative solution to achieve a 43% speedup in performance. We will discuss the benefits and challenges of this approach, as well as the techniques and technologies used to build the PyTorch Python compiler. This talk will provide valuable insights into the development process of and offer attendees a deeper understanding of how PyTorch continues to evolve and innovate.