Description
Title: Supercharging your Jupyter and Machine Learning Workflow with PrimeHub – Chia-liang Kao
Day 2, 10:35-11:20
Abstract
Jupyter has become a critical component of the machine learning life cycle. However scaled enterprise deployments and making the Data Science Experience frictionless remain challenging. We address a few common issues with PrimeHub, an open-source enterprise offering based on JupyterHub, with the capability to work with popular ML tooling like mlflow, labelstudio, and streamlit. This talk also investigates the MLOps trends adjacent to the Jupyter ecosystem.
Description
This talk is intended for audience interested in larger scale Jupyter environment deployment in their organisation, particularly for machine learning applications.
PrimeHub is an open-source enterprise offering based on JupyterHub, which also allows orchestrating tools like mlflow, labelstudio, and streamlit, to assemble your own end-to-end ML toolbox for your team. We address a few common hurdles:
advanced resource scheduling for heterogenous GPU clusters multi-tenancy and project isolations customizable and consistent ML environments per project managing jupyterlab extensions for teams hybrid development environment such as vscode and ssh within notebook instances authorization and data access management usage reporting: allocation and utilization We also investigate a few trends adjacent to the day-to-day jupyter environment used by data scientists and data engineers, where the roles become more cross functional in the age of MLOps:
Working with job schedulers from within Notebook Managing ML model deployments, CI/CD integration Model monitoring and retraining
Slides not uploaded by the speaker. HackMD: https://hackmd.io/@pycontw/2021/%2F%40pycontw%2Frk-phz9zF
Speaker: Chia-liang Kao
clkao (Chia-liang Kao) has been an open source software developer since 2000. He believes that good collaboration model and tools drive innovation. In 2013, he created SVK, a distributed version control system that helps developers collaborate. He co-founded the g0v.tw community in 2012, advocating information transparency and digital-activism through open source model. g0v.tw was awarded as "Digital Communities: Award of Distinction" by Prix Ars Electronica 2018. He started InfuseAI in 2018 to enable data scientists to thrive, and to help wider adoption of AI across industries.