Contribute Media
A thank you to everyone who has made this possible: Read More

Designing a Python-based Data Analysis / Online Coding Platform for Researchers and Educators

Description

In modern science and engineering, it is essential to use data analysis and calculation using computers. However, large-scale computing is labor-intensive to achieve extensibility and manageability of computing resources. We are designing and implementing a cloud platform that standardizes development, running, and sharing of the data processing tasks using cloud technologies and Python 3. We are also adding research/education services on top of it.

In this talk, we are going to share what we have learned during 2 months of development experiences. In particular, it will include the architecture of our platform, experiences in the design and implementation process, and common caveats to care when you do a similar work. We hope to share our motivation that allowed our pathway over such a mine field with you.

Summary

  • Modern science and Python
    • Online programming playground for researchers and educators
  • Technical challenges
    • Sandboxing!
    • Resource consolidation
    • Fast uploads/downloads of data
  • Choice of tech stack
    • Advantages and disadvantages of Python
    • It's time to go Python 3
    • On-premise vs. Hosting vs. Cloud (AWS / Azure / GCE)
    • Docker containers 와 kubernetes
  • "Shoveling"
    • Warring states of Javascript frameworks + UI frontends
    • Entangling Waltz of Polymer, webcomponents, and Django
    • Backends: Mad-max around Docker
    • Security holes in ipython/Jupyter in the perspective of developer and hacker
    • etc.

Details

Improve this page