Contribute Media
A thank you to everyone who makes this possible: Read More

Automated deployment of Python packages for development


Andrew MacDonald

The ability to automatically deploy development and test versions of software supports a rapid development/release cycle.

Within our section of the Bureau of Meteorology we have a number of internal Python packages, ranging from small simple packages to large applications that are dependent on the smaller packages. We manage the development cycle to ease deployment of these packages and applications into development, test, and production environments.

Elements of our process are:

  • Source code management (git)
  • Code review (Gerrit)
  • Continuous integration (Jenkins)
  • Internal PyPi servers (Apache)
  • A development environment for automatic deployment of every Gerrit approved commit (Anaconda environment via Jenkins)
  • Versioning (git tags + versioneer)
  • Test environment for every tagged version (Anaconda environment via Jenkins)
  • Production environment for specified releases (Anaconda environment via Ansible)

A key benefit of this process is that we have a deployed Python environment for the latest development version of all packages, a pinned collection of packages for testing, and an approved stable collection for production. The development and test environments are rapidly updated when commits are approved in Gerrit or tagged. The production environment is then readily updated with specific versions after a period of testing in the test environment.

This presentation will discuss our development process, how it works for us and how we leverage Python packaging to do it.


Improve this page