Description
We all have to package Python based applications for various environments, starting from command line tools, to web applications. And depending on the users, it can be installed on thousands on computers or on a selected few systems. https://pypi.org is our goto place for finding any dependencies and also in most of the time we install binary wheels directly from there, thus saving a lot time.
But, Python is also being used in many environments where security is the utter most important, and validating the dependencies of project is also very critical along with the actual project source code. Many of noticed the recent incident where people were being able to steal bticoins using a popular library.This talk will take SecureDrop client application for journalists as an example project and see how we tried to tackle the similar problem. SecureDrop is an Open Source whistleblower system which is deployed over 75 news organizations all over the world. Our threat model has nation state actors as possible threats, so, security and privacy of the users of the system is a very important point of the whole project. The tools in this case are build and packaged into reproducible Debian deb packages and are installed on Qubes OS in the final end user systems.
There are two basic ways we handle Python project dependencies, for most of the development work, we use a virtualenv, and directly install the dependencies using wheels built from pypi.org. When we package the application for the end users, many times we package them using a operating system based package manager and ask the users to install using those (say RPM or Debian's deb package). In the second case, all the dependencies come as separate packages (and most of the time from the OS itself). The dependency is being handled by the OS package manager itself. That case, we can not update the dependencies fast enough if required, it depends on the packagers from the community who maintains those said packages in the distribution.
We use dh-virtualenv project to help us to use our own wheels + a virtualenv for the project to be packaged inside the debian .deb package. This talk will go throuh the process of building wheels from known (based on sha256sum) source tarballs, and then having a gpg signed list of updated wheels and a private index for the same. And also how we are verifying the wheels' sha256sum (and the signature of that list) during the build process. The final output is reproducible Debian packages.