HPy: The Future of Python Native Extensions

[EuroPython 2023 — Forum Hall on 2023-07-19]

Updating Python versions often forces us to update native extensions at the

same time. But what if you need to update Python because of a security issue,

but cannot (yet) move to a newer version of a dependency? Or you are running a

proprietary binary extension that cannot easily be recompiled?

The HPy project provides a better C extension API for Python. It compiles to

binaries that work across all versions of CPython, PyPy, GraalPy. HPy makes

porting from the existing C API easy and its design ensures that the binaries

we produce today stay binary compatible with future Python versions.

NumPy is the single largest direct user of the CPython C API we know of. After

over 2 years of work and more than 30k lines of code ported, we can demonstrate

NumPy running its tests and benchmarks with HPy. We will show the same NumPy

binary run on multiple CPython versions and GraalPy. And we will discuss

performance characteristics of this port across CPython, GraalPy, and PyPy.

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License http://creativecommons.org/licenses/by-nc-sa/4.0/