Summary
Good solutions to hard problems require both domain and algorithmic expertise. Domain experts know what to do and computer scientists know how to do it well. This talk discusses challenges and experiences trying to reconcile these two groups, particularly within SymPy. It proposes concrete approaches including multiple dispatch, pattern matching, and programmatic strategies.
Description
Good solutions to hard problems require both domain and algorithmic expertise. Domain experts know what to do and computer scientists know how to do it well. Coordination between the algorithmic and domain programmer is challenging to do well and difficult to scale. It is also arguably one of the most relevant blocks to scientific progress today.
This talk draws from experience supporting mathematical programmers in the SymPy project. SymPy is a computer algebra system, a complex problem that requires the graph manipulation algorithms of a modern compiler alongside the mathematics of several PhD theses. SymPy draws from a broad developer base with experienced and novice developers alike and so struggles to maintain a cohesive organized codebase.
We approach this development problem by separating software engineering into a collection of small functions, written by domain experts, alongside an abstract control system, written by algorithmic programmers. We facilitate this division with techniques taken from other languages and compiler technologies. Notably we motivate the use of a few general purpose libraries for multiple dispatch, pattern matching, and programmatic control.