Description
In this short talk, I will show some examples of how sparse dtypes in dataframes can be great for memory saving when working with sparsely populated data.
Showing the benefits of using metadata— alongside the pitfalls and the conversion costs—we can compare the pandas approach versus the pure NumPy/ SciPy approach.
Finally, using linear systems problems (with LCA backgrounds) as examples, we can compare a few methods and see what stands out.