Description
For some Python Objects it is difficult to understand its contents, even if type annotations are used. For example, PyTorch's Tensor or Pandas's DataFrame have many different "shapes" to their data structures, but their type annotations are always the same. This can make it very difficult to understand functions that utilize these objects as parameters or return these objects. When reviewing code, it is very common to ask these sorts of questions:
- What is the shape of the Tensor I need to pass in?
- What columns does this DataFrame need to have?
In this talk, we will describe how to utilize typing metadata and Pydantic to not only provide additional context for these data structures, but to validate the input/output of functions as well.
Links for the talk:
- Notebook used
in talk; dev container compatible
- ds_validator, the
package demonstrated in the talk