Mastodon Feed: Post

Mastodon Feed

jonny@neuromatch.social ("jonny (nonvenomous)") wrote:

aw ye whats up my bozos it's static type checking for array shape and dtype constraints so you no longer have to toil in the mines of infinity incomprehensible (ndarray)->ndarray functions like bunch of clowns face smacking in a field of rakes

https://numpydantic.readthedocs.io/en/latest/typecheckers.html

#python

[E.g. say you have some analysis function that only accepts grayscale images: [python code follows. a type for 2D grayscale images and 3D RGB images are declared. then functions that return RGB and grayscale images, and finally a functino that only accepts grayscale image. demonstrates that the function fails a type chck when called with an RGB image] import numpy as np from numpydantic import NDArray, Shape GRAYSCALE = NDArray[Shape["* x, * y"], np.uint8] RGB = NDArray[Shape["* x, * y, 3 rgb"], np.uint8] def read_rgb() -> RGB: return np.ones((1920, 1080, 3), dtype=np.uint8) def read_grayscale() -> GRAYSCALE: return np.ones((1920, 1080), dtype=np.uint8) def grayscale_mask(frame: GRAYSCALE) -> GRAYSCALE: # Probably something fancier than this... mask = np.zeros((frame.shape[0], frame.shape1), np.uint8) mask[frame > 5] = 1 return mask # this works grayscale_mask(read_grayscale()) # this doesn't grayscale_mask(read_rgb()) examples/incorrect/rgb_gray_frame.py:28: error: Argument 1 to "grayscale_mask" has incompatible type "ndarray[tuple[int, int, Literal3], dtype[unsignedinteger[_8Bit]]]"; expected "ndarray[tuple[int, int], dtype[unsignedinteger[_8Bit]]]" [arg-type] grayscale_mask(read_rgb()) ^~~~~~~~~~ Found 1 error in 1 file (checked 1 source file)]4
[Shape checking Scalars Scalar shapes are checked as expected, where the shapes must match exactly. [python code follows. a correct example shows an NDArray annotation with a Shape[1, 2, 3] (three dimensions with sizes 1, 2, and 3 respectively) as a return value coming from a np.ones() constructor and assigned to x as passing a type check. an incorrect example shows that the type checker fails both between the numpy constructor and return type and the return type and the assignment] Correct def make_array() -> NDArray[Shape[1, 2, 3], np.uint8]: return np.ones((1, 2, 3), dtype=np.uint8) x: NDArray[Shape[1, 2, 3], np.uint8] = make_array() Incorrect def make_array() -> NDArray[Shape[2, 3, 4], np.uint8]: return np.ones((1, 2, 3), dtype=np.uint8) x: NDArray[Shape[5, 6, 7], np.uint8] = make_array() error: Incompatible return value type (got "ndarray[int, dtype[_8Bit]]", expected "ndarray[tuple[Literal2, Literal3, Literal4], dtype[_8Bit]]") [return-value] return np.ones((1, 2, 3), dtype=np.uint8) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: Incompatible types in assignment (expression has type "ndarray[tuple[Literal2, Literal3, Literal4], dtype[8Bit]]", variable has type "ndarray[tuple[Literal5, Literal6, Literal[7]], dtype[_8Bit]]") [assignment] x: NDArray[Shape[5, 6, 7], np.uint8] = make_array() ^~~~~~~~~~~~ Found 2 errors in 1 file (checked 1 source file)]5
[So, if enabled, return values for non-numpy interfaces can declare how to infer their shapes and dtypes: [python code follows, demonstrates that numpy and zarr constructors are correctly typed from the sizes and dtypes in their constructors (dask is broken, note at bottom). without the plugin they are just generic arrays and Any types] from typing import reveal_type import dask.array as da import numpy as np import zarr x = np.zeros((3, 4, 5), dtype=np.uint8) y = da.zeros((3, 4, 5), dtype=np.uint8) z = zarr.zeros((3, 4, 5), another=int, dtype=np.uint8) reveal_type(x) reveal_type(y) reveal_type(z) Without the plugin note: Revealed type is "numpy.ndarray[tuple[int, int, int], numpy.dtype[numpy.unsignedinteger[numpy._typing._nbit_base._8Bit]]]" note: Revealed type is "Any" note: Revealed type is "Any" With the plugin note: Revealed type is "numpy.ndarray[tuple[Literal3, Literal4, Literal5, fallback=int], numpy.dtype[numpy.unsignedinteger[numpy._typing._nbit_base._8Bit]]]" note: Revealed type is "Any" note: Revealed type is "numpy.ndarray[tuple[Literal3, Literal4, Literal5, fallback=int], numpy.dtype[numpy.unsignedinteger[numpy._typing._nbit_base._8Bit]]]" Dask is broken Note that dask’s constructor inference doesn’t work at the moment. This is due to dask’s array creation routines being positively haunted, an untyped wrapped dynamic construction of a function that creates a class that creates a class. PRs welcome re: figuring out how to type that.]6