Supported DataFrame LibrariesĀ¶
Pandera started out as a pandas-specific dataframe validation library, and moving forward its core functionality will continue to support pandas. However, panderaās adoption has resulted in the realization that it can be a much more powerful tool by supporting other dataframe-like formats.
DataFrame Library SupportĀ¶
Pandera supports validation of the following DataFrame libraries:
Validate pandas dataframes. This is the original dataframe library supported by pandera. |
|
Validate Polars dataframes, the blazingly fast dataframe library. |
|
A data processing library for large-scale data. |
Validating Pandas-like DataFramesĀ¶
Pandera provides multiple ways of scaling up data validation of pandas-like dataframes that donāt fit into memory. Fortunately, pandera doesnāt have to re-invent the wheel. Standing on shoulders of giants, it integrates with the existing ecosystem of libraries that allow you to perform validations on out-of-memory pandas-like dataframes. The following libraries are supported via panderaās pandas validation backend:
Apply pandera schemas to Dask dataframe partitions. |
|
A pandas drop-in replacement, distributed using a Ray or Dask backend. |
|
The pandas-like interface exposed by pyspark. |
Domain-specific Data ValidationĀ¶
The pandas ecosystem provides support for domain-specific data manipulation, and by extension pandera can provide access to data types, methods, and data container types specific to these libraries.
An extension of pandas that adds geospatial data processing capabilities. |
Alternative Acceleration FrameworksĀ¶
Pandera works with other dataframe-agnostic libraries that allow for distributed dataframe validation:
Apply pandera schemas to distributed dataframe partitions with Fugue. |
Note
Donāt see a library that you want supported? Check out the github issues to see if that library is in the roadmap. If it isnāt, open up a new issue to add support for it!