Pydantic#
new in 0.8.0
Using Pandera Schemas in Pydantic Models#
DataFrameModel
is fully compatible with
pydantic. You can specify
a DataFrameModel
in a pydantic BaseModel
as you would
any other field:
import pandas as pd
import pandera as pa
from pandera.typing import DataFrame, Series
import pydantic
class SimpleSchema(pa.DataFrameModel):
str_col: Series[str] = pa.Field(unique=True)
class PydanticModel(pydantic.BaseModel):
x: int
df: DataFrame[SimpleSchema]
valid_df = pd.DataFrame({"str_col": ["hello", "world"]})
PydanticModel(x=1, df=valid_df)
invalid_df = pd.DataFrame({"str_col": ["hello", "hello"]})
PydanticModel(x=1, df=invalid_df)
Traceback (most recent call last):
...
ValidationError: 1 validation error for PydanticModel
df
series 'str_col' contains duplicate values:
1 hello
Name: str_col, dtype: object (type=value_error)
Other pandera components are also compatible with pydantic:
Note
The SeriesSchema
, DataFrameSchema
and schema_components
types
validates the type of a schema object, e.g. if your pydantic
BaseModel
contained a schema object, not a pandas
object.
Using Pydantic Models in Pandera Schemas#
new in 0.10.0
You can also use a pydantic BaseModel
in a pandera schema. Suppose you had
a Record
model:
from pydantic import BaseModel
import pandera as pa
class Record(BaseModel):
name: str
xcoord: int
ycoord: int
The PydanticModel
datatype enables you to
specify the Record
model as a row-wise type.
import pandas as pd
from pandera.engines.pandas_engine import PydanticModel
class PydanticSchema(pa.DataFrameModel):
"""Pandera schema using the pydantic model."""
class Config:
"""Config with dataframe-level data type."""
dtype = PydanticModel(Record)
coerce = True # this is required, otherwise a SchemaInitError is raised
Note
By combining dtype=PydanticModel(...)
and coerce=True
, pandera will
apply the pydantic model validation process to each row of the dataframe,
converting the model back to a dictionary with the BaseModel.dict() method.
The equivalent pandera schema would look like this:
class PanderaSchema(pa.DataFrameModel):
"""Pandera schema that's equivalent to PydanticSchema."""
name: pa.typing.Series[str]
xcoord: pa.typing.Series[int]
ycoord: pa.typing.Series[int]
Note
Since the PydanticModel
datatype
applies the BaseModel
constructor to each row of the dataframe, using
PydanticModel
might not scale well with larger datasets.
If you want to help benchmark, consider contributing a benchmark script