Minimal Usage Example¶
This provides an minimal usage example for pydantic-cereal
.
To start, use the following imports and create a global cereal
object:
"""Minimal imports to get started."""
from fsspec import AbstractFileSystem
from pydantic import BaseModel, ConfigDict
from pydantic_cereal import Cereal
cereal = Cereal() # global variable
We have a custom type MyType
, in this case it's just an wrapper for a str
:
class MyType(object):
"""My custom type, which isn't a Pydantic model."""
def __init__(self, value: str):
"""Initialize the object."""
self.value = str(value)
def __repr__(self) -> str:
"""Represent the string."""
return f"MyType({self.value})"
def __str__(self) -> str:
"""Return the internal string."""
return str(self.value)
We must add reader and writer classes for it.
These must accept fsspec
file system
and path (within that filesystem) as inputs.
We can register these with our cereal
object by creating a wrapped (Annotated
) type.
def my_reader(fs: AbstractFileSystem, path: str) -> MyType:
"""Read a MyType from an fsspec URI."""
res = fs.read_text(path)
if isinstance(res, bytes):
res = res.decode("utf8")
else:
res = str(res)
return MyType(value=res)
def my_writer(obj: MyType, fs: AbstractFileSystem, path: str) -> None:
"""Write a MyType object to an fsspec URI."""
fs.write_text(path, obj.value)
MyWrappedType = cereal.wrap_type(MyType, reader=my_reader, writer=my_writer)
Note that your type checker should recognize MyWrappedType
as exactly MyType
.
from typing import get_args # noqa
assert get_args(MyWrappedType)[0] == MyType
We can use this type in a Pydantic model:
class ExampleModel(BaseModel):
"""Example model."""
model_config = ConfigDict(arbitrary_types_allowed=True)
fld: MyWrappedType # NOTE: Make sure to use the wrapped type!
value: str = "default_value"
You can instantiate objects as usual:
mdl = ExampleModel(fld=MyType("my_field"))
Now, you can write your model to an arbitrary directory-like fsspec
URI.
In this example, we're writing to a temporary MemoryFileSystem
:
cereal.write_model(mdl, "memory://example_model")
'/example_model'
And we can load another object from there:
obj = cereal.read_model("memory://example_model")
assert isinstance(obj, ExampleModel)
obj.fld
MyType(my_field)
If you require a specific type (or base type), you can specify this in read_model
:
cereal.read_model("memory://example_model", supercls=ExampleModel).fld
MyType(my_field)
Inspecting the path, you can see the file structure:
from fsspec.implementations.memory import MemoryFileSystem # noqa
fs = MemoryFileSystem()
fs.glob("example_model/*")
['/example_model/faecc838ddc54489a066df1654c48c85', '/example_model/model.json', '/example_model/model.schema.json']