在运行时验证Python TypedDict



我在Python 3.8+Django/Rest Framework环境中工作,在新代码中强制执行类型,但建立在许多未类型化的遗留代码和数据之上。我们广泛使用TypedDicts,以确保我们生成的数据以正确的数据类型传递到TypeScript前端。

MyPy/PyCharm等。在检查我们的新代码是否吐出符合要求的数据方面做得很好,但我们想测试我们的许多RestSerializer/ModelSerializer的输出是否符合TypeDict。如果我有一个序列化程序并键入dict,比如:

class PersonSerializer(ModelSerializer):
class Meta:
model = Person
fields = ['first', 'last']
class PersonData(TypedDict):
first: str
last: str
email: str

然后运行类似的代码

person_dict: PersonData = PersonSerializer(Person.objects.first()).data

静态类型检查器无法确定person_dict缺少所需的email密钥,因为(根据PEP-589的设计(它只是一个普通的dict。但我可以写这样的东西:

annotations = PersonData.__annotations__
for k in annotations:
assert k in person_dict  # or something more complex.
assert isinstance(person_dict[k], annotations[k])

并且它将发现CCD_ 4从串行器的数据中丢失。在这种情况下,这是很好的,因为from __future__ import annotations没有引入任何更改(不确定这是否会破坏它(,并且我所有的类型注释都是空类型。但如果PersonData定义为:

class PersonData(TypedDict):
email: Optional[str]
affiliations: Union[List[str], Dict[int, str]]

则CCD_ 7不足以检查数据是否通过(因为"预订的泛型不能与类和实例检查一起使用"(。

我想知道的是,是否已经存在一个可调用的函数/方法(在mypy或另一个检查器中(,它允许我根据注释验证TypedDict(甚至是单个变量,因为我自己可以迭代dict(,并查看它是否有效?

我不关心速度等,因为这样做的目的是检查我们所有的数据/方法/函数一次,然后在我们对当前数据的有效性感到满意后删除检查。

我发现的最简单的解决方案使用pydantic。

pydantic v2的解决方案

import pydantic
from pydantic import TypeAdapter, ValidationError
from typing_extensions import TypedDict # Required by pydantic for python < 3.12
class SomeDict(TypedDict):
val: int
name: str

SomeDictValidator = TypeAdapter(SomeDict)
# this could be a valid/invalid declaration
obj: SomeDict = {
'val': 12,
'name': 'John',
}
# validate with pydantic
try:
obj = SomeDictValidator.validate_python(obj)
except ValidationError as exc: 
print(f"ERROR: Invalid schema: {exc}")

有关更多信息,请参阅TypeAdapter文档。

pydantic v1的解决方案

from typing import cast, TypedDict 
import pydantic
class SomeDict(TypedDict):
val: int
name: str
# this could be a valid/invalid declaration
obj: SomeDict = {
'val': 12,
'name': 'John',
}
# validate with pydantic
try:
obj = cast(SomeDict, pydantic.create_model_from_typeddict(SomeDict)(**obj).dict())
except pydantic.ValidationError as exc: 
print(f"ERROR: Invalid schema: {exc}")

EDIT:当类型检查此项时,它当前返回一个错误,但工作正常。请参见此处:https://github.com/samuelcolvin/pydantic/issues/3008

您可能想看看https://pypi.org/project/strongtyping/.这可能会有所帮助。

在文档中,您可以找到以下示例:

from typing import List, TypedDict
from strongtyping.strong_typing import match_class_typing

@match_class_typing
class SalesSummary(TypedDict):
sales: int
country: str
product_codes: List[str]
# works like expected
SalesSummary({"sales": 10, "country": "Foo", "product_codes": ["1", "2", "3"]})
# will raise a TypeMisMatch
SalesSummary({"sales": "Foo", "country": 10, "product_codes": [1, 2, 3]})

有点破解,但您可以使用mypy命令行-c选项检查两种类型。只需将其封装在python函数中:

import subprocess
def is_assignable(type_to, type_from) -> bool:
"""
Returns true if `type_from` can be assigned to `type_to`,
e. g. type_to := type_from
Example:
>>> is_assignable(bool, str) 
False
>>> from typing import *
>>> is_assignable(Union[List[str], Dict[int, str]], List[str])
True
"""
code = "n".join((
f"import typing",
f"type_to: {type_to}",
f"type_from: {type_from}",
f"type_to = type_from",
))
return subprocess.call(("mypy", "-c", code)) == 0

您可以这样做:

def validate(typ: Any, instance: Any) -> bool:
for property_name, property_type in typ.__annotations__.items():
value = instance.get(property_name, None)
if value is None:
# Check for missing keys
print(f"Missing key: {property_name}")
return False
elif property_type not in (int, float, bool, str):
# check if property_type is object (e.g. not a primitive)
result = validate(property_type, value)
if result is False:
return False
elif not isinstance(value, property_type):
# Check for type equality
print(f"Wrong type: {property_name}. Expected {property_type}, got {type(value)}")
return False
return True

然后测试一些对象,例如传递给REST端点的对象:

class MySubModel(TypedDict):
subfield: bool

class MyModel(TypedDict):
first: str
last: str
email: str
sub: MySubModel
m = {
'email': 'JohnDoeAtDoeishDotCom',
'first': 'John'
}
assert validate(MyModel, m) is False

这个会打印第一个错误并返回bool,您可以将其更改为异常,可能会丢失所有的键。您还可以将其扩展为在模型定义之外的其他键上失败。

我会使用typing.get_type_hints函数,它从TypeDict(在python 3.8下测试(返回一个dict:

from typing import TypedDict, get_type_hints
def checkdict(value: object, typedict: type) -> None:
"""
Raise a TypeError if value does not check the TypeDict.
:param value: the value to check
:param typedict: the TypeDict type
"""
if not isinstance(value, dict):
raise TypeError(f'Value must be a dict not a: {type(value).__name__}')
d = get_type_hints(typedict)
diff = d.keys() ^ value.keys()
if diff: # must have the same fields
raise TypeError(f"Invalid dict fields: {' '.join(diff)}")
for k, v in get_type_hints(typedict).items():
if not isinstance(value[k], v): # must have same types
raise TypeError(
f"Invalid type: '{k}' should be {v.__name__} "
f"but is {type(value[k]).__name__}"
)
class TargetDict(TypedDict):
name: str
integer: int
obj: dict = {
'name': 'John',
'integer': '3',
}
checkdict(
obj, TargetDict
)  # TypeError: Invalid type: 'integer' should be int but is str

我喜欢你的解决方案!。为了避免某些用户的迭代修复,我在您的解决方案中添加了一些代码:D

def validate_custom_typed_dict(instance: Any, custom_typed_dict:TypedDict) -> bool|Exception:
key_errors = []
type_errors = []
for property_name, type_ in my_typed_dict.__annotations__.items():
value = instance.get(property_name, None)
if value is None:
# Check for missing keys
key_errors.append(f"t- Missing property: '{property_name}' n")
elif type_ not in (int, float, bool, str):
# check if type is object (e.g. not a primitive)
result = validate_custom_typed_dict(type_, value)
if result is False:
type_errors.append(f"t- '{property_name}' expected {type_}, got {type(value)}n")
elif not isinstance(value, type_):
# Check for type equality
type_errors.append(f"t- '{property_name}' expected {type_}, got {type(value)}n")
if len(key_errors) > 0 or len(type_errors) > 0:
error_message = f'n{"".join(key_errors)}{"".join(type_errors)}'
raise Exception(error_message)

return True

一些控制台输出:

Exception: 
- Missing property: 'Combined_cycle' 
- Missing property: 'Solar_PV' 
- Missing property: 'Hydro' 
- 'timestamp' expected <class 'str'>, got <class 'int'>
- 'Diesel_engines' expected <class 'float'>, got <class 'int'>

最新更新