对多个YAML文件使用Pydantic设置配置



我正在考虑使用pydantic_settings_yaml将yaml配置文件加载到pydantic模型中。理想情况下,我会有一个包含与环境无关的配置的global.yml,然后是一个用于环境特定配置的env.yml。目前尚不清楚pydantic_settings_yaml如何处理此问题。我希望避免每次加载配置时都创建merged.yml。

以下代码将读取Pydantic模型中的dev.yml或live.yml,但不读取global.yml.

app_config.py

env = os.getenv("ENVIRONMENT", "").lower()
match env:
case "dev":
yaml_filename = "dev.yml"
secrets_sub_dir = "dev"
case "live":
yaml_filename = "live.yml"
secrets_sub_dir = "live"
case _:
raise Exception(f"Unrecognised environment: {env}")

class Settings(YamlBaseSettings):
test_api_one: Dict[str, str]
class Config:
secrets_dir = f"/secrets/{secrets_sub_dir}"
yaml_file = Path(__file__).parent / yaml_filename

config = Settings()

dev.yml

'test_api_one':
'uri': 'https://dev_api/'
'secret': <file:dev_secret>

live.yml

'test_api_one':
'uri': 'https://live_api/'
'secret': <file:live_secret>

global.yml

'test_api_one':
'dateFormat': '%Y-%m-%d'
'dateTimeFormat': '%Y-%m-%dT%H:%M:%SZ'

由于我对您提到的包有疑问(请参阅上面的评论),我建议您自己实现它。

Pydantic文档解释了如何自定义设置源。为了确保嵌套字典被更新;porperly";,您还可以使用非常方便的pydantic.utils.deep_update函数。

显然,您需要安装pyyaml才能正常工作。

这里有一个例子:

from pathlib import Path
from typing import Any
from pydantic import BaseSettings as PydanticBaseSettings
from pydantic.env_settings import SettingsSourceCallable
from pydantic.utils import deep_update
from yaml import safe_load

THIS_DIR = Path(__file__).parent

class BaseSettings(PydanticBaseSettings):
test_api_one: dict[str, str]
class Config:
config_files = [
Path(THIS_DIR, "global.yaml"),
Path(THIS_DIR, "dev.yaml"),
]
@classmethod
def customise_sources(
cls,
init_settings: SettingsSourceCallable,
env_settings: SettingsSourceCallable,
file_secret_settings: SettingsSourceCallable
) -> tuple[SettingsSourceCallable, ...]:
return init_settings, env_settings, config_file_settings

def config_file_settings(settings: PydanticBaseSettings) -> dict[str, Any]:
config: dict[str, Any] = {}
if not isinstance(settings, BaseSettings):
return config
for path in settings.Config.config_files:
if not path.is_file():
print(f"No file found at `{path.resolve()}`")
continue
print(f"Reading config file `{path.resolve()}`")
if path.suffix in {".yaml", ".yml"}:
config = deep_update(config, load_yaml(path))
else:
print(f"Unknown config file extension `{path.suffix}`")
return config

def load_yaml(path: Path) -> dict[str, Any]:
with Path(path).open("r") as f:
config = safe_load(f)
if not isinstance(config, dict):
raise TypeError(
f"Config file has no top-level mapping: {path}"
)
return config

现在假设这些是有问题的文件:

global.yaml

test_api_one:
date_format: '%Y-%m-%d'
date_time_format: '%Y-%m-%dT%H:%M:%SZ'

dev.yaml

test_api_one:
uri: 'https://dev_api/'
secret: dev

print(BaseSettings().json(indent=4))的结果是:

Reading config file `.../global.yaml`
Reading config file `.../dev.yaml`
{
"test_api_one": {
"date_format": "%Y-%m-%d",
"date_time_format": "%Y-%m-%dT%H:%M:%SZ",
"uri": "https://dev_api/",
"secret": "dev"
}
}
感谢您的回答。我确实试着听从了你的建议,我认为代码可能已经过时了。

最后,我选择了一个更简单的解决方案:

class Settings(BaseModel):
field: str = "sqlite:///./unset.db"
@staticmethod
def load() -> "Settings":
config_data:dict = {}
for path in ["path1","path2"]:
with open(path, "r") as yaml_file:
config_data.update(yaml.safe_load(yaml_file))
return sSettings(**config_data)

然后我就用在一个地方读取设置

settings = Settings.load()

最新更新