如何使用巴泽尔的py_library导入参数

Question

我正在尝试在 Bazel 构建的项目中使用 Boto3，但似乎无法为库获得正确的导入。由于 Boto git 存储库，所有源都位于名为 botocore 的文件夹中，boto3位于存储库的根目录中。导入都是boto3.boto3的，第一个对应于外部依赖项的名称，第二个是驻留的根文件夹。如何使用py_binary和py_library规则的 imports 属性从内部boto3而不是另一个导入？

这是我的工作空间的样子：

//WORKSPACE

BOTOCORE_BUILD_FILE = """
py_library(
    name = "botocore",
    srcs = glob([ "botocore/**/*.py" ]),
    imports = [ "botocore" ],
    visibility = [ "//visibility:public" ],
)
"""
_BOTO3_BUILD_FILE = """
py_library(
    name = "boto3",
    srcs = glob([ "boto3/**/*.py" ]),
    imports = [ "boto3" ],
    deps = [ "@botocore//:botocore" ],
    visibility = [ "//visibility:public" ],
)
"""
new_git_repository(
    name = "botocore",
    commit = "cc3da098d06392c332a60427ff434aa51ba31699",
    remote = "https://github.com/boto/botocore.git",
    build_file_content = _BOTOCORE_BUILD_FILE,
)
new_git_repository(
    name = "boto3",
    commit = "8227503d7b1322b45052a16b197ac41fedd634e9", # 1.4.4
    remote = "https://github.com/boto/boto3.git",
    build_file_content = _BOTO3_BUILD_FILE,
)

//BUILD

py_binary(
    name = "example",
    srcs = [ "example.py" ],
    deps = [
        "@boto3//:boto3",
    ],
)

//example.py

import boto3
boto3.client('')

检查生成文件夹的内容

$ ls bazel-bin/example.runfiles/*
bazel-bin/example.runfiles/__init__.py bazel-bin/example.runfiles/MANIFEST
bazel-bin/example.runfiles/boto3:
boto3  __init__.py
bazel-bin/example.runfiles/botocore:
botocore  __init__.py

当我尝试运行示例脚本时，我得到了我可以import boto3.boto3 AttributeError: 'module' object has no attribute 'client'但是使用其中的任何内容都会导致缺少依赖项，例如boto3.sessions因为所有内容都嵌套在<target-name>.boto3

Answer 1

我认为你走在正确的轨道上，但由于python sys.path的排序，你遇到了一个微妙的问题。

如果我运行您的示例并在 example.py 中打印出 sys.path，我看到路径按顺序包含：

bazel-out/local-fastbuild/bin/example.runfiles
bazel-out/local-fastbuild/bin/example.runfiles/boto3/boto3
bazel-out/local-fastbuild/bin/example.runfiles/boto3

第二行是由于工作区文件中的imports = ['boto3']。

我认为您希望第三行是你获得import boto3的地方，因为您希望python看到bazel-out/local-fastbuild/bin/example.runfiles/boto3/boto3/__init__.py。

因此，当python评估import boto3时，它从第一个条目中看到bazel-out/local-fastbuild/bin/example.runfiles/boto3/__init__.py并使用它，而不是从第三个条目中bazel-out/local-fastbuild/bin/example.runfiles/boto3/boto3/__init__.py。

我认为这里的答案是将您的"工作区"命名为它包含的目录以外的名称。例如：

# WORKSPACE
new_git_repository(
  name = "boto3_archive",
  commit = "8227503d7b1322b45052a16b197ac41fedd634e9", # 1.4.4
  remote = "https://github.com/boto/boto3.git",
  build_file_content = _BOTO3_BUILD_FILE,
)
# BUILD
py_binary(
  name = "example",
  srcs = [ "example.py" ],
  deps = [
    "@boto3_archive//:boto3",
  ],
)

当我在您的示例中执行此操作时，我收到以下错误：ImportError: No module named dateutil.parser，我认为这是进步。

Answer 2

文件

$ tree
.
├── BUILD
├── WORKSPACE
├── requirements.txt
└── src
    └── main.py

要求.txt

$ cat requirements.txt 
boto3==1.13.4

工作

$ cat WORKSPACE 
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
    name = "rules_python",
    url = "https://github.com/bazelbuild/rules_python/releases/download/0.0.2/rules_python-0.0.2.tar.gz",
    strip_prefix = "rules_python-0.0.2",
    sha256 = "b5668cde8bb6e3515057ef465a35ad712214962f0b3a314e551204266c7be90c",
)
load("@rules_python//python:repositories.bzl", "py_repositories")
py_repositories()
# Only needed if using the packaging rules.
load("@rules_python//python:pip.bzl", "pip_repositories")
pip_repositories()

# Python external packages installation
load(
    "@rules_python//python:pip.bzl", "pip3_import"
)
pip3_import(
    name = "lambda_deps",
    requirements = "//:requirements.txt", # Top level requirements.txt file
)
load("@lambda_deps//:requirements.bzl", "pip_install")
pip_install()

建

$ cat BUILD
load(
    "@lambda_deps//:requirements.bzl",
    "requirement"
)
py_binary(
    name = 's3_test',
    main = 'main.py',
    srcs = ['src/main.py'],
    deps = [
        requirement('boto3')
    ]
)

src/main.py

$ cat src/main.py
import boto3

def hello_boto3():
    print('hello', boto3.client('s3'))

if __name__ == '__main__':
    hello_boto3()

构建和运行

$ bazel clean && bazel build //:s3_test
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.
INFO: Analyzed target //:s3_test (24 packages loaded, 1234 targets configured).
INFO: Found 1 target...
Target //:s3_test up-to-date:
  bazel-bin/s3_test
INFO: Elapsed time: 11.666s, Critical Path: 0.25s
INFO: 0 processes.
INFO: Build completed successfully, 5 total actions
$ ./bazel-bin/s3_test
hello <botocore.client.S3 object at 0x7ff1b6686b38>