我正在尝试使用 rules_k8s 让 Bazel 部署到我的 Kubernetes 集群。
因此，我有这个cloudbuild.yaml文件，它由Google Cloud Build执行：
steps:
- name: gcr.io/cloud-builders/bazel
args: ['run', '//:kubernetes.apply']
(//:kubernetes只是一个k8s_objects(
在我的本地计算机上运行bazel run //:kubernetes.apply工作正常，但是 尽管 Google Cloud Build 成功，但它会记录这些错误。所以配置没有应用于我的 Kubernetes 集群：
Target //:kubernetes.apply up-to-date:
bazel-bin/kubernetes.apply
INFO: Elapsed time: 29.863s, Critical Path: 0.14s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/kubernetes.apply
INFO: Build Event Protocol files produced successfully.
INFO: Build completed successfully, 1 total action
kubectl toolchain was not properly configured so k8s_deployment.apply cannot be executed.
kubectl toolchain was not properly configured so k8s_service.apply cannot be executed.
kubectl toolchain was not properly configured so projection_database_k8s_deployment.apply cannot be executed.
kubectl toolchain was not properly configured so projection_database_k8s_service.apply cannot be executed.
kubectl toolchain was not properly configured so k8s_deployment.apply cannot be executed.
kubectl toolchain was not properly configured so k8s_service.apply cannot be executed.
kubectl toolchain was not properly configured so k8s_deployment.apply cannot be executed.
kubectl toolchain was not properly configured so k8s_service.apply cannot be executed.
kubectl toolchain was not properly configured so event_store_k8s_deployment.apply cannot be executed.
kubectl toolchain was not properly configured so event_store_k8s_service.apply cannot be executed.
kubectl toolchain was not properly configured so k8s_deployment.apply cannot be executed.
kubectl toolchain was not properly configured so k8s_service.apply cannot be executed.
kubectl toolchain was not properly configured so event_store_k8s_deployment.apply cannot be executed.
kubectl toolchain was not properly configured so event_store_k8s_service.apply cannot be executed.
kubectl toolchain was not properly configured so k8s_deployment.apply cannot be executed.
kubectl toolchain was not properly configured so k8s_service.apply cannot be executed.
kubectl toolchain was not properly configured so event_store_k8s_deployment.apply cannot be executed.
kubectl toolchain was not properly configured so event_store_k8s_service.apply cannot be executed.
我还收到来自 bazel 缓存的警告：
DEBUG: /builder/home/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/external/io_bazel_rules_k8s/toolchains/kubectl/kubectl_toolchain.bzl:28:9: No kubectl tool was found or built, executing run for rules_k8s targets might not work.
PS：使用//:kubernetes.create时我收到相同的错误
我的设置
部署
load("@io_bazel_rules_k8s//k8s:object.bzl", "k8s_object")
k8s_object(
name = "k8s_deployment",
kind = "deployment",
cluster = "gke_cents-ideas_europe-west3-a_cents-ideas",
template = ":ideas.deployment.yaml",
images = {
"gcr.io/cents-ideas/ideas:latest": ":image"
},
)
服务业
k8s_object(
name = "k8s_service",
kind = "service",
cluster = "gke_cents-ideas_europe-west3-a_cents-ideas",
template = ":ideas.service.yaml",
)
聚合
load("@io_bazel_rules_k8s//k8s:objects.bzl", "k8s_objects")
k8s_objects(
name = "k8s",
objects = [
":k8s_deployment",
":k8s_service",
]
)
最终组成
k8s_objects(
name = "kubernetes",
objects = [
"//services/ideas:k8s",
# ...
]
)
更新
我现在尝试使用 Bazel 和 kubectl 制作自己的 docker 镜像：
FROM gcr.io/cloud-builders/bazel
RUN curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
RUN chmod +x ./kubectl
RUN mv ./kubectl /usr/local/bin/kubectl
我将其推送到 GCR 并将cloudbuild.yaml更改为：
steps:
- name: eu.gcr.io/cents-ideas/bazel-kubectl
args: ["run", "//:kubernetes.apply"]
我首先注意到，这一步比以前花了更长的时间。但是在最后它抛出一个错误：
$ /usr/local/bin/kubectl --kubeconfig= --cluster=gke_cents-ideas_europe-west3-a_cents-ideas --context= --user= apply -f -
error: cluster "gke_cents-ideas_europe-west3-a_cents-ideas" does not exist
这是完整的日志。
至于更新的问题，现在您需要以某种方式向容器内的 GKE 进行身份验证。
首先，我建议将gcloud工具安装到您的容器中。 顺便说一句，至于巨大的容器大小1.2 GB，那是因为cloud-builders/bazel是巨大的:)
看看我们关于超薄边框容器版本的示例： https://github.com/aspect-development/bazel-k8s-example/blob/master/tools/Dockerfile.dazel
这是用于安装 gcloud 和 kubectl 的 Dockerfile，因此您可以从这两个文件中获取所需的部分： https://github.com/GoogleCloudPlatform/cloud-builders/blob/master/gcloud/Dockerfile
第二件事是身份验证，安装 gcloud 后应该很容易。 整体云构建步骤应如下所示：
- name: <link to your container>
entrypoint: /bin/sh
args:
- -c
- |
gcloud container clusters get-credentials cents-ideas --zone europe-west3-a --project cents-ideas
bazel run //:kubernetes.apply
它抱怨在您的计算机上找不到kubectl。如果使用 GKE，还需要安装gcloudsdk。
还要检查是否为这些工具配置了身份验证：kubectl auth，GKE auth。
主要问题是kubectl没有附带gcr.io/cloud-builders/bazeldocker 映像。这就是为什么找不到它的原因。
但是，有人努力将kubectl工具链手动实现到Bazel中：https://github.com/bazelbuild/rules_k8s/tree/master/toolchains/kubectl#kubectl-toolchain。但由于这些功能目前处于实验状态，因此它们通常不起作用。
更多信息可以在本期中找到：https://github.com/bazelbuild/rules_k8s/issues/512
目前，最好的选择是在容器中安装 kubectl。