Cloud Composer v2 API Service Agent Extension角色可能缺失

Question

我正在尝试使用下面的一组地形脚本代码库来启动GCP Cloud Composer:

resource "google_composer_environment" "test" {
name   = "example-composer-env-tf-c2"
region = "us-central1"
config {
software_config {
image_version = "composer-2-airflow-2"
}
workloads_config {
scheduler {
cpu        = 0.5
memory_gb  = 1.875
storage_gb = 1
count      = 1
}
web_server {
cpu        = 0.5
memory_gb  = 1.875
storage_gb = 1
}
worker {
cpu        = 0.5
memory_gb  = 1.875
storage_gb = 1
min_count  = 1
max_count  = 3
}

}
environment_size = "ENVIRONMENT_SIZE_SMALL"
node_config {
network         = google_compute_network.test.id
subnetwork      = google_compute_subnetwork.test.id
service_account = google_service_account.test.name
}
}
}
resource "google_compute_network" "test" {
name                    = "composer-test-network3"
auto_create_subnetworks = false
}
resource "google_compute_subnetwork" "test" {
name          = "composer-test-subnetwork"
ip_cidr_range = "10.2.0.0/16"
region        = "us-central1"
network       = google_compute_network.test.id
}
resource "google_service_account" "test" {
account_id   = "composer-env-account"
display_name = "Test Service Account for Composer Environment"
}
resource "google_project_iam_member" "composer-worker" {
project = "inlaid-ally-373906"
role    = "roles/composer.worker"
member  = "serviceAccount:${google_service_account.test.email}"
}
resource "google_project_iam_member" "composer-service-agent-v2-ext" {
project = "inlaid-ally-373906"
role    = "roles/composer.ServiceAgentV2Ext"
member  = "serviceAccount:${google_service_account.test.email}"
}`

However, while executing terraform apply, I am facing below err:

╷
│ Error: googleapi: Error 400: Composer API Service Agent service account ([e-mail]) does not have required permissions set. Cloud Composer v2 API Service Agent Extension role might be missing. Please refer to https://cloud.google.com/composer/docs/composer-2/create-environments#grant-permissions and Composer Creation Troubleshooting pages to resolve this issue., failedPrecondition
│ 
│   with google_composer_environment.test,
│   on main.tf line 49, in resource "google_composer_environment" "test":
│   49: resource "google_composer_environment" "test" {
│ 

我参考了这个文档，但没有找到解决上述问题的方法。有什么方法来修复这个错误吗?

尝试使用下面的一组地形脚本代码库启动GCP Cloud Composer，但面临以下错误:

╷│ Error: googleapi: Error 400: Composer API Service Agent service account ([e-mail]) does not have required permissions set. Cloud Composer v2 API Service Agent Extension role might be missing. Please refer to ``https://cloud.google.com/composer/docs/composer-2/create-environments#grant-permissions`` and Composer Creation Troubleshooting pages to resolve this issue., failedPrecondition││ with google_composer_environment.test,│ on main.tf line 49, in resource "google_composer_environment" "test":│ 49: resource "google_composer_environment" "test" {│

Answer 1

第一次创建Cloud Composer集群时，必须将roles/composer.ServiceAgentV2Ext提供给Composer默认的服务帐户，例如gcloudcli:

gcloud projects add-iam-policy-binding composer-env-account@{your_project}.iam.gserviceaccount.com 
--member [e-mail] 
--role roles/composer.ServiceAgentV2Ext

用您的项目ID替换{your_project}。

默认Composer业务帐号为:[e-mail]

您提供给Composer的服务帐户将在运行时用于Airflowdag:composer-env-account

Answer 2

我在启动cloud composer v2时遇到了同样的问题。我必须使用角色roles/composer. serviceagentv2ext为composer环境配置中使用的服务帐户设置iam策略绑定。在类似的情况下，以下命令应该可以工作:

gcloud iam service-accounts add-iam-policy-binding
composer-env-account@{YOUR_PROJECT_ID}.iam.gserviceaccount.com 
--member serviceAccount:[e-mail] 
--role roles/composer.ServiceAgentV2Ext
resource "google_service_account_iam_member" "custom_service_account" {
provider = google-beta
service_account_id = "[e-mail]"
role = "roles/composer.ServiceAgentV2Ext"
member = "serviceAccount:[e-mail]"
}

相关链接:

1. https://cloud.google.com/composer/docs/how-to/access-control composer-sa
2. 以上代码的文档:https://cloud.google.com/composer/docs/composer-2/create-environments#terraform_2:~:text=Project%20number.-，Example，-%3A

Answer 3

有用于terraform的composer模块，我发现它更容易使用，因为模块可以处理权限。

module "composer" {
source  = "terraform-google-modules/composer/google"
version = "~> 3.4"
project_id        = "<PROJECT ID>"
region            = "us-central1"
composer_env_name = "composer-env-test"
network           = "test-network"
subnetwork        = "composer-subnet"
enable_private_endpoint = false
}

resource "google_project_iam_member" "composer_agent_service_account" {
count   = var.grant_sa_agent_permission ? 1 : 0
project = data.google_project.project.project_id
role    = "roles/composer.ServiceAgentV2Ext"
member  = format("serviceAccount:%s", local.cloud_composer_sa)
}

https://registry.terraform.io/modules/terraform-google-modules/composer/google/latest