我正在尝试使用terraform在DataBricks工作空间中自动化和提供DataBricks集群。
我们有预定义的库,需要在集群创建时安装,我们确实有大约20个库
variable"libraries"{
default = ["pypi","jar"]
}
variable"pythonpackages"{
default = ["conda","pymongo"]
}
variable"jarpackages"{
default = ["etc1","etc1"]
}
variable"repositories"{
default = ["internal1","internal2"]
}
resource "databricks_cluster" "shared_autoscaling" {
cluster_name = "Shared Autoscaling"
spark_version = data.databricks_spark_version.latest_lts.id
node_type_id = data.databricks_node_type.smallest.id
autotermination_minutes = 20
autoscale {
min_workers = 1
max_workers = 50
}
library {
pypi {
package = "dnspython"
}
}
library {
pypi {
package = "condo"
}
}
library {
pypi {
package = "kedro"
repo = "artifactory-url"
}
}
}
我正在尝试删除库属性的重复性。我尝试了在库属性内使用for_each
,但它没有正确工作。
我想达到的目标
variable"libraries"{
default = ["pypi","jar"]
}
variable"pythonpackages"{
default = ["conda","pymongo"]
}
variable"jarpackages"{
default = ["etc1","etc1"]
}
variable"repositories"{
default = ["internal1","internal2"]
}
resource "databricks_cluster" "shared_autoscaling" {
cluster_name = "Shared Autoscaling"
spark_version = data.databricks_spark_version.latest_lts.id
node_type_id = data.databricks_node_type.smallest.id
autotermination_minutes = 20
autoscale {
min_workers = 1
max_workers = 50
}
library {
pypi {
package = ["condo","pymongo","listofpackages"]
}
}
}
您可以使用动态块来重复嵌套块。
例如,要重复库块,可以这样写:resource "databricks_cluster" "shared_autoscaling" {
cluster_name = "Shared Autoscaling"
spark_version = data.databricks_spark_version.latest_lts.id
node_type_id = data.databricks_node_type.smallest.id
autotermination_minutes = 20
autoscale {
min_workers = 1
max_workers = 50
}
dynamic "library" {
for_each = ["condo","pymongo","listofpackages"]
content {
pypi {
package = library.value
}
}
}
}