Terraform:获取nginx入口控制器负载均衡器IP

Question

我正在尝试在Azure AKS中获取nginx入口控制器负载均衡器ip。我想我会使用kubernetes提供商通过：

data "kubernetes_service" "nginx_service" {
metadata {
name      = "${local.ingress_name}-ingress-nginx-controller"
namespace = local.ingress_ns
}
depends_on = [helm_release.ingress]
}

然而，我没有看到IP地址，这是我得到的：

nginx_service = [
+ {
+ cluster_ip                  = "10.0.165.249"
+ external_ips                = []
+ external_name               = ""
+ external_traffic_policy     = "Local"
+ health_check_node_port      = 31089
+ load_balancer_ip            = ""
+ load_balancer_source_ranges = []
+ port                        = [
+ {
+ name        = "http"
+ node_port   = 30784
+ port        = 80
+ protocol    = "TCP"
+ target_port = "http"
},
+ {
+ name        = "https"
+ node_port   = 32337
+ port        = 443
+ protocol    = "TCP"
+ target_port = "https"
},
]
+ publish_not_ready_addresses = false
+ selector                    = {
+ "app.kubernetes.io/component" = "controller"
+ "app.kubernetes.io/instance"  = "nginx-ingress-internal"
+ "app.kubernetes.io/name"      = "ingress-nginx"
}
+ session_affinity            = "None"
+ type                        = "LoadBalancer"
},
]

然而，当我通过kubectl关闭服务时，我可以通过获得IP地址

kubectl get svc nginx-ingress-internal-ingress-nginx-controller -n nginx-ingress -o json | jq -r '.status.loadBalancer.ingress[].ip'
10.141.100.158

这是kubernetes提供商对AKS的限制吗？如果是，其他人使用了什么变通方法？我的最终目标是使用IP来配置应用程序网关后端。

我想我可以使用local-exec，但这似乎很难。然而，这可能是我目前唯一的选择。

谢谢，

Jerry

Answer 1

尽管我强烈建议不要使用Terraform在Kubernetes中创建资源，但您可以这样做：

使用Terraform创建公共IP->使用Terraform在Kubernetes中创建ingress nginx，并使用Terraform资源中的数据传递annotations和loadBalancerIP。最后的清单应该是这样的：

apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/azure-load-balancer-resource-group: myResourceGroup
name: ingress-nginx-controller
spec:
loadBalancerIP: <YOUR_STATIC_IP>
type: LoadBalancer

Terraform可能是这样的：

resource "kubernetes_service" "ingress_nginx" {
metadata {
name = "tingress-nginx-controller"

annotations {
"service.beta.kubernetes.io/azure-load-balancer-resource-group" = "${azurerm_resource_group.YOUR_RG.name}"
}
spec {
selector = {
app = <PLACEHOLDER>
}
port {
port        = <PLACEHOLDER>
target_port = <PLACEHOLDER>
}
type = "LoadBalancer"
load_balancer_ip = "${azurerm_public_ip.YOUR_IP.ip_address}"
}
}

Answer 2

不幸的是，这是用于内部入口，而不是面向公共的，并且IP是动态分配的。我们目前不想使用静态ips

这就是我想到的：

module "load_balancer_ip" {
count = local.create_ingress ? 1 : 0
source  = "github.com/matti/terraform-shell-resource?ref=v1.5.0"
command = "./scripts/get_load_balancer_ip.sh"
environment = {
KUBECONFIG = base64encode(module.aks.kube_admin_config_raw)
}
depends_on = [local_file.load_balancer_ip_script]
}
resource "local_file" "load_balancer_ip_script" {
count = local.create_ingress ? 1 : 0
filename = "./scripts/get_load_balancer_ip.sh"
content  = <<-EOT
#!/bin/bash
echo $KUBECONFIG | base64 --decode > kubeconfig
kubectl get svc -n ${local.ingress_ns} ${local.ingress_name}-ingress-nginx-controller --kubeconfig kubeconfig -o=jsonpath='{.status.loadBalancer.ingress[0].ip}'
rm -f kubeconfig 2>&1 >/dev/null
EOT
}
output nginx_ip {
description = "IP address of the internal nginx controller"
value = local.create_ingress ? module.load_balancer_ip[0].content : null
}