我们使用IAP, GCP L7负载均衡器与nginx-ingress控制器(版本0.49.3)。我们部署了自托管的GitLab，我们得到了"ERR_TOO_MANY_REDIRECTS"
经过几天紧张的故障排除，我们注意到
POST /api/v4/jobs/request HTTP/1.1
Host: gitlab.ci.g.nakhoda.ai
X-Request-ID: dfa874d97ed06e0e7a7cf17c0a4ae2c0
X-Real-IP: 35.191.12.183
X-Forwarded-For: 35.191.12.183
X-Forwarded-Host: gitlab.ci.g.nakhoda.ai
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Scheme: http
X-Scheme: http
X-Original-Forwarded-For: 62.189.73.245, 35.186.225.221
Content-Length: 714
User-Agent: gitlab-runner 15.6.1 (15-6-stable; go1.18.8; linux/amd64)
Accept: application/json
Content-Type: application/json
Accept-Encoding: gzip
X-Cloud-Trace-Context: 4efdd65224a5882999fd0fb26a888bfd/2273898499790060164
Via: 1.1 google
X-Forwarded-Proto: http和X-Forwarded-Scheme: http在初始重定向中设置为http。在摆弄了一下之后，我们在nginx-ingress-controller的容器中执行了一下，并编辑了nginx.conf，使它们都具有https，并添加了注释ssl-redirect: false。
之后的请求看起来像这样:
POST /api/v4/jobs/request HTTP/1.1
Host: gitlab.ci.g.nakhoda.ai
X-Request-ID: f103bba96d47527dae15087d1dd1d476
X-Real-IP: 35.191.12.180
X-Forwarded-For: 35.191.12.180
X-Forwarded-Host: gitlab.ci.g.nakhoda.ai
X-Forwarded-Port: 80
X-Forwarded-Proto: https
X-Forwarded-Scheme: https
X-Scheme: http
X-Original-Forwarded-For: 194.203.216.4, 35.186.225.221
Content-Length: 714
User-Agent: gitlab-runner 15.6.1 (15-6-stable; go1.18.8; linux/amd64)
Accept: application/json
Content-Type: application/json
Accept-Encoding: gzip
X-Cloud-Trace-Context: 3ceb1d9d95fa28be4da929dea1f3ac95/3016269448751760533
Via: 1.1 google
通过手动编辑，现在我们可以访问GitLab，没有任何问题，但问题是，这是我们添加的手动修复，所以每次管道进行部署时，它都会被重置。
问题是我们寻找添加自定义头来覆盖[nginx.conf](https://kubernetes.github)。. io/Ingress -nginx/examples/customization/custom-headers/不是覆盖正常的nginx.conf配置)，而是在添加了带有特定内容的ConfigMap之后，或者只是在nginx-ingress控制器的Ingress或ConfigMap中添加类似的注释:
Gitlab入口
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: gitlab-chart-webservice-default
namespace: gitlab
labels:
app: webservice
app.kubernetes.io/managed-by: Helm
chart: webservice-6.6.2
gitlab.com/webservice-name: default
heritage: Helm
release: gitlab-chart
annotations:
kubernetes.io/ingress.provider: nginx
meta.helm.sh/release-name: gitlab-chart
meta.helm.sh/release-namespace: gitlab
nginx.ingress.kubernetes.io/proxy-body-size: 512m
nginx.ingress.kubernetes.io/proxy-connect-timeout: '15'
nginx.ingress.kubernetes.io/proxy-read-timeout: '600'
nginx.ingress.kubernetes.io/service-upstream: 'true'
nginx.ingress.kubernetes.io/ssl-redirect: 'false'
status:
loadBalancer:
ingress:
- ip: 1.1.1.1
- ip: 1.1.1.1
spec:
ingressClassName: stable-protected
tls:
- hosts:
- gitlab.ci.example.com
secretName: gitlab-cert
rules:
- host: gitlab.ci.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: gitlab-chart-webservice-default
port:
number: 8181
配置nginx控制器
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx
namespace: ingress-stable-protected
annotations:
refresh-me: |
This 'refresh-me' annotation is purely used as a placeholder so we can
modify its value in order to manually force the nginx ingress-controller
to reload its configuration.
Reload.
data:
enable-vts-status: 'true'
hide-headers: Strict-Transport-Security
hsts: 'true'
hsts-include-subdomains: 'false'
hsts-preload: 'false'
proxy-body-size: 2048m
proxy-buffer-size: 16k
proxy-connect-timeout: '5'
proxy-next-upstream: 'off'
proxy-read-timeout: '600'
proxy-send-timeout: '600'
server-name-hash-bucket-size: '256'
server-tokens: 'false'
nginx.ingress.kubernetes.io/x-forwarded-proto=https
nginx.ingress.kubernetes.io/x-forwarded-scheme=https
它仍然需要nginx.confhttp，所以我们有点困于这个临时修复。
尝试添加一堆自定义标头和x-forward -headers，但无济于事。
这个问题是在nginx-ingress的0.24.0版本之后，有一个注释的变化，所以通过添加以下到nginx的配置图，没有更多的错误。
更深入的描述可以在这张票
中找到use-forwarded-headers: "true"
use-proxy-protcol: "false"
完整的示例:
apiVersion: v1
data:
enable-vts-status: "true"
hide-headers: Strict-Transport-Security
hsts-preload: "false"
proxy-body-size: 2048m
proxy-buffer-size: 16k
proxy-connect-timeout: "5"
proxy-next-upstream: "off"
proxy-read-timeout: "600"
proxy-send-timeout: "600"
server-name-hash-bucket-size: "256"
server-tokens: "false"
use-forwarded-headers: "true"
use-proxy-protcol: "false"
kind: ConfigMap
metadata:
annotations:
refresh-me: |
This 'refresh-me' annotation is purely used as a placeholder so we can
modify its value in order to manually force the nginx ingress-controller
to reload its configuration.
labels:
app.kubernetes.io/managed-by: pulumi
name: nginx
namespace: namespace