带TLS HA和Raft的k8s上的Vault



我一直试图让Vault在OpenShift 3.11中使用Raft存储以HA模式运行,但在尝试将第二个节点加入集群时,我总是遇到以下错误:

  • 无法加入raft集群:引导初始化调用期间出错:Puthttps://vault3.vault3.svc:8200/v1/sys/storage/raft/bootstrap/challenge:x509:由未知权威机构签署的证书

我按照这个例子创建了一个证书和密钥,并将它们与Kubernetes CA一起存储为Kubernete机密(命名为vault服务器tls(。

Helm图表values.yaml有一个部分引用了这个秘密:

extraVolumes:
- type: secret
name: vault-server-tls 

我相信这是正确的,因为我可以进入吊舱,保险库服务器tls机密正在安装,3个文件如预期显示:

/ $ ls -l /vault/userconfig/vault-server-tls
total 0
lrwxrwxrwx    1 root     root            15 Apr  7 19:26 vault.ca -> ..data/vault.ca
lrwxrwxrwx    1 root     root            16 Apr  7 19:26 vault.crt -> ..data/vault.crt
lrwxrwxrwx    1 root     root            16 Apr  7 19:26 vault.key -> ..data/vault.key

Helm chart values.yaml侦听器配置引用了cert、key和ca文件:

raft:
# Enables Raft integrated storage
enabled: false
config: |
ui = true
cluster_addr = "https://POD_IP:8201"
listener "tcp" {
tls_disable = 0
address = "[::]:8200"
cluster_address = "[::]:8201"
tls_cert_file = "/vault/userconfig/vault-server-tls/vault.crt"
tls_key_file  = "/vault/userconfig/vault-server-tls/vault.key"
tls_client_ca_file = "/vault/userconfig/vault-server-tls/vault.ca"
}
storage "raft" {
path = "/vault/data"
}

第一个吊舱开启良好:

oc exec -ti vault3-0 -- vault operator unseal -tls-skip-verify 
Key                    Value
---                    -----
Seal Type              shamir
Initialized            true
Sealed                 false
Total Shares           1
Threshold              1
Version                1.3.4
Cluster Name           vault-cluster-945d1fc5
Cluster ID             2ac97997-b596-4dcc-4926-2d8acec56ed5
HA Enabled             true
HA Cluster             n/a
HA Mode                standby
Active Node Address    <none>

但是,当我尝试将第二个pod加入集群时,我得到了一个x509:由未知权威机构消息签名的证书:

oc exec -ti vault3-1 -- vault operator raft join -tls-skip-verify https://vault3.vault3.svc:8200
Error joining the node to the raft cluster: Error making API request.
URL: POST https://127.0.0.1:8200/v1/sys/storage/raft/join
Code: 500. Errors:
* failed to join raft cluster: error during bootstrap init call: Put https://vault3.vault3.svc:8200/v1/sys/storage/raft/bootstrap/challenge: x509: certificate signed by unknown authority

在第二个pod中,我可以看到VAULT_CACERT环境值存在并且设置正确(如上所述,3个文件已按预期就位(:

/ $ printenv VAULT_CACERT
/vault/userconfig/vault-server-tls/vault.ca
/ $ printenv VAULT_ADDR
https://127.0.0.1:8200
/ $

这是我的CA:

-----BEGIN CERTIFICATE-----
MIIC6jCCAdKgAwIBAgIBATANBgkqhkiG9w0BAQsFADAmMSQwIgYDVQQDDBtvcGVu
c2hpZnQtc2lnbmVyQDE1NjcxNzQyODkwHhcNMTkwODMwMTQxMTI4WhcNMjQwODI4
MTQxMTI5WjAmMSQwIgYDVQQDDBtvcGVuc2hpZnQtc2lnbmVyQDE1NjcxNzQyODkw
ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDTE3YoGObzoRFLTcLR4Ocv
FPy8XrWvWxBGnb1U9ct2QUaTjxuubcazGZOr8QuovwnFzp3Is8xS3pz9wBDyZqpS
JX1Z3Ja/ngi9472aAPLZnalRnGWkrTl8pAargLTUmnAcS7eEfRs+NH6aZdpefhX5
PQ3Ll1xMuu3U/PVVtfwkLje85LsevlVs5VBjDdc36UOU0BhEJM4pXbvLDG2pLpnM
BhILR2RgOX5oca27K6YDVVTHwxw83EgWvUHwpeoZuyEMPBQqhHJ3fzPvxfx4rCgd
/iV8KO2zP+/3t/kmrkJM7avgWbc7QqWPeKbB7WkKOpaTMwhpe1U1ZNjTnt2tzo8f
AgMBAAGjIzAhMA4GA1UdDwEB/wQEAwICpDAPBgNVHRMBAf8EBTADAQH/MA0GCSqG
SIb3DQEBCwUAA4IBAQC6hJ/74qvFztZ0YQBT5ZhGdwgUQtFYttcgLMRcv6QX8S+M
WK8TnmlMb6xgklzsc63nFzPl1pPHMQ7pEiclUUZCJdivzgVlDGLrjXUxYVLA2TGu
UqJJer9XAkAtrAPZ/ppx3iroinDWA+8mQBs4h9nmmFQS98et7A1Of0Josl/3UE6R
9jQDvadOLi01t3UPqSLvsYfpl5I4cuqpiCSnphTHLIl1AL5PNwuZAgLpOIVVsi9h
7OaIy285vYoGK12L3yUXYj5EixY94CmYMu6LDfiQdeN6P9nE78O4OXPS3uXm2M1/
Trm1rY3ulqyj6jwLwmvPQpwtO81A48GrNTvcoFq+
-----END CERTIFICATE-----

这是我的证书:

-----BEGIN CERTIFICATE-----
MIIDgjCCAmqgAwIBAgIUQ3RqBAahTzugd9PfQ29Y1nAT5EcwDQYJKoZIhvcNAQEL
BQAwJjEkMCIGA1UEAwwbb3BlbnNoaWZ0LXNpZ25lckAxNTc4NDI5MzQwMB4XDTIw
MDQwNzE3NDkwMFoXDTIxMDQwNzE3NDkwMFowHDEaMBgGA1UEAxMRdmF1bHQzLnZh
dWx0My5zdmMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC5/cyUmBJM
c3ngjqZmc5LDVp7Ft7UNeb2RF3GedRiefp5/S+0xTSGRItigv/qJ4QrGuujkFtcI
O9zwxPFAjXy+HFW3gjUzqqZKdY1aur9QYtVYeZxFHxSy67jLQIT2NIcN2c4HC8yk
FKDTY/sviv5LZy90w7Spw514sdy5bQZApfE9PWrhDMNHYbiU6aq2uWmvlkVTfXdB
v/5DMFE691WcH1x0kJxHW3TMp8vh9Y7N5tm7PTeFPuHc/OimHSvzuMPOqrCPmcoY
h8fvQTBmjOthIY9ATSELXPYnRsOmwqYEnHh2PLIugVRQ/iAT4zSuN0eoo8ja05Oa
hFpxeOfBZWWrAgMBAAGjgbEwga4wDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQMMAoG
CCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFO2ZvIrxIeG1BFYTMMhQ
T4VJALNjMFoGA1UdEQRTMFGCBnZhdWx0M4INdmF1bHQzLnZhdWx0M4IRdmF1bHQz
LnZhdWx0My5zdmOCH3ZhdWx0My52YXVsdDMuc3ZjLmNsdXN0ZXIubG9jYWyHBH8A
AAEwDQYJKoZIhvcNAQELBQADggEBAFzKuf7vGxowvEqCDLB044DWUFnx5iSJ3t9q
+xJTza3a7aIVZtbHy9wz8z2u/3WZKXKfepQJW8BvFfr1LkZd7bA73/RgysJkLclo
E2gPwRTqIcYdNc0za5GB5WV2HH6TM/Ybfvp+UdJ42YzvhdFr8FL8CZS9IOjJ/yyl
nRp77c4O10OFNwQIxGqsvhcDmKKw9/ISER5opWT9vRKoOlW7vDcsCgOg+td8m0//
WX6IOOQz5gYvBTQFXK4t8cNrA2P8Fb7LICdHD9v8yNshwa0ytYCgRhqoXmSR7fXf
Rw8c2ZjzLNlZbRJprkTafI9lDZBpoltKf0qpfDWfeeEXab2toyw=
-----END CERTIFICATE-----

这是我的钥匙:

-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAuf3MlJgSTHN54I6mZnOSw1aexbe1DXm9kRdxnnUYnn6ef0vt
MU0hkSLYoL/6ieEKxrro5BbXCDvc8MTxQI18vhxVt4I1M6qmSnWNWrq/UGLVWHmc
RR8Usuu4y0CE9jSHDdnOBwvMpBSg02P7L4r+S2cvdMO0qcOdeLHcuW0GQKXxPT1q
4QzDR2G4lOmqtrlpr5ZFU313Qb/+QzBROvdVnB9cdJCcR1t0zKfL4fWOzebZuz03
hT7h3Pzoph0r87jDzqqwj5nKGIfH70EwZozrYSGPQE0hC1z2J0bDpsKmBJx4djyy
LoFUUP4gE+M0rjdHqKPI2tOTmoRacXjnwWVlqwIDAQABAoIBAGfn/1QAhTCKzssC
Rroz1QkmKjthP1fQ7HPTehlBZ8icCmYpf9CsO5V+tzFPL2O6ArA1mYhbrjQeZXdp
PzKGGOuInuaParN7ob01YQwQCnOZU4FDJ2eCTqkKgcmjOOTnQZAeXziWgfQkxhhy
4dCUwS5U7jE4aITzENVt4FcDLpwMZHNZBnfI9npB5/UhaDEl8X6DB/B3lDwB6lWf
mfA20FB3qaLA4tQYRL3DrhD2e5uKje7ajfd6IUHEnvCIfMrrjNrfTQY6I0YfubvB
scs2pr0pyVgZkt4vWZHpwlw2vjMrBy93HtCoAyPN+aSLD9CM4j+HXWJ4Bew5EjFQ
kBEcuCECgYEA6zVVwYWp92iOJ5lhH7tLgkTCBLDCOAdLMuPBJVhQdO1K0FaMwbHV
2Su/RmOPpgYZGAlicdgrm9MS77zsrhWneV3Oo82IpD8j/ah7bP0RGxS7BZDSy+cK
/QI7mYTrNG7Re8+4WdAL9seO4gcS6+bPOBuf5wR7cFeXacJHo72CLvsCgYEAym60
q3hLiC+RZifuK4z9KN3hMQ3MDtbYrjSGROzZF0SWEM1xJke4fX/jjSGt/IMK8p9N
HxcYWD1gVfeP5Vxnj297JdBbi/knmoKoDN075IVP6j1P7Q9UG4XVEK5qgvmYizU8
kd+IbyMa5OXT93M5aw2NgeEoe/JntfEA22cCJRECgYAVQd29PrpMvOtUEt2fQ4sg
e9xZFiyHaclXERRsrp2e469GQvw3qT3dgcGot+jMpXJxJK/8AAB49cuZVSbC2Pwo
0NyTG0lFJtu22hpFkF2SZ/47E4qpmPj6QtBmIIgtVfKi0PQlUdMy+3gjX2ZLYbHK
rVx3QYVycsgha8iTuNXiLwKBgQCwrYS0L27E8rdVUL53djsyIs07kg4qWWuOR7t0
hr9Gpo7PJW9++JPVvPvunpmKzRiN/2lBHFgcE510Cnilt0uPjb4Ol9Z+yTu+iBCC
AckXPx8rks2iWoGO7/Sw9XlyzMNNpG4z5sPeM+ZyJwEkdIWFoLODyu8Zlszbp/eW
hkYB8QKBgQDp9x7rWaWUAeCFTUkEUWNiVXGp/BrTfGGkunZFJXgZrqJHHKSmdeRI
VarplRVJzyeD8qf7qyoS34oaXB/3o+ZdPHxM1+bLp2tQtqwgQtcX36FmyimHggym
71YkHfWj9ABUX0Gczdwj8atJcBrNSiHzbpDvdi1F5vNFymOvfqI9hA==
-----END RSA PRIVATE KEY-----

我遵循了Helm TLS示例中的示例(在本文的顶部(,可能有10次不同的错误,但仍然会得到相同的错误。

我不确定我做错了什么,但如果能提供任何帮助,我将不胜感激。

我在加入集群时遇到了与您类似的错误,但问题是我试图加入https://IP:8200证书在SAN上没有IP,只有DNS。

关于您的错误:* failed to join raft cluster: error during bootstrap init call: Put https://vault3.vault3.svc:8200/v1/sys/storage/raft/bootstrap/challenge: x509: certificate signed by unknown authority我建议通过操作系统对CA的实际信任进行故障排除。I

这两个命令中的一个命令的输出应该会使实际错误和openssl s_client -showcerts -connect vault3.vault3.svc:8200/

curl -v https://vault3.vault3.svc:8200

请参阅一个示例,该示例显示正在使用哪个CA证书存储

curl -v https://vault01:8200
* Rebuilt URL to: https://vault01:8200/
*   Trying 192.168.1.151...
* TCP_NODELAY set
* Connected to vault01 (192.168.1.151) port 8200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, [no content] (0):

最新更新