我正在尝试使用包含Java工具的容器在Kubernetes作业中对MySQL数据库进行一些DB迁移。
当我在Docker本地运行容器时(使用同一网络中的MySQL容器)，该工具按预期运行。如果我使用容器创建一个Pod，并将命令参数设置为指向在同一名称空间中运行的mysql服务，它也会这样做。
但是，如果我将该Pod规范转换为Job，则由于某种原因，创建的容器不能再连接到MySQL服务。
容器基于amazoncorretto:8-al2-jdk，只是将JAR拷贝到/opt/。
MySQL数据库通过集群的mysql服务可用:
$ kubectl describe service mysql -n <namespace>
Name: mysql
Namespace: <namespace>
Labels: app=mysql
Annotations: <none>
Selector: app=mysql
Type: ClusterIP
IP Families: <none>
IP: <ip>
IPs: <ip>
Port: mysql 3306/TCP
TargetPort: 3306/TCP
Endpoints: <ip>:3306
Session Affinity: None
Events: <none>
这些是Pod的规格:
apiVersion: v1
kind: Pod
metadata:
name: java-tool-pod
spec:
containers:
- name: javatool
image: <registry>/<image-name>:<version>
command: [ "/bin/sh" ]
args: [ "-x", "-c", "/usr/bin/java -jar /opt/<tool>.jar "jdbc:mysql://mysql:3306/<db>" -u <user> -p<password>" ]
imagePullSecrets:
- name: <secret>
将容器作为Pod运行:
$ kubectl apply -f /tmp/as-pod.yaml -n <namespace>
pod/java-tool-pod created
$ kubectl logs pod/java-tool-pod -n <namespace>
+ /usr/bin/java -jar /opt/<tool>.jar jdbc:mysql://mysql:3306/<db> -u <user> -p<password>
DB Migration Tool
Database Schema, 3.30.0.3300024390, built Wed Jul 14 12:13:52 UTC 2021
Driver class: com.mysql.jdbc.Driver
INFO Flyway 3.2.1 by Boxfuse
INFO Database: jdbc:mysql://mysql:3306/<db> (MySQL 5.7)
INFO Validated 721 migrations (execution time 00:00.253s)
INFO Current version of schema `<db>`: 3.29.0.10859.10
WARN outOfOrder mode is active. Migration of schema `<db>` may not be reproducible.
INFO Schema `<db>` is up to date. No migration necessary.
这些是工作的规格:
$ cat /tmp/as-job.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: javatool-job
spec:
template:
spec:
containers:
- name: javatool
image: <registry>/<image-name>:<version>
command: [ "/bin/sh" ]
args: [ "-x", "-c", "/usr/bin/java -jar /opt/<tool>.jar "jdbc:mysql://mysql:3306/<db>" -u <user -p<password>" ]
imagePullSecrets:
- name: <secret>
restartPolicy: Never
将容器作为作业运行:
$ kubectl apply -f /tmp/as-job.yaml -n <namespace>
job.batch/javatool-job created
$ kubectl logs job.batch/javatool-job -n <namespace>
+ /usr/bin/java -jar /opt/<tool>.jar jdbc:mysql://mysql:3306/<db> -u <user> -p<password>
DB Migration Tool
Database Schema, 3.30.0.3300024390, built Wed Jul 14 12:13:52 UTC 2021
Driver class: com.mysql.jdbc.Driver
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:983)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:339)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2252)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2285)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2084)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:795)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:44)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:400)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:327)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at org.springframework.jdbc.datasource.DriverManagerDataSource.getConnectionFromDriverManager(DriverManagerDataSource.java:173)
at org.springframework.jdbc.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:164)
at org.springframework.jdbc.datasource.AbstractDriverBasedDataSource.getConnectionFromDriver(AbstractDriverBasedDataSource.java:153)
at org.springframework.jdbc.datasource.AbstractDriverBasedDataSource.getConnection(AbstractDriverBasedDataSource.java:119)
at com.nordija.itv.db.FlywayMigrationSchemaData.isNotFlywaySchemaVersion(FlywayMigrationSchemaData.java:58)
[...]
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:607)
at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:214)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:298)
... 22 more
INFO Flyway 3.2.1 by Boxfuse
Unable to obtain Jdbc connection from DataSource
[...]
我没有看到正在创建的容器有任何显著的差异。我唯一能想到的是某种字符编码问题，但我不明白为什么这只发生在为Job创建的Pod中，而不是直接创建的Pod中。
提前感谢任何帮助与这个问题!
编辑:我忘了提到Istio在命名空间上是活动的，这是导致问题的原因。
问题是Istio不能很好地与Kubernetes工作(我忘了提到Istio在命名空间上是活动的，抱歉)。
一旦我添加了一个短延迟(启动Java工具之前的sleep 5)，连接就可以建立了。
但是我又遇到了另一个问题:在容器成功终止后，Job不会被标记为已完成。
原因还是Istio。一旦所有作业被认为完成Pod被终止，Istio Sidecar是一个不会终止的服务Pod。在找到这篇文章之后，我最终将他们的scuttle工具集成到容器中，现在Job可以成功完成了。
错误信息为"通信链路失败"。所以我认为你应该看看你的网络策略。