如何将一些外部jar库配置到flinkdocker容器中

Question

我正在运行一个具有以下配置的flink docker映像。

version: '2.1'
services:
jobmanager:
build: .
image: flink
volumes:
- .:/usr/local/lib/python3.7/site-packages/pyflink/lib
hostname: "jobmanager"
expose:
- "6123"
ports:
- "8081:8081"
command: jobmanager
environment:
- JOB_MANAGER_RPC_ADDRESS=jobmanager
taskmanager:
image: flink
volumes:
- .:/usr/local/lib/python3.7/site-packages/pyflink/lib
expose:
- "6121"
- "6122"
depends_on:
- jobmanager
command: taskmanager
links:
- jobmanager:jobmanager
environment:
- JOB_MANAGER_RPC_ADDRESS=jobmanager

运行日志如下

taskmanager_1  | 2020-10-11 10:34:03,714 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner      [] -  Classpath: 
/opt/flink/lib/flink-csv-1.11.0.jar:
/opt/flink/lib/flink-json-1.11.0.jar:
/opt/flink/lib/flink-shaded-zookeeper-3.4.14.jar:
/opt/flink/lib/flink-table-blink_2.12-1.11.0.jar:
/opt/flink/lib/flink-table_2.12-1.11.0.jar:
/opt/flink/lib/log4j-1.2-api-2.12.1.jar:
/opt/flink/lib/log4j-api-2.12.1.jar:
/opt/flink/lib/log4j-core-2.12.1.jar:
/opt/flink/lib/log4j-slf4j-impl-2.12.1.jar:
/opt/flink/lib/flink-dist_2.12-1.11.0.jar:::

从日志中，您可以看到一些默认库已加载到系统中，但我想添加一些jar，如flink-jdbc_2.11-1.9.0.jar，它位于我的本地文件系统中。

我的本地jar文件夹是/usr/local/lib/python3.7/sitepackages/pyflink/lib

我试过

volumes:
- /opt/flink/lib/:/usr/local/lib/python3.7/site-packages/pyflink/lib

错误是

ERROR: for keras-flask-deploy-webapp-master_jobmanager_1  Cannot start service jobmanager: Mounts denied: 
The path /opt/flink/lib is not shared from the host and is not known to Docker.
You can configure shared paths from Docker -> Preferences... -> Resources -> File Sharing.

我试过这个，但仍然不能工作

volumes:
- /usr/local/lib/python3.7/site-packages/pyflink/lib:/opt/flink/lib
[ERROR] Flink distribution jar not found in /opt/flink/lib.
taskmanager_1  | [ERROR] The execution result is empty.

如何将jdbc jar库链接到flink的docker容器？

Answer 1

这种情况下的解决方法可以是将带有特定jar的卷附加到容器中的某个临时位置，并覆盖run命令以将附加的文件复制到/opt/flink/libflink类路径文件夹中：

volumes:
- /usr/local/lib/python3.7/site-packages/pyflink/lib:/opt/flink/lib-custom
command: cp -a /opt/flink/lib-custom/. /opt/flink/lib/ && jobmanager

我想说，如果你从那里运行应用程序，调整jobmanager容器就足够了，但如果没有，也可以在taskmanager中重复同样的操作。