Apache Flink Operator语言 - enable azure-fs-hadoop

Question

我试图执行一个flink作业，使用flink操作符(https://github:com/apache/flink-kubernetes-operator)在k8s上，使用使用连接到Azure Blob存储在这里描述:https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/deployment/filesystems/azure/

按照指南，我需要将jar文件flink-azure-fs-hadoop-1.15.0.jar从一个目录复制到另一个目录。

我已经尝试过通过podTemplate和命令功能来做到这一点，但不幸的是它不起作用，并且文件没有出现在目标目录中。

你能指导我如何正确地做这件事吗?下面你可以找到我的FlinkDeployment文件。

apiVersion: flink.apache.org/v1beta1
kind: FlinkDeployment
metadata:
namespace: flink
name: basic-example
spec:
image: flink:1.15
flinkVersion: v1_15
flinkConfiguration:
taskmanager.numberOfTaskSlots: "2"
serviceAccount: flink
podTemplate:
apiVersion: v1
kind: Pod
metadata:
name: pod-template
spec:
serviceAccount: flink
containers:
- name: flink-main-container
volumeMounts:
- mountPath: /opt/flink/data
name: flink-data
#          command:
#            - "touch"
#            - "/tmp/test.txt"
volumes:
- name: flink-data
emptyDir: { }
jobManager:
resource:
memory: "2048m"
cpu: 1
podTemplate:
apiVersion: v1
kind: Pod
metadata:
name: job-manager-pod-template
spec:
initContainers:
- name: fetch-jar
image: cirrusci/wget
volumeMounts:
- mountPath: /opt/flink/data
name: flink-data
command:
- "wget"
- "LINK_TO_CUSTOM_JAR_FILE_ON_AZURE_BLOB_STORAGE"
- "-O"
- "/opt/flink/data/test.jar"
containers:
- name: flink-main-container
command:
- "touch"
- "/tmp/test.txt"
taskManager:
resource:
memory: "2048m"
cpu: 1
job:
jarURI: local:///opt/flink/data/test.jar
parallelism: 2
upgradeMode: stateless
state: running
ingress:
template: "CUSTOM_LINK_TO_AZURE"
annotations:
cert-manager.io/cluster-issuer: letsencrypt
kubernetes.io/ingress.allow-http: 'false'
traefik.ingress.kubernetes.io/router.entrypoints: websecure
traefik.ingress.kubernetes.io/router.tls: 'true'
traefik.ingress.kubernetes.io/router.tls.options: default

Answer 1

因为你使用的是Flink 1.15的镜像，所以这个Azure文件系统插件是内置的。您可以通过设置ENABLE_BUILT_IN_PLUGINS环境变量来启用它。

spec:
podTemplate:
containers:
# Do not change the main container name
- name: flink-main-container
env:
- name: ENABLE_BUILT_IN_PLUGINS
value: flink-azure-fs-hadoop-1.15.0.jar

https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/resource-providers/standalone/docker/using-filesystem-plugins