Apache Flink Google Pub/Sub Connector 运行在 intellij idea 中，抛出

Question

我正在尝试将Google Pub/Sub连接器与Flink一起使用，请参阅此处。

但是，我得到了

Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.google.cloud.pubsub.v1.stub.GrpcPublisherStub

我有连接器的依赖项版本是 1.9.0

<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-gcp-pubsub_2.12</artifactId>
<version>1.9.0</version>
</dependency>

完整的例外堆栈

Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:638)
at org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:123)
at btc.da.StreamingJob.main(StreamingJob.java:87)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.google.cloud.pubsub.v1.stub.GrpcPublisherStub
at com.google.cloud.pubsub.v1.Publisher.<init>(Publisher.java:149)
at com.google.cloud.pubsub.v1.Publisher.<init>(Publisher.java:78)
at com.google.cloud.pubsub.v1.Publisher$Builder.build(Publisher.java:614)
at org.apache.flink.streaming.connectors.gcp.pubsub.PubSubSink.open(PubSubSink.java:111)
at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
at org.apache.flink.streaming.api.operators.StreamSink.open(StreamSink.java:48)
at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:424)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:290)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
at java.base/java.lang.Thread.run(Thread.java:835)

Flink 生成的项目也使用 maven 阴影插件

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.0.0</version>
<executions>
<!-- Run shade goal on package phase -->
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<artifactSet>
<excludes>
<exclude>org.apache.flink:force-shading</exclude>
<exclude>com.google.code.findbugs:jsr305</exclude>
<exclude>org.slf4j:*</exclude>
<exclude>log4j:*</exclude>
</excludes>
</artifactSet>
<filters>
<filter>
<!-- Do not copy the signatures in the META-INF folder.
Otherwise, this might cause SecurityExceptions when using the JAR. -->
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>btc.da.StreamingJob</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
</plugins> 

似乎罐子有

macbookOOO:btc-utx-da aghan$ jar tf target/btc-utx-da-0.1.jar | grep GrpcPublisherStub
com/google/cloud/pubsub/v1/stub/GrpcPublisherStub.class
macbook5434:btc-utx-da aghan$ 

Answer 1

好的，经过一些调试并使用强大的 Intellij Idea 后，我能够意识到问题在于缺少对以下方面的依赖：

<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>3.9.1</version>
</dependency>