数据流错误:io.grpc.StatusRuntimeException:UNAVAILABLE



我正在尝试构建一个从BigQuery读取并向Pubsub主题发送消息的Dataflow作业。我一直在努力解决相互依赖关系,这是最新的问题。

Dataflow作业启动良好,但由于以下异常而失败:

com.google.cloud.pubsub.PubSubException:io.grpc.StatusRuntimeException:UNAVAILABLE at com.google.ccloud.pubsub.spi.DefaultPubSubRpc$1.apply(DefaultPubSubRpc.java:174)at com.google.common.util.concurrent.Futures$CatchingFuture.doFallback(Futures.java:842)atcom.google.common.util.concurrent.F期货$CatchingFuture.doFallback(期货.java:834),网址为com.google.commmon.util/concurrent.FFutures$AbstractCatchingFuture.run(期货.java:789)com.google.common.util.concurrent.AbstractFuture.completecom.google.common.util.concurrent.Foretures$6.run(Futures.java:1764),网址为com.google.commmon.util/concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456),网址:com.google.coommon.util.conconcurrent.AbstractFuture.executeListener(AbstractFuture.java:817)com.google.common.util.concurrent.AbstractFuture.setExceptioncom.google.common.util.concurrent.MoreExecutors$DirectExecutor.executeio.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:382在java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)在java.util/concurrent.FFutureTask.run(FutureTask.java:266com.google.api.gax.grpc.api异常:io.grpc.StatusRuntimeException:在com.google.ai.gax.grpc.ExceptionTransformingCallable$1.onFailure(ExceptionTransformingtCallable.ja:91)在com.google.common.util.concurrent.Forets$6.run(Futures.java:1764)在com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.ja:456)在com.google.common.util.concurrent.AbstractFuture.executeListener(ClientCalls.java:358)位于io.grpc.internal。ClientCallImpl$ClientStreamListenerImpl$3。runInContext(ClientCallImpl.java:462)位于io[grpc.internal.SerializationExecutior$TaskRunner.run(ContextRunnable.java:54)位于java.util.concurrent.FutureTask.run(FutureTask.java:266)在java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)在java.util.concurrent.ThreadPoolExecutor$ScheduedFutureTask.run(ScheduledTthreadPoolExecutior.java:293)(ThreadPoolExecutor.java:617)在java.lang.Thread.run(Thread.java:745)由以下原因引起:io.grpc.StatusRuntimeException:在io.grpc.Status.asRuntimeException(Status.java:431)处不可用…11更多由以下原因造成:javax.net.ssl.ssl握手异常:在sun.security.ssl.Handshaker.activate(Handshaker.java:503)处没有合适的协议(协议被禁用或密码套件不合适)sun.security.ssl.SSLEngineImpl.kickstartHandshake(SSLEngineImpl.java:729)at sun.security.ssl.SLLEngineImpl.beginHandshake

我怀疑这与正在使用的netty/ssl库有关,但我不知道正确的操作方法是什么。我花了很多时间试图从产生的类重叠的着色jar中排除某些库的某些版本,但最终出现了未找到类的异常。

这些是我的依赖项:

<dependencies>
<dependency>
<groupId>com.google.cloud.dataflow</groupId>
<artifactId>google-cloud-dataflow-java-sdk-all</artifactId>
<version>1.8.0</version>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-pubsub</artifactId>
<version>0.3.0</version>
</dependency>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>3.0.0-beta-1</version>
</dependency>
</dependencies>

这就是依赖树:

[INFO] --- maven-dependency-plugin:2.1:tree (default-cli) @ dataflow-bulk-load ---
[INFO] com.google.lindsaysmith.titan:dataflow-bulk-load:jar:0.0.1-SNAPSHOT
[INFO] +- com.google.cloud.dataflow:google-cloud-dataflow-java-sdk-all:jar:1.8.0:compile
[INFO] |  +- com.google.apis:google-api-services-dataflow:jar:v1b3-rev36-1.22.0:compile
[INFO] |  +- io.grpc:grpc-all:jar:0.13.1:compile
[INFO] |  |  +- io.grpc:grpc-auth:jar:0.13.1:compile
[INFO] |  |  +- io.grpc:grpc-netty:jar:0.13.1:compile
[INFO] |  |  |  - io.netty:netty-codec-http2:jar:4.1.0.CR1:compile
[INFO] |  |  |     - io.netty:netty-codec-http:jar:4.1.0.CR1:compile
[INFO] |  |  +- io.grpc:grpc-protobuf:jar:0.13.1:compile
[INFO] |  |  +- io.grpc:grpc-core:jar:0.13.1:compile
[INFO] |  |  +- io.grpc:grpc-okhttp:jar:0.13.1:compile
[INFO] |  |  |  +- com.squareup.okio:okio:jar:1.6.0:compile
[INFO] |  |  |  - com.squareup.okhttp:okhttp:jar:2.5.0:compile
[INFO] |  |  +- io.grpc:grpc-protobuf-nano:jar:0.13.1:compile
[INFO] |  |  |  - com.google.protobuf.nano:protobuf-javanano:jar:3.0.0-alpha-5:compile
[INFO] |  |  - io.grpc:grpc-stub:jar:0.13.1:compile
[INFO] |  +- io.netty:netty-handler:jar:4.1.0.CR1:compile
[INFO] |  |  +- io.netty:netty-buffer:jar:4.1.0.CR1:compile
[INFO] |  |  |  - io.netty:netty-common:jar:4.1.0.CR1:compile
[INFO] |  |  +- io.netty:netty-transport:jar:4.1.0.CR1:compile
[INFO] |  |  |  - io.netty:netty-resolver:jar:4.1.0.CR1:compile
[INFO] |  |  - io.netty:netty-codec:jar:4.1.0.CR1:compile
[INFO] |  +- com.google.api.grpc:grpc-pubsub-v1:jar:0.0.2:compile
[INFO] |  |  - com.google.api.grpc:grpc-core-proto:jar:0.0.3:compile
[INFO] |  +- com.google.auth:google-auth-library-oauth2-http:jar:0.4.0:compile
[INFO] |  |  - com.google.auth:google-auth-library-credentials:jar:0.4.0:compile
[INFO] |  +- com.google.cloud.bigtable:bigtable-protos:jar:0.3.0:compile
[INFO] |  |  - com.google.guava:guava:jar:19.0:compile
[INFO] |  +- com.google.api-client:google-api-client:jar:1.22.0:compile
[INFO] |  +- com.google.apis:google-api-services-bigquery:jar:v2-rev295-1.22.0:compile
[INFO] |  +- com.google.apis:google-api-services-clouddebugger:jar:v2-rev8-1.22.0:compile
[INFO] |  +- com.google.apis:google-api-services-pubsub:jar:v1-rev10-1.22.0:compile
[INFO] |  +- com.google.apis:google-api-services-storage:jar:v1-rev71-1.22.0:compile
[INFO] |  +- com.google.http-client:google-http-client:jar:1.22.0:compile
[INFO] |  |  - org.apache.httpcomponents:httpclient:jar:4.0.1:compile
[INFO] |  |     +- org.apache.httpcomponents:httpcore:jar:4.0.1:compile
[INFO] |  |     +- commons-logging:commons-logging:jar:1.1.1:compile
[INFO] |  |     - commons-codec:commons-codec:jar:1.3:compile
[INFO] |  +- com.google.http-client:google-http-client-jackson2:jar:1.22.0:compile
[INFO] |  +- com.google.oauth-client:google-oauth-client-java6:jar:1.22.0:compile
[INFO] |  +- com.google.oauth-client:google-oauth-client:jar:1.22.0:compile
[INFO] |  +- com.google.apis:google-api-services-datastore-protobuf:jar:v1beta2-rev1-4.0.0:compile
[INFO] |  +- com.google.cloud.bigdataoss:gcsio:jar:1.4.5:compile
[INFO] |  |  +- com.google.api-client:google-api-client-java6:jar:1.22.0:compile
[INFO] |  |  - com.google.api-client:google-api-client-jackson2:jar:1.22.0:compile
[INFO] |  +- com.google.cloud.datastore:datastore-v1-proto-client:jar:1.1.0:compile
[INFO] |  |  +- com.google.http-client:google-http-client-protobuf:jar:1.22.0:compile
[INFO] |  |  - com.google.http-client:google-http-client-jackson:jar:1.22.0:compile
[INFO] |  +- com.google.cloud.datastore:datastore-v1-protos:jar:1.0.1:compile
[INFO] |  +- com.google.cloud.bigdataoss:util:jar:1.4.5:compile
[INFO] |  +- com.google.code.findbugs:jsr305:jar:3.0.1:compile
[INFO] |  +- com.fasterxml.jackson.core:jackson-core:jar:2.7.0:compile
[INFO] |  +- com.fasterxml.jackson.core:jackson-annotations:jar:2.7.0:compile
[INFO] |  +- com.fasterxml.jackson.core:jackson-databind:jar:2.7.0:compile
[INFO] |  +- org.slf4j:slf4j-api:jar:1.7.14:compile
[INFO] |  +- org.apache.avro:avro:jar:1.7.7:compile
[INFO] |  |  +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] |  |  +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] |  |  - com.thoughtworks.paranamer:paranamer:jar:2.3:compile
[INFO] |  +- org.xerial.snappy:snappy-java:jar:1.1.2.1:compile
[INFO] |  +- org.apache.commons:commons-compress:jar:1.9:compile
[INFO] |  - joda-time:joda-time:jar:2.4:compile
[INFO] +- com.google.cloud:google-cloud-pubsub:jar:0.3.0:compile
[INFO] |  +- io.netty:netty-tcnative-boringssl-static:jar:1.1.33.Fork17:compile
[INFO] |  +- com.google.cloud:google-cloud-core:jar:0.3.0:compile
[INFO] |  |  +- com.google.api-client:google-api-client-appengine:jar:1.21.0:compile
[INFO] |  |  |  +- com.google.oauth-client:google-oauth-client-appengine:jar:1.21.0:compile
[INFO] |  |  |  |  +- com.google.oauth-client:google-oauth-client-servlet:jar:1.21.0:compile
[INFO] |  |  |  |  |  - com.google.http-client:google-http-client-jdo:jar:1.21.0:compile
[INFO] |  |  |  |  - javax.servlet:servlet-api:jar:2.5:compile
[INFO] |  |  |  +- com.google.api-client:google-api-client-servlet:jar:1.21.0:compile
[INFO] |  |  |  |  - javax.jdo:jdo2-api:jar:2.3-eb:compile
[INFO] |  |  |  |     - javax.transaction:transaction-api:jar:1.1:compile
[INFO] |  |  |  - com.google.http-client:google-http-client-appengine:jar:1.21.0:compile
[INFO] |  |  +- org.json:json:jar:20151123:compile
[INFO] |  |  +- com.google.api:gax:jar:0.0.16:compile
[INFO] |  |  |  - com.google.inject:guice:jar:4.0:compile
[INFO] |  |  |     +- javax.inject:javax.inject:jar:1:compile
[INFO] |  |  |     - aopalliance:aopalliance:jar:1.0:compile
[INFO] |  |  - com.google.api.grpc:grpc-google-common-protos:jar:0.0.7:compile
[INFO] |  +- com.google.api.grpc:grpc-google-pubsub-v1:jar:0.0.7:compile
[INFO] |  - com.google.auto.value:auto-value:jar:1.1:compile
[INFO] - com.google.protobuf:protobuf-java:jar:3.0.0-beta-1:compile
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.749s
[INFO] Finished at: Sun Dec 04 16:50:41 CET 2016
[INFO] Final Memory: 16M/972M
[INFO] ------------------------------------------------------------------------

当我尝试使用PubSub库为DataFlow中的每个处理项目手动发送消息时,出现了此问题。

我没有这样做,而是改用DataFlow的PubSub-IO-Sink类来发送消息,这一操作完美无瑕。

文件:

https://cloud.google.com/dataflow/model/pubsub-io#writing-带有pubsubio

相关内容

  • 没有找到相关文章

最新更新