我是Spark的新手，我的Spark主机是在GCP Kubernetes标准集群上。我正在使用gremlin客户端来触发spark作业，在我的本地机器配置上它工作得很好，作业完成了，但是在Kubernetes上，我看到了下面的错误。
Gremlin查询和错误
==>tinkerpop.hadoop activated
gremlin> :plugin use tinkerpop.spark
==>tinkerpop.spark activated
gremlin> graph = GraphFactory.open('/opt/janusgraph-0.6.2/conf/hadoop-graph/read-hbase-standalone-cluster-stg-bigtable-spark.properties')
==>hadoopgraph[hbaseinputformat->nulloutputformat]
gremlin>
gremlin> g = graph.traversal().withComputer(SparkGraphComputer)
==>graphtraversalsource[hadoopgraph[hbaseinputformat->nulloutputformat], sparkgraphcomputer]
gremlin> g.V().count()
14:01:48 WARN org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer - class org.apache.hadoop.mapreduce.lib.output.NullOutputFormat does not implement PersistResultGraphAware and thus, persistence options are unknown -- assuming all options are possible
14:02:03 WARN org.apache.spark.scheduler.TaskSetManager - Lost task 6.0 in stage 0.0 (TID 6, XX.XX.XX.XX, executor 0): java.io.InvalidClassException: org.apache.tinkerpop.gremlin.hadoop.structure.HadoopConfiguration; unable to create instance
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2198)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.readArray(ObjectInputStream.java:2117)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1658)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.readArray(ObjectInputStream.java:2117)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1658)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.readArray(ObjectInputStream.java:2117)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1658)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:488)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:446)
at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:488)
at jdk.internal.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at java.base/java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1046)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2340)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:488)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:446)
at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:488)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at java.base/java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1046)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2340)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:488)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:446)
at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:76)
at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:115)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:85)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:52)
at org.apache.spark.scheduler.Task.run(Task.scala:127)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:446)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1377)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:449)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.reflect.InvocationTargetException
at jdk.internal.reflect.GeneratedSerializationConstructorAccessor270.newInstance(Unknown Source)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at java.base/java.io.ObjectStreamClass.newInstance(ObjectStreamClass.java:963)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2195)
... 77 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.commons.configuration2.interpol.ConfigurationInterpolator
at org.apache.commons.configuration2.AbstractConfiguration.installDefaultInterpolator(AbstractConfiguration.java:455)
at org.apache.commons.configuration2.AbstractConfiguration.<init>(AbstractConfiguration.java:132)
... 81 more
14:02:03 ERROR org.apache.spark.scheduler.TaskSetManager - Task 8 in stage 0.0 failed 4 times; aborting job
org.apache.spark.SparkException: Job aborted due to stage failure: Task 8 in stage 0.0 failed 4 times, most recent failure: Lost task 8.3 in stage 0.0 (TID 32, 10.12.11.7, executor 1): java.io.InvalidClassException: org.apache.tinkerpop.gremlin.hadoop.structure.HadoopConfiguration; unable to create instance
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2198)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.readArray(ObjectInputStream.java:2117)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1658)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.readArray(ObjectInputStream.java:2117)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1658)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.readArray(ObjectInputStream.java:2117)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1658)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:488)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:446)
at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:488)
at jdk.internal.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at java.base/java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1046)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2340)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:488)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:446)
at scala.collection.immutable.List$SerializationProxy.readObject(List.scala:488)
at jdk.internal.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at java.base/java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1046)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2340)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2479)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2373)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2211)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1670)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:488)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:446)
at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:76)
at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:115)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:85)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:52)
at org.apache.spark.scheduler.Task.run(Task.scala:127)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:446)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1377)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:449)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.reflect.InvocationTargetException
at jdk.internal.reflect.GeneratedSerializationConstructorAccessor238.newInstance(Unknown Source)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at java.base/java.io.ObjectStreamClass.newInstance(ObjectStreamClass.java:963)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2195)
... 76 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.commons.configuration2.interpol.ConfigurationInterpolator
at org.apache.commons.configuration2.AbstractConfiguration.installDefaultInterpolator(AbstractConfiguration.java:455)
at org.apache.commons.configuration2.AbstractConfiguration.<init>(AbstractConfiguration.java:132)
... 80 more
配置文件——比;read-hbase-standalone-cluster-stg-bigtable-spark.properties
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Hadoop Graph Configuration
#
gremlin.graph=org.apache.tinkerpop.gremlin.hadoop.structure.HadoopGraph
gremlin.hadoop.graphReader=org.janusgraph.hadoop.formats.hbase.HBaseInputFormat
gremlin.hadoop.graphWriter=org.apache.hadoop.mapreduce.lib.output.NullOutputFormat
gremlin.hadoop.jarsInDistributedCache=true
gremlin.hadoop.inputLocation=none
gremlin.hadoop.outputLocation=output
gremlin.spark.persistContext=true
#
# JanusGraph HBase InputFormat configuration
#
#janusgraphmr.ioformat.conf.storage.backend=hbase
#janusgraphmr.ioformat.conf.storage.hostname=localhost
#janusgraphmr.ioformat.conf.storage.port=8586
#janusgraphmr.ioformat.conf.storage.hbase.table=janusgraph
janusgraphmr.ioformat.conf.storage.backend=hbase
janusgraphmr.ioformat.conf.storage.hbase.ext.hbase.client.connection.impl=com.google.cloud.bigtable.hbase2_x.BigtableConnection
janusgraphmr.ioformat.conf.storage.hbase.ext.google.bigtable.project.id=**********
janusgraphmr.ioformat.conf.storage.hbase.ext.google.bigtable.instance.id=*********
janusgraphmr.ioformat.conf.storage.hbase.table=**********
janusgraphmr.ioformat.conf.storage.hbase.ext.hbase.regionsizecalculator.enable=false
# This defines the indexing backend configuration used while writing data to JanusGraph.
janusgraphmr.ioformat.conf.index.search.backend=elasticsearch
janusgraphmr.ioformat.conf.index.search.hostname=**********
#
# SparkGraphComputer Configuration
#
spark.master=spark://spark-master:7077
spark.executor.memory=1g
spark.executor.extraClassPath=/opt/janusgraph-0.6.2/lib/*
spark.serializer=org.apache.spark.serializer.KryoSerializer
spark.kryo.registrator=org.janusgraph.hadoop.serialize.JanusGraphKryoRegistrator```
参考这个https://li-boxuan.medium.com/configure-janusgraph-0-6-0-for-spark-64bd7628fb77博客。spark.executor.extraClassPath=/opt/janusgraph-0.6.2/lib/*在gremlin参考配置文件中提到，这些库应该存在于主节点和工作节点中。