堆叠交换和Giraph的新手,因此请忽略错误并提出任何澄清问题。
操作系统:ubuntu 13.10
Hadoop/Yarn:Hadoop-2.20/(2节点集群)
Giraph:1.0.0(编辑:trunk)
当我尝试运行以下示例时,我得到了一个NullPointerException(NPE):
$hadoop jar$GIRAPH_HOME/GIRAPH examples/target/GIRAPH-examples-1.1.0-SNAPSHOT-for-hadoop-2.20-jar-with-dependences.jarorg.apache.giraph.GiraphRunnerorg.apache.giraph.examples.SimpleShortestPathsComputation-viforg.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertex输入格式-vip/user/hduser/rrdata/tiny_graph.txt-vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat-op/user/hduser/rrdata/output/tiny_graph.out-w 1
堆栈跟踪:
线程"main"java.lang.NullPointerException中的异常org.apache.giraph.syarn.GiraphYarnClient.checkJobLocalZooKeeperSupported(GiraphYarnClient.java:460)在org.apache.giraph.syarn.GiraphYarnClient.run(GiraphYarnClient.java:116)网址:org.apache.giraph.GiraphRunner.run(GiraphRunner.java:96)org.apache.hoop.util.ToolRunner.run(ToolRunner.java:70)org.apache.hoop.util.ToolRunner.run(ToolRunner.java:84)org.apache.giraph.GiraphRunner.main(GiraphRunner.java:126)位于的sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)在sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)在java.lang.reflect.Method.ioke(Method.java:606)org.apache.hadop.util.RunJar.main(RunJar.java:212)
这似乎与动物园管理员有关。我安装了zookeeper,但之前没有使用过它,看起来配置是错误的。我尝试过-Dgiraph.zkList=hostname:port
和相关选项,但出现了"无法识别的选项"异常。
正在为此场景查找正确的动物园管理员设置。如果我想明白了,我会给你回复的。
这是一个如何指定-D标志的示例:
hadoop jar giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2-jar-with-dependencies.jar org.apache.giraph.GiraphRunner -D giraph.zkList="zkNode.net:2081" org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/rav/giraph/input/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/rav/giraph/output/shortestpaths -w 1
顺便说一句,Giraph还不支持本地动物园管理员(GiraphYarnClient):
/**
* Check if the job's configuration is for a local run. These can all be
* removed as we expand the functionality of the "pure YARN" Giraph profile.
*/
private void checkJobLocalZooKeeperSupported() {
final boolean isZkExternal = giraphConf.isZookeeperExternal();
final String checkZkList = giraphConf.getZookeeperList();
if (!isZkExternal || checkZkList.isEmpty()) {
throw new IllegalArgumentException("Giraph on YARN does not currently" +
"support Giraph-managed ZK instances: use a standalone ZooKeeper.");
}
}
不幸的是,checkZkList为NULL,因此您永远不会看到此异常:)
NPE的原因可能是缺少检查ZK设置的giraphConf。我认为这是由于早期的问题。看起来examples jar不是使用-yj参数提供的。使用"hadoopjar"运行的jar通常是giraph核心本身。
祝你好运,如果你有更多的麻烦,请在Giraph用户列表上发帖。