我正在阅读Tom White的Hadoop-The definitive guide
第3版。我已经成功地将一个sequenceFile
写入了HDFS
。我照着作者在书中所举的例子去做。但当我试图运行sort
(第138页),我得到classCastException
。下面是堆栈跟踪。
这里有什么问题,需要什么修复?
hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar sort -r 1 -inFormat org.apache.hadoop.mapred.SequenceFileInputFormat -outFormat org.apache.hadoop.mapred.SequenceFileOutputFormat -outKey org.apache.hadoop.io.IntWritable -outValue org.apache.hadoop.io.Text /output/seqfile /output/sortedfile
14/07/09 10:51:53 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/07/09 10:51:53 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
14/07/09 10:51:53 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
java.lang.ClassCastException: class org.apache.hadoop.mapred.SequenceFileInputFormat
at java.lang.Class.asSubclass(Class.java:3075)
at org.apache.hadoop.examples.Sort.run(Sort.java:104)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.examples.Sort.main(Sort.java:191)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
这可能是因为您正在使用旧的map/reduce序列文件类。而不是使用
-inFormat org.apache.hadoop.mapred.SequenceFileInputFormat
-outFormat org.apache.hadoop.mapred.SequenceFileOutputFormat
尝试使用
-inFormat org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat;
-outFormat org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat;