hadoop, java.lang.RuntimeException: java.lang.ClassNotFoundE

Question

我遵循这本书(权威指南hadoop)。

，而试图执行一个例子在书与localhost，我已经面临的错误，

14/06/13 22:24:57 WARN mapred.JobClient: No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
****hdfs://localhost/usr/kim/input/ncdc
14/06/13 22:24:57 INFO input.FileInputFormat: Total input paths to process : 3
14/06/13 22:24:57 WARN snappy.LoadSnappy: Snappy native library is available
14/06/13 22:24:57 INFO util.NativeCodeLoader: Loaded the native-hadoop library
14/06/13 22:24:57 INFO snappy.LoadSnappy: Snappy native library loaded
14/06/13 22:24:57 INFO mapred.JobClient: Running job: job_201406132100_0011
14/06/13 22:24:58 INFO mapred.JobClient:  map 0% reduce 0%
14/06/13 22:25:11 INFO mapred.JobClient: Task Id : attempt_201406132100_0011_m_000000_0, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: mapred.MaxTemperatureMapper_v1
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:867)
    at org.apache.hadoop.mapreduce.JobContext.getMapperClass(JobContext.java:199)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:719)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.ClassNotFoundException: mapred.MaxTemperatureMapper_v1
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:270)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:865)
    ... 8 more

i使用命令行执行作业，hadoop jar MaxTemperatureDriver.jar mapred.MaxTemperatureDriver -conf /hadoop_conf/Hadoop-localhost.xml /input/ncdc /max-temp

jar文件中有2个文件夹META-INF, mapred, mapred文件夹中有5个类(这些类在mapred包中)

1. MaxTemeratureReducer.class
2. MaxTemperatureDriver.class
3. MaxTemperatureMapper_v1.class
4. MaxTemperatureMapper_v1美元Temperature.class
5. NcdcRecordParser.class

这些是配置文件，MaxTemperatureDriver和MaxTemperatureMapper_v1

<?xml version="1.0"?>
<configuration>
    <property>
        <name>fs.default.name</name>
        <value>hdfs://localhost/</value>
    </property>
    <property>
        <name>mapred.job.tracker</name>
        <value>localhost:8021</value>
    </property>
    <property>
        <name>dfs.replication</name>    
        <value>1</value>
    </property>
</configuration>

.

public class MaxTemperatureDriver extends Configured implements Tool{
    @Override
    public int run(String[] args) throws Exception{
        if(args.length != 2){
            System.err.printf("Usage: %s [generic options] <input> <output>n", getClass().getSimpleName());
            ToolRunner.printGenericCommandUsage(System.err);
            return -1;
        }
        Job job = new Job(getConf(), "Max Temperature");
        job.setJarByClass(getClass());
        FileInputFormat.addInputPath(job, new Path(args[0]));
        FileOutputFormat.setOutputPath(job,  new Path(args[1]));
        job.setReducerClass(MaxTemeratureReducer.class);
        job.setMapperClass(MaxTemperatureMapper_v1.class);
        job.setCombinerClass(MaxTemeratureReducer.class);
        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(IntWritable.class);
        return job.waitForCompletion(true) ? 0 : 1;
    }
}

.

public class MaxTemperatureMapper_v1 extends Mapper<LongWritable, Text, Text, IntWritable>{
    enum Temperature{
        OVER_100
    }
    private NcdcRecordParser parser = new NcdcRecordParser();
    @Override
    public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException{
        parser.parse(value);    //Text.toString()
        if(parser.isValidTemperature()){
            int airTemperature = parser.getAirTemperature();
            if(airTemperature > 1000){
                System.err.println("Temperature over 100 degrees for input: " + value);
                context.setStatus("Detected possibly corrupt record: see logs.");
                context.getCounter(Temperature.OVER_100).increment(1);
            }
            context.write(new Text(parser.getYear()), new IntWritable(parser.getAirTemperature()));
        }
    }
}

Answer 1

按类名设置jar有时会给出错误，并设置输入格式和输出格式，希望这对您有所帮助。

job.setJarByClass(MaxTemperatureDriver.class);
job.setInputFormatClass(TextInputFormat.class);
job.setOutputFormatClass(TextOutputFormat.class);

Answer 2

job。setJarByClass("主类");

主类意味着有 Main ()方法