我正在尝试做Amazon EMR(Elastic MapReduce)的教程https://www.youtube.com/embed/ZSu8N2PoVbM?rel=0&hd=1;自动播放=1
我下载了教程示例http://elasticmapreduce.s3.amazonaws.com/training/wikipedia-lab.tgz,它统计维基百科文档集合中的bigram的数量。我一开始就陷入了困境:
2. Build the job jar
[On your dev machine]
% ant clean job
This will create the wikipedia-ngrams-job.jar Hadoop job jar
file in your build sub-directory. If you have Hadoop installed on
your development machine, you can try running it locally via:
% hadoop jar build/wikipedia-ngrams-job.jar -inputfile src/test/resources/enwiki-split.xml -outputdir build/test
This will generate text output files in build/test/raw-counts and build/test/sorted-counts.
To view the results, you can dump the output (these are text files), e.g.
% cat build/test/sorted-counts/part-r-00000
这是终端输出:
> ant clean job
Buildfile: /Users/jfk/work/data/aws_s3/wikipedia-lab/build.xml
clean:
[echo] cleaning wikipedia-ngrams
mvn-init:
[artifact:dependencies] [WARNING] POM for 'org.codehaus.janino:janino:pom:2.5.16:compile' is invalid. It will be ignored for artifact resolution. Reason: Not a v4.0.0 POM. for project org.codehaus.janino:janino at /Users/jfk/.m2/repository/org/codehaus/janino/janino/2.5.16/janino-2.5.16.pom
[…]
compile:
[mkdir] Created dir: /Users/jfk/work/data/aws_s3/wikipedia-lab/build/classes-main
[javac] Compiling 14 source files to /Users/jfk/work/data/aws_s3/wikipedia-lab/build/classes-main
[javac] error: error reading /Users/jfk/.m2/repository/org/codehaus/janino/janino/2.5.16/janino-2.5.16.jar; cannot read zip file
[javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/amazon/aws/training/emr/wikipedia/NgramsJobOptions.java:9: package org.apache.commons.lang.builder does not exist
[javac] import org.apache.commons.lang.builder.ReflectionToStringBuilder;
[javac] ^
[javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/amazon/aws/training/emr/wikipedia/NgramsJobOptions.java:10: package org.apache.commons.lang.builder does not exist
[javac] import org.apache.commons.lang.builder.ToStringStyle;
[javac] ^
[javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/ProcessXmlOptions.java:3: package org.apache.commons.lang.builder does not exist
[javac] import org.apache.commons.lang.builder.ReflectionToStringBuilder;
[javac] ^
[javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/ProcessXmlOptions.java:4: package org.apache.commons.lang.builder does not exist
[javac] import org.apache.commons.lang.builder.ToStringStyle;
[javac] ^
[javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/SplitXmlOptions.java:3: package org.apache.commons.lang.builder does not exist
[javac] import org.apache.commons.lang.builder.ReflectionToStringBuilder;
[javac] ^
[javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/SplitXmlOptions.java:4: package org.apache.commons.lang.builder does not exist
[javac] import org.apache.commons.lang.builder.ToStringStyle;
[javac] ^
[javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/SplitXmlTool.java:17: package org.apache.commons.io does not exist
[javac] import org.apache.commons.io.IOUtils;
[javac] ^
[javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/amazon/aws/training/emr/wikipedia/NgramsJobOptions.java:84: cannot find symbol
[javac] symbol : variable ToStringStyle
[javac] location: class com.amazon.aws.training.emr.wikipedia.NgramsJobOptions
[javac] return ReflectionToStringBuilder.toString(this, ToStringStyle.MULTI_LINE_STYLE);
[javac] ^
[javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/amazon/aws/training/emr/wikipedia/NgramsJobOptions.java:84: cannot find symbol
[javac] symbol : variable ReflectionToStringBuilder
[javac] location: class com.amazon.aws.training.emr.wikipedia.NgramsJobOptions
[javac] return ReflectionToStringBuilder.toString(this, ToStringStyle.MULTI_LINE_STYLE);
[javac] ^
[javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/ProcessXmlOptions.java:41: cannot find symbol
[javac] symbol : variable ToStringStyle
[javac] location: class com.scaleunlimited.wikipedia.ProcessXmlOptions
[javac] return ReflectionToStringBuilder.toString(this, ToStringStyle.MULTI_LINE_STYLE);
[javac] ^
[javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/ProcessXmlOptions.java:41: cannot find symbol
[javac] symbol : variable ReflectionToStringBuilder
[javac] location: class com.scaleunlimited.wikipedia.ProcessXmlOptions
[javac] return ReflectionToStringBuilder.toString(this, ToStringStyle.MULTI_LINE_STYLE);
[javac] ^
[javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/SplitXmlOptions.java:52: cannot find symbol
[javac] symbol : variable ToStringStyle
[javac] location: class com.scaleunlimited.wikipedia.SplitXmlOptions
[javac] return ReflectionToStringBuilder.toString(this, ToStringStyle.MULTI_LINE_STYLE);
[javac] ^
[javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/SplitXmlOptions.java:52: cannot find symbol
[javac] symbol : variable ReflectionToStringBuilder
[javac] location: class com.scaleunlimited.wikipedia.SplitXmlOptions
[javac] return ReflectionToStringBuilder.toString(this, ToStringStyle.MULTI_LINE_STYLE);
[javac] ^
[javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/SplitXmlTool.java:128: cannot find symbol
[javac] symbol : variable IOUtils
[javac] location: class com.scaleunlimited.wikipedia.SplitXmlTool
[javac] InputStream is = IOUtils.toInputStream(xml, "UTF-8");
[javac] ^
[javac] Note: /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/ProcessXmlTool.java uses or overrides a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] 15 errors
BUILD FAILED
/Users/jfk/work/data/aws_s3/wikipedia-lab/build.xml:70: Compile failed; see the compiler error output for details.
Total time: 2 seconds
欢迎任何帮助。
删除.m2/repository目录然后执行mvn clean install帮助我