mapper无法读取gz.parquet文件



org.apache.hadoop.mapred.maptask:映射输出的开始冲洗

2017-04-20 20:53:20,101 warn [main] org.apache.hadoop.mapred.yarnchild:例外运行子女:java.lang.nullpointerexception atrg.apache.parquet.avro.avroschemaconverter.convertfield(avroschemaconverter.java:294( at org.apache.parquet.avro.avroschemaconverter.convertfields(avroschemaconverter.java:204( atorg.apache.parquet.avro.avroschemaconverter.convert(avroschemaconverter.java:198( atrg.apache.parquet.avro.avroreadsupport.prepareforread(avroreadsupport.java:105( at org.apache.parquet.hadoop.internalparquetrecordreader.initialize(internalParquetreCordReader.java:174( at org.apache.parquet.hadoop.parquetrecordreader.initializeInsizeInternalReader(parquetrecordreader.java:192( atorg.apache.parquet.hadoop.parquetrecordreader.initialize(parquetrecordreader.java:140( at org.apache.hadoop.mapreduce.lib.input.delegatingRecordReader.initialize(delegatingRecordReader.java:84( atorg.apache.hadoop.mapred.maptask $ newTrackingRecordReader.initialize(mappask.java:548( atrg.apache.hadoop.mapred.maptask.runnewmapper(maptask.java:786( atrg.apache.hadoop.mapred.maptask.run(maptask.java:341( at org.apache.hadoop.mapred.yarnchild $ 2. run(yarnchild.java:163( 在java.security.accesscontroller.doprivileged(本机方法( 在javax.security.auth.subject.doas(object.java:415( atrg.apache.hadoop.security.usergroupinformation.doas(userGroupInformation.java:1657( atorg.apache.hadoop.mapred.yarnchild.main(yarnchild.java:158(

我正在尝试阅读映射器中的filename.gz.paquet,并具有同一MapReduce作业,我可以读取filename.snappy.parquet文件。

我可以解决问题。好吧,我已更新到Avroversion 1.8.1的新罐子,Parquetversion 1.9.0,ParquetFormatversion 2.3.1和Hiveversion 1.2.2。

最新更新