Pig：解析失败：输入不匹配'id'期望设置空值

Question

我使用的是Pig 0.12.1，并有以下Pig代码:

C = LOAD '$file' USING myCustomLoader();
D = FOREACH C GENERATE key#id;

我正在使用自定义加载器加载文件。然后，我想生成存储在key(一个映射)中的所有id。

为什么我得到以下错误信息:

14/06/27 16:56:21 ERROR pig.PigServer: exception during parsing: Error during parsing.     <line 3, column 28>  mismatched input 'id' expecting set null
Failed to parse: <line 3, column 28>  mismatched input 'id' expecting set null

下面是完整的堆栈跟踪:

14/06/27 16:56:21 ERROR pig.PigServer: exception during parsing: Error during parsing.     <line 3, column 28>  mismatched input 'id' expecting set null
Failed to parse: <line 3, column 28>  mismatched input 'id' expecting set null
        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:241)
        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:179)
        at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1676)
        at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1623)
        at org.apache.pig.PigServer.registerQuery(PigServer.java:575)
        at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1093)
        at org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
        at org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
        at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:170)
        at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:249)
        at com.testpig.PigTest.testPig(PigTest.java:159)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
        at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
        at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
        at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
        at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
        at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
        at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:172)
        at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:104)
        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:70)
14/06/27 16:56:21 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/06/27 16:56:21 INFO compress.CodecPool: Got brand-new compressor

Answer 1

问题中的错误代码是:

C = LOAD '$file' USING myCustomLoader();
D = FOREACH C GENERATE key#id;

正确的代码是:

C = LOAD '$file' USING myCustomLoader();
D = FOREACH C GENERATE key#'id';

在Pig中，要访问map的键，必须在键周围使用单引号。

有关更多信息，请参阅Philip (flip) Kromer的帖子https://www.mail-archive.com/[e-mail]/msg24691.html:

Omitting the quotes on the key dereference gives a very unhelpful error message.
{code}
users = FOREACH user_hashes GENERATE info#userid AS userid:chararray;
-- 400   ERROR: ERROR 1200: <file ./foo.pig, line 8, column 42>  [...] 
mismatched input 'userid' expecting set null
{code}
It may be that the user forgot the quotes, or may instead be assuming that Pig 
allows dereferencing a map by the value of an alias or expression:
{code}
users = FOREACH user_hashes GENERATE
  info#'username',               -- works
  info#username,                 -- need quotes around literal
  info#fullref,                  -- no, can't use an alias' value to deref
  info#(CONCAT('user',shortref)) -- and can't use an expression to deref
  ;
{code}