我在Azure HDInsight (Hadoop)上运行一个简单的Map/Reduce作业。映射器失败，出现以下错误(来自JobTracker日志)…
2014-03-27 16:42:50,330 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201403262112_0004_m_000000_0: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 255
at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:362)
at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:576)
at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:135)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:365)
at org.apache.hadoop.mapred.Child$4.run(Child.java:266)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1233)
at org.apache.hadoop.mapred.Child.main(Child.java:260)
我正在使用Hadoop连接到集群。连接方法…
// Connect to the HDInsight cluster
var hadoop = Hadoop.Connect(hadoopClusterUri,
hadoopUserName,
localHadoopUser,
hadoopPassword,
asvAccount,
asvKey,
asvContainer,
createContinerIfNotExist);
由于作业已经启动并记录，我假设连接成功。作为一个简单的测试，我发出少量的数据…
context.EmitKeyValue("This is a key", "This is a value");
完全难住了,比尔
这个MapReduce生成两个作业。第一个操作成功完成，第二个操作失败，出现以下异常…
Unhandled Exception: Microsoft.Hadoop.MapReduce.StreamingException: The user type could not be loaded. DLL=MapReduce.exe, Type=PrescienceRx.MapReduce.BeneficiaryMapper ---> System.BadImageFormatException: Could not load file or assembly 'file:///c:appstemphdfsmapredlocaltaskTrackeradminjobcachejob_201403262112_0006attempt_201403262112_0006_m_000001_0workMapReduce.exe' or one of its dependencies. An attempt was made to load a program with an incorrect format.
at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.RuntimeAssembly.InternalLoadFrom(String assemblyFile, Evidence securityEvidence, Byte[] hashValue, AssemblyHashAlgorithm hashAlgorithm, Boolean forIntrospection, Boolean suppressSecurityChecks, StackCrawlMark& stackMark)
at System.Reflection.Assembly.LoadFrom(String assemblyFile)
at Microsoft.Hadoop.MapReduce.MapperMain.Main()
--- End of inner exception stack trace ---
在Microsoft.Hadoop.MapReduce.MapperMain.Main ()在Microsoft.HdInsight.MapDriver.Program。Main (String [] args
看起来System.BadImageFormatException: Could not load file or assembly可能是因为加载器的。net版本与您的程序集不匹配。尝试将其设置回4.0，然后在NuGet软件包管理器窗口中执行update-package -reinstall Microsoft.Hadoop.MapReduce以确保您拥有所有正确的版本。
我在解决方案的配置管理器中将目标CPU设置为x64，这就解决了问题。