文件:File:/C:/Python/HDFS/program1/mapper.py不可读



在执行启动Hadoop作业时出现以下错误

文件:File:/C:/Python/HDFS/program1/mapper.py不可读。

使用以下命令

hadoop jar C:hadoopsharehadooptoolslibhadoop-streaming-3.2.0.jar -file C:/Python/HDFS/program1/mapper.py -file C:/Python/HDFS/program1/reducer.py -mapper "python mapper.py" -reducer "python reducer.py" -input /sample/input_word.txt -output /sample/owc1.txt

这是我的mapper.py文件内容

import sys
for line in sys.stdin:
line = line.strip()
words = line.split()
for word in words:
print ("%st%s" % (word, 1))

并将.py内容减少到以下

import sys
import collections
counter = collections.Counter()
for line in sys.stdin:
word, count = line.strip().split("t", 1)
counter[word] += int(count)
for x in counter.most_common(9999):
print(x[0],"t",x[1])

我正在使用以下环境:

Java版本:Java版本";1.8.0_291";Hadoop版本:Hadoop 3.2.0和Windows 10

尝试用最旧的版本更改hadoop的版本,我的版本是3.2.4,当我尝试执行hadoop作业命令时也遇到了同样的错误,直到我用版本2.10.2更改它,问题才得到解决。

最新更新