> start.sh
copy_file.sh
开始并传递 2 个参数
yeasterday_with_dash=2017-01-31
today_without_dash=20170201
echo "-----------RUN copy mta-------------"
bash copy_file.sh mta $today_without_dash
echo "-----------RUN copy rcr-------------"
bash copy_file.sh rcr $today_without_dash
echo "-----------RUN copy sub-------------"
bash copy_file.sh sub $today_without_dash
copy_file.sh
被支持检查目录是否存在,如果不存在,则创建它。
hdfs_dir=/apps/hive/warehouse/mydb.db/fct_evkuzmin/${2}/file_${1}
if hadoop fs -test -d $hdfs_dir ; then
echo "Directory "$hdfs_dir" exists"
else
hadoop fs -mkdir $hdfs_dir
echo "Creating "$hdfs_dir
fi
脚本似乎运行成功。
日志。
yeasterday_with_dash=2017-01-31
today_without_dash=20170201
-----------RUN copy mta-------------
mkdir: `/apps/hive/warehouse/mydb.db/fct_evkuzmin/20170201/file_mta': No such file or directory
Creating /apps/hive/warehouse/mydb.db/fct_evkuzmin/20170201/file_mta
end copy mta files
-----------RUN copy rcr-------------
mkdir: `/apps/hive/warehouse/mydb.db/fct_evkuzmin/20170201/file_rcr': No such file or directory
Creating /apps/hive/warehouse/mydb.db/fct_evkuzmin/20170201/file_rcr
end copy rcr files
-----------RUN copy sub-------------
mkdir: `/apps/hive/warehouse/mydb.db/fct_evkuzmin/20170201/file_sub': No such file or directory
Creating /apps/hive/warehouse/mydb.db/fct_evkuzmin/20170201/file_sub
end copy sub files
但是没有创建实际的目录。怎么了?
将mkdir
与-p
参数一起使用也可以创建父目录。
hadoop fs -mkdir -p etc