Bash-基于时间戳的搜索(以毫秒为单位)



我在一个时间间隔内搜索所有Hadoop完成的(100个)作业。此时间间隔以毫秒为单位。

以下是格式:

JobId      State           StartTime      UserName           Queue      Priority       UsedContainers  RsvdContainers      UsedMem         RsvdMem        NeededMem                 AM info
job_xxxxxxx  SUCCEEDED       1458844667431    default    default          NORMAL                  N/A             N/A          N/A           N/A        N/A             http://xxxxxxxx:8088/proxy/application_xxxxxxxxxx/jobhistory/job/job_xxxxxxxx
job_xxxxxxx  SUCCEEDED       1459449718363    default    default         NORMAL                  N/A             N/A        N/A             N/A              N/A        http://xxxxxx.xxxxx.com:8088/proxy/application_xxxxxxxxx/jobhistory/job/job_xxxxx

以下是我的格式:

STARTTIME="Tue Apr 12 10:24:29 EDT 2016"
ENDTIME="Tue Apr 12 15:24:29 EDT 2016"
CONVERTTIME_1=`date --date="$STARTTIME" +%s%3N`
CONVERTTIME_2=`date --date="$ENDTIME" +%s%3N`
echo $CONVERTTIME_1, $CONVERTTIME_2
mapred job -list all | sed -n '/$CONVERTTIME_1/,/$CONVERTTIME_2/p' > out

输出:timerange中的所有类似上述的作业。

有人能帮忙拿这些吗?

映射作业的输出-列出所有

映射作业-列出所有

 job_1457613852865_5163  SUCCEEDED       1459199337140    zzzzzzzzzz    uuuuuuuu_critical          NORMAL                  N/A             N/A      N/A           N/A        N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_5163/jobhistory/job/job_1457613852865_5163
 job_1457613852865_4633  SUCCEEDED       1458992402216       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_4633/jobhistory/job/job_1457613852865_4633
 job_1457613852865_4821  SUCCEEDED       1459078845580       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_4821/jobhistory/job/job_1457613852865_4821
 job_1457613852865_0322  SUCCEEDED       1457717313217        zzzddd    uuuuuuuu_critical          NORMAL                  N/A             N/A      N/A           N/A        N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_0322/jobhistory/job/job_1457613852865_0322
 job_1457613852865_5304  SUCCEEDED       1459254375921       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_5304/jobhistory/job/job_1457613852865_5304
 job_1457613852865_8744  SUCCEEDED       1460195126188       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_8744/jobhistory/job/job_1457613852865_8744
 job_1457613852865_3384  SUCCEEDED       1458649020794       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_3384/jobhistory/job/job_1457613852865_3384
 job_1457613852865_9038  SUCCEEDED       1460291694279       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_9038/jobhistory/job/job_1457613852865_9038
 job_1457613852865_8487  SUCCEEDED       1460115319590       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_8487/jobhistory/job/job_1457613852865_8487
 job_1457613852865_8321  SUCCEEDED       1460038991587        dddyyy    uuuuuuuu_critical          NORMAL                  N/A             N/A      N/A           N/A        N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_8321/jobhistory/job/job_1457613852865_8321
 job_1457613852865_4661  SUCCEEDED       1458994901619       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_4661/jobhistory/job/job_1457613852865_4661
 job_1457613852865_1975  SUCCEEDED       1458216683800       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_1975/jobhistory/job/job_1457613852865_1975

我用过这个:

#!/bin/bash

STARTTIME="Tue Apr 12 10:13:01 EDT 2016"
ENDTIME="Tue Apr 12 10:13:59 EDT 2016"
start=$(date -d "$STARTTIME" '+%s%3N')
end=$(date -d "$ENDTIME" '+%s%3N')
echo "start=$start :: end=$end"
mapred job -list all | awk -v start="$start" -v end="$end" '$3>=start && $3<=end'

得到一份额外的工作:

job_1457613852865_9785  SUCCEEDED       1460470436726       yyyyyyyyyy    nnnnnnnnnn       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://888888.xxxxxxxxxx.com:8088/proxy/application_1457613852865_9785/jobhistory/job/job_1457613852865_9785

您可以将日期转换为毫秒值,然后使用awk过滤数据:

STARTTIME="Tue Apr 12 10:24:29 EDT 2016"
ENDTIME="Tue Apr 12 15:24:29 EDT 2016"
start=$(date -d "$STARTTIME" '+%s%3N')
end=$(date -d "$ENDTIME" '+%s%3N')"
echo "start=$start :: end=$end"
mapred job -list all | awk -v start="$start" -v end="$end" '$3>=start && $3<=end'

最新更新