使用AWK Linux操纵文件

Question

我有一个1.txt文件(带有字段分隔仪为 ||o||(：

[e-mail]||o||bb1e6b92d60454122037f302359d8a53||o||Aida||o||Aida||o||Muji?
[e-mail]||o||bcfddb5d06bd02b206ac7f9033f34677||o||Aida||o||Aida||o||Muji?
[e-mail]||o||bf6265003ae067b19b88fa4359d5c392||o||Aida||o||Aida||o||Garic Gara
[e-mail]||o||d3a6a8b1ed3640188e985f8a1efbfe22||o||Aida||o||Aida||o||Muji?
[e-mail]||o||14f87ec1e760d16c0380c74ec7678b04||o||Aida||o||Aida||o||Rodriguez Puerto

2.txt(以 :为单位分离器(：

bf6265003ae067b19b88fa4359d5c392:hyworebu:@
14f87ec1e760d16c0380c74ec7678b04:sujycugu

i有一个result.txt文件(将1.txt的第二列与2.txt的第一列匹配，如果结果匹配，它将用2.txt的第二列替换1.txt的第二列(

[e-mail]||o||hyworebu:@||o||Aida||o||Aida||o||Garic Gara
[e-mail]||o||sujycugu||o||Aida||o||Aida||o||Rodriguez Puerto

和一个left.txt文件(其中包含来自1.txt中没有匹配的1.txt的无与伦比的行(：

[e-mail]||o||d3a6a8b1ed3640188e985f8a1efbfe22||o||Aida||o||Aida||o||Muji?
[e-mail]||o||bb1e6b92d60454122037f302359d8a53||o||Aida||o||Aida||o||Muji?
[e-mail]||o||bcfddb5d06bd02b206ac7f9033f34677||o||Aida||o||Aida||o||Muji?

我正在尝试的脚本是：

awk -F '[|][|]o[|][|]' -v s1="||o||"  '
NR==FNR {
a[$2] = $1; 
b[$2]= $3s1$4s1$5; 
next
} 
($1 in a){
$1 = "";
sub(/:/, "")
print a[$1]s1$2s1b[$1] > "result.txt";
next
}' 1.txt 2.txt

问题是脚本在2.txt中使用 ||o||也是由于我得到错误的结果。

编辑

修改后的脚本：

awk -v s1="||o||"  '
NR==FNR {
a[$2] = $1; 
b[$2]= $3s1$4s1$5; 
next
} 
($1 in a){
$1 = "";
sub(/:/, "")
print a[$1]s1$2s1b[$1] > "result.txt";
next
}' FS = "||o||" 1.txt FS = ":" 2.txt

现在，我会得到以下错误：

awk：致命：无法打开文件``fs''(没有这样的文件或 目录(

Answer 1

我已经修改了您的原始脚本：

awk -F'[|][|]o[|][|]' -v s1="||o||" '
NR == FNR {
    a[$2] = $1; 
    b[$2] = $3 s1 $4 s1 $5;
    c[$2] = $0;                     # keep the line for left.txt
}
NR != FNR {
    split($0, d, ":");
    r = substr($0, index($0, ":") + 1);     # right side of the 1st ":"
    if (a[d[1]] != "") {
        print a[d[1]] s1 r s1 b[d[1]] > "result.txt";
            c[d[1]] = "";           # drop from the list of left.txt
    }
}
END {
    for (var in c) {
        if (c[var] != "") {
            print c[var] > "left.txt"
        }
    }
}' 1.txt 2.txt

下一个方面更改文件读取顺序以减少内存消耗：

awk -F'[|][|]o[|][|]' -v s1="||o||" '
NR == FNR {
    split($0, a, ":");
        r = substr($0, index($0, ":") + 1); # right side of the 1st ":"
    map[a[1]] = r; 
}
NR != FNR {
    if (map[$2] != "") {
        print $1 s1 map[$2] s1 $3 s1 $4 s1 $5 > "result.txt";
    } else {
        print $0 > "left.txt"
    }
}' 2.txt 1.txt

和最终版本利用基于文件的数据库，该数据库最小化DRAM消耗，尽管我不确定您的系统中Perl是否可以接受。

perl -e '
use DB_File;
$file1 = "1.txt";
$file2 = "2.txt";
$result = "result.txt";
$left = "left.txt";
my $dbfile = "tmp.db";
tie(%db, "DB_File", $dbfile, O_CREAT|O_RDWR, 0644) or die "$dbfile: $!";
open(FH, $file2) or die "$file2: $!";
while (<FH>) {
    chop;
    @_ = split(/:/, $_, 2);
    $db{$_[0]} = $_[1];
}
close FH;
open(FH, $file1) or die "$file1: $!";
open(RESULT, "> $result") or die "$result: $!";
open(LEFT, "> $left") or die "$left: $!";
while (<FH>) {
    @_ = split(/||o||/, $_);
    if (defined $db{$_[1]}) {
        $_[1] = $db{$_[1]};
        print RESULT join("||o||", @_);
    } else {
        print LEFT $_;
    }
}
close FH;
untie %db;
'
rm tmp.db