Bash循环一个文件，并根据在行中查找字符串在另一个文件中放置行

Question

我试图处理一个大文件4 Gig，并希望每行去一个特定的文件，但似乎不能得到它。有人能帮我找出Ubuntu 20中最适合这个的吗?我已经研究了几个小时了，但似乎还是不明白。我还差得远吗?

test.log

go to file 1
should go to file 2
should be seen in file 3
another for file 3 I belong in file 3
file 2 is my place

processLog.sh

while read -r LINE
do
grep -h "file 1" > file1.txt
grep -h "file 2" > file2.txt
grep -h "file 3" > file3.txt
done < test.log

预期结果

file1.txt
go to file 1
file2.txt
should go to file 2
file 2 is my place
file3.txt
should be seen in file 3
another for file 3 I belong in file 3

最终使用

@William Pursell谢谢你的回答，它像光剑穿过黄油一样撕裂了4Gig文件!

最后，我使用了以下语句;为了帮助别人，它在这里是完整的。在bash CLI上调用，这就是为什么后面有反斜杠。注意，第一行是一个否定(!)，所以它得到所有没有"INSERT INTO "的内容。其余的行查找特定的字符串并将其放入特定的文件中。awk快乐!

awk '
!/INSERT INTO / {print > "DATE_mysql_om.sql"; next } 
/INSERT INTO `objecttype`/ {print > "insert_om_objecttype.sql"; next } 
/INSERT INTO `objecttag`/ {print > "insert_om_objecttag.sql"; next } 
/INSERT INTO `object_objecttag`/ {print > "insert_om_object_objecttag.sql"; next } 
/INSERT INTO `indexlogger`/ {print > "insert_om_indexlogger.sql";}' 
*_mysqldump.sql

Answer 1

grep是错误的工具。如果您想要匹配字符串"file 1"要写入file1.txt等，您可以这样做:

awk '/file 1/ { print > "file1.txt"; next } /file 2/ {print > "file2.txt"; next }` input-file

Answer 2

这个awk应该在使用match函数的单个动作块中为您工作:

awk 'match($0, /file [0-9]+/) {
print > (substr($0, RSTART, 4) substr($0, RSTART+5, RLENGTH-5) ".txt")
}' file