这是我的输入字符串：
myserver:/etc/fail2ban # cat one_line.txt
12.123.123.12 - - [25/Jul/2017:09:26:40 +0200] "GET /a/path/file.html HTTP/1.1" 200 1450 "http://example.com/a/path/file.html" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"
我试图演变一个以^<HOST>.*$开头并以^<HOST> - - [2.*$结束的正则表达式，但是当它结束时发生了一些奇怪的事情。
上次模式迭代不再匹配。 当我这样陈述时，我可以强制比赛^<HOST> - - [D.*$.
咦？。这意味着左方括号后面有一个非数字字符！
我打印我采取的步骤，以便向您展示它是如何演变的。
希望有人能解释我这种奇怪的行为。
环境信息：
myserver:/etc/fail2ban # fail2ban-regex --version
fail2ban-regex 0.9.4
myserver:/etc/fail2ban # python --version
Python 2.7.17
现在遵循我的过滤器表达式的迭代演变：
迭代 1：匹配 - 如预期
myserver:/etc/fail2ban # fail2ban-regex -l heavydebug one_line.txt '^<HOST>.*$'
Running tests
=============
D: Setting usedns = warn for Filter(None)
D: Created Filter(None)
Use failregex line : ^<HOST>.*$
Use log file : one_line.txt
Use encoding : UTF-8
D: Working on line u'12.123.123.12 - - [25/Jul/2017:09:26:40 +0200] "GET /a/path/file.html HTTP/1.1" 200 1450 "http://example.com/a/path/file.html" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"'
D: Matched time template Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)?
D: Got time 1500967600.000000 for "u'25/Jul/2017:09:26:40 +0200'" using template Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)?
D: Looking for failregex match of [(u'12.123.123.12 - - [', u'25/Jul/2017:09:26:40 +0200', u'] "GET /a/path/file.html HTTP/1.1" 200 1450 "http://example.com/a/path/file.html" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"')]
D: Matched FailRegex('^(?:::f{4,6}:)?(?P<host>[\w\-.^_]*\w).*$')
D: Sorting the template list
D: Winning template: Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)? with 1 hits
Results
=======
Failregex: 1 total
|- #) [# of hits] regular expression
| 1) [1] ^<HOST>.*$
`-
Ignoreregex: 0 total
Date template hits:
|- [# of hits] date format
| [1] Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)?
`-
Lines: 1 lines, 0 ignored, 1 matched, 0 missed
[processed in 0.00 sec]
迭代 2：匹配 - 如预期的那样
myserver:/etc/fail2ban # fail2ban-regex -l heavydebug one_line.txt '^<HOST> - - .*$'
Running tests
=============
D: Setting usedns = warn for Filter(None)
D: Created Filter(None)
Use failregex line : ^<HOST> - - .*$
Use log file : one_line.txt
Use encoding : UTF-8
D: Working on line u'12.123.123.12 - - [25/Jul/2017:09:26:40 +0200] "GET /a/path/file.html HTTP/1.1" 200 1450 "http://example.com/a/path/file.html" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"'
D: Matched time template Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)?
D: Got time 1500967600.000000 for "u'25/Jul/2017:09:26:40 +0200'" using template Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)?
D: Looking for failregex match of [(u'12.123.123.12 - - [', u'25/Jul/2017:09:26:40 +0200', u'] "GET /a/path/file.html HTTP/1.1" 200 1450 "http://example.com/a/path/file.html" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"')]
D: Matched FailRegex('^(?:::f{4,6}:)?(?P<host>[\w\-.^_]*\w) - - .*$')
D: Sorting the template list
D: Winning template: Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)? with 1 hits
Results
=======
Failregex: 1 total
|- #) [# of hits] regular expression
| 1) [1] ^<HOST> - - .*$
`-
Ignoreregex: 0 total
Date template hits:
|- [# of hits] date format
| [1] Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)?
`-
Lines: 1 lines, 0 ignored, 1 matched, 0 missed
[processed in 0.00 sec]
迭代 3：匹配 - 如预期
myserver:/etc/fail2ban # fail2ban-regex -l heavydebug one_line.txt '^<HOST> - - [.*$'
Running tests
=============
D: Setting usedns = warn for Filter(None)
D: Created Filter(None)
Use failregex line : ^<HOST> - - [.*$
Use log file : one_line.txt
Use encoding : UTF-8
D: Working on line u'12.123.123.12 - - [25/Jul/2017:09:26:40 +0200] "GET /a/path/file.html HTTP/1.1" 200 1450 "http://example.com/a/path/file.html" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"'
D: Matched time template Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)?
D: Got time 1500967600.000000 for "u'25/Jul/2017:09:26:40 +0200'" using template Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)?
D: Looking for failregex match of [(u'12.123.123.12 - - [', u'25/Jul/2017:09:26:40 +0200', u'] "GET /a/path/file.html HTTP/1.1" 200 1450 "http://example.com/a/path/file.html" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"')]
D: Matched FailRegex('^(?:::f{4,6}:)?(?P<host>[\w\-.^_]*\w) - - \[.*$')
D: Sorting the template list
D: Winning template: Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)? with 1 hits
Results
=======
Failregex: 1 total
|- #) [# of hits] regular expression
| 1) [1] ^<HOST> - - [.*$
`-
Ignoreregex: 0 total
Date template hits:
|- [# of hits] date format
| [1] Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)?
`-
Lines: 1 lines, 0 ignored, 1 matched, 0 missed
[processed in 0.00 sec]
迭代 4.1：失败 - 为什么???
myserver:/etc/fail2ban # fail2ban-regex -l heavydebug one_line.txt '^<HOST> - - [2.*$'
Running tests
=============
D: Setting usedns = warn for Filter(None)
D: Created Filter(None)
Use failregex line : ^<HOST> - - [2.*$
Use log file : one_line.txt
Use encoding : UTF-8
D: Working on line u'12.123.123.12 - - [25/Jul/2017:09:26:40 +0200] "GET /a/path/file.html HTTP/1.1" 200 1450 "http://example.com/a/path/file.html" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"'
D: Matched time template Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)?
D: Got time 1500967600.000000 for "u'25/Jul/2017:09:26:40 +0200'" using template Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)?
D: Looking for failregex match of [(u'12.123.123.12 - - [', u'25/Jul/2017:09:26:40 +0200', u'] "GET /a/path/file.html HTTP/1.1" 200 1450 "http://example.com/a/path/file.html" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"')]
D: Sorting the template list
D: Winning template: Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)? with 1 hits
Results
=======
Failregex: 0 total
Ignoreregex: 0 total
Date template hits:
|- [# of hits] date format
| [1] Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)?
`-
Lines: 1 lines, 0 ignored, 0 matched, 1 missed
[processed in 0.00 sec]
|- Missed line(s):
| 12.123.123.12 - - [25/Jul/2017:09:26:40 +0200] "GET /a/path/file.html HTTP/1.1" 200 1450 "http://example.com/a/path/file.html" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"
`-
迭代 4.2：失败 - 为什么???
myserver:/etc/fail2ban # fail2ban-regex -l heavydebug one_line.txt '^<HOST> - - [d.*$'
Running tests
=============
D: Setting usedns = warn for Filter(None)
D: Created Filter(None)
Use failregex line : ^<HOST> - - [d.*$
Use log file : one_line.txt
Use encoding : UTF-8
D: Working on line u'12.123.123.12 - - [25/Jul/2017:09:26:40 +0200] "GET /a/path/file.html HTTP/1.1" 200 1450 "http://example.com/a/path/file.html" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"'
D: Matched time template Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)?
D: Got time 1500967600.000000 for "u'25/Jul/2017:09:26:40 +0200'" using template Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)?
D: Looking for failregex match of [(u'12.123.123.12 - - [', u'25/Jul/2017:09:26:40 +0200', u'] "GET /a/path/file.html HTTP/1.1" 200 1450 "http://example.com/a/path/file.html" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"')]
D: Sorting the template list
D: Winning template: Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)? with 1 hits
Results
=======
Failregex: 0 total
Ignoreregex: 0 total
Date template hits:
|- [# of hits] date format
| [1] Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)?
`-
Lines: 1 lines, 0 ignored, 0 matched, 1 missed
[processed in 0.00 sec]
|- Missed line(s):
| 12.123.123.12 - - [25/Jul/2017:09:26:40 +0200] "GET /a/path/file.html HTTP/1.1" 200 1450 "http://example.com/a/path/file.html" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"
`-
迭代 4.3： 匹配 - 为什么???
myserver:/etc/fail2ban # fail2ban-regex -l heavydebug one_line.txt '^<HOST> - - [D.*$'
Running tests
=============
D: Setting usedns = warn for Filter(None)
D: Created Filter(None)
Use failregex line : ^<HOST> - - [D.*$
Use log file : one_line.txt
Use encoding : UTF-8
D: Working on line u'12.123.123.12 - - [25/Jul/2017:09:26:40 +0200] "GET /a/path/file.html HTTP/1.1" 200 1450 "http://example.com/a/path/file.html" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"'
D: Matched time template Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)?
D: Got time 1500967600.000000 for "u'25/Jul/2017:09:26:40 +0200'" using template Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)?
D: Looking for failregex match of [(u'12.123.123.12 - - [', u'25/Jul/2017:09:26:40 +0200', u'] "GET /a/path/file.html HTTP/1.1" 200 1450 "http://example.com/a/path/file.html" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"')]
D: Matched FailRegex('^(?:::f{4,6}:)?(?P<host>[\w\-.^_]*\w) - - \[\D.*$')
D: Sorting the template list
D: Winning template: Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)? with 1 hits
Results
=======
Failregex: 1 total
|- #) [# of hits] regular expression
| 1) [1] ^<HOST> - - [D.*$
`-
Ignoreregex: 0 total
Date template hits:
|- [# of hits] date format
| [1] Day(?P<_sep>[-/])MON(?P=_sep)Year[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)?
`-
Lines: 1 lines, 0 ignored, 1 matched, 0 missed
[processed in 0.00 sec]
提前感谢。
根据 https://fail2ban.readthedocs.io/en/latest/filters.html#regular-expressions
[...]
Regular expressions (failregex, ignoreregex) assume that the date/time has been removed from the log line (this is just how fail2ban works internally ATM).
If the format is like ‘<date...> error 1.2.3.4 is evil’ then you need to match the < at the start so regex should be similar to ‘^<> <HOST> is evil$’ using <HOST> where the IP/domain name appears in the log line.
[...]
就是这样！