午夜时分,一个cronjob启动logrotate来旋转4个关键日志文件。rsyslog还将这4个日志文件发送到日志聚合服务器。尽管在每个日志旋转后都运行rsyslog logrotate命令,但有时服务器会遇到一个"错误";权限被拒绝";阻止rsyslog发送属于新文件的日志的错误。我不明白为什么这种情况发生得如此不可靠。
以下是rsyslog报告的错误。请注意,它们只在某些日期出现:
Jul 31 00:00:01 hostname rsyslogd[26343]: file '/home/ubuntu/remote_logs/celery.log': open error: Permission denied [v8.32.0 try http://www.rsyslog.com/e/2433 ]
Aug 01 00:00:01 hostname rsyslogd[26343]: file '/home/ubuntu/remote_logs/celery.log': open error: Permission denied [v8.32.0 try http://www.rsyslog.com/e/2433 ]
Aug 02 00:00:04 hostname rsyslogd[26343]: file '/home/ubuntu/remote_logs/nginx.log': open error: Permission denied [v8.32.0 try http://www.rsyslog.com/e/2433 ]
Aug 09 00:00:01 hostname rsyslogd[26343]: file '/home/ubuntu/remote_logs/celery.log': open error: Permission denied [v8.32.0 try http://www.rsyslog.com/e/2433 ]
Aug 10 00:00:01 hostname rsyslogd[26343]: file '/home/ubuntu/remote_logs/django.log': open error: Permission denied [v8.32.0 try http://www.rsyslog.com/e/2433 ]
Aug 10 00:00:01 hostname rsyslogd[26343]: imfile: error with inotify API, ignoring file '/home/ubuntu/remote_logs/django.log': Permission denied [v8.32.0]
Aug 16 00:00:01 hostname rsyslogd[26343]: file '/home/ubuntu/remote_logs/celery.log': open error: Permission denied [v8.32.0 try http://www.rsyslog.com/e/2433 ]
Aug 16 00:00:02 hostname rsyslogd[26343]: file '/home/ubuntu/remote_logs/django.log': open error: Permission denied [v8.32.0 try http://www.rsyslog.com/e/2433 ]
Aug 19 00:00:03 hostname rsyslogd[856]: imfile: error with inotify API, ignoring file '/home/ubuntu/remote_logs/django.log': Permission denied [v8.32.0]
Aug 20 00:00:06 hostname rsyslogd[856]: imfile: error with inotify API, ignoring file '/home/ubuntu/remote_logs/app.log': Permission denied [v8.32.0]
Aug 30 00:00:01 hostname rsyslogd[856]: file '/home/ubuntu/remote_logs/celery.log': open error: Permission denied [v8.32.0 try http://www.rsyslog.com/e/2433 ]
Sep 03 00:00:01 hostname rsyslogd[856]: imfile: error with inotify API, ignoring file '/home/ubuntu/remote_logs/celery.log': Permission denied [v8.32.0]
Sep 03 00:00:04 hostname rsyslogd[856]: imfile: error with inotify API, ignoring file '/home/ubuntu/remote_logs/django.log': Permission denied [v8.32.0]
Logrotate由cron作业触发,使用根用户。以下是logrotate配置:
/home/ubuntu/remote_logs/*.log {
rotate 365
daily
compress
missingok
notifempty
dateext
dateformat .%Y-%m-%d
dateyesterday
postrotate
/usr/lib/rsyslog/rsyslog-rotate
endscript
}
/usr/lib/rsyslog/rsyslog-rotate
#!/bin/sh
if [ -d /run/systemd/system ]; then
systemctl kill -s HUP rsyslog.service
else
invoke-rc.d rsyslog rotate > /dev/null
fi
我知道logrotate本身是成功的,因为logrotate详细日志表明了这一点,我每天都会看到旋转的日志文件。rsyslog似乎无法访问新创建的文件。
以下是logrotate日志:
Handling 13 logs
rotating pattern: /home/ubuntu/remote_logs/*.log after 1 days (365 rotations)
empty log files are not rotated, old logs are removed
switching euid to 0 and egid to 106
considering log /home/ubuntu/remote_logs/celery.log
Now: 2020-10-14 00:00
Last rotated at 2020-10-13 00:00
log needs rotating
considering log /home/ubuntu/remote_logs/django.log
Now: 2020-10-14 00:00
Last rotated at 2020-10-13 00:00
log needs rotating
considering log /home/ubuntu/remote_logs/app.log
Now: 2020-10-14 00:00
Last rotated at 2020-10-13 00:00
log needs rotating
considering log /home/ubuntu/remote_logs/nginx.log
Now: 2020-10-14 00:00
Last rotated at 2020-10-13 00:00
log needs rotating
rotating log /home/ubuntu/remote_logs/celery.log, log->rotateCount is 365
Converted ' .%Y-%m-%d' -> '.%Y-%m-%d'
dateext suffix '.2020-10-13'
glob pattern '.[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
renaming /home/ubuntu/remote_logs/celery.log to /home/ubuntu/remote_logs/celery.log.2020-10-13
creating new /home/ubuntu/remote_logs/celery.log mode = 0644 uid = 102 gid = 106
running postrotate script
switching euid to 0 and egid to 0
compressing log with: /bin/gzip
switching uid to 0 and gid to 106
rotating log /home/ubuntu/remote_logs/django.log, log->rotateCount is 365
Converted ' .%Y-%m-%d' -> '.%Y-%m-%d'
dateext suffix '.2020-10-13'
glob pattern '.[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
renaming /home/ubuntu/remote_logs/django.log to /home/ubuntu/remote_logs/django.log.2020-10-13
creating new /home/ubuntu/remote_logs/django.log mode = 0644 uid = 102 gid = 106
running postrotate script
switching euid to 0 and egid to 0
compressing log with: /bin/gzip
switching uid to 0 and gid to 106
rotating log /home/ubuntu/remote_logs/app.log, log->rotateCount is 365
Converted ' .%Y-%m-%d' -> '.%Y-%m-%d'
dateext suffix '.2020-10-13'
glob pattern '.[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
renaming /home/ubuntu/remote_logs/app.log to /home/ubuntu/remote_logs/app.log.2020-10-13
creating new /home/ubuntu/remote_logs/app.log mode = 0644 uid = 102 gid = 106
running postrotate script
switching euid to 0 and egid to 0
compressing log with: /bin/gzip
switching uid to 0 and gid to 106
rotating log /home/ubuntu/remote_logs/nginx.log, log->rotateCount is 365
Converted ' .%Y-%m-%d' -> '.%Y-%m-%d'
dateext suffix '.2020-10-13'
glob pattern '.[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
renaming /home/ubuntu/remote_logs/nginx.log to /home/ubuntu/remote_logs/nginx.log.2020-10-13
creating new /home/ubuntu/remote_logs/nginx.log mode = 0644 uid = 102 gid = 106
running postrotate script
switching euid to 0 and egid to 0
compressing log with: /bin/gzip
switching uid to 0 and gid to 106
switching euid to 0 and egid to 0
rsyslog发生了什么,导致它无法偶尔访问日志文件?
看起来有些文件没有创建权限。您需要在配置文件中添加create 0644 root root
,以便获得旋转(修改(文件内容的必要权限
您还可以指定组名称create 0644 root grouname
。但我建议使用根
/home/ubuntu/remote_logs/*.log {
rotate 365
daily
compress
missingok
create 0644 root root
notifempty
dateext
dateformat .%Y-%m-%d
dateyesterday
postrotate
/usr/lib/rsyslog/rsyslog-rotate
endscript
}