我试图在Kubernetes中配置Fluentbit，以从应用程序POD/Docker Containers中获取日志，并使用GELF格式将此日志消息发送到Graylog，但这不起作用。
请参阅下面的堆栈：
输入
- Docker 1.13.1版本
- Docker日志格式=>JSON
- Docker日志驱动程序=>日志=>系统
- Fluent bit 1.3在Kubernetes中作为Daemoset运行
- Kubernetes 1.17
- 操作系统主机：CentOS 7
输出
- 消息输出格式：GELF 1.1
- 集中式日志=>灰色日志3
问题是fluentbit没有从systemd读取日志我在两个输出(systemd，Stdout(中都没有得到任何日志，Stdout只是为了帮助进行故障排除。
我不知道为什么我不能从systemd中阅读。我完全遵循了文档https://docs.fluentbit.io/manual/input/systemd
我的K8S配置：
流体-比特-figmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: fluent-bit-config
namespace: log
labels:
k8s-app: fluent-bit
data:
# Configuration files: server, input, filters and output
# ======================================================
fluent-bit.conf: |
[SERVICE]
Flush 1
Log_Level debug
Daemon off
@INCLUDE input-systemd.conf
@INCLUDE output-stdout.conf
input-systemd.conf: |
[INPUT]
Name systemd
Tag host.*
Parser json
Systemd_Filter _SYSTEMD_UNIT=docker.service
output-graylog.conf: |
[OUTPUT]
Name gelf
Match *
Host 10.142.15.214
Port 12201
Mode tcp
Gelf_Short_Message_Key log
output-stdout.conf: |
[OUTPUT]
Name stdout
Match *
fluent-bit-ds.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: fluent-bit
namespace: log
labels:
k8s-app: fluent-bit-logging
version: v1
kubernetes.io/cluster-service: "true"
spec:
selector:
matchLabels:
k8s-app: fluent-bit-logging
version: v1
kubernetes.io/cluster-service: "true"
template:
metadata:
labels:
k8s-app: fluent-bit-logging
version: v1
kubernetes.io/cluster-service: "true"
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "2020"
prometheus.io/path: /api/v1/metrics/prometheus
spec:
containers:
- name: fluent-bit
image: fluent/fluent-bit:1.3.5
imagePullPolicy: Always
ports:
- containerPort: 2020
volumeMounts:
- name: varlog
mountPath: /var/log
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
- name: fluent-bit-config
mountPath: /fluent-bit/etc/
terminationGracePeriodSeconds: 10
volumes:
- name: varlog
hostPath:
path: /var/log
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
- name: fluent-bit-config
configMap:
name: fluent-bit-config
serviceAccountName: fluent-bit
tolerations:
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule
- operator: "Exists"
effect: "NoExecute"
- operator: "Exists"
effect: "NoSchedule"
流体-比特-油-标记.yaml
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: fluent-bit-read
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: fluent-bit-read
subjects:
- kind: ServiceAccount
name: fluent-bit
namespace: log
流体-位-油.yaml
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: fluent-bit-read
rules:
- apiGroups: [""]
resources:
- namespaces
- pods
verbs: ["get", "list", "watch"]
流动比特服务账户
apiVersion: v1
kind: ServiceAccount
metadata:
name: fluent-bit
namespace: log
我的Fluentbit OUTPUT(STDOUT(仅用于调试：
$ kubectl logs -f fluent-bit-2bzxb -n log
[2020/02/20 18:54:23] [Warning] [config] I cannot open /fluent-bit/etc/..2020_02_20_18_54_22.252769193/parsers_custom.conf file
[2020/02/20 18:54:23] [ info] [storage] initializing...
[2020/02/20 18:54:23] [ info] [storage] in-memory
[2020/02/20 18:54:23] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2020/02/20 18:54:23] [ info] [engine] started (pid=1)
[2020/02/20 18:54:23] [ info] [filter_kube] https=1 host=kubernetes.default.svc port=443
[2020/02/20 18:54:23] [ info] [filter_kube] local POD info OK
[2020/02/20 18:54:23] [ info] [filter_kube] testing connectivity with API server...
[2020/02/20 18:54:23] [ info] [filter_kube] API server connectivity OK
[2020/02/20 18:54:23] [ info] [sp] stream processor started
问题是我没有从带有此配置的systemd获得任何日志
谢谢@edsiper我修复了我的Daemoset添加的"path:/run/log">
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: fluent-bit
namespace: log
labels:
k8s-app: fluent-bit-logging
version: v1
kubernetes.io/cluster-service: "true"
spec:
selector:
matchLabels:
k8s-app: fluent-bit-logging
version: v1
kubernetes.io/cluster-service: "true"
template:
metadata:
labels:
k8s-app: fluent-bit-logging
version: v1
kubernetes.io/cluster-service: "true"
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "2020"
prometheus.io/path: /api/v1/metrics/prometheus
spec:
containers:
- name: fluent-bit
image: fluent/fluent-bit:1.3.5
imagePullPolicy: Always
ports:
- containerPort: 2020
env:
volumeMounts:
- name: varlog
mountPath: /var/log
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
- name: systemdlog
mountPath: /run/log
- name: fluent-bit-config
mountPath: /fluent-bit/etc/
terminationGracePeriodSeconds: 10
volumes:
- name: varlog
hostPath:
path: /var/log
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
- name: systemdlog
hostPath:
path: /run/log
- name: fluent-bit-config
configMap:
name: fluent-bit-config
serviceAccountName: fluent-bit
tolerations:
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule
- operator: "Exists"
effect: "NoExecute"
- operator: "Exists"
effect: "NoSchedule"
Fluent Bit容器是否可以访问Systemd日志路径？
没有足够的因果报应来发布评论，所以发布作为对@edsiper的回答："您的Fluent Bit容器是否可以访问Systemd日志路径"在默认设置下(否(，它不会。当我试图解决这个问题时，我偶然发现了以下线索：https://github.com/fluent/fluent-bit/issues/497
长话短说：
-
您需要以root身份运行fluent bit容器，因为访问日志需要root权限
-
将docker中的机器id设置为与根机器中的相同
-
bind/run/log/journal:/run/log/日志所以：
fluent-bit: image: 'bitnami/fluent-bit:latest' restart: always user: root #give root access network_mode: host command: /fluent-bit/bin/fluent-bit -c /fluent-bit/etc/fluent-bit.conf volumes: - ./service/config/fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf - /etc/machine-id:/etc/machine-id:ro #set the machine id - /run/log/journal:/run/log/journal #give access to logs
然后，在fluent-bit.conf中，如果需要编辑INPUTPath:
[INPUT]
Name systemd
Tag *
Path /run/log/journal
Systemd_Filter _SYSTEMD_UNIT=docker.service
Systemd_Filter _SYSTEMD_UNIT=kubelet.service