我正在使用AWS EKS 1.21与Fargate(无服务器)。我试图将Fluentd作为守护进程运行，但是守护进程根本没有运行。
所有其他对象，如role, rolebinding, serviceaccount, configmap已经在集群中存在。
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
aws-node 0 0 0 0 0 <none> 8d
fluentd-cloudwatch 0 0 0 0 0 <none> 3m36s
kube-proxy 0 0 0 0 0 <none> 8d
这是我的守护进程:-
apiVersion: apps/v1 #Latest support AWS EKS 1.21
kind: DaemonSet
metadata:
labels:
k8s-app: fluentd-cloudwatch
name: fluentd-cloudwatch
namespace: kube-system
spec:
selector:
matchLabels:
k8s-app: fluentd-cloudwatch
template:
metadata:
labels:
k8s-app: fluentd-cloudwatch
spec:
containers:
- env:
- name: REGION
value: us-east-1 # Correct AWS EKS region should be verified before running this Daemonset
- name: CLUSTER_NAME
value: eks-fargate-alb-demo # AWS EKS Cluster Name should be verified before running this Daemonset
image: fluent/fluentd-kubernetes-daemonset:v1.1-debian-cloudwatch
imagePullPolicy: IfNotPresent
name: fluentd-cloudwatch
resources:
limits:
memory: 200Mi
requests:
cpu: 100m
memory: 200Mi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /config-volume
name: config-volume
- mountPath: /fluentd/etc
name: fluentdconf
- mountPath: /var/log
name: varlog
- mountPath: /var/lib/docker/containers
name: varlibdockercontainers
readOnly: true
- mountPath: /run/log/journal
name: runlogjournal
readOnly: true
dnsPolicy: ClusterFirst
initContainers:
- command:
- sh
- -c
- cp /config-volume/..data/* /fluentd/etc
image: busybox
imagePullPolicy: Always
name: copy-fluentd-config
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /config-volume
name: config-volume
- mountPath: /fluentd/etc
name: fluentdconf
serviceAccount: fluentd
serviceAccountName: fluentd
terminationGracePeriodSeconds: 30
volumes:
- configMap:
defaultMode: 420
name: fluentd-config
name: config-volume
- emptyDir: {}
name: fluentdconf
- hostPath:
path: /var/log
type: ""
name: varlog
- hostPath:
path: /var/lib/docker/containers
type: ""
name: varlibdockercontainers
- hostPath:
path: /run/log/journal
type: ""
name: runlogjournal
当我描述它时，我也没有看到任何事件。我可以在这个集群上运行其他pod，比如Nginx等，但这个根本没有运行。
kubectl describe ds fluentd-cloudwatch -n kube-system
Name: fluentd-cloudwatch
Selector: k8s-app=fluentd-cloudwatch
Node-Selector: <none>
Labels: k8s-app=fluentd-cloudwatch
Annotations: deprecated.daemonset.template.generation: 1
Desired Number of Nodes Scheduled: 0
Current Number of Nodes Scheduled: 0
Number of Nodes Scheduled with Up-to-date Pods: 0
Number of Nodes Scheduled with Available Pods: 0
Number of Nodes Misscheduled: 0
Pods Status: 0 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
Labels: k8s-app=fluentd-cloudwatch
Service Account: fluentd
Init Containers:
copy-fluentd-config:
Image: busybox
Port: <none>
Host Port: <none>
Command:
sh
-c
cp /config-volume/..data/* /fluentd/etc
Environment: <none>
Mounts:
/config-volume from config-volume (rw)
/fluentd/etc from fluentdconf (rw)
Containers:
fluentd-cloudwatch:
Image: fluent/fluentd-kubernetes-daemonset:v1.1-debian-cloudwatch
Port: <none>
Host Port: <none>
Limits:
memory: 200Mi
Requests:
cpu: 100m
memory: 200Mi
Environment:
REGION: us-east-1
CLUSTER_NAME: eks-fargate-alb-demo
Mounts:
/config-volume from config-volume (rw)
/fluentd/etc from fluentdconf (rw)
/run/log/journal from runlogjournal (ro)
/var/lib/docker/containers from varlibdockercontainers (ro)
/var/log from varlog (rw)
Volumes:
config-volume:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: fluentd-config
Optional: false
fluentdconf:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
varlog:
Type: HostPath (bare host directory volume)
Path: /var/log
HostPathType:
varlibdockercontainers:
Type: HostPath (bare host directory volume)
Path: /var/lib/docker/containers
HostPathType:
runlogjournal:
Type: HostPath (bare host directory volume)
Path: /run/log/journal
HostPathType:
Events: <none>
ConfigMap: -
apiVersion: v1
data:
containers.conf: |
<source>
@type tail
@id in_tail_container_logs
@label @containers
path /var/log/containers/*.log
pos_file /var/log/fluentd-containers.log.pos
tag *
read_from_head true
<parse>
@type json
time_format %Y-%m-%dT%H:%M:%S.%NZ
</parse>
</source>
<label @containers>
<filter **>
@type kubernetes_metadata
@id filter_kube_metadata
</filter>
<filter **>
@type record_transformer
@id filter_containers_stream_transformer
<record>
stream_name ${tag_parts[3]}
</record>
</filter>
<match **>
@type cloudwatch_logs
@id out_cloudwatch_logs_containers
region "#{ENV.fetch('REGION')}"
log_group_name "/k8s-nest/#{ENV.fetch('CLUSTER_NAME')}/containers"
log_stream_name_key stream_name
remove_log_stream_name_key true
auto_create_stream true
<buffer>
flush_interval 5
chunk_limit_size 2m
queued_chunks_limit_size 32
retry_forever true
</buffer>
</match>
</label>
fluent.conf: |
@include containers.conf
@include systemd.conf
<match fluent.**>
@type null
</match>
systemd.conf: |
<source>
@type systemd
@id in_systemd_kubelet
@label @systemd
filters [{ "_SYSTEMD_UNIT": "kubelet.service" }]
<entry>
field_map {"MESSAGE": "message", "_HOSTNAME": "hostname", "_SYSTEMD_UNIT": "systemd_unit"}
field_map_strict true
</entry>
path /run/log/journal
pos_file /var/log/fluentd-journald-kubelet.pos
read_from_head true
tag kubelet.service
</source>
<source>
@type systemd
@id in_systemd_kubeproxy
@label @systemd
filters [{ "_SYSTEMD_UNIT": "kubeproxy.service" }]
<entry>
field_map {"MESSAGE": "message", "_HOSTNAME": "hostname", "_SYSTEMD_UNIT": "systemd_unit"}
field_map_strict true
</entry>
path /run/log/journal
pos_file /var/log/fluentd-journald-kubeproxy.pos
read_from_head true
tag kubeproxy.service
</source>
<source>
@type systemd
@id in_systemd_docker
@label @systemd
filters [{ "_SYSTEMD_UNIT": "docker.service" }]
<entry>
field_map {"MESSAGE": "message", "_HOSTNAME": "hostname", "_SYSTEMD_UNIT": "systemd_unit"}
field_map_strict true
</entry>
path /run/log/journal
pos_file /var/log/fluentd-journald-docker.pos
read_from_head true
tag docker.service
</source>
<label @systemd>
<filter **>
@type record_transformer
@id filter_systemd_stream_transformer
<record>
stream_name ${tag}-${record["hostname"]}
</record>
</filter>
<match **>
@type cloudwatch_logs
@id out_cloudwatch_logs_systemd
region "#{ENV.fetch('REGION')}"
log_group_name "/k8s-nest/#{ENV.fetch('CLUSTER_NAME')}/systemd"
log_stream_name_key stream_name
auto_create_stream true
remove_log_stream_name_key true
<buffer>
flush_interval 5
chunk_limit_size 2m
queued_chunks_limit_size 32
retry_forever true
</buffer>
</match>
</label>
kind: ConfigMap
metadata:
labels:
k8s-app: fluentd-cloudwatch
name: fluentd-config
namespace: kube-system
请让我知道问题在哪里，谢谢
正如您所设想的，EKS/Fargate不支持守护进程(因为没有[真正的]节点)。实际上，您不需要在每个pod中运行FluentBit作为侧车。EKS/Fargate支持一个名为Firelens的日志记录功能，允许你只配置你想要记录的地方(目的地)，Fargate将配置一个"隐藏汽车"。在后端(对用户不可见)执行此操作。详情请参阅此文档页
片段:
Amazon EKS on Fargate offers a built-in log router based on Fluent Bit. This means that you don't explicitly run a Fluent Bit container as a sidecar, but Amazon runs it for you. All that you have to do is configure the log router. The configuration happens through a dedicated ConfigMap....
经过研究，我发现AWS中还不支持Fargate的Kubernetes Daemon set对象。现在剩下的选项是:-A)在pod中与其他容器一起运行Fluentd作为sidecar模式B)基于
将集群从Fargate更改为NodeGroup