使用fluent插件grok解析器与splunk-hec图像



我正在尝试创建一个基于fluentd hec映像安装grok解析器的映像。

这是我正在使用的Dockerfile:

FROM splunk/fluentd-hec:1.2.4
USER root
RUN gem install fluent-plugin-grok-parser
RUN chown -R fluent:fluent /usr/local/share/gems/gems/fluent-plugin-grok-parser-*
USER fluent

这是构建的输出


Step 1/5 : FROM splunk/fluentd-hec:1.2.4
---> ac49b85acc6a
Step 2/5 : USER root
---> Running in 4ee81880e92a
Removing intermediate container 4ee81880e92a
---> e3748059e604
Step 3/5 : RUN gem install fluent-plugin-grok-parser
---> Running in 2a1debb084ec
Successfully installed bundler-2.2.5
Building native extensions. This could take a while...
Successfully installed msgpack-1.3.3
Building native extensions. This could take a while...
Successfully installed yajl-ruby-1.4.1
Building native extensions. This could take a while...
Successfully installed cool.io-1.7.0
Successfully installed sigdump-0.2.4
Successfully installed serverengine-2.2.2
Building native extensions. This could take a while...
Successfully installed http_parser.rb-0.6.0
Successfully installed concurrent-ruby-1.1.7
Successfully installed tzinfo-2.0.4
Successfully installed tzinfo-data-1.2020.6
Building native extensions. This could take a while...
Successfully installed strptime-0.2.5
Successfully installed fluentd-1.12.0
Successfully installed fluent-plugin-grok-parser-2.6.2
13 gems installed
Removing intermediate container 2a1debb084ec
---> c5155932810c
Step 4/5 : RUN chown -R fluent:fluent /usr/local/share/gems/gems/fluent-plugin-grok-parser-*
---> Running in 1c2550dcac74
Removing intermediate container 1c2550dcac74
---> 7e216a676427
Step 5/5 : USER fluent
---> Running in 5ee31ea2e78a
Removing intermediate container 5ee31ea2e78a
---> ea8bdee73ee5
Successfully built ea8bdee73ee5

configmap的片段是:

@id snow
@type tail
@label @SPLUNK
tag tail.snow.*
path /opt/snow/data/*.log
pos_file /var/log/splunk-snow.log.pos
path_key source
<parse>
@type grok
grok_failure_key grokfailure
<grok>
pattern %{TIMESTAMP_ISO8601:time};%{SPACE}%{GREEDYDATA:log}
</grok>
</parse>
</source>

当我使用新映像部署守护程序集时,我会收到错误config error file="/fluentd/etc/fluent.conf" error_class=Fluent::ConfigError error="Unknown parser plugin 'grok'. Run 'gem search -rd fluent-plugin' to find plugins"我在EKS v1.18以及docker桌面中尝试过这一点,但它们都有相同的问题。

还有什么需要添加到dockerfile中以便使用额外的插件吗?

我非常感谢在这方面的任何帮助!

我已经重写了配置,所以不使用grok,所以这个问题不再需要了

最新更新