FlumeNG中的正则表达式确认

Question

我正在尝试使用Flume ng（1.2）将数据从平面文件（日志文件）加载到Hbase中。平面文件有多个列，每个列都用冒号（：）分隔，它们都需要加载到Hbase中的单独列中。我在查看论坛时发现Apache有一个jar可以解决这个问题（org.Apache.flumes.sink.hbase.RegexHbaseEventSerializer），但我找不到任何配置文件或在互联网上的用法。如果有人能帮我处理配置文件，那将对很有帮助

平面文件中的内容1:nn2:pp3:mm

感谢

Answer 1

RegexHbaseEventSerializer有三个配置参数可以设置（如源代码中所述）；这些是：

/** Regular expression used to parse groups from event data. */
public static final String REGEX_CONFIG = "regex";
/** Whether to ignore case when performing regex matches. */
public static final String IGNORE_CASE_CONFIG = "regexIgnoreCase";
/** Comma separated list of column names to place matching groups in. */
public static final String COL_NAME_CONFIG = "colNames";

使用RegexHbaseEventSerializer的示例配置如下（部分引用Cloudera的Flume和HBase演示）：

host1.sources = src1
host1.sinks = sink1
host1.channels = ch1
host1.sources.src1.type = seq
host1.sources.src1.port = 25001
host1.sources.src1.bind = localhost
host1.sources.src1.channels = ch1
host1.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
host1.sinks.sink1.channel = ch1
host1.sinks.sink1.table = test3
host1.sinks.sink1.columnFamily = testing
host1.sinks.sink1.serializer = org.apache.flume.sink.hbase.RegexHbaseEventSerializer
host1.sinks.sink1.serializer.regex = X
host1.sinks.sink1.serializer.regexIgnoreCase = true
host1.sinks.sink1.serializer.colNames = column_1,column_2,column_3
host1.channels.ch1.type=memory10