Flume仍然保留.TMP文件，而不将文件完全复制到HDFS

Question

嗨，我正在使用Flume将文件从spooling目录复制到使用文件作为频道的HDF。

#Component names
a1.sources = src
a1.channels = c1
a1.sinks = k1
#Source details
a1.sources.src.type = spooldir
a1.sources.src.channels = c1
a1.sources.src.spoolDir = /home/cloudera/onetrail
a1.sources.src.fileHeader = false
a1.sources.src.basenameHeader = true
# a1.sources.src.basenameHeaderKey = basename
a1.sources.src.fileSuffix = .COMPLETED
a1.sources.src.threads = 4
a1.sources.src.interceptors = newint
a1.sources.src.interceptors.newint.type = timestamp
#Sink details
a1.sinks.k1.type = hdfs
a1.sinks.k1.channel = c1
a1.sinks.k1.hdfs.path = hdfs:///data/contentProviders/cnet/%Y%m%d/
# a1.sinks.k1.hdfs.round = false
# a1.sinks.k1.hdfs.roundValue = 1
# a1.sinks.k1.hdfs.roundUnit = second
a1.sinks.k1.hdfs.writeFormat = Text
a1.sinks.k1.hdfs.fileType = DataStream
#a1.sinks.k1.hdfs.file.Type = DataStream
a1.sinks.k1.hdfs.filePrefix = %{basename}
# a1.sinks.k1.hdfs.fileSuffix = .xml
a1.sinks.k1.threadsPoolSize = 4
# use a single file at a time
a1.sinks.k1.hdfs.maxOpenFiles = 1
# rollover file based on maximum size of 10 MB
a1.sinks.k1.hdfs.rollCount = 0
a1.sinks.k1.hdfs.rollInterval = 0
a1.sinks.k1.hdfs.rollSize = 0
a1.sinks.k1.hdfs.batchSize = 12
# Channel details
a1.channels.c1.type = file
a1.channels.c1.checkpointDir = /tmp/flume/checkpoint/
a1.channels.c1.dataDirs = /tmp/flume/data/
# Bind the source and sink to the channel
a1.sources.src.channels = c1
a1.sinks.k1.channels = c1

使用上述配置，它可以将文件复制到HDFS，但是我面临的问题是一个文件是保持为.tmp而不复制完整的文件内容。

一些人可以帮助我问题。

Answer 1

.tmp文件将其重命名为最终名称，一旦将其"滚动"。

您所有的翻转设置都是0，这意味着"永远保持流"。

将一个或多个设置为非零值确定何时将Flume视为文件完成，以便可以将其关闭并打开下一个。

有关更多详细信息，请参见Flume Anns上的文档。