FlumeSink到Kafka遇到的问题
1、版本问题
2018-12-02 15:23:06,334 (conf-file-poller-0) [ERROR - org.de.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.jav a:427)] Sink k1 has been removed due to an error during configuration
org.f.ConfigurationException: brokerList must contain at least one Kafka broker
at org.apache.flume.sink.kafka.KafkaSinkUtil.addDocumentedKafkaProps(KafkaSinkUtil.java:55)
at org.apache.flume.sink.KafkaProperties(KafkaSinkUtil.java:37)
at org.apache.flume.sink.figure(KafkaSink.java:217)
at org.figure(Configurables.java:41)
at org.de.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:413)
at org.Configuration(AbstractConfigurationProvider.java:98)
at org.de.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)        at urrent.Executors$RunnableAdapter.call(Executors.java:511)
at urrent.FutureTask.runAndReset(FutureTask.java:308)
at urrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at urrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at urrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at urrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
查看官⽹，发现是因为使⽤了最新版本的Flume（1.8.0）的配置信息，⽽导致了上⾯的⼀个错误，使数据⽆法写⼊Kafka。
使⽤相对应的版本的配置即可。
所以配置⾥的a1.sinks.k1.kafka.bootstrap.servers = master:9092,slave1:9092,slave2:9092换成a1.sinks.k1.brokerList=
master:9092,slave1:9092,slave2:9092
Flume1.8.0有以下过期配置：
Deprecated Properties
kafka最新版本
Property Name Default Description brokerList–Use kafka.bootstrap.servers topic default-flume-topic pic batchSize100Use kafka.flumeBatchSize requiredAcks1Use kafka.producer.acks 其他可能因为版本出现的警告：
WARN kafka.KafkaSink: The Property 'topic' is not set. Using the default topic name: default-flume-topic

版权声明：本站内容均来自互联网，仅供演示用，请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系QQ:729038198，我们将在24小时内删除。