kafka监控指标项
监控配置
kafka基本分为broker、producer、consumer三个⼦项，每⼀项的启动都需要⽤到$KAFKA_HOME/bin/kafka-run-class.sh脚本，在该脚本中，存在以下语句：
if ...
KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false  -Dcom.sun.management.jmxremote.ssl=false"
fi
if ...
KAFKA_JMX_OPTS="$KAFKA_JMX_OPTS -Dcom.sun.management.jmxremote.port=$JMX_PORT "
fi
在启动kafka的过程中，只要指定JMX_PORT的值，即可对broker、producer、consumer进⾏监控。⽬前
有两种⽅法，在$KAFKA_HOME/bin/kafka-server-start.sh、$KAFKA_HOME/bin/kafka-console-consumer.sh、$KAFKA_HOME/bin/kafka-console-producer.sh三个脚本中分别添加$JMX_PORT=XXXX语句，但是只适⽤于使⽤console⽅式对topic进⾏使⽤的情况。
修改$KAFKA_HOME/bin/kafka-run-class.sh脚本中的上述语句，使其端⼝随机变化，可以通过ps -ef |grep kafka命令来获取随机的端⼝号，来进⾏监控
指标项来源
以下端⼝均随机获得。
主机名类别端⼝号
192.168.20.10broker9183
192.168.20.10producer9108
192.168.20.10consumer9173
kafka监控项
OS监控项
objectName指标项说明
java.lang:type=OperatingSystem FreePhysicalMemorySize空闲物理内存
java.lang:type=OperatingSystem SystemCpuLoad系统CPU利⽤率
java.lang:type=OperatingSystem ProcessCpuLoad进程CPU利⽤率
java.lang:type=GarbageCollector,
CollectionCount GC次数
name=G1 Young Generation
broker指标
objectName指标项说明
kafka.server:type=BrokerTopicMetrics,
Count每秒输⼊的流量
name=BytesInPerSec
kafka.server:type=BrokerTopicMetrics,
Count每秒输出的流量
name=BytesOutPerSec
kafka.server:type=BrokerTopicMetrics,
Count每秒扔掉的流量
name=BytesRejectedPerSec
kafka.server:type=BrokerTopicMetrics,
Count每秒的消息写⼊总量
name=MessagesInPerSec
kafka.server:type=BrokerTopicMetrics,
Count当前机器每秒fetch请求失败的数量
name=FailedFetchRequestsPerSec
kafka.server:type=BrokerTopicMetrics,
Count当前机器每秒produce请求失败的数量
name=FailedProduceRequestsPerSec
kafka.server:type=ReplicaManager,
Value该broker上的partition的数量
name=PartitionCount
kafka.server:type=ReplicaManager,
Value Leader的replica的数量
name=LeaderCount
kafkawork:type=RequestMetrics,
Count⼀个请求FetchConsumer耗费的所有时间
name=TotalTimeMs,request=FetchConsumer
kafkawork:type=RequestMetrics,
Count⼀个请求FetchFollower耗费的所有时间
name=TotalTimeMs,request=FetchFollower
name=TotalTimeMs,request=FetchFollower
kafkawork:type=RequestMetrics, name=TotalTimeMs,request=Produce Count⼀个请求Produce耗费的所有时间
objectName指标项说明
producer以及topic指标
objectName指标项官⽹说明译⽂说明
kafka.producer:type=producer-metrics,client-id=console-producer(client-id会变化)incoming-
byte-rate
The average number of incoming bytes
received per second from all servers.
producer每秒的平均写
⼊流量
kafka.producer:type=producer-metrics,client-id=console-producer(client-id会变化)outgoing-
byte-ratekafka命令
The average number of outgoing bytes
sent per second to all servers.
producer每秒的输出流
量
kafka.producer:type=producer-metrics,client-id=console-producer(client-id会变化)request-
rate
The average number of requests sent per
second to the broker.
producer每秒发给
broker的平均request次
数
kafka.producer:type=producer-metrics,client-id=console-producer(client-id会变化)response-
rate
The average number of responses
received per second from the broker.
producer每秒发给
broker的平均response
次数
kafka.producer:type=producer-metrics,client-id=console-producer(client-id会变化)request-
latency-
avg
The average time taken for a fetch
request.
⼀个fetch请求的平均时
间
kafka.producer:type=producer-topic-metrics,client-
id=console-producer,topic=testjmx(client-id和topic名称会变化)record-
send-rate
The average number of records sent per
second for a topic.
每秒从topic发送的平均
记录数
kafka.producer:type=producer-topic-metrics,client-
id=console-producer,topic=testjmx(client-id和topic名称会变化)record-
retry-total
The total number of retried record sends重试发送的消息总数量
kafka.producer:type=producer-topic-metrics,client-
id=console-producer,topic=testjmx(client-id和topic名称会变化)record-
error-total
The total number of record sends that
resulted in errors发送错误的消息总数量
consumer指标
objectName指标项官⽹说明说明
max
Number of messages the consumer lags behind the
producer by. Published by the consumer, not broker.
由consumer提交
的消息消费lag
consumed-
rate
The average number of records consumed per second每秒平均消费的
消息数量

版权声明：本站内容均来自互联网，仅供演示用，请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系QQ:729038198，我们将在24小时内删除。