kafkaconnect常见问题
ERROR WorkerSourceTask{id=mysql-source-binlog-336-jobId-0} Task threw an uncaught and unrecoverable exception
(org.t.runtime.WorkerTask:179)
org.t.errors.ConnectException: Unrecoverable exception from producer send callback
at org.t.runtime.WorkerSourceTask.maybeThrowProducerSendException(WorkerSourceTask.java:252)
at org.t.ute(WorkerSourceTask.java:221)
at org.t.runtime.WorkerTask.doRun(WorkerTask.java:177)
at org.t.runtime.WorkerTask.run(WorkerTask.java:227)
at urrent.Executors$RunnableAdapter.call(Executors.java:511)
at urrent.FutureTask.run(FutureTask.java:266)
at urrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at urrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.s.RecordTooLargeException: The message is 3900896 bytes when serialized which is larger than the maximum request size you have configured with quest.size configuration.
查看 kafka connect的⽇志 INFO ProducerConfig values中quest.size 的值
mysql-source-binlog-336-jobId-dbhistory  的quest.siz改掉了。但是数据的quest.siz没改依然报错
connector-producer-mysql-source-binlog-336-jobId-0
参考
加⼊参数
"database.history.quest.size": "157286400",
"quest.size": "157286400"
到source中解决
---------------------------------------------------------------------------------------------------------------------
{"code":500,"data":null,"msg":"java.lang.IllegalArgumentException: URI is not absolute"}
---------------------------------------------------------------------------------------------------------------------
[2019-11-14 14:10:10,064] ERROR WorkerSourceTask{id=mysql-source-binlog-336-jobId-0} Task threw an uncaught and unrecoverable exception (org.t.runtime.WorkerTask:179)
org.t.errors.ConnectException: Tolerance exceeded in error handler
at
org.AndHandleError(RetryWithToleranceOperator.java:17
8)
at org.ute(RetryWithToleranceOperator.java:104)
at org.t.vertTransformedRecord(WorkerSourceTask.java:281)
at org.t.runtime.WorkerSourceTask.sendRecords(WorkerSourceTask.java:309)
at org.t.ute(WorkerSourceTask.java:234)
at org.t.runtime.WorkerTask.doRun(WorkerTask.java:177)
at org.t.runtime.WorkerTask.run(WorkerTask.java:227)
at urrent.Executors$RunnableAdapter.call(Executors.java:511)
at urrent.FutureTask.run(FutureTask.java:266)
memset函数的用法举例at urrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at urrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.t.errors.DataException: Failed to serialize Avro data from topic mysql-source-binlog-336-jobId.slow_log.slow_log :
t.avro.AvroConverter.fromConnectData(AvroConverter.java:83)
at
org.t.runtime.WorkerSourceTask.lambda$convertTransformedRecord$1(WorkerSourceTask.java:281)
at
org.AndRetry(RetryWithToleranceOperator.java:128)
at
org.AndHandleError(RetryWithToleranceOperator.java:16 2)
... 11 more
Caused by: org.s.SerializationException: Error registering Avro schema: {"type":"record","name":"Key","namespace":"mysql_source_binlog_336_jobId.slow_log.slow_log","fields": [{"name":"id","type":"string"}],"connect.name":"mysql_source_binlog_336_jobId.slow_log.slow_log.Key"}
学院网页设计模板Caused by: io.confluent.kafka.st.exceptions.RestClientException: Error while forwarding register schema request to the master; error code: 50003
fluent.kafka.st.RestService.sendHttpRequest(RestService.java:230)
fluent.kafka.st.RestService.httpRequest(RestService.java:256)
fluent.kafka.isterSchema(RestService.java:356)
fluent.kafka.isterSchema(RestService.java:348)
fluent.kafka.isterSchema(RestService.java:334)
编程猫社区是用来干嘛的
at
fluent.kafka.schemaregistry.ister(CachedSchemaRegistryClient.java:222)
fluent.kafka.schemaregistry.ister(CachedSchemaRegistryClient.java:198)
fluent.kafka.serializers.AbstractKafkaAvroSerializer.serializeImpl(AbstractKafkaAvroSerializer.java:70)
t.avro.AvroConverter$Serializer.serialize(AvroConverter.java:131)
t.avro.AvroConverter.fromConnectData(AvroConverter.java:80)
at
org.t.runtime.WorkerSourceTask.lambda$convertTransformedRecord$1(WorkerSourceTask.java:281)
at
org.AndRetry(RetryWithToleranceOperator.java:128)
at
org.AndHandleError(RetryWithToleranceOperator.java:16 2)
at org.ute(RetryWithToleranceOperator.java:104)
at org.t.vertTransformedRecord(WorkerSourceTask.java:281)
at org.t.runtime.WorkerSourceTask.sendRecords(WorkerSourceTask.java:309)
at org.t.ute(WorkerSourceTask.java:234)
at org.t.runtime.WorkerTask.doRun(WorkerTask.java:177)
at org.t.runtime.WorkerTask.run(WorkerTask.java:227)
at urrent.Executors$RunnableAdapter.call(Executors.java:511)
at urrent.FutureTask.run(FutureTask.java:266)
at urrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at urrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
html语言中的知识点查看SchemaRegister⽇志没有发现明显错误，看到hostname不通
---------------------------------------------------------------------------------------------------------------------
[2019-11-21 16:32:16,681] ERROR Error during binlog processing. Last offset stored = {ts_sec=1574325000, file=mysqld-bin.008559, pos=419190460, row=1, server_
id=463306, event=3}, binlog reader near position = mysqld-bin.008559/430041542
(sql.BinlogReader:1054)
[2019-11-21 16:32:16,681] ERROR Failed due to error: Error processing binlog event
(sql.BinlogReader:209)
org.t.errors.ConnectException: Error recording the DDL statement(s) in the database history Kafka sql-source-binlog-333-jobId:0 using brokers at null: truncate table mysql.slow_log
at sql.AbstractReader.wrap(AbstractReader.java:230)
at sql.AbstractReader.failed(AbstractReader.java:208)
at sql.BinlogReader.handleEvent(BinlogReader.java:508)
at com.sql.ifyEventListeners(BinaryLogClient.java:1095)
at com.sql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:943)
at com.sql.t(BinaryLogClient.java:580)
at com.sql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:825)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.t.errors.ConnectException: Error recording the DDL statement(s) in the database history Kafka sql-source-binlog-333-jobId:0 using brokers at null: truncate table mysql.slow_log
at sql.MySqlSchema.applyDdl(MySqlSchema.java:361)
at sql.BinlogReader.handleQueryEvent(BinlogReader.java:694)
mysqlschema作用at sql.BinlogReader.handleEvent(BinlogReader.java:492)
... 5 more
Caused by: lational.history.DatabaseHistoryException: urrent.ExecutionException:
showheader是什么意思
org.s.TimeoutException: Expiring 1 record(s) sql-source-binlog-333-jobId-0:120001 ms has passed since batch creation
at lational.history.KafkaDatabaseHistory.storeRecord(KafkaDatabaseHistory.java:198)
at lational.d(AbstractDatabaseHistory.java:66)
at lational.d(AbstractDatabaseHistory.java:60)
at sql.MySqlSchema.applyDdl(MySqlSchema.java:356)
... 7 more
Caused by: urrent.ExecutionException: org.s.TimeoutException: Expiring 1 record(s) sql-source-binlog-333-jobId-0:120001 ms has passed since batch creation
at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.valueOrError(FutureRecordMetadata.java:98)
at org.apache.kafka.clients.producer.(FutureRecordMetadata.java:67)
at org.apache.kafka.clients.producer.(FutureRecordMetadata.java:30)
at lational.history.KafkaDatabaseHistory.storeRecord(KafkaDatabaseHistory.java:188)
... 10 more
Caused by: org.s.TimeoutException: Expiring 1 record(s) sql-source-binlog-333-jobId-0:120001 ms has passed since batch creation
---------------------------------------------------------------------------------------------------------------------
ERROR WorkerSourceTask{id=mysql-source-binlog-358-jobId-0} Task threw an uncaught and unrecoverable exception
(org.
ect.runtime.WorkerTask:179)
org.t.errors.ConnectException: log event entry exceeded max_allowed_packet; Increase
max_allowed_packet on master; the first event 'mysqld-
bin.000060' at 910723425, the last event read from '/data/binlog/qdpp/mysqld-bin.000060' at 123, the last byte read from
'/data/binlog/qdpp/mysqld-bin.000060'
at 910723444. Error code: 1236; SQLSTATE: HY000.
at sql.AbstractReader.wrap(AbstractReader.java:230)
at sql.AbstractReader.failed(AbstractReader.java:197)
at
sql.CommunicationFailure(BinlogReader.java:1041)
at com.sql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:950)
at com.sql.t(BinaryLogClient.java:580)
at com.sql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:825)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.sql.binlogwork.ServerException: log event entry exceeded max_allowed_packet; Increase max_allowed_packet on master; the fir
st event 'mysqld-bin.000060' at 910723425, the last event read from '/data/binlog/qdpp/mysqld-bin.000060' at 123, the last byte read from '/data/binlog/qdpp/m
ysqld-bin.000060' at 910723444.
at com.sql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:914)
... 3 more
单个事物⽇志超过max_allowed_packet配置的限制⼤⼩
同时Error code: 1236代表BINLOG不存在
---------------------------------------------------------------------------------------------------------------------
ERROR WorkerSinkTask{id=elasticsearch-sink-binlog-364-taskId-0} Task threw an uncaught and unrecoverable exception. Task is being ki
lled and will not recover until manually restarted. (org.t.runtime.WorkerSinkTask:558)
org.t.errors.ConnectException: java.SocketTimeoutException: Read timed out

版权声明：本站内容均来自互联网，仅供演示用，请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系QQ:729038198，我们将在24小时内删除。