Caused by org apache kafka connect errors DataException Converting byte to Kafka Connect data failed due to serialization error

0 votes

I tried to read JSON data from a topic. But I am getting the below error.

$ curl -s "http://localhost:8083/connectors/file_sink_01/status"| \
    jq -c -M '[.name,.tasks[].state]'
["file_sink_01","FAILED"]
org.apache.kafka.connect.errors.ConnectException: Tolerance exceeded in error handler
at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:178)
…
Caused by: org.apache.kafka.connect.errors.DataException: Converting byte[] to Kafka Connect data failed due to serialization error:
at org.apache.kafka.connect.json.JsonConverter.toConnectData(JsonConverter.java:334)
…
Caused by: org.apache.kafka.common.errors.SerializationException: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('b' (code 98)): was expecting double-quote to start field name
at [Source: (byte[])"{brokenjson-:"bar 1"}"; line: 1, column: 3]

Thank You

Apr 3, 2020 in Apache Kafka by akhtar
• 38,170 points
1,015 views

1 answer to this question.

0 votes

Hi@akhtar,

To fix the pipeline, you need to resolve the issue with the message on the source topic. Kafka Connect will not simply “skip” the bad message unless we tell it to do.

The default behavior of Kafka Connect.

errors.tolerance = none

But it can be set explicitly with the following.

errors.tolerance = all

Now try again, it will work.

Thank You

answered Apr 3, 2020 by MD
• 95,060 points

Related Questions In Apache Kafka

0 votes
1 answer

How to commit message offsets in Kafka for reliable data pipeline?

You can use auto.commit.enable to allow Kafka ...READ MORE

answered Jul 9, 2018 in Apache Kafka by Shubham
• 13,480 points
1,464 views
0 votes
1 answer

Explain to me the functionality of Kafka in a Big-Data Cluster

Let me explain to you about Apache ...READ MORE

answered Apr 30, 2019 in Apache Kafka by ravikiran
• 4,620 points
342 views
+1 vote
1 answer
0 votes
1 answer
0 votes
1 answer

Error: pass --path /path/to/confluent flag or set environment variable CONFLUENT_HOME

Hi@akhtar, Before starting confluent kafka in your system, ...READ MORE

answered Apr 3, 2020 in Apache Kafka by MD
• 95,060 points
878 views
0 votes
1 answer

How to read batch messages in confluent-kafka-python?

Hi@akhtar, You have to read the messages normally ...READ MORE

answered Apr 3, 2020 in Apache Kafka by MD
• 95,060 points
1,728 views
+1 vote
14 answers

How can I delete all the messages from a Kafka topic?

To delete all the messages from a Kafka topic.  There are many approach ...READ MORE

answered Jun 19, 2020 in Apache Kafka by Pawan
• 380 points
144,855 views
0 votes
1 answer

How to purge Kafka Topic?

For your case what you can do ...READ MORE

answered Sep 4, 2018 in Apache Kafka by nitinrawat895
• 11,380 points
1,564 views
0 votes
1 answer

How can i connect to an api which gives me weather data to Apache kafka?

Hi, To connect an API in kafka, you ...READ MORE

answered Apr 13, 2020 in Apache Kafka by MD
• 95,060 points
265 views