Caused by org apache kafka connect errors DataException Converting byte to Kafka Connect data failed due to serialization error

0 votes

I tried to read JSON data from a topic. But I am getting the below error.

$ curl -s "http://localhost:8083/connectors/file_sink_01/status"| \
    jq -c -M '[.name,.tasks[].state]'
["file_sink_01","FAILED"]
org.apache.kafka.connect.errors.ConnectException: Tolerance exceeded in error handler
at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:178)
…
Caused by: org.apache.kafka.connect.errors.DataException: Converting byte[] to Kafka Connect data failed due to serialization error:
at org.apache.kafka.connect.json.JsonConverter.toConnectData(JsonConverter.java:334)
…
Caused by: org.apache.kafka.common.errors.SerializationException: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('b' (code 98)): was expecting double-quote to start field name
at [Source: (byte[])"{brokenjson-:"bar 1"}"; line: 1, column: 3]

Thank You

Apr 3, 2020 in Apache Kafka by akhtar
• 38,230 points
4,758 views

1 answer to this question.

0 votes

Hi@akhtar,

To fix the pipeline, you need to resolve the issue with the message on the source topic. Kafka Connect will not simply “skip” the bad message unless we tell it to do.

The default behavior of Kafka Connect.

errors.tolerance = none

But it can be set explicitly with the following.

errors.tolerance = all

Now try again, it will work.

Thank You

answered Apr 3, 2020 by MD
• 95,440 points

Related Questions In Apache Kafka

0 votes
1 answer

How to commit message offsets in Kafka for reliable data pipeline?

You can use auto.commit.enable to allow Kafka ...READ MORE

answered Jul 10, 2018 in Apache Kafka by Shubham
• 13,490 points
3,316 views
0 votes
1 answer

Explain to me the functionality of Kafka in a Big-Data Cluster

Let me explain to you about Apache ...READ MORE

answered Apr 30, 2019 in Apache Kafka by ravikiran
• 4,620 points
842 views
+1 vote
1 answer
0 votes
1 answer
0 votes
1 answer

Error: pass --path /path/to/confluent flag or set environment variable CONFLUENT_HOME

Hi@akhtar, Before starting confluent kafka in your system, ...READ MORE

answered Apr 3, 2020 in Apache Kafka by MD
• 95,440 points
3,427 views
0 votes
1 answer

How to read batch messages in confluent-kafka-python?

Hi@akhtar, You have to read the messages normally ...READ MORE

answered Apr 3, 2020 in Apache Kafka by MD
• 95,440 points
4,132 views
0 votes
14 answers

How can I delete all the messages from a Kafka topic?

To delete all the messages from a Kafka topic.  There are many approach ...READ MORE

answered Jun 19, 2020 in Apache Kafka by Pawan
• 380 points
226,447 views
0 votes
1 answer

How to purge Kafka Topic?

For your case what you can do ...READ MORE

answered Sep 4, 2018 in Apache Kafka by nitinrawat895
• 11,380 points
2,488 views
0 votes
1 answer

How can i connect to an api which gives me weather data to Apache kafka?

Hi, To connect an API in kafka, you ...READ MORE

answered Apr 13, 2020 in Apache Kafka by MD
• 95,440 points
1,127 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP