Caused by: org.apache.kafka.connect.errors.DataException: Converting byte[] to Kafka Connect data failed due to serialization error:

0 votes

I tried to read JSON data from a topic. But I am getting the below error.

$ curl -s "http://localhost:8083/connectors/file_sink_01/status"| \
    jq -c -M '[.name,.tasks[].state]'
["file_sink_01","FAILED"]
org.apache.kafka.connect.errors.ConnectException: Tolerance exceeded in error handler
at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:178)
…
Caused by: org.apache.kafka.connect.errors.DataException: Converting byte[] to Kafka Connect data failed due to serialization error:
at org.apache.kafka.connect.json.JsonConverter.toConnectData(JsonConverter.java:334)
…
Caused by: org.apache.kafka.common.errors.SerializationException: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('b' (code 98)): was expecting double-quote to start field name
at [Source: (byte[])"{brokenjson-:"bar 1"}"; line: 1, column: 3]

Thank You

Apr 3 in Apache Kafka by akhtar
• 10,580 points
125 views

1 answer to this question.

0 votes

Hi@akhtar,

To fix the pipeline, you need to resolve the issue with the message on the source topic. Kafka Connect will not simply “skip” the bad message unless we tell it to do.

The default behavior of Kafka Connect.

errors.tolerance = none

But it can be set explicitly with the following.

errors.tolerance = all

Now try again, it will work.

Thank You

answered Apr 3 by MD
• 23,050 points

Related Questions In Apache Kafka

0 votes
1 answer

How to commit message offsets in Kafka for reliable data pipeline?

You can use auto.commit.enable to allow Kafka ...READ MORE

answered Jul 9, 2018 in Apache Kafka by Shubham
• 13,380 points
787 views
0 votes
1 answer

Explain to me the functionality of Kafka in a Big-Data Cluster

Let me explain to you about Apache ...READ MORE

answered Apr 30, 2019 in Apache Kafka by ravikiran
• 4,600 points
236 views
0 votes
1 answer

Error: pass --path /path/to/confluent flag or set environment variable CONFLUENT_HOME

Hi@akhtar, Before starting confluent kafka in your system, ...READ MORE

answered Apr 3 in Apache Kafka by MD
• 23,050 points
106 views
0 votes
1 answer

How to read batch messages in confluent-kafka-python?

Hi@akhtar, You have to read the messages normally ...READ MORE

answered Apr 3 in Apache Kafka by MD
• 23,050 points
89 views
+1 vote
12 answers

How can I delete all the messages from a Kafka topic?

As of kafka 2.3.0 version, there is ...READ MORE

answered Sep 8, 2019 in Apache Kafka by Brajkishore
• 240 points
87,634 views
0 votes
1 answer

How to purge Kafka Topic?

For your case what you can do ...READ MORE

answered Sep 4, 2018 in Apache Kafka by nitinrawat895
• 10,920 points
967 views
0 votes
1 answer

How can i connect to an api which gives me weather data to Apache kafka?

Hi, To connect an API in kafka, you ...READ MORE

answered Apr 13 in Apache Kafka by MD
• 23,050 points
67 views