Caused by org apache kafka connect errors DataException Converting byte to Kafka Connect data failed due to serialization error

Question

I tried to read JSON data from a topic. But I am getting the below error.

$ curl -s "http://localhost:8083/connectors/file_sink_01/status"| \
    jq -c -M '[.name,.tasks[].state]'
["file_sink_01","FAILED"]
org.apache.kafka.connect.errors.ConnectException: Tolerance exceeded in error handler
at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:178)
…
Caused by: org.apache.kafka.connect.errors.DataException: Converting byte[] to Kafka Connect data failed due to serialization error:
at org.apache.kafka.connect.json.JsonConverter.toConnectData(JsonConverter.java:334)
…
Caused by: org.apache.kafka.common.errors.SerializationException: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('b' (code 98)): was expecting double-quote to start field name
at [Source: (byte[])"{brokenjson-:"bar 1"}"; line: 1, column: 3]

Thank You

Answer 1

Hi@akhtar,

To fix the pipeline, you need to resolve the issue with the message on the source topic. Kafka Connect will not simply “skip” the bad message unless we tell it to do.

The default behavior of Kafka Connect.

errors.tolerance = none

But it can be set explicitly with the following.

errors.tolerance = all

Now try again, it will work.

Thank You