How can I run a blacklisted task

Question

Hi. One of my tasks has been blacklisted because it caused too many task failures. Now I want to run this task but Spark is stoppoing it from running. How can I run it?

Answer 1

When a task results in too many failures, it is blacklisted and spark prevents it from running again. To run this task, you can disable the property that prevents spark from running the blacklisted tasks. Something like this:

val sc = new SparkContext(new SparkConf())

./bin/spark-submit <all your existing options> --spark.blacklist.enabled=false