How to connect to Zookeeper after setting Spark recovery mode

Question

I am facing some problem with Spark application. I have enabled Spark recovery and set the mode to Zookeeper. But I don't know why Spark is not getting connected to Zookeeper for connection. How to make the connection?

Answer 1

You have set Zookeeper as the recovery mode but I think you forgot to set the URL for Zookeeper to connect to. When you set Zookeeper for recovery, you also need to mention the Zookeeper's URL that the Spark application should connect to when it needs to make the recovery. Use the below command and replace <URL> with your Zookeeper's URL:

val sc = new SparkContext(new SparkConf())

./bin/spark-submit <all your existing options> --spark.deploy.zookeeper.url=<URL>