Change heart beat time to R process

Question

I want to change the speed of the heart beat signal that is sent from Spark to R process. I want the heartbeat signals to be sent faster. Where should I change this value of speed?

Answer 1

The heartbeat interval is assigned to the spark.heartBeatInterval property. By default, this value is set to 100. To change this, run the following command in the spark shell:

val sc = new SparkContext(new SparkConf())

./bin/spark-submit <all your existing options> --spark.r.heartBeatInterval=80