How to launch spark application in cluster mode in Spark?

0 votes
Can anyone explain how to launch spark application in cluster mode in spark?
Aug 2, 2019 in Apache Spark by Raima
1,082 views

1 answer to this question.

0 votes

Hi,

To launch spark application in cluster mode, you have to use spark-submit command. You cannot run yarn-cluster mode via spark-shell because when you will run spark application, driver program will be running as part application master container/process. So it is not possible to run cluster mode via spark-shell:

spark-submit –class com.df.SparkWordCount SparkWC.jar yarn-client
-> spark-submit –class com.df.SparkWordCount SparkWC.jar yarn-cluster
answered Aug 2, 2019 by Gitika
• 26,390 points

Related Questions In Apache Spark

0 votes
1 answer

How to change scheduling mode in Spark?

You can change the scheduling mode as ...READ MORE

answered Mar 11, 2019 in Apache Spark by Raj
440 views
0 votes
1 answer

How to increase worker timeout in Spark application?

By default, the timeout is set to ...READ MORE

answered Mar 25, 2019 in Apache Spark by Hari
1,381 views
0 votes
1 answer

How to use ftp scheme using Yarn in Spark application?

In case Yarn does not support schemes ...READ MORE

answered Mar 28, 2019 in Apache Spark by Raj
209 views
0 votes
4 answers

How to change the spark Session configuration in Pyspark?

You can dynamically load properties. First create ...READ MORE

answered Dec 10, 2018 in Apache Spark by Vini
25,465 views
+1 vote
2 answers
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,870 points
4,580 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,870 points
651 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
25,983 views
0 votes
1 answer

How to run spark in Standalone client mode?

Hi, These are the steps to run spark in ...READ MORE

answered Jul 5, 2019 in Apache Spark by Gitika
• 26,390 points
177 views
0 votes
1 answer

How to check if a particular keyword exists in Apache Spark?

Hey, You can try this code to get ...READ MORE

answered Jul 22, 2019 in Apache Spark by Gitika
• 26,390 points
255 views