Spark Installation problem

0 votes

I've downloaded Spark, and unzipped and used the cd command to bin directory but it's showing an error on invoking.

[edureka@localhost ~]$ cd /home/edureka/Downloads/spark-2.4.0-bin-hadoop2.7/bin/[edureka@localhost bin]$ spark-shell
bash: spark-shell: command not found
Jul 5, 2019 in Apache Spark by Jin
4,586 views

1 answer to this question.

0 votes

After downloading Spark, you need to set the path for SPARK_HOME in the .bashrc file. Execute the below command to open the .bashrc file,

vi ~/.bashrc

After this hit the insert button on your keyboard to edit the .bashrc file. Now, set the SPARK_HOME path as below,

SPARK_HOME=<enter the spark directory path here> 
export PATH=$SPARK_HOME/bin:$PATH

After this, source the .bashrc file with the below command,

source ~/.bashrc

Now, everything is set up. You can run the spark shell now.

answered Jul 5, 2019 by Rishi

Related Questions In Apache Spark

0 votes
1 answer

Changing Column position in spark dataframe

Yes, you can reorder the dataframe elements. You need ...READ MORE

answered Apr 19, 2018 in Apache Spark by Ashish
• 2,650 points
13,708 views
0 votes
1 answer

Is there any way to check the Spark version?

There are 2 ways to check the ...READ MORE

answered Apr 19, 2018 in Apache Spark by nitinrawat895
• 11,380 points
8,489 views
+1 vote
2 answers

Hadoop 3 compatibility with older versions of Hive, Pig, Sqoop and Spark

Hadoop 3 is not widely used in ...READ MORE

answered Apr 20, 2018 in Apache Spark by kurt_cobain
• 9,390 points
5,864 views
0 votes
1 answer

Efficient way to read specific columns from parquet file in spark

As parquet is a column based storage ...READ MORE

answered Apr 20, 2018 in Apache Spark by kurt_cobain
• 9,390 points
7,804 views
+1 vote
2 answers
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
10,979 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
2,497 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
108,548 views
+5 votes
11 answers

Concatenate columns in apache spark dataframe

its late but this how you can ...READ MORE

answered Mar 21, 2019 in Apache Spark by anonymous
72,294 views
+1 vote
1 answer

How can I write a text file in HDFS not from an RDD, in Spark program?

Yes, you can go ahead and write ...READ MORE

answered May 29, 2018 in Apache Spark by Shubham
• 13,490 points
8,413 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP