Difference between RDD as val and var

0 votes

RDD is considered immutable. I tried to create an RDD with val and var like given below. I can see I was able to change RDD definition created using var. If it"s immutable why was I able to use var to create an RDD?

scala> var df = sc.textFile("/user/saifu/problem5/text")
df: org.apache.spark.rdd.RDD[String] = /user/saifu/problem5/text MapPartitionsRDD[13] at textFile at :27

scala> val db = sc.textFile("/user/saifu/problem5/text")
db: org.apache.spark.rdd.RDD[String] = /user/saifu/problem5/text MapPartitionsRDD[15] at textFile at :27

scala> df = sc.textFile("/user/sarfu/problem5/text-uncompress")
df: org.apache.spark.rdd.RDD[String] = /user/sarfu/problem5/text-uncompress MapPartitionsRDD[17] at textFile at :29
May 23 in Apache Spark by SHeen
140 views

1 answer to this question.

0 votes

Variable declaration can be done in two ways

1. val --> immutable variable

Refer to the screenshot below:

image

2. var --> mutable variable

Refer to the screenshot below:

image

I hope this helps.

answered May 23 by Arun

Related Questions In Apache Spark

0 votes
1 answer

Difference between createOrReplaceTempView and registerTempTable

createOrReplaceTempView() creates/replaces a local temp view with the dataframe provided. Lifetime of this ...READ MORE

answered Apr 25, 2018 in Apache Spark by kurt_cobain
• 9,280 points
2,505 views
0 votes
1 answer

What's the difference between 'filter' and 'where' in Spark SQL?

Both 'filter' and 'where' in Spark SQL ...READ MORE

answered May 23, 2018 in Apache Spark by nitinrawat895
• 10,760 points
8,216 views
0 votes
1 answer

What is the difference between persist() and cache() in apache spark?

Hi, persist () allows the user to specify ...READ MORE

answered Jul 3 in Apache Spark by Gitika
• 25,420 points
526 views
0 votes
1 answer

Difference between cogroup and full outer join in spark

Please go through the below explanation : Full ...READ MORE

answered Jul 13 in Apache Spark by Kiran
1,164 views
+1 vote
1 answer
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,760 points
3,515 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,760 points
431 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
17,842 views
+1 vote
3 answers

What is the difference between rdd and dataframes in Apache Spark ?

Comparison between Spark RDD vs DataFrame 1. Release ...READ MORE

answered Aug 27, 2018 in Apache Spark by shams
• 3,580 points
17,823 views
0 votes
1 answer