Most voted questions in Apache Spark

0 votes
1 answer

How to access variables in s string interpolation in Scala?

Hey, You can use below code to access variables ...READ MORE

Jul 31 in Apache Spark by Gitika
• 25,360 points
37 views
0 votes
1 answer

How to work with Matrix Multiplication in Apache Spark?

Hey, You can follow this below solution for ...READ MORE

Jul 31 in Apache Spark by Gitika
• 25,360 points
704 views
0 votes
1 answer

Spark Error: StackOverflowError : Exception in thread "main" java.lang.StackOverflowError at org.apache.spark.rdd.UnionRDD$$anonfun$1.apply

Hey, It already has SparkContent.union and it does know how to ...READ MORE

Jul 31 in Apache Spark by Gitika
• 25,360 points
218 views
0 votes
1 answer

Spark:error:throws stack overflow when union a lot.

Hey, Use SparkContext.union(...) instead to union many RDDs at once You ...READ MORE

Jul 31 in Apache Spark by Gitika
• 25,360 points
130 views
0 votes
1 answer

What is 'TRAITS' in Scala

Hi, Traits are basically Scala's workaround for the ...READ MORE

Jul 31 in Apache Spark by Gitika
• 25,360 points
42 views
0 votes
1 answer

Scala: Loading a csv file

Refer to the below command: val input_df = ...READ MORE

Jul 31 in Apache Spark by Emma
34 views
0 votes
1 answer

What is the use of App class in Scala?

Hi, Scala provides a helper class, called App, that ...READ MORE

Jul 31 in Apache Spark by Gitika
• 25,360 points
111 views
0 votes
1 answer

What are these in scala : _* & @_*

As is widely used, and has different ...READ MORE

Jul 31 in Apache Spark by Turic
66 views
0 votes
1 answer

How to create singleton classes in Scala?

Hey, Scala introduces a new object keyword, which is used ...READ MORE

Jul 31 in Apache Spark by Gitika
• 25,360 points
47 views
0 votes
1 answer
0 votes
1 answer

Removing the header of a text file in SparkRDD

1) First we loaded the data to ...READ MORE

Jul 31 in Apache Spark by Namitha
39 views
0 votes
1 answer

How to format a string in Scala?

Hey, To format a string, use the .format ...READ MORE

Jul 30 in Apache Spark by Gitika
• 25,360 points
30 views
0 votes
1 answer

How 'Scala Higher Order function' works in Scala?

Hey, Scala allows the definition of a higher-order function. These ...READ MORE

Jul 30 in Apache Spark by Gitika
• 25,360 points
27 views
0 votes
1 answer

How to work with multidimensional arrays in Scala?

Hi, Here is an example you can follow: scala> ...READ MORE

Jul 30 in Apache Spark by Gitika
• 25,360 points
60 views
0 votes
1 answer

How to concatenate sets in Scala?

Hey, Yes, there are two ways of doing ...READ MORE

Jul 30 in Apache Spark by Gitika
• 25,360 points
37 views
0 votes
1 answer

How to find values common to two sets in Scala

Hey, There are two ways to find the ...READ MORE

Jul 30 in Apache Spark by Gitika
• 25,360 points
31 views
0 votes
1 answer

How do find Max and Min values in a set in Scala?

Hey, Here is the example of which will return ...READ MORE

Jul 30 in Apache Spark by Gitika
• 25,360 points
61 views
0 votes
1 answer

How to declare sets with value in Scala?

Hey, You can follow this code: scala> var s=Set(1,4,4,3) s: ...READ MORE

Jul 30 in Apache Spark by Gitika
• 25,360 points
35 views
0 votes
1 answer

PySpark not starting: No active sparkcontext

Seems like Spark hadoop daemons are not ...READ MORE

Jul 30 in Apache Spark by Jishan
115 views
0 votes
1 answer

Unable to use ml library in pyspark

The error message you have shared with ...READ MORE

Jul 30 in Apache Spark by Karan
27 views
0 votes
1 answer
0 votes
1 answer

Unable to run the java- spark standalone program

Though there is nothing wrong with the ...READ MORE

Jul 30 in Apache Spark by Lohit
63 views
0 votes
1 answer

Spark: Dataframe vs Dataset

Recently, there are two new data abstractions ...READ MORE

Jul 29 in Apache Spark by Jackie
195 views
0 votes
1 answer

Date formats : how to cast string to date?

Try this, it should work: > from pyspark.sql.functions ...READ MORE

Jul 29 in Apache Spark by Niall
91 views
0 votes
1 answer

How to fit multiple assignment statements into one statement in Scala?

Hi, You can assign a Tuple to a ...READ MORE

Jul 29 in Apache Spark by Gitika
• 25,360 points
20 views
0 votes
1 answer

How to search for a Key in a Scala Map?

Hey, The Map.contains() method will tell you if ...READ MORE

Jul 29 in Apache Spark by Gitika
• 25,360 points
30 views
0 votes
1 answer

How to concatenate Maps in Scala?

Hey, You can concatenate/join two Maps in more than ...READ MORE

Jul 29 in Apache Spark by Gitika
• 25,360 points

edited Jul 29 by Gitika 31 views
0 votes
1 answer

How to declare a Empty Scala Map?

Hi, You can either declare an empty Scala ...READ MORE

Jul 29 in Apache Spark by Gitika
• 25,360 points

edited Jul 29 by Gitika 30 views
0 votes
1 answer

"main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataInputStream

1. We will check whether master and ...READ MORE

Jul 29 in Apache Spark by Yogi
185 views
0 votes
1 answer

How the if-else statement is different for Scala and Java?

Hey, There is one main difference. This statement ...READ MORE

Jul 29 in Apache Spark by Gitika
• 25,360 points
27 views
0 votes
1 answer

What is the differentiate between equals() and == in Scala. Is the latter the same as == in Java?

Hey, When we try to compare two instances ...READ MORE

Jul 29 in Apache Spark by Gitika
• 25,360 points
55 views
0 votes
1 answer

Scala: 30: error: value partitions is not a member of String

Try this code: val rdd= sc.textFile (“file.txt”, 5) rdd.partitions.size Output ...READ MORE

Jul 29 in Apache Spark by Nijit
52 views
0 votes
1 answer

What is Lazy val in Scala?

Hey, Scala executes a val when we define ...READ MORE

Jul 29 in Apache Spark by Gitika
• 25,360 points
30 views
0 votes
1 answer

What is Case Class in Scala?

Hey, Scala case classes let us model immutable data. ...READ MORE

Jul 29 in Apache Spark by Gitika
• 25,360 points
28 views
0 votes
1 answer

what is an anonymous function Scala?

Hi, An anonymous function in Scala is a ...READ MORE

Jul 29 in Apache Spark by Gitika
• 25,360 points
29 views
0 votes
1 answer

Consider for-comprehensions in Scala. What are they syntactic sugars for?

Hi, A for-comprehension is one way to carry ...READ MORE

Jul 29 in Apache Spark by Gitika
• 25,360 points
14 views
0 votes
1 answer

What is Options in Scala?

Hey, A Scala Option is a kind of ...READ MORE

Jul 26 in Apache Spark by Gitika
• 25,360 points
29 views
0 votes
1 answer

What is tail-recursion in Scala?

Hey, Recursion is when a function makes a ...READ MORE

Jul 26 in Apache Spark by Gitika
• 25,360 points
34 views
0 votes
1 answer

What is the work of Yield Keyword in Scala?

Hey, Used with a loop, yield produces a value for ...READ MORE

Jul 26 in Apache Spark by Gitika
• 25,360 points
43 views
0 votes
1 answer

How to append a list in Scala?

Hey, For this purpose, we use the single ...READ MORE

Jul 26 in Apache Spark by Gitika
• 25,360 points
77 views
0 votes
1 answer

Different types of identifiers in Scala?

Hey, We have four kinds of identifiers in ...READ MORE

Jul 26 in Apache Spark by Gitika
• 25,360 points
22 views
0 votes
1 answer

How does Closure function works in Scala?

Hi, A closure in Scala is a function ...READ MORE

Jul 26 in Apache Spark by Gitika
• 25,360 points
30 views
0 votes
1 answer

What is a lens in Scala?

Hi, A lens is an abstraction from functional ...READ MORE

Jul 26 in Apache Spark by Gitika
• 25,360 points
31 views
0 votes
1 answer

What is function currying in Scala?

Hi, With Scala Currying, we can take a ...READ MORE

Jul 26 in Apache Spark by Gitika
• 25,360 points
31 views
0 votes
1 answer

How to call the Debug Mode in PySpark?

As far as I understand your intentions ...READ MORE

Jul 26 in Apache Spark by ravikiran
• 4,560 points
79 views
0 votes
1 answer

How to use nested function in Scala?

Hey, With Scala, we can define a Scala ...READ MORE

Jul 26 in Apache Spark by Gitika
• 25,360 points
67 views
0 votes
1 answer

How to use Scala anonymous functions and why do we use it?

Hi, Anonymous functions in Scala is the lightweight ...READ MORE

Jul 26 in Apache Spark by Gitika
• 25,360 points
70 views
0 votes
1 answer

Spark Submit: class does not exists

In the command, you have mentioned the ...READ MORE

Jul 26 in Apache Spark by Jimmy
45 views
0 votes
1 answer

Spark: Read from Hive, store in HDFS

Below is an example of reading data ...READ MORE

Jul 26 in Apache Spark by Lohit
60 views