Trending questions in Apache Spark

0 votes
1 answer

What is the differentiate between equals() and == in Scala. Is the latter the same as == in Java?

Hey, When we try to compare two instances ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
• 65,910 points
1,029 views
0 votes
1 answer

How to create scala project in intellij?

You have to install Intellij with scala plugin. ...READ MORE

Jul 5, 2019 in Apache Spark by Jimmy
2,034 views
0 votes
1 answer

What is “Unit” and “()” in Scala?

Hey, Unit is a subtype of scala.anyval and ...READ MORE

Jul 24, 2019 in Apache Spark by Gitika
• 65,910 points
1,200 views
0 votes
1 answer

What is Lazy val in Scala?

Hey, Scala executes a val when we define ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
• 65,910 points
974 views
0 votes
1 answer

How to concatinating Arrays in scala?

Hey, We can append a Scala array to ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
• 65,910 points
1,127 views
0 votes
1 answer

How do I access the Map Task ID in Spark?

You can access task information using TaskContext: import org.apache.spark.TaskContext sc.parallelize(Seq[Int](), ...READ MORE

Jul 23, 2019 in Apache Spark by ravikiran
• 4,620 points
1,205 views
0 votes
1 answer

How to access variables in s string interpolation in Scala?

Hey, You can use below code to access variables ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
• 65,910 points
861 views
0 votes
1 answer

How to reverse a Scala list?

Hi, This reverses the order of elements in ...READ MORE

Aug 1, 2019 in Apache Spark by Gitika
• 65,910 points
838 views
0 votes
1 answer

Scala: Loading a csv file

Refer to the below command: val input_df = ...READ MORE

Jul 31, 2019 in Apache Spark by Emma
842 views
0 votes
1 answer

How to start spark history server?

Hi, You can use this command to start ...READ MORE

Aug 6, 2019 in Apache Spark by Gitika
• 65,910 points
568 views
0 votes
1 answer

How to find the number of null contain in dataframe?

Hey there! You can use the select method of the ...READ MORE

May 3, 2019 in Apache Spark by Omkar
• 69,210 points
4,620 views
0 votes
1 answer

What is polyglot in spark?

Hi, Spark provides a high-level API in Java, ...READ MORE

Jul 1, 2019 in Apache Spark by Gitika
• 65,910 points
2,056 views
0 votes
1 answer

How the if-else statement is different for Scala and Java?

Hey, There is one main difference. This statement ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
• 65,910 points
845 views
0 votes
0 answers

Why doesn't my Spark Yarn client runs on all available worker machines?

I am running an application on Spark ...READ MORE

Feb 22, 2019 in Apache Spark by Uzair Ahmad

edited Feb 22, 2019 by Omkar 7,654 views
0 votes
1 answer

what is an anonymous function Scala?

Hi, An anonymous function in Scala is a ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
• 65,910 points
820 views
0 votes
1 answer

How to search for a Key in a Scala Map?

Hey, The Map.contains() method will tell you if ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
• 65,910 points
800 views
0 votes
1 answer

Different types of identifiers in Scala?

Hey, We have four kinds of identifiers in ...READ MORE

Jul 26, 2019 in Apache Spark by Gitika
• 65,910 points
928 views
0 votes
1 answer

What is Case Class in Scala?

Hey, Scala case classes let us model immutable data. ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
• 65,910 points
782 views
0 votes
1 answer

Scala: when to use x(2) and x._2?

In the above statement, x(2) is specifying an array ...READ MORE

Jul 22, 2019 in Apache Spark by Yogi
1,078 views
0 votes
1 answer

Copy all files from local (Windows) to HDFS with Scala code

Please try the following Scala code: import org.apache.hadoop.conf.Configuration import ...READ MORE

May 22, 2019 in Apache Spark by Karan
3,696 views
0 votes
1 answer

What is Map and flatMap in Spark?

Hi, The map is a specific line or ...READ MORE

Jul 3, 2019 in Apache Spark by Gitika
• 65,910 points
1,867 views
0 votes
1 answer

How to work with multidimensional arrays in Scala?

Hi, Here is an example you can follow: scala> ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
• 65,910 points
648 views
0 votes
1 answer

What is a lens in Scala?

Hi, A lens is an abstraction from functional ...READ MORE

Jul 26, 2019 in Apache Spark by Gitika
• 65,910 points
815 views
0 votes
1 answer

What is 'TRAITS' in Scala

Hi, Traits are basically Scala's workaround for the ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
• 65,910 points
586 views
0 votes
1 answer

Code to compute average in Apache Spark?

Hi, You can compute the average using this ...READ MORE

Jul 22, 2019 in Apache Spark by Gitika
• 65,910 points
956 views
0 votes
1 answer

How to fit multiple assignment statements into one statement in Scala?

Hi, You can assign a Tuple to a ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
• 65,910 points
651 views
0 votes
1 answer

How to use nested function in Scala?

Hey, With Scala, we can define a Scala ...READ MORE

Jul 26, 2019 in Apache Spark by Gitika
• 65,910 points

edited Jun 1, 2023 by Srinath 778 views
0 votes
1 answer

How to use yield keyword in scala and why it is used instead of println?

Hi, The yield keyword is used because the ...READ MORE

Jul 6, 2019 in Apache Spark by Gitika
• 65,910 points
1,679 views
0 votes
1 answer

What is Hive on Spark?

Hi, Hive contains significant support for Apache Spark, ...READ MORE

Aug 2, 2019 in Apache Spark by Gitika
• 65,910 points
477 views
0 votes
1 answer

Load CSV file data inot MySQL

If you are trying to load file ...READ MORE

Jul 25, 2019 in Apache Spark by Sita
820 views
0 votes
1 answer

Pyspark is taking default path

The HDFS path for MyLab is /user/edureka_id. ...READ MORE

Jul 16, 2019 in Apache Spark by Khushi
1,195 views
0 votes
1 answer

How to format a string in Scala?

Hey, To format a string, use the .format ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
• 65,910 points
562 views
0 votes
1 answer

How 'Scala Higher Order function' works in Scala?

Hey, Scala allows the definition of a higher-order function. These ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
• 65,910 points
559 views
0 votes
1 answer

Spark memory processing on a not temporary table

Temporary table is more like an index ...READ MORE

Jul 14, 2019 in Apache Spark by Suri
1,280 views
0 votes
1 answer

Which Scala library is used for functional programming?

Hi, Scala library has purely functional data structures ...READ MORE

Jul 24, 2019 in Apache Spark by Gitika
• 65,910 points
779 views
0 votes
1 answer

How shallow copy carry out using Scala?

Hey, Scala uses the method copy() to carry ...READ MORE

Aug 1, 2019 in Apache Spark by Gitika
• 65,910 points
464 views
0 votes
1 answer

How to use Scala anonymous functions and why do we use it?

Hi, Anonymous functions in Scala is the lightweight ...READ MORE

Jul 26, 2019 in Apache Spark by Gitika
• 65,910 points
642 views
0 votes
1 answer

What is recursion in Scala function? How does it works?

Hey, A Scala function involves recursion when it ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
• 65,910 points
685 views
0 votes
1 answer

Query regarding Operator Overloading in Scala

All prefix operators' symbols are predefined: +, -, ...READ MORE

Jul 10, 2019 in Apache Spark by Karan
1,333 views
0 votes
0 answers

Load data to mysql from local

I'm trying to load data to mysql ...READ MORE

Jul 23, 2019 in Apache Spark by Yogi
814 views
0 votes
1 answer

How to create Scala function using "Def" keyword?

Hi, To create a Scala function, we use ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
• 65,910 points
677 views
0 votes
1 answer

Which syntax to use to take the sum of list of collection in scala?

Hi, You can see this example to get ...READ MORE

Jul 5, 2019 in Apache Spark by Gitika
• 65,910 points
1,519 views
0 votes
1 answer

What is function currying in Scala?

Hi, With Scala Currying, we can take a ...READ MORE

Jul 26, 2019 in Apache Spark by Gitika
• 65,910 points
613 views
0 votes
1 answer

Spark foldbykey doubt

Please have a look below for your ...READ MORE

Jun 19, 2019 in Apache Spark by Tina
2,210 views
0 votes
1 answer

What is the work of Yield Keyword in Scala?

Hey, Used with a loop, yield produces a value for ...READ MORE

Jul 26, 2019 in Apache Spark by Gitika
• 65,910 points
577 views
0 votes
1 answer

What is exception propagation in Scala?

Hi, When a function experiences an exception, it ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
• 65,910 points
625 views
0 votes
1 answer

What is tail-recursion in Scala?

Hey, Recursion is when a function makes a ...READ MORE

Jul 26, 2019 in Apache Spark by Gitika
• 65,910 points
560 views
0 votes
1 answer

How to declare sets with value in Scala?

Hey, You can follow this code: scala> var s=Set(1,4,4,3) s: ...READ MORE

Jul 30, 2019 in Apache Spark by Gitika
• 65,910 points
374 views
0 votes
0 answers

How to define SparkConf?

Can anyone explain how to define SparkConf? READ MORE

Aug 1, 2019 in Apache Spark by Danish
315 views