Most voted questions in Apache Spark

0 votes
1 answer

How Foreach Operation works in Apache Spark?

Hi, foreach() operation is an action. It does not ...READ MORE

Aug 2, 2019 in Apache Spark by Gitika
 • 65,910 points • 6,016 views
0 votes
1 answer

How SortBykey() operation works in Spark?

Hey, sortByKey() is a transformation. It returns an RDD sorted ...READ MORE

Aug 2, 2019 in Apache Spark by Gitika
 • 65,910 points • 5,818 views
0 votes
1 answer

In how many modes Apache spark can run?

Hey, You can launch spark application in four ...READ MORE

Aug 2, 2019 in Apache Spark by Gitika
 • 65,910 points • 1,331 views
0 votes
1 answer

How to launch spark application in cluster mode in Spark?

Hi, To launch spark application in cluster mode, ...READ MORE

Aug 2, 2019 in Apache Spark by Gitika
 • 65,910 points • 4,728 views
0 votes
1 answer

How to create paired RDD using subString method in Spark?

Hi, If you have a file with id ...READ MORE

Aug 2, 2019 in Apache Spark by Gitika
 • 65,910 points • 2,325 views
0 votes
1 answer

what is Paired RDD and how to create paired RDD in Spark?

Hi, Paired RDD is a distributed collection of ...READ MORE

Aug 2, 2019 in Apache Spark by Gitika
 • 65,910 points • 9,045 views
0 votes
1 answer

Join in RDD using keys

Suppose you have two dataset results( id, ...READ MORE

Aug 2, 2019 in Apache Spark by Trisha
 • 7,949 views
0 votes
1 answer

Scala: save filtered data row by row using saveAsTextFile

Try this code, it worked for me: val ...READ MORE

Aug 2, 2019 in Apache Spark by Karan
 • 1,614 views
0 votes
1 answer

What is Hive on Spark?

Hi, Hive contains significant support for Apache Spark, ...READ MORE

Aug 2, 2019 in Apache Spark by Gitika
 • 65,910 points • 486 views
0 votes
1 answer

Can anyone explain the sparse vector in Spark?

Hey, A sparse vector is used for storing ...READ MORE

Aug 2, 2019 in Apache Spark by Gitika
 • 65,910 points • 5,288 views
0 votes
0 answers

How to define SparkConf?

Can anyone explain how to define SparkConf? READ MORE

Aug 1, 2019 in Apache Spark by Danish
 • 322 views
0 votes
1 answer

Scala - Error in Inheritance: <console>:: error: not found: value

You need to declare the variable which ...READ MORE

Aug 1, 2019 in Apache Spark by Karan
 • 4,420 views
0 votes
1 answer

Pyspark dataframe with random values

Hey @Esha, you can use this code. ...READ MORE

Aug 1, 2019 in Apache Spark by Zed
 • 8,515 views
0 votes
1 answer

Spark + Hive connectivity

The problem is probably with the command. ...READ MORE

Aug 1, 2019 in Apache Spark by Rishni
 • 1,918 views
0 votes
1 answer

How to reverse a Scala list?

Hi, This reverses the order of elements in ...READ MORE

Aug 1, 2019 in Apache Spark by Gitika
 • 65,910 points • 853 views
0 votes
1 answer

How to use uniform list in Scala?

Hey, The method List.fill() creates a list and ...READ MORE

Aug 1, 2019 in Apache Spark by Gitika
 • 65,910 points • 1,253 views
0 votes
1 answer

How shallow copy carry out using Scala?

Hey, Scala uses the method copy() to carry ...READ MORE

Aug 1, 2019 in Apache Spark by Gitika
 • 65,910 points • 492 views
0 votes
1 answer

How to access variables in s string interpolation in Scala?

Hey, You can use below code to access variables ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
 • 65,910 points • 881 views
0 votes
1 answer

How to work with Matrix Multiplication in Apache Spark?

Hey, You can follow this below solution for ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
 • 65,910 points • 7,304 views
0 votes
1 answer

Spark Error: StackOverflowError : Exception in thread "main" java.lang.StackOverflowError at org.apache.spark.rdd.UnionRDD$$anonfun$1.apply

Hey, It already has SparkContent.union and it does know how to ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
 • 65,910 points • 4,118 views
0 votes
1 answer

Spark:error:throws stack overflow when union a lot.

Hey, Use SparkContext.union(...) instead to union many RDDs at once You ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
 • 65,910 points • 4,476 views
0 votes
1 answer

What is 'TRAITS' in Scala

Hi, Traits are basically Scala's workaround for the ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
 • 65,910 points • 593 views
0 votes
1 answer

Scala: Loading a csv file

Refer to the below command: val input_df = ...READ MORE

Jul 31, 2019 in Apache Spark by Emma
 • 850 views
0 votes
1 answer

What is the use of App class in Scala?

Hi, Scala provides a helper class, called App, that ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
 • 65,910 points • 10,766 views
0 votes
1 answer

What are these in scala : _* & @_*

As is widely used, and has different ...READ MORE

Jul 31, 2019 in Apache Spark by Turic
 • 2,306 views
0 votes
1 answer

How to create singleton classes in Scala?

Hey, Scala introduces a new object keyword, which is used ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
 • 65,910 points • 1,566 views
0 votes
1 answer

Cannot load file to spark: "org.apache.spark.sql.AnalysisException: Path does not exist"

Since the file is in HDFS so ...READ MORE

Jul 31, 2019 in Apache Spark by Tina
 • 10,910 views
0 votes
1 answer

Removing the header of a text file in SparkRDD

1) First we loaded the data to ...READ MORE

Jul 31, 2019 in Apache Spark by Namitha
 • 6,190 views
0 votes
1 answer

How to format a string in Scala?

Hey, To format a string, use the .format ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
 • 65,910 points • 576 views
0 votes
1 answer

How 'Scala Higher Order function' works in Scala?

Hey, Scala allows the definition of a higher-order function. These ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
 • 65,910 points • 571 views
0 votes
1 answer

How to work with multidimensional arrays in Scala?

Hi, Here is an example you can follow: scala> ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
 • 65,910 points • 684 views
0 votes
1 answer

How to concatenate sets in Scala?

Hey, Yes, there are two ways of doing ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
 • 65,910 points • 1,282 views
0 votes
1 answer

How to find values common to two sets in Scala

Hey, There are two ways to find the ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
 • 65,910 points • 1,401 views
0 votes
1 answer

How do find Max and Min values in a set in Scala?

Hey, Here is the example of which will return ...READ MORE

Jul 30, 2019 in Apache Spark by Gitika
 • 65,910 points • 3,888 views
0 votes
1 answer

How to declare sets with value in Scala?

Hey, You can follow this code: scala> var s=Set(1,4,4,3) s: ...READ MORE

Jul 30, 2019 in Apache Spark by Gitika
 • 65,910 points • 383 views
0 votes
1 answer

PySpark not starting: No active sparkcontext

Seems like Spark hadoop daemons are not ...READ MORE

Jul 30, 2019 in Apache Spark by Jishan
 • 4,111 views
0 votes
1 answer

Unable to use ml library in pyspark

The error message you have shared with ...READ MORE

Jul 30, 2019 in Apache Spark by Karan
 • 2,483 views
0 votes
1 answer

Scala: org.apache.poi.openxml4j.exceptions.InvalidFormatException: Your InputStream was neither an OLE2 stream, nor an OOXML stream

Try executing the below code, def readExcel(file: String): ...READ MORE

Jul 30, 2019 in Apache Spark by Raman
 • 1,461 views
0 votes
1 answer

Unable to run the java- spark standalone program

Though there is nothing wrong with the ...READ MORE

Jul 30, 2019 in Apache Spark by Lohit
 • 1,112 views
0 votes
1 answer

Date formats : how to cast string to date?

Try this, it should work: > from pyspark.sql.functions ...READ MORE

Jul 29, 2019 in Apache Spark by Niall
 • 5,715 views
0 votes
1 answer

How to fit multiple assignment statements into one statement in Scala?

Hi, You can assign a Tuple to a ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
 • 65,910 points • 663 views
0 votes
1 answer

How to search for a Key in a Scala Map?

Hey, The Map.contains() method will tell you if ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
 • 65,910 points • 839 views
0 votes
1 answer

How to concatenate Maps in Scala?

Hey, You can concatenate/join two Maps in more than ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
 • 65,910 points
edited Jul 29, 2019 by Gitika • 2,469 views
0 votes
1 answer

How to declare a Empty Scala Map?

Hi, You can either declare an empty Scala ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
 • 65,910 points
edited Jul 29, 2019 by Gitika • 1,845 views
0 votes
1 answer

"main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataInputStream

1. We will check whether master and ...READ MORE

Jul 29, 2019 in Apache Spark by Yogi
 • 5,874 views
0 votes
1 answer

How the if-else statement is different for Scala and Java?

Hey, There is one main difference. This statement ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
 • 65,910 points • 858 views
0 votes
1 answer

What is the differentiate between equals() and == in Scala. Is the latter the same as == in Java?

Hey, When we try to compare two instances ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
 • 65,910 points • 1,053 views
0 votes
1 answer

Scala: 30: error: value partitions is not a member of String

Try this code: val rdd= sc.textFile (“file.txt”, 5) rdd.partitions.size Output ...READ MORE

Jul 29, 2019 in Apache Spark by Nijit
 • 2,792 views
0 votes
1 answer

What is Lazy val in Scala?

Hey, Scala executes a val when we define ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
 • 65,910 points • 984 views
0 votes
1 answer

What is Case Class in Scala?

Hey, Scala case classes let us model immutable data. ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
 • 65,910 points • 809 views
Page:

Most popular tags

"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.