Latest questions in Apache Spark

0 votes
1 answer

How to work with multidimensional arrays in Scala?

Hi, Here is an example you can follow: scala> ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
 • 65,910 points • 676 views
0 votes
1 answer

How to concatenate sets in Scala?

Hey, Yes, there are two ways of doing ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
 • 65,910 points • 1,271 views
0 votes
1 answer

How to find values common to two sets in Scala

Hey, There are two ways to find the ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
 • 65,910 points • 1,392 views
0 votes
1 answer

How do find Max and Min values in a set in Scala?

Hey, Here is the example of which will return ...READ MORE

Jul 30, 2019 in Apache Spark by Gitika
 • 65,910 points • 3,884 views
0 votes
1 answer

How to declare sets with value in Scala?

Hey, You can follow this code: scala> var s=Set(1,4,4,3) s: ...READ MORE

Jul 30, 2019 in Apache Spark by Gitika
 • 65,910 points • 381 views
0 votes
1 answer

PySpark not starting: No active sparkcontext

Seems like Spark hadoop daemons are not ...READ MORE

Jul 30, 2019 in Apache Spark by Jishan
 • 4,102 views
0 votes
1 answer

Unable to use ml library in pyspark

The error message you have shared with ...READ MORE

Jul 30, 2019 in Apache Spark by Karan
 • 2,478 views
0 votes
1 answer

Scala: org.apache.poi.openxml4j.exceptions.InvalidFormatException: Your InputStream was neither an OLE2 stream, nor an OOXML stream

Try executing the below code, def readExcel(file: String): ...READ MORE

Jul 30, 2019 in Apache Spark by Raman
 • 1,458 views
0 votes
1 answer

Unable to run the java- spark standalone program

Though there is nothing wrong with the ...READ MORE

Jul 30, 2019 in Apache Spark by Lohit
 • 1,111 views
+1 vote
2 answers

Spark: Dataframe vs Dataset

Recently, there are two new data abstractions ...READ MORE

Jul 29, 2019 in Apache Spark by Jackie
 • 45,144 views
0 votes
1 answer

Date formats : how to cast string to date?

Try this, it should work: > from pyspark.sql.functions ...READ MORE

Jul 29, 2019 in Apache Spark by Niall
 • 5,713 views
0 votes
1 answer

How to fit multiple assignment statements into one statement in Scala?

Hi, You can assign a Tuple to a ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
 • 65,910 points • 658 views
0 votes
1 answer

How to search for a Key in a Scala Map?

Hey, The Map.contains() method will tell you if ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
 • 65,910 points • 823 views
0 votes
1 answer

How to concatenate Maps in Scala?

Hey, You can concatenate/join two Maps in more than ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
 • 65,910 points
edited Jul 29, 2019 by Gitika • 2,456 views
0 votes
1 answer

How to declare a Empty Scala Map?

Hi, You can either declare an empty Scala ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
 • 65,910 points
edited Jul 29, 2019 by Gitika • 1,836 views
0 votes
1 answer

"main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataInputStream

1. We will check whether master and ...READ MORE

Jul 29, 2019 in Apache Spark by Yogi
 • 5,866 views
0 votes
1 answer

How the if-else statement is different for Scala and Java?

Hey, There is one main difference. This statement ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
 • 65,910 points • 855 views
0 votes
1 answer

What is the differentiate between equals() and == in Scala. Is the latter the same as == in Java?

Hey, When we try to compare two instances ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
 • 65,910 points • 1,048 views
0 votes
1 answer

Scala: 30: error: value partitions is not a member of String

Try this code: val rdd= sc.textFile (“file.txt”, 5) rdd.partitions.size Output ...READ MORE

Jul 29, 2019 in Apache Spark by Nijit
 • 2,783 views
0 votes
1 answer

What is Lazy val in Scala?

Hey, Scala executes a val when we define ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
 • 65,910 points • 983 views
0 votes
1 answer

What is Case Class in Scala?

Hey, Scala case classes let us model immutable data. ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
 • 65,910 points • 803 views
0 votes
1 answer

what is an anonymous function Scala?

Hi, An anonymous function in Scala is a ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
 • 65,910 points • 850 views
0 votes
1 answer

Consider for-comprehensions in Scala. What are they syntactic sugars for?

Hi, A for-comprehension is one way to carry ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
 • 65,910 points • 393 views
0 votes
1 answer

What is Options in Scala?

Hey, A Scala Option is a kind of ...READ MORE

Jul 26, 2019 in Apache Spark by Gitika
 • 65,910 points • 397 views
0 votes
1 answer

What is tail-recursion in Scala?

Hey, Recursion is when a function makes a ...READ MORE

Jul 26, 2019 in Apache Spark by Gitika
 • 65,910 points • 569 views
0 votes
1 answer

What is the work of Yield Keyword in Scala?

Hey, Used with a loop, yield produces a value for ...READ MORE

Jul 26, 2019 in Apache Spark by Gitika
 • 65,910 points • 584 views
0 votes
1 answer

How to append a list in Scala?

Hey, For this purpose, we use the single ...READ MORE

Jul 26, 2019 in Apache Spark by Gitika
 • 65,910 points • 5,432 views
0 votes
1 answer

Different types of identifiers in Scala?

Hey, We have four kinds of identifiers in ...READ MORE

Jul 26, 2019 in Apache Spark by Gitika
 • 65,910 points • 939 views
0 votes
1 answer

How does Closure function works in Scala?

Hi, A closure in Scala is a function ...READ MORE

Jul 26, 2019 in Apache Spark by Gitika
 • 65,910 points • 448 views
0 votes
1 answer

What is a lens in Scala?

Hi, A lens is an abstraction from functional ...READ MORE

Jul 26, 2019 in Apache Spark by Gitika
 • 65,910 points • 836 views
0 votes
1 answer

What is function currying in Scala?

Hi, With Scala Currying, we can take a ...READ MORE

Jul 26, 2019 in Apache Spark by Gitika
 • 65,910 points • 619 views
0 votes
1 answer

How to call the Debug Mode in PySpark?

As far as I understand your intentions ...READ MORE

Jul 26, 2019 in Apache Spark by ravikiran
 • 4,620 points • 5,553 views
0 votes
1 answer

How to use nested function in Scala?

Hey, With Scala, we can define a Scala ...READ MORE

Jul 26, 2019 in Apache Spark by Gitika
 • 65,910 points
edited Jun 1, 2023 by Srinath • 786 views
0 votes
1 answer

How to use Scala anonymous functions and why do we use it?

Hi, Anonymous functions in Scala is the lightweight ...READ MORE

Jul 26, 2019 in Apache Spark by Gitika
 • 65,910 points • 646 views
0 votes
1 answer

Spark Submit: class does not exists

In the command, you have mentioned the ...READ MORE

Jul 26, 2019 in Apache Spark by Jimmy
 • 1,435 views
0 votes
1 answer

Spark: Read from Hive, store in HDFS

Below is an example of reading data ...READ MORE

Jul 26, 2019 in Apache Spark by Lohit
 • 2,475 views
–1 vote
0 answers

what is the benefit of repartition(1) and coalesce(1). When we save data we use df.repartition(1).so how many partition it will create

what is the benefit of repartition(1) and ...READ MORE

Jul 26, 2019 in Apache Spark by kumar Ma
 • 5,022 views
0 votes
1 answer

How to create Scala Array with Range?

Hey, The range() method will give us integers ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
 • 65,910 points • 552 views
0 votes
1 answer

How to concatinating Arrays in scala?

Hey, We can append a Scala array to ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
 • 65,910 points • 1,151 views
0 votes
1 answer

How to process an Array in Scala?

Hi, We can use loop control structures to ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
 • 65,910 points • 424 views
0 votes
1 answer

How to declare Scala array?

Hi, We can declare Scala arrays in two ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
 • 65,910 points • 533 views
0 votes
1 answer

At which point of time Scala will take default arguments?

Hi, If we omit an argument in a ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
 • 65,910 points • 385 views
0 votes
1 answer

What is recursion in Scala function? How does it works?

Hey, A Scala function involves recursion when it ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
 • 65,910 points • 699 views
0 votes
1 answer

How to create Scala function using "Def" keyword?

Hi, To create a Scala function, we use ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
 • 65,910 points • 684 views
0 votes
1 answer

What is Streams in Scala?

Hi, A stream is a lazy list as ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
 • 65,910 points • 392 views
0 votes
1 answer

What is BitSet in Scala?

Hey, A BitSet is a set of non-negative ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
 • 65,910 points • 543 views
0 votes
1 answer

What is exception propagation in Scala?

Hi, When a function experiences an exception, it ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
 • 65,910 points • 634 views
0 votes
1 answer

Read multiple xml files in Spark

You can do this using globbing. See ...READ MORE

Jul 25, 2019 in Apache Spark by Jack
 • 5,302 views
0 votes
1 answer

RDD word count with line numbers

df = spark.createDataFrame([("A", 2000), ("A", 2002), ("A", ...READ MORE

Jul 25, 2019 in Apache Spark by Siri
 • 2,327 views
0 votes
1 answer

Load CSV file data inot MySQL

If you are trying to load file ...READ MORE

Jul 25, 2019 in Apache Spark by Sita
 • 838 views
Page:

Most popular tags

"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.