Trending questions in Apache Spark

0 votes
1 answer

What's the difference between 'filter' and 'where' in Spark SQL?

Both 'filter' and 'where' in Spark SQL ...READ MORE

May 23, 2018 in Apache Spark by nitinrawat895
• 11,380 points
19,583 views
0 votes
1 answer

Scala: 30: error: value partitions is not a member of String

Try this code: val rdd= sc.textFile (“file.txt”, 5) rdd.partitions.size Output ...READ MORE

Jul 29, 2019 in Apache Spark by Nijit
913 views
0 votes
1 answer

How to handle data shuffle in Spark

Hi, You can do it using map partition ...READ MORE

Aug 6, 2019 in Apache Spark by Gitika
• 65,870 points
503 views
0 votes
1 answer

Scala: save filtered data row by row using saveAsTextFile

Try this code, it worked for me: val ...READ MORE

Aug 2, 2019 in Apache Spark by Karan
678 views
0 votes
1 answer

How to check if a particular keyword exists in Apache Spark?

Hey, You can try this code to get ...READ MORE

Jul 22, 2019 in Apache Spark by Gitika
• 65,870 points
1,088 views
0 votes
1 answer

Unable to use ml library in pyspark

The error message you have shared with ...READ MORE

Jul 30, 2019 in Apache Spark by Karan
743 views
0 votes
1 answer

How to use uniform list in Scala?

Hey, The method List.fill() creates a list and ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
• 65,870 points
671 views
0 votes
1 answer

Monitoring Spark application

Spark-submit jobs are also run from client/edge ...READ MORE

Aug 9, 2019 in Apache Spark by Umesh
237 views
0 votes
1 answer

error: identified expected but integer literal found.

Hi, You can resolve this error with a ...READ MORE

Jul 4, 2019 in Apache Spark by Gitika
• 65,870 points
1,809 views
0 votes
1 answer

error: identifier expected but ']' found.

Hi, You can try this remove brackets from ...READ MORE

Jul 3, 2019 in Apache Spark by Gitika
• 65,870 points
1,826 views
0 votes
1 answer

In how many modes Apache spark can run?

Hey, You can launch spark application in four ...READ MORE

Aug 2, 2019 in Apache Spark by Gitika
• 65,870 points
520 views
0 votes
1 answer

what are the spark job and spark task and spark staging ?

In a Spark application, when you invoke ...READ MORE

Mar 18, 2019 in Apache Spark by Pavan
6,453 views
0 votes
1 answer

Scala: org.apache.poi.openxml4j.exceptions.InvalidFormatException: Your InputStream was neither an OLE2 stream, nor an OOXML stream

Try executing the below code, def readExcel(file: String): ...READ MORE

Jul 30, 2019 in Apache Spark by Raman
654 views
0 votes
1 answer

How to create singleton classes in Scala?

Hey, Scala introduces a new object keyword, which is used ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
• 65,870 points
567 views
+1 vote
1 answer

How to install Scala Build Tool (SBT) on ubuntu?

Hey, To install SBT on Ubuntu first you need ...READ MORE

Jul 23, 2019 in Apache Spark by Gitika
• 65,870 points
763 views
+1 vote
1 answer

Scala: CSV file to Save data into HBase

Check the reference code mentioned below: def main(args: ...READ MORE

Jul 25, 2019 in Apache Spark by Hari
682 views
0 votes
1 answer

Unable to run the java- spark standalone program

Though there is nothing wrong with the ...READ MORE

Jul 30, 2019 in Apache Spark by Lohit
509 views
0 votes
1 answer

RDD word count with line numbers

df = spark.createDataFrame([("A", 2000), ("A", 2002), ("A", ...READ MORE

Jul 25, 2019 in Apache Spark by Siri
715 views
0 votes
1 answer

How to load data of .csv file in MySQL Database Table?

You can do it using a code ...READ MORE

Jul 22, 2019 in Apache Spark by Vishwa
805 views
0 votes
1 answer

How to start spark history server?

Hi, You can use this command to start ...READ MORE

Aug 6, 2019 in Apache Spark by Gitika
• 65,870 points
141 views
0 votes
1 answer

Scala: error: value unary_+ is not a member of (Int, Int)

All prefix operators' symbols are predefined: +, -, ...READ MORE

Jul 22, 2019 in Apache Spark by karan
784 views
0 votes
1 answer

Average function is not commutative and associative?

Hey, I guess the only problem with the ...READ MORE

Jul 22, 2019 in Apache Spark by Gitika
• 65,870 points
767 views
–2 votes
1 answer

What is the difference in Java’s “If..Else” and Scala’s “If..Else”? [closed]

Hey, Java’s “If. Else”: In Java, “If. Else” is a statement, ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
• 65,870 points
444 views
0 votes
1 answer

Scala: Loading a csv file

Refer to the below command: val input_df = ...READ MORE

Jul 31, 2019 in Apache Spark by Emma
364 views
0 votes
1 answer

Spark to Hive Table creation

There's an easier way to achieve your ...READ MORE

Jul 23, 2019 in Apache Spark by Dinesh
709 views
0 votes
1 answer

Spark Streaming Pyspark code not working

The address you are using in the ...READ MORE

Jul 11, 2019 in Apache Spark by Shir
1,186 views
0 votes
1 answer

How to declare a Empty Scala Map?

Hi, You can either declare an empty Scala ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
• 65,870 points

edited Jul 29, 2019 by Gitika 362 views
0 votes
1 answer

Query regarding Appending " to a string in Scala

You can perform this task in two ...READ MORE

Jul 10, 2019 in Apache Spark by Esha
1,177 views
0 votes
1 answer

What is Lazy val in Scala?

Hey, Scala executes a val when we define ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
• 65,870 points
346 views
0 votes
1 answer

How to reverse a Scala list?

Hi, This reverses the order of elements in ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
• 65,870 points
240 views
0 votes
1 answer

error:error: only classes can have declared but undefined members.

Hi, This happens in Scala whenever you won't ...READ MORE

Jul 24, 2019 in Apache Spark by Gitika
• 65,870 points
537 views
0 votes
1 answer

Spark Submit: class does not exists

In the command, you have mentioned the ...READ MORE

Jul 26, 2019 in Apache Spark by Jimmy
446 views
0 votes
1 answer

Explain the for loop for printing the Map values in Scala in Apache Spark?

Hey, You can see this following code to ...READ MORE

Jul 22, 2019 in Apache Spark by Gitika
• 65,870 points
590 views
0 votes
1 answer

How to create dataframe for the comma delimited file?

 Refer to the below command used: val df ...READ MORE

Jul 5, 2019 in Apache Spark by karan
1,339 views
0 votes
1 answer

How to work with multidimensional arrays in Scala?

Hi, Here is an example you can follow: scala> ...READ MORE

Jul 30, 2019 in Apache Spark by Gitika
• 65,870 points
243 views
0 votes
1 answer

How to access variables in s string interpolation in Scala?

Hey, You can use below code to access variables ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
• 65,870 points
200 views
0 votes
1 answer

What is Case Class in Scala?

Hey, Scala case classes let us model immutable data. ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
• 65,870 points
281 views
0 votes
1 answer

What is Hive on Spark?

Hi, Hive contains significant support for Apache Spark, ...READ MORE

Aug 2, 2019 in Apache Spark by Gitika
• 65,870 points
107 views
0 votes
1 answer

How to concatenate sets in Scala?

Hey, Yes, there are two ways of doing ...READ MORE

Jul 30, 2019 in Apache Spark by Gitika
• 65,870 points
199 views
0 votes
1 answer

What is 'TRAITS' in Scala

Hi, Traits are basically Scala's workaround for the ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
• 65,870 points
164 views
0 votes
1 answer

How SparkSQL is different from HQL and SQL?

Hi, SparkSQL is a special component on the ...READ MORE

Jul 3, 2019 in Apache Spark by Gitika
• 65,870 points
1,374 views
0 votes
1 answer

what is an anonymous function Scala?

Hi, An anonymous function in Scala is a ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
• 65,870 points
233 views
0 votes
1 answer

How 'Scala Higher Order function' works in Scala?

Hey, Scala allows the definition of a higher-order function. These ...READ MORE

Jul 30, 2019 in Apache Spark by Gitika
• 65,870 points
178 views
0 votes
1 answer

How shallow copy carry out using Scala?

Hey, Scala uses the method copy() to carry ...READ MORE

Jul 31, 2019 in Apache Spark by Gitika
• 65,870 points
127 views
0 votes
1 answer

How to find values common to two sets in Scala

Hey, There are two ways to find the ...READ MORE

Jul 30, 2019 in Apache Spark by Gitika
• 65,870 points
167 views
0 votes
1 answer

How to format a string in Scala?

Hey, To format a string, use the .format ...READ MORE

Jul 30, 2019 in Apache Spark by Gitika
• 65,870 points
160 views
0 votes
1 answer

What is the differentiate between equals() and == in Scala. Is the latter the same as == in Java?

Hey, When we try to compare two instances ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
• 65,870 points
205 views
0 votes
0 answers

How to define SparkConf?

Can anyone explain how to define SparkConf? READ MORE

Aug 1, 2019 in Apache Spark by Danish
106 views
0 votes
1 answer

How to fit multiple assignment statements into one statement in Scala?

Hi, You can assign a Tuple to a ...READ MORE

Jul 29, 2019 in Apache Spark by Gitika
• 65,870 points
181 views