Trending questions in Apache Spark

0 votes
1 answer

Spark:error:throws stack overflow when union a lot.

Hey, Use SparkContext.union(...) instead to union many RDDs at once You ...READ MORE

Jul 31 in Apache Spark by Gitika
 • 25,360 points • 131 views
0 votes
1 answer

Spark: Dataframe vs Dataset

Recently, there are two new data abstractions ...READ MORE

Jul 29 in Apache Spark by Jackie
 • 211 views
0 votes
1 answer

In how many modes Apache spark can run?

Hey, You can launch spark application in four ...READ MORE

Aug 2 in Apache Spark by Gitika
 • 25,360 points • 33 views
0 votes
1 answer

Join in RDD using keys

Suppose you have two dataset results( id, ...READ MORE

Aug 2 in Apache Spark by Trisha
 • 37 views
0 votes
1 answer

What is the use of App class in Scala?

Hi, Scala provides a helper class, called App, that ...READ MORE

Jul 31 in Apache Spark by Gitika
 • 25,360 points • 113 views
0 votes
1 answer

What is Hive on Spark?

Hi, Hive contains significant support for Apache Spark, ...READ MORE

Aug 2 in Apache Spark by Gitika
 • 25,360 points • 28 views
0 votes
1 answer

"main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataInputStream

1. We will check whether master and ...READ MORE

Jul 29 in Apache Spark by Yogi
 • 191 views
0 votes
1 answer

Spark + Hive connectivity

The problem is probably with the command. ...READ MORE

Aug 1 in Apache Spark by Rishni
 • 63 views
0 votes
1 answer

How to use uniform list in Scala?

Hey, The method List.fill() creates a list and ...READ MORE

Jul 31 in Apache Spark by Gitika
 • 25,360 points • 90 views
0 votes
1 answer

Read multiple xml files in Spark

You can do this using globbing. See ...READ MORE

Jul 25 in Apache Spark by Jack
 • 353 views
0 votes
1 answer

PySpark not starting: No active sparkcontext

Seems like Spark hadoop daemons are not ...READ MORE

Jul 30 in Apache Spark by Jishan
 • 118 views
0 votes
1 answer

What are these in scala : _* & @_*

As is widely used, and has different ...READ MORE

Jul 31 in Apache Spark by Turic
 • 66 views
0 votes
1 answer

Scala: org.apache.poi.openxml4j.exceptions.InvalidFormatException: Your InputStream was neither an OLE2 stream, nor an OOXML stream

Try executing the below code, def readExcel(file: String): ...READ MORE

Jul 30 in Apache Spark by Raman
 • 96 views
0 votes
1 answer

How to reverse a Scala list?

Hi, This reverses the order of elements in ...READ MORE

Jul 31 in Apache Spark by Gitika
 • 25,360 points • 37 views
0 votes
1 answer

How to create singleton classes in Scala?

Hey, Scala introduces a new object keyword, which is used ...READ MORE

Jul 31 in Apache Spark by Gitika
 • 25,360 points • 48 views
0 votes
1 answer

What is 'TRAITS' in Scala

Hi, Traits are basically Scala's workaround for the ...READ MORE

Jul 31 in Apache Spark by Gitika
 • 25,360 points • 43 views
0 votes
1 answer

How to access variables in s string interpolation in Scala?

Hey, You can use below code to access variables ...READ MORE

Jul 31 in Apache Spark by Gitika
 • 25,360 points • 37 views
0 votes
1 answer

How shallow copy carry out using Scala?

Hey, Scala uses the method copy() to carry ...READ MORE

Jul 31 in Apache Spark by Gitika
 • 25,360 points • 28 views
0 votes
1 answer

Removing the header of a text file in SparkRDD

1) First we loaded the data to ...READ MORE

Jul 31 in Apache Spark by Namitha
 • 39 views
0 votes
1 answer

How do find Max and Min values in a set in Scala?

Hey, Here is the example of which will return ...READ MORE

Jul 30 in Apache Spark by Gitika
 • 25,360 points • 67 views
0 votes
1 answer

Scala: Loading a csv file

Refer to the below command: val input_df = ...READ MORE

Jul 31 in Apache Spark by Emma
 • 34 views
0 votes
0 answers

How to define SparkConf?

Can anyone explain how to define SparkConf? READ MORE

Aug 1 in Apache Spark by Danish
 • 21 views
0 votes
1 answer

How to work with multidimensional arrays in Scala?

Hi, Here is an example you can follow: scala> ...READ MORE

Jul 30 in Apache Spark by Gitika
 • 25,360 points • 61 views
0 votes
1 answer

Unable to run the java- spark standalone program

Though there is nothing wrong with the ...READ MORE

Jul 30 in Apache Spark by Lohit
 • 64 views
0 votes
1 answer

Error : split value is not a member of org.apache.spark.sql.Row

spark.read.csv is used when loading into a ...READ MORE

Jul 22 in Apache Spark by Firoz
 • 398 views
0 votes
1 answer

Date formats : how to cast string to date?

Try this, it should work: > from pyspark.sql.functions ...READ MORE

Jul 29 in Apache Spark by Niall
 • 94 views
0 votes
1 answer

How to concatenate sets in Scala?

Hey, Yes, there are two ways of doing ...READ MORE

Jul 30 in Apache Spark by Gitika
 • 25,360 points • 37 views
0 votes
1 answer

How to declare sets with value in Scala?

Hey, You can follow this code: scala> var s=Set(1,4,4,3) s: ...READ MORE

Jul 30 in Apache Spark by Gitika
 • 25,360 points • 35 views
0 votes
1 answer

How to format a string in Scala?

Hey, To format a string, use the .format ...READ MORE

Jul 30 in Apache Spark by Gitika
 • 25,360 points • 30 views
0 votes
1 answer

How to find values common to two sets in Scala

Hey, There are two ways to find the ...READ MORE

Jul 30 in Apache Spark by Gitika
 • 25,360 points • 31 views
0 votes
1 answer

How 'Scala Higher Order function' works in Scala?

Hey, Scala allows the definition of a higher-order function. These ...READ MORE

Jul 30 in Apache Spark by Gitika
 • 25,360 points • 27 views
0 votes
1 answer

Unable to use ml library in pyspark

The error message you have shared with ...READ MORE

Jul 30 in Apache Spark by Karan
 • 27 views
0 votes
1 answer

What is the differentiate between equals() and == in Scala. Is the latter the same as == in Java?

Hey, When we try to compare two instances ...READ MORE

Jul 29 in Apache Spark by Gitika
 • 25,360 points • 57 views
0 votes
1 answer

Scala: 30: error: value partitions is not a member of String

Try this code: val rdd= sc.textFile (“file.txt”, 5) rdd.partitions.size Output ...READ MORE

Jul 29 in Apache Spark by Nijit
 • 53 views
–2 votes
1 answer

What is the difference in Java’s “If..Else” and Scala’s “If..Else”? [closed]

Hey, Java’s “If. Else”: In Java, “If. Else” is a statement, ...READ MORE

Jul 31 in Apache Spark by Gitika
 • 25,360 points • 45 views
0 votes
1 answer

How to concatenate Maps in Scala?

Hey, You can concatenate/join two Maps in more than ...READ MORE

Jul 29 in Apache Spark by Gitika
 • 25,360 points
edited Jul 29 by Gitika • 31 views
0 votes
1 answer

How to search for a Key in a Scala Map?

Hey, The Map.contains() method will tell you if ...READ MORE

Jul 29 in Apache Spark by Gitika
 • 25,360 points • 30 views
0 votes
1 answer

How to declare a Empty Scala Map?

Hi, You can either declare an empty Scala ...READ MORE

Jul 29 in Apache Spark by Gitika
 • 25,360 points
edited Jul 29 by Gitika • 30 views
0 votes
1 answer

Spark, Scala: Load custom delimited file

You can load a DAT file into ...READ MORE

Jul 16 in Apache Spark by Shri
 • 599 views
0 votes
1 answer

load/save text file in spark

The reason you are able to load ...READ MORE

Jul 22 in Apache Spark by Giri
 • 332 views
0 votes
1 answer

What is Lazy val in Scala?

Hey, Scala executes a val when we define ...READ MORE

Jul 29 in Apache Spark by Gitika
 • 25,360 points • 30 views
0 votes
1 answer

what is an anonymous function Scala?

Hi, An anonymous function in Scala is a ...READ MORE

Jul 29 in Apache Spark by Gitika
 • 25,360 points • 29 views
0 votes
1 answer

How the if-else statement is different for Scala and Java?

Hey, There is one main difference. This statement ...READ MORE

Jul 29 in Apache Spark by Gitika
 • 25,360 points • 27 views
0 votes
1 answer

What is Case Class in Scala?

Hey, Scala case classes let us model immutable data. ...READ MORE

Jul 29 in Apache Spark by Gitika
 • 25,360 points • 28 views
0 votes
1 answer

How to fit multiple assignment statements into one statement in Scala?

Hi, You can assign a Tuple to a ...READ MORE

Jul 29 in Apache Spark by Gitika
 • 25,360 points • 20 views
0 votes
1 answer

Copy file from local to hdfs from the spark job in yarn mode

Refer to the below code: import org.apache.hadoop.conf.Configuration import org.apache.hadoop.fs.FileSystem import ...READ MORE

Jul 24 in Apache Spark by Yogi
 • 232 views
0 votes
1 answer

Consider for-comprehensions in Scala. What are they syntactic sugars for?

Hi, A for-comprehension is one way to carry ...READ MORE

Jul 29 in Apache Spark by Gitika
 • 25,360 points • 15 views
+1 vote
1 answer

How to install Scala Build Tool (SBT) on ubuntu?

Hey, To install SBT on Ubuntu first you need ...READ MORE

Jul 23 in Apache Spark by Gitika
 • 25,360 points • 171 views
0 votes
1 answer

How to append a list in Scala?

Hey, For this purpose, we use the single ...READ MORE

Jul 26 in Apache Spark by Gitika
 • 25,360 points • 80 views
0 votes
1 answer

How to call the Debug Mode in PySpark?

As far as I understand your intentions ...READ MORE

Jul 26 in Apache Spark by ravikiran
 • 4,560 points • 80 views
Page:

Most popular tags

"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.