Most answered questions in Apache Spark

0 votes
1 answer

At which point of time Scala will take default arguments?

Hi, If we omit an argument in a ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
 • 65,910 points • 392 views
0 votes
1 answer

What is recursion in Scala function? How does it works?

Hey, A Scala function involves recursion when it ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
 • 65,910 points • 719 views
0 votes
1 answer

How to create Scala function using "Def" keyword?

Hi, To create a Scala function, we use ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
 • 65,910 points • 702 views
0 votes
1 answer

What is Streams in Scala?

Hi, A stream is a lazy list as ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
 • 65,910 points • 416 views
0 votes
1 answer

What is BitSet in Scala?

Hey, A BitSet is a set of non-negative ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
 • 65,910 points • 562 views
0 votes
1 answer

What is exception propagation in Scala?

Hi, When a function experiences an exception, it ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
 • 65,910 points • 650 views
0 votes
1 answer

Read multiple xml files in Spark

You can do this using globbing. See ...READ MORE

Jul 25, 2019 in Apache Spark by Jack
 • 5,343 views
0 votes
1 answer

RDD word count with line numbers

df = spark.createDataFrame([("A", 2000), ("A", 2002), ("A", ...READ MORE

Jul 25, 2019 in Apache Spark by Siri
 • 2,353 views
0 votes
1 answer

Load CSV file data inot MySQL

If you are trying to load file ...READ MORE

Jul 25, 2019 in Apache Spark by Sita
 • 878 views
+1 vote
1 answer

Scala: CSV file to Save data into HBase

Check the reference code mentioned below: def main(args: ...READ MORE

Jul 25, 2019 in Apache Spark by Hari
 • 1,306 views
0 votes
1 answer

Spark: Error while instantiating "org.apache.spark.sql.hive.HiveSessionState"

Seems like you have not started the ...READ MORE

Jul 25, 2019 in Apache Spark by Rohit
 • 7,807 views
0 votes
1 answer

How to start spark history server?

Hey, You can use this command to start​ ...READ MORE

Jul 25, 2019 in Apache Spark by Gitika
 • 65,910 points • 2,784 views
0 votes
1 answer

Explain vararg arguments.

Hey, With varargs, we can pass a variable ...READ MORE

Jul 24, 2019 in Apache Spark by Gitika
 • 65,910 points • 435 views
0 votes
1 answer

Why do we need App in Scala?

Hey, The app is a helper class that ...READ MORE

Jul 24, 2019 in Apache Spark by Gitika
 • 65,910 points • 499 views
0 votes
1 answer

How is Val different from var in Scala?

Hey, In this language, val is a value and var is ...READ MORE

Jul 24, 2019 in Apache Spark by Gitika
 • 65,910 points • 635 views
0 votes
1 answer

What is ofDim in Scala?

Hey, ofDim() is a method in Scala that ...READ MORE

Jul 24, 2019 in Apache Spark by Gitika
 • 65,910 points • 4,248 views
0 votes
1 answer

What is the advantage of having immutability in design for Scala programming language?

Hi, Scala uses immutability by default in most ...READ MORE

Jul 24, 2019 in Apache Spark by Gitika
 • 65,910 points • 553 views
0 votes
1 answer

Which Scala library is used for functional programming?

Hi, Scala library has purely functional data structures ...READ MORE

Jul 24, 2019 in Apache Spark by Gitika
 • 65,910 points • 802 views
0 votes
1 answer

What is “Unit” and “()” in Scala?

Hey, Unit is a subtype of scala.anyval and ...READ MORE

Jul 24, 2019 in Apache Spark by Gitika
 • 65,910 points • 1,260 views
+1 vote
1 answer

Reading a text file through spark data frame

Try this: val df = sc.textFile("HDFS://nameservice1/user/edureka_168049/Structure_IT/samplefile.txt") df.collect() val df = ...READ MORE

Jul 24, 2019 in Apache Spark by Suri
 • 25,863 views
0 votes
1 answer

error:error: only classes can have declared but undefined members.

Hi, This happens in Scala whenever you won't ...READ MORE

Jul 24, 2019 in Apache Spark by Gitika
 • 65,910 points • 1,967 views
0 votes
1 answer

Spark: java.sql.SQLException: No suitable driver

The missing driver is the JDBC one ...READ MORE

Jul 24, 2019 in Apache Spark by John
 • 16,710 views
0 votes
1 answer

Load custom delimited file in Spark

Refer to the following code: val sqlContext = ...READ MORE

Jul 24, 2019 in Apache Spark by Ritu
 • 1,709 views
0 votes
1 answer

Copy file from local to hdfs from the spark job in yarn mode

Refer to the below code: import org.apache.hadoop.conf.Configuration import org.apache.hadoop.fs.FileSystem import ...READ MORE

Jul 24, 2019 in Apache Spark by Yogi
 • 3,458 views
0 votes
1 answer

How do I access the Map Task ID in Spark?

You can access task information using TaskContext: import org.apache.spark.TaskContext sc.parallelize(Seq[Int](), ...READ MORE

Jul 23, 2019 in Apache Spark by ravikiran
 • 4,620 points • 1,275 views
+1 vote
1 answer

How to install Scala Build Tool (SBT) on ubuntu?

Hey, To install SBT on Ubuntu first you need ...READ MORE

Jul 23, 2019 in Apache Spark by Gitika
 • 65,910 points • 1,467 views
0 votes
1 answer

Error: Loading data to mysql

You have to use sqoop to export data ...READ MORE

Jul 23, 2019 in Apache Spark by Janan
 • 460 views
0 votes
1 answer

How to add package com.databricks.spark.avro in spark?

Start spark shell using below line of ...READ MORE

Jul 23, 2019 in Apache Spark by Ritu
 • 3,322 views
0 votes
1 answer

Spark to Hive Table creation

There's an easier way to achieve your ...READ MORE

Jul 23, 2019 in Apache Spark by Dinesh
 • 1,450 views
0 votes
1 answer

Appending " to a string in Scala

1) Use the concat() function. Refer to the below ...READ MORE

Jul 23, 2019 in Apache Spark by Ritu
 • 519 views
0 votes
1 answer

Explain the for loop for printing the Map values in Scala in Apache Spark?

Hey, You can see this following code to ...READ MORE

Jul 23, 2019 in Apache Spark by Gitika
 • 65,910 points • 1,792 views
0 votes
1 answer

How to save RDD in Apache Spark?

Hey, There are few methods provided by the ...READ MORE

Jul 23, 2019 in Apache Spark by Gitika
 • 65,910 points • 3,334 views
0 votes
1 answer

How to check if a particular keyword exists in Apache Spark?

Hey, You can try this code to get ...READ MORE

Jul 23, 2019 in Apache Spark by Gitika
 • 65,910 points • 4,383 views
0 votes
1 answer

How to compute the square root of sum of squares of numbers?

Hey, You need to follow some steps to complete ...READ MORE

Jul 23, 2019 in Apache Spark by Gitika
 • 65,910 points • 5,508 views
0 votes
1 answer

Average function is not commutative and associative?

Hey, I guess the only problem with the ...READ MORE

Jul 23, 2019 in Apache Spark by Gitika
 • 65,910 points • 2,187 views
0 votes
1 answer

Scala: when to use x(2) and x._2?

In the above statement, x(2) is specifying an array ...READ MORE

Jul 22, 2019 in Apache Spark by Yogi
 • 1,130 views
0 votes
1 answer

Scala: error: value unary_+ is not a member of (Int, Int)

All prefix operators' symbols are predefined: +, -, ...READ MORE

Jul 22, 2019 in Apache Spark by karan
 • 3,001 views
0 votes
1 answer

Load .xlsx files to hive tables with spark scala

This should work: def readExcel(file: String): DataFrame = ...READ MORE

Jul 22, 2019 in Apache Spark by Kishan
 • 4,086 views
0 votes
1 answer

Code to compute average in Apache Spark?

Hi, You can compute the average using this ...READ MORE

Jul 22, 2019 in Apache Spark by Gitika
 • 65,910 points • 1,017 views
0 votes
1 answer

How fault tolerance is achieved in Apache Spark?

Hey, In Apache Spark, the data storage model is ...READ MORE

Jul 22, 2019 in Apache Spark by Gitika
 • 65,910 points • 8,203 views
0 votes
1 answer

How to remove the elements with a key present in any other RDD?

Hey, You can use the subtractByKey () function to ...READ MORE

Jul 22, 2019 in Apache Spark by Gitika
 • 65,910 points • 3,831 views
0 votes
1 answer

Error : split value is not a member of org.apache.spark.sql.Row

spark.read.csv is used when loading into a ...READ MORE

Jul 22, 2019 in Apache Spark by Firoz
 • 2,819 views
0 votes
1 answer

load/save text file in spark

The reason you are able to load ...READ MORE

Jul 22, 2019 in Apache Spark by Giri
 • 3,217 views
0 votes
1 answer

Create dataframe for Avro file

Yes, we can work with Avro files ...READ MORE

Jul 22, 2019 in Apache Spark by Rishi
 • 2,513 views
0 votes
1 answer

How to load data of .csv file in MySQL Database Table?

You can do it using a code ...READ MORE

Jul 22, 2019 in Apache Spark by Vishwa
 • 1,688 views
0 votes
1 answer

Spark Null Pointer Exception.

I used Spark 1.5.2 with Hadoop 2.6 ...READ MORE

Jul 19, 2019 in Apache Spark by ravikiran
 • 4,620 points • 5,785 views
0 votes
1 answer

Spark on Yarn

If you just want to get your ...READ MORE

Jul 18, 2019 in Apache Spark by ravikiran
 • 4,620 points • 646 views
+1 vote
1 answer

Need to load 40 GB data to elasticsearch using spark

Did you find any documents or example ...READ MORE

Nov 5, 2019 in Apache Spark by Begum
 • 1,121 views
0 votes
1 answer

Pyspark is taking default path

The HDFS path for MyLab is /user/edureka_id. ...READ MORE

Jul 16, 2019 in Apache Spark by Khushi
 • 1,246 views
+1 vote
1 answer

Spark: java.io.FileNotFoundException

Hello, From the error I get that the ...READ MORE

Dec 13, 2019 in Apache Spark by Alexandru
 • 510 points • 3,679 views
Page:

Most popular tags

"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.