Most voted questions in Apache Spark

0 votes
1 answer

How to create Scala Array with Range?

Hey, The range() method will give us integers ...READ MORE

Jul 25 in Apache Spark by Gitika
• 25,360 points
29 views
0 votes
1 answer

How to concatinating Arrays in scala?

Hey, We can append a Scala array to ...READ MORE

Jul 25 in Apache Spark by Gitika
• 25,360 points
26 views
0 votes
1 answer

How to process an Array in Scala?

Hi, We can use loop control structures to ...READ MORE

Jul 25 in Apache Spark by Gitika
• 25,360 points
24 views
0 votes
1 answer

How to declare Scala array?

Hi, We can declare Scala arrays in two ...READ MORE

Jul 25 in Apache Spark by Gitika
• 25,360 points
36 views
0 votes
1 answer

At which point of time Scala will take default arguments?

Hi, If we omit an argument in a ...READ MORE

Jul 25 in Apache Spark by Gitika
• 25,360 points
24 views
0 votes
1 answer

What is recursion in Scala function? How does it works?

Hey, A Scala function involves recursion when it ...READ MORE

Jul 25 in Apache Spark by Gitika
• 25,360 points
27 views
0 votes
1 answer

How to create Scala function using "Def" keyword?

Hi, To create a Scala function, we use ...READ MORE

Jul 25 in Apache Spark by Gitika
• 25,360 points
22 views
0 votes
1 answer

What is Streams in Scala?

Hi, A stream is a lazy list as ...READ MORE

Jul 25 in Apache Spark by Gitika
• 25,360 points
37 views
0 votes
1 answer

What is BitSet in Scala?

Hey, A BitSet is a set of non-negative ...READ MORE

Jul 25 in Apache Spark by Gitika
• 25,360 points
37 views
0 votes
1 answer

What is exception propagation in Scala?

Hi, When a function experiences an exception, it ...READ MORE

Jul 25 in Apache Spark by Gitika
• 25,360 points
31 views
0 votes
1 answer

Read multiple xml files in Spark

You can do this using globbing. See ...READ MORE

Jul 25 in Apache Spark by Jack
353 views
0 votes
1 answer

RDD word count with line numbers

df = spark.createDataFrame([("A", 2000), ("A", 2002), ("A", ...READ MORE

Jul 25 in Apache Spark by Siri
62 views
0 votes
1 answer

Load CSV file data inot MySQL

If you are trying to load file ...READ MORE

Jul 25 in Apache Spark by Sita
28 views
0 votes
1 answer

Scala: CSV file to Save data into HBase

Check the reference code mentioned below: def main(args: ...READ MORE

Jul 25 in Apache Spark by Hari
118 views
0 votes
1 answer

Spark: Error while instantiating "org.apache.spark.sql.hive.HiveSessionState"

Seems like you have not started the ...READ MORE

Jul 25 in Apache Spark by Rohit
409 views
0 votes
1 answer

How to start spark history server?

Hey, You can use this command to start​ ...READ MORE

Jul 24 in Apache Spark by Gitika
• 25,360 points
49 views
0 votes
1 answer

Explain vararg arguments.

Hey, With varargs, we can pass a variable ...READ MORE

Jul 24 in Apache Spark by Gitika
• 25,360 points
28 views
0 votes
1 answer

Why do we need App in Scala?

Hey, The app is a helper class that ...READ MORE

Jul 24 in Apache Spark by Gitika
• 25,360 points
29 views
0 votes
1 answer

How is Val different from var in Scala?

Hey, In this language, val is a value and var is ...READ MORE

Jul 24 in Apache Spark by Gitika
• 25,360 points
34 views
0 votes
1 answer

What is ofDim in Scala?

Hey, ofDim() is a method in Scala that ...READ MORE

Jul 24 in Apache Spark by Gitika
• 25,360 points
65 views
0 votes
1 answer

What is the advantage of having immutability in design for Scala programming language?

Hi, Scala uses immutability by default in most ...READ MORE

Jul 24 in Apache Spark by Gitika
• 25,360 points
44 views
0 votes
1 answer

Which Scala library is used for functional programming?

Hi, Scala library has purely functional data structures ...READ MORE

Jul 24 in Apache Spark by Gitika
• 25,360 points
32 views
0 votes
1 answer

What is “Unit” and “()” in Scala?

Hey, Unit is a subtype of scala.anyval and ...READ MORE

Jul 24 in Apache Spark by Gitika
• 25,360 points
42 views
0 votes
1 answer

Reading a text file through spark data frame

Try this: val df = sc.textFile("HDFS://nameservice1/user/edureka_168049/Structure_IT/samplefile.txt") df.collect() val df = ...READ MORE

Jul 24 in Apache Spark by Suri
2,058 views
0 votes
1 answer

error:error: only classes can have declared but undefined members.

Hi, This happens in Scala whenever you won't ...READ MORE

Jul 24 in Apache Spark by Gitika
• 25,360 points
83 views
0 votes
1 answer

Spark: java.sql.SQLException: No suitable driver

The missing driver is the JDBC one ...READ MORE

Jul 24 in Apache Spark by John
451 views
0 votes
1 answer

Load custom delimited file in Spark

Refer to the following code: val sqlContext = ...READ MORE

Jul 24 in Apache Spark by Ritu
42 views
0 votes
1 answer

Copy file from local to hdfs from the spark job in yarn mode

Refer to the below code: import org.apache.hadoop.conf.Configuration import org.apache.hadoop.fs.FileSystem import ...READ MORE

Jul 24 in Apache Spark by Yogi
232 views
0 votes
0 answers

What is immutabiliity in Spark?

Can anyone explain what is immutability in ...READ MORE

Jul 23 in Apache Spark by Risha
24 views
0 votes
1 answer

How do I access the Map Task ID in Spark?

You can access task information using TaskContext: import org.apache.spark.TaskContext sc.parallelize(Seq[Int](), ...READ MORE

Jul 23 in Apache Spark by ravikiran
• 4,560 points
53 views
0 votes
1 answer

Error: Loading data to mysql

You have to use sqoop to export data ...READ MORE

Jul 23 in Apache Spark by Janan
44 views
0 votes
1 answer

How to add package com.databricks.spark.avro in spark?

Start spark shell using below line of ...READ MORE

Jul 23 in Apache Spark by Ritu
203 views
0 votes
0 answers

Load data to mysql from local

I'm trying to load data to mysql ...READ MORE

Jul 23 in Apache Spark by Yogi
35 views
0 votes
1 answer

Spark to Hive Table creation

There's an easier way to achieve your ...READ MORE

Jul 23 in Apache Spark by Dinesh
37 views
0 votes
1 answer

Appending " to a string in Scala

1) Use the concat() function. Refer to the below ...READ MORE

Jul 23 in Apache Spark by Ritu
26 views
0 votes
1 answer

Explain the for loop for printing the Map values in Scala in Apache Spark?

Hey, You can see this following code to ...READ MORE

Jul 22 in Apache Spark by Gitika
• 25,360 points
53 views
0 votes
1 answer

How to save RDD in Apache Spark?

Hey, There are few methods provided by the ...READ MORE

Jul 22 in Apache Spark by Gitika
• 25,360 points
144 views
0 votes
1 answer

How to check if a particular keyword exists in Apache Spark?

Hey, You can try this code to get ...READ MORE

Jul 22 in Apache Spark by Gitika
• 25,360 points
83 views
0 votes
1 answer

How to compute the square root of sum of squares of numbers?

Hey, You need to follow some steps to complete ...READ MORE

Jul 22 in Apache Spark by Gitika
• 25,360 points
108 views
0 votes
1 answer

Average function is not commutative and associative?

Hey, I guess the only problem with the ...READ MORE

Jul 22 in Apache Spark by Gitika
• 25,360 points
108 views
0 votes
1 answer

Scala: when to use x(2) and x._2?

In the above statement, x(2) is specifying an array ...READ MORE

Jul 22 in Apache Spark by Yogi
64 views
0 votes
1 answer

Scala: error: value unary_+ is not a member of (Int, Int)

All prefix operators' symbols are predefined: +, -, ...READ MORE

Jul 22 in Apache Spark by karan
84 views
0 votes
1 answer

Load .xlsx files to hive tables with spark scala

This should work: def readExcel(file: String): DataFrame = ...READ MORE

Jul 22 in Apache Spark by Kishan
221 views
0 votes
1 answer

Code to compute average in Apache Spark?

Hi, You can compute the average using this ...READ MORE

Jul 22 in Apache Spark by Gitika
• 25,360 points
27 views
0 votes
1 answer

How fault tolerance is achieved in Apache Spark?

Hey, In Apache Spark, the data storage model is ...READ MORE

Jul 22 in Apache Spark by Gitika
• 25,360 points
72 views
0 votes
1 answer

How to remove the elements with a key present in any other RDD?

Hey, You can use the subtractByKey () function to ...READ MORE

Jul 22 in Apache Spark by Gitika
• 25,360 points
121 views
0 votes
1 answer

Error : split value is not a member of org.apache.spark.sql.Row

spark.read.csv is used when loading into a ...READ MORE

Jul 22 in Apache Spark by Firoz
398 views
0 votes
1 answer

load/save text file in spark

The reason you are able to load ...READ MORE

Jul 22 in Apache Spark by Giri
332 views
0 votes
1 answer

Create dataframe for Avro file

Yes, we can work with Avro files ...READ MORE

Jul 22 in Apache Spark by Rishi
184 views
0 votes
1 answer

How to load data of .csv file in MySQL Database Table?

You can do it using a code ...READ MORE

Jul 22 in Apache Spark by Vishwa
41 views