Trending questions in Apache Spark

0 votes
1 answer

What is RDD Lineage in Spark?

Hey, Lineage is an RDD process to reconstruct ...READ MORE

Jul 4 in Apache Spark by Gitika
 • 25,340 points • 61 views
0 votes
1 answer

error: identified expected but integer literal found.

Hi, You can resolve this error with a ...READ MORE

Jul 4 in Apache Spark by Gitika
 • 25,340 points • 61 views
0 votes
1 answer

How SparkSQL is different from HQL and SQL?

Hi, SparkSQL is a special component on the ...READ MORE

Jul 3 in Apache Spark by Gitika
 • 25,340 points • 90 views
0 votes
1 answer

error: reassingment to val

Hi, This error will only generate when you ...READ MORE

Jul 4 in Apache Spark by Gitika
 • 25,340 points • 17 views
0 votes
1 answer

What is polyglot in spark?

Hi, Spark provides a high-level API in Java, ...READ MORE

Jul 1 in Apache Spark by Gitika
 • 25,340 points • 148 views
0 votes
1 answer

How to create RDD from an external file source in scala?

Hi, To create an RDD from external file ...READ MORE

Jul 3 in Apache Spark by Gitika
 • 25,340 points • 54 views
0 votes
1 answer

How to create RDD from parallelized collection in scala?

Hi, You can check this example in your ...READ MORE

Jul 3 in Apache Spark by Gitika
 • 25,340 points • 53 views
0 votes
1 answer

error: identifier expected but ']' found.

Hi, You can try this remove brackets from ...READ MORE

Jul 3 in Apache Spark by Gitika
 • 25,340 points • 63 views
0 votes
1 answer

What does reduce action do in Spark?

Hey, Reduce action converts an RDD to a ...READ MORE

Jul 4 in Apache Spark by Gitika
 • 25,340 points • 20 views
0 votes
1 answer

What is SparkCore functionalities?

Hey, Spark Core is a base engine of ...READ MORE

Jul 4 in Apache Spark by Gitika
 • 25,340 points • 14 views
0 votes
0 answers

How to create RDD as string file?

Can anyone suggest how to create RDD ...READ MORE

Jul 4 in Apache Spark by anand
 • 26 views
0 votes
1 answer

Why Partitions are immutable in Spark?

Hi, Every transformation generates a new partition. Partitions ...READ MORE

Jul 3 in Apache Spark by Gitika
 • 25,340 points • 41 views
0 votes
1 answer

What is reduce() action in Spark?

Hey, It takes a function that operates on two ...READ MORE

Jul 2 in Apache Spark by Gitika
 • 25,340 points • 81 views
0 votes
1 answer

What is Piping in Spark?

Hi, Spark provides a pipe() method on RDDs. ...READ MORE

Jul 3 in Apache Spark by Gitika
 • 25,340 points • 32 views
0 votes
1 answer

When we create an RDD, does it bring the data and load it into the memory?

Hey, No, an RDD is made up of ...READ MORE

Jul 3 in Apache Spark by Gitika
 • 25,340 points • 28 views
0 votes
1 answer

What is Action in Spark?

Hi, Actions are RDD’s operation, that value returns ...READ MORE

Jul 3 in Apache Spark by Gitika
 • 25,340 points • 23 views
0 votes
1 answer

What is meant by Transformation? Give some examples.

Hi, The transformations are the functions that are ...READ MORE

Jul 3 in Apache Spark by Gitika
 • 25,340 points • 17 views
0 votes
1 answer

What is a Parquet file in Spark?

Hey, Parquet is a columnar format file supported ...READ MORE

Jul 2 in Apache Spark by Gitika
 • 25,340 points • 50 views
0 votes
1 answer

What is RDD in Apache spark?

Hi, RDD in spark stands for REsilient distributed ...READ MORE

Jul 1 in Apache Spark by Gitika
 • 25,340 points • 79 views
0 votes
1 answer

What is persist() in Spark?

Hi, Spark’s RDDs are by default recomputed each ...READ MORE

Jul 2 in Apache Spark by Gitika
 • 25,340 points • 40 views
0 votes
0 answers

How to create RDD from existing RDD in scala?

Can anyone suggest how to create RDD ...READ MORE

Jul 3 in Apache Spark by Nihal
 • 17 views
0 votes
1 answer

How to calculate the result of formula with Scala?

Hi, You can use a simple mathematical calculation ...READ MORE

Jul 1 in Apache Spark by Gitika
 • 25,340 points • 45 views
0 votes
1 answer

How to execute a function in apache-scala?

Hi, Here is a simple example of how ...READ MORE

Jul 1 in Apache Spark by Gitika
 • 25,340 points • 43 views
0 votes
1 answer

By which components spark ecosystem libraries are composed of?

Hi, Spark ecosystem libraries are composed of various ...READ MORE

Jul 1 in Apache Spark by Gitika
 • 25,340 points • 33 views
0 votes
1 answer

Spark foldbykey doubt

Please have a look below for your ...READ MORE

Jun 19 in Apache Spark by Tina
 • 44 views
0 votes
1 answer

Scala pass input data as arguments

Please refer to the below code as ...READ MORE

Jun 19 in Apache Spark by Lisa
 • 37 views
0 votes
1 answer

Scala: Add user input to array

You can try this:  object printarray { ...READ MORE

Jun 19 in Apache Spark by Dinesha
 • 28 views
0 votes
1 answer

Doubt in display(id, name, salary) before display function

The statement display(id, name, salary) is written before the display function ...READ MORE

Jun 19 in Apache Spark by Ritu
 • 26 views
0 votes
1 answer

Spark CLI issue

For spark.read.textFile we need spark-2.x. Please try ...READ MORE

Jun 19 in Apache Spark by Maahi
 • 18 views
0 votes
2 answers

map() vs flatMap() in Spark

Spark map function expresses a one-to-one transformation. ...READ MORE

Jun 17 in Apache Spark by vishal
 • 160 points • 2,031 views
0 votes
7 answers

How to print the contents of RDD in Apache Spark?

Simple and easy: line.foreach(println) READ MORE

Dec 10, 2018 in Apache Spark by Kuber
 • 10,943 views
0 votes
1 answer

Spark Error: java.lang.IllegalStateException: Cannot call methods on a stopped SparkContext.

There seems to be a problem with ...READ MORE

May 24 in Apache Spark by Jishan
 • 505 views
0 votes
0 answers

_spark_metadata/0 doesn't exist while Compacting batch 9 Structured streaming error

We have Streaming Application implemented using Spark ...READ MORE

May 31 in Apache Spark by AzimKangda
 • 120 points • 199 views
0 votes
1 answer

How do I get number of columns in each line from a delimited file??

Instead of spliting on '\n'. You should ...READ MORE

Aug 7 in Apache Spark by ashish
 • 461 views
0 votes
1 answer

Copy all files from local (Windows) to HDFS with Scala code

Please try the following Scala code: import org.apache.hadoop.conf.Configuration import ...READ MORE

May 22 in Apache Spark by Karan
 • 269 views
0 votes
1 answer

Error while reading multiline Json

peopleDF: org.apache.spark.sql.DataFrame = [_corrupt_record: string] The above that ...READ MORE

May 23 in Apache Spark by Conny
 • 139 views
0 votes
1 answer

Difference between RDD as val and var

Variable declaration can be done in two ...READ MORE

May 23 in Apache Spark by Arun
 • 79 views
0 votes
1 answer

Starting Spark Scala console

To get command prompt for Scala open ...READ MORE

May 24 in Apache Spark by Cassy
 • 29 views
0 votes
1 answer

Spark: Saving file csv

 If you need a single output file ...READ MORE

May 22 in Apache Spark by Rishi
 • 46 views
0 votes
1 answer

Starting Spark in Windows

Run below commands spark-class org.apache.spark.deploy.master.Master spark-class org.apache.spark.deploy.worker.Worker spark://192.168.254.1:7077 NOTE: The ...READ MORE

May 22 in Apache Spark by Reshma
 • 26 views
0 votes
0 answers

WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable [closed]

Hi All I am running Scala program on ...READ MORE

May 5 in Apache Spark by Vishal
closed May 6 by Omkar • 191 views
0 votes
1 answer

How to find the number of null contain in dataframe?

Hey there! You can use the select method of the ...READ MORE

May 3 in Apache Spark by Omkar
 • 67,620 points • 218 views
0 votes
1 answer

How can we use spark shell for scala without cluster?

You can run the Spark shell for ...READ MORE

Apr 28 in Apache Spark by Giri
 • 58 views
0 votes
1 answer

Spark comparing two big data files using scala

Try this and see if this does ...READ MORE

Apr 2 in Apache Spark by Omkar
 • 67,620 points • 428 views
0 votes
1 answer

How to increase worker timeout in Spark application?

By default, the timeout is set to ...READ MORE

Mar 25 in Apache Spark by Hari
 • 507 views
0 votes
1 answer

How to set extra JVM options for Spark application?

You cans set extra JVM options that ...READ MORE

Mar 28 in Apache Spark by Raj
 • 384 views
0 votes
1 answer

How to store files in executor's working directory?

You have to specify a comma-separated list ...READ MORE

Mar 28 in Apache Spark by Raj
 • 338 views
0 votes
1 answer

Changing Yarn queue in Spark application

To change the default queue to which ...READ MORE

Mar 28 in Apache Spark by Raj
 • 315 views
0 votes
1 answer

what are the spark real time issues ?

Some of the issues I have faced ...READ MORE

Mar 18 in Apache Spark by Sharman
 • 667 views
0 votes
1 answer

How to use Spark jars for Yarn distribution?

First, store upload this archive to hdfs and ...READ MORE

Mar 28 in Apache Spark by Raj
 • 180 views
Page:

Most popular tags

"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.