Recent questions tagged apache-spark

0 votes
1 answer

Difference between map() and mapPartitions() function in Spark.

Jan 23, 2020 in Apache Spark by akhtar
 • 38,260 points • 8,123 views
+1 vote
0 answers

How to create a list of RDDs(or RDD of RDDs, if possible) from a single JavaRDD<List<Integers>> in Java?

Jan 11, 2020 in Apache Spark by itsroops
 • 130 points • 4,051 views
+1 vote
1 answer

How to assign a column in Spark Dataframe (PySpark) as a Primary Key?

Jan 9, 2020 in Apache Spark by anonymous
 • 150 points • 17,398 views
+2 votes
1 answer

Spark code takes too much time to run on cluster

Jan 4, 2020 in Apache Spark by asif
 • 140 points • 2,427 views
+1 vote
1 answer

In spark streaming with kafka, i am getting error “Exception in thread ”main“ java.lang.NoSuchMethodError: org.apache.spark.internal.Logging.$init$”

Nov 7, 2019 in Apache Spark by c.kothamasu
 • 160 points • 5,116 views
+1 vote
2 answers

sparkstream.textfilstreaming(localpathdirectory). I am getting empty results

Nov 7, 2019 in Apache Spark by c.kothamasu
 • 160 points • 2,008 views
+1 vote
1 answer

Is there any efficient way of dealing null values during concat functionality of pyspark.sql version 2.3.4?

Nov 5, 2019 in Apache Spark by aizhar
 • 130 points • 46,417 views
+1 vote
1 answer

Cannot resolve Error In Spark when filter records with two where condition

Oct 1, 2019 in Apache Spark by anonymous
 • 130 points • 3,964 views
0 votes
1 answer

Primary keys in Apache Spark

Sep 11, 2019 in Apache Spark by nitinrawat895
 • 11,380 points • 2,604 views
0 votes
1 answer

How do I connect to a HIVE Meta store through a program in SparkSQL?

Sep 5, 2019 in Apache Spark by nitinrawat895
 • 11,380 points • 5,472 views
+1 vote
1 answer

What are all the Data quality checks we do in our real time Big Data projects.

Sep 4, 2019 in Big Data Hadoop by Madhan
 • 130 points • 3,086 views
+1 vote
1 answer

How to convert JSON file to AVRO file and vise versa

Aug 24, 2019 in Apache Spark by Prashant
 • 4,644 views
+1 vote
1 answer

How to extract record from one RDD using another RDD

Aug 23, 2019 in Apache Spark by anonymous
 • 3,486 views
+2 votes
1 answer

Type mismatch error in scala

Aug 16, 2019 in Apache Spark by anonymous
 • 14,641 views
+1 vote
2 answers

Spark: Can we add column to dataframe?

Aug 9, 2019 in Apache Spark by Chirag
 • 5,888 views
0 votes
1 answer

Monitoring Spark application

Aug 9, 2019 in Apache Spark by Rishi
 • 1,927 views
+1 vote
1 answer

How to read a data from text file in Spark?

Aug 6, 2019 in Apache Spark by Dipshika
 • 6,281 views
0 votes
1 answer

How to start spark history server?

Aug 6, 2019 in Apache Spark by Sunita
 • 1,670 views
0 votes
1 answer

What is Spark UI and how to monitor a spark job?

Aug 6, 2019 in Apache Spark by Dhanus
 • 4,030 views
0 votes
1 answer

How to handle data shuffle in Spark

Aug 6, 2019 in Apache Spark by Dinisha
 • 2,418 views
0 votes
1 answer

How Foreach Operation works in Apache Spark?

Aug 2, 2019 in Apache Spark by Nidhi
 • 7,696 views
0 votes
1 answer

How SortBykey() operation works in Spark?

Aug 2, 2019 in Apache Spark by Rashi
 • 7,648 views
0 votes
1 answer

What filter transformation in apache spark?

Aug 2, 2019 in Apache Kafka by Badri
 • 2,304 views
0 votes
1 answer

In how many modes Apache spark can run?

Aug 2, 2019 in Apache Spark by Sunny
 • 2,301 views
0 votes
1 answer

How to launch spark application in cluster mode in Spark?

Aug 2, 2019 in Apache Spark by Raima
 • 5,933 views
0 votes
1 answer

How to create paired RDD using subString method in Spark?

Aug 2, 2019 in Apache Spark by Riddhi
 • 3,947 views
0 votes
1 answer

what is Paired RDD and how to create paired RDD in Spark?

Aug 2, 2019 in Apache Spark by disha
 • 11,032 views
0 votes
1 answer

Join in RDD using keys

Aug 2, 2019 in Apache Spark by Jishan
 • 9,364 views
0 votes
1 answer

Scala: save filtered data row by row using saveAsTextFile

Aug 2, 2019 in Apache Spark by Hari
 • 2,595 views
0 votes
1 answer

What is Hive on Spark?

Aug 2, 2019 in Apache Spark by Jaidev
 • 1,593 views
0 votes
1 answer

Can anyone explain the sparse vector in Spark?

Aug 2, 2019 in Apache Spark by NIrja
 • 7,042 views
0 votes
0 answers

How to define SparkConf?

Aug 1, 2019 in Apache Spark by Danish
 • 1,204 views
0 votes
1 answer

Scala - Error in Inheritance: <console>:: error: not found: value

Aug 1, 2019 in Apache Spark by Ramya
 • 5,953 views
0 votes
1 answer

Pyspark dataframe with random values

Aug 1, 2019 in Apache Spark by Esha
 • 10,224 views
0 votes
1 answer

Spark + Hive connectivity

Aug 1, 2019 in Apache Spark by Jill
 • 3,123 views
+1 vote
1 answer

Scala: Convert text file data into ORC format using data frame

Aug 1, 2019 in Apache Spark by Rishi
 • 4,772 views
0 votes
1 answer

How to reverse a Scala list?

Aug 1, 2019 in Apache Spark by Niraj
 • 2,391 views
0 votes
1 answer

How to use uniform list in Scala?

Aug 1, 2019 in Apache Spark by Reshma
 • 2,586 views
0 votes
1 answer

How shallow copy carry out using Scala?

Aug 1, 2019 in Apache Spark by Sikha
 • 1,633 views
0 votes
1 answer

How to access variables in s string interpolation in Scala?

Jul 31, 2019 in Apache Spark by Pavitra
 • 2,181 views
–2 votes
1 answer

What is the difference in Java’s “If..Else” and Scala’s “If..Else”?

Jul 31, 2019 in Apache Spark by Dhiman
 • 3,445 views
0 votes
1 answer

How to work with Matrix Multiplication in Apache Spark?

Jul 31, 2019 in Apache Spark by Amrita
 • 9,258 views
0 votes
1 answer

Spark Error: StackOverflowError : Exception in thread "main" java.lang.StackOverflowError at org.apache.spark.rdd.UnionRDD$$anonfun$1.apply

Jul 31, 2019 in Apache Spark by Disha
 • 5,579 views
0 votes
1 answer

Spark:error:throws stack overflow when union a lot.

Jul 31, 2019 in Apache Spark by Sunny
 • 6,638 views
0 votes
1 answer

What is 'TRAITS' in Scala

Jul 31, 2019 in Apache Spark by pavitra
 • 1,875 views
0 votes
1 answer

Scala: Loading a csv file

Jul 31, 2019 in Apache Spark by Trisha
 • 2,032 views
0 votes
1 answer

What is the use of App class in Scala?

Jul 31, 2019 in Apache Spark by Rashmi
 • 12,918 views
0 votes
1 answer

What are these in scala : _* & @_*

Jul 31, 2019 in Apache Spark by Kunal
 • 3,677 views
0 votes
1 answer

How to create singleton classes in Scala?

Jul 31, 2019 in Apache Spark by Nimisha
 • 2,770 views
0 votes
1 answer

Cannot load file to spark: "org.apache.spark.sql.AnalysisException: Path does not exist"

Jul 31, 2019 in Apache Spark by Ritu
 • 13,624 views
Page:

Most popular tags

"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.