Recent questions tagged spark-dataframe

+1 vote
1 answer

How to write Spark DataFrame to Avro Data File?

Nov 4, 2020 in Apache Spark by akhtar
 • 38,230 points • 2,798 views
0 votes
1 answer

How to read a dataframe based on an avro schema?

Oct 30, 2020 in Apache Spark by anonymous
 • 120 points • 2,828 views
0 votes
1 answer

How to insert data into Cassandra table using Spark DataFrame?

Sep 21, 2020 in Apache Spark by akhtar
 • 38,230 points • 3,450 views
+1 vote
1 answer

How to convert pyspark Dataframe to pandas Dataframe?

May 7, 2020 in Apache Spark by akhtar
 • 38,230 points • 7,960 views
0 votes
1 answer

What are Dstreams?

Feb 4, 2020 in Apache Spark by akhtar
 • 38,230 points • 747 views
+1 vote
1 answer

How to assign a column in Spark Dataframe (PySpark) as a Primary Key?

Jan 9, 2020 in Apache Spark by anonymous
 • 150 points • 12,296 views
+1 vote
1 answer

Is there any efficient way of dealing null values during concat functionality of pyspark.sql version 2.3.4?

Nov 5, 2019 in Apache Spark by aizhar
 • 130 points • 38,184 views
+1 vote
1 answer

Cannot resolve Error In Spark when filter records with two where condition

Oct 1, 2019 in Apache Spark by anonymous
 • 130 points • 2,307 views
+1 vote
2 answers

Spark: Can we add column to dataframe?

Aug 9, 2019 in Apache Spark by Chirag
 • 4,449 views
+1 vote
1 answer

Reading a text file through spark data frame

Jul 24, 2019 in Apache Spark by Kiran
 • 25,789 views
0 votes
1 answer

Scala join comma delimited file as tables

Jul 9, 2019 in Apache Spark by Lohit
 • 749 views
0 votes
1 answer

How to create dataframe for the comma delimited file?

Jul 5, 2019 in Apache Spark by Yogi
 • 2,988 views
0 votes
1 answer

How to find the number of null contain in dataframe?

May 3, 2019 in Apache Spark by anonymous
 • 120 points • 4,672 views
+1 vote
2 answers

How do I get number of columns in each line from a delimited file??

Mar 8, 2019 in Apache Spark by Vijay Dixon
 • 190 points • 5,037 views
0 votes
1 answer

How to convert Spark data into CSV?

Nov 21, 2018 in Big Data Hadoop by digger
 • 26,740 points • 4,010 views
+1 vote
2 answers

How can I convert Spark Dataframe to Spark RDD?

Jul 4, 2018 in Apache Spark by code799
 • 19,895 views
0 votes
3 answers

Sorting rows in descending order in Spark SQL

Jul 4, 2018 in Apache Spark by code799
 • 16,345 views
+2 votes
14 answers

How to create new column with function in Spark Dataframe?

Jun 26, 2018 in Apache Spark by coldcode
 • 2,080 points • 87,596 views
0 votes
1 answer

Minimizing Data Transfers in Spark

Jun 19, 2018 in Apache Spark by shams
 • 3,670 points • 1,167 views
0 votes
1 answer

How to find the number of elements present in the array in a Spark DataFame column?

Jun 6, 2018 in Apache Spark by code799
 • 21,809 views
0 votes
3 answers

Filtering a row in Spark DataFrame based on matching values from a list

Jun 6, 2018 in Apache Spark by code799
 • 92,001 views
0 votes
1 answer

Convert the given Spar rdd object to Spark DataFrame.

Jun 6, 2018 in Apache Spark by code799
 • 842 views
0 votes
1 answer

Different Spark Ecosystem

Jun 4, 2018 in Apache Spark by shams
 • 3,670 points • 711 views
+1 vote
8 answers

How to replace null values in Spark DataFrame?

May 31, 2018 in Apache Spark by kurt_cobain
 • 9,390 points • 74,247 views
0 votes
1 answer

What is Spark Piping?

May 31, 2018 in Apache Spark by Data_Nerd
 • 2,390 points • 2,015 views
0 votes
1 answer

How to convert rdd object to dataframe in spark

May 30, 2018 in Apache Spark by Shubham
 • 13,490 points • 3,717 views
0 votes
3 answers

How to transpose Spark DataFrame?

May 24, 2018 in Apache Spark by anonymous
 • 19,141 views
0 votes
2 answers

In a Spark DataFrame how can I flatten the struct?

May 24, 2018 in Apache Spark by code799
 • 5,738 views
0 votes
3 answers

How to connect Spark to a remote Hive server?

May 17, 2018 in Big Data Hadoop by code799
 • 12,166 views
+2 votes
4 answers

use length function in substring in spark

May 3, 2018 in Apache Spark by Data_Nerd
 • 2,390 points • 41,970 views

Most popular tags

"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.