Recent questions tagged spark-sql

+1 vote
1 answer

How to write Spark DataFrame to Avro Data File?

Nov 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 3,806 views
0 votes
1 answer

How to read a dataframe based on an avro schema?

Oct 30, 2020 in Apache Spark by anonymous
 • 120 points • 3,853 views
0 votes
1 answer

How to insert data into Cassandra table using Spark DataFrame?

Sep 21, 2020 in Apache Spark by akhtar
 • 38,260 points • 4,288 views
0 votes
1 answer

Py4JJavaError: An error occurred while calling o310.csv. : java.net.ConnectException: Call From master/192.168.56.101 to master:9000

May 7, 2020 in Apache Spark by akhtar
 • 38,260 points • 7,979 views
+1 vote
1 answer

How to convert pyspark Dataframe to pandas Dataframe?

May 7, 2020 in Apache Spark by akhtar
 • 38,260 points • 8,513 views
0 votes
1 answer

Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D

Feb 5, 2020 in Big Data Hadoop by akhtar
 • 38,260 points • 9,367 views
0 votes
1 answer

What are Dstreams?

Feb 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 1,467 views
+1 vote
1 answer

How to assign a column in Spark Dataframe (PySpark) as a Primary Key?

Jan 9, 2020 in Apache Spark by anonymous
 • 150 points • 15,347 views
+1 vote
1 answer

Is there any efficient way of dealing null values during concat functionality of pyspark.sql version 2.3.4?

Nov 5, 2019 in Apache Spark by aizhar
 • 130 points • 41,272 views
+1 vote
1 answer

Cannot resolve Error In Spark when filter records with two where condition

Oct 1, 2019 in Apache Spark by anonymous
 • 130 points • 3,114 views
+1 vote
2 answers

Spark: Can we add column to dataframe?

Aug 9, 2019 in Apache Spark by Chirag
 • 5,062 views
+1 vote
1 answer

Reading a text file through spark data frame

Jul 24, 2019 in Apache Spark by Kiran
 • 26,865 views
0 votes
1 answer

Spark Vs Hive LLAP Question

Jul 16, 2019 in Big Data Hadoop by Vishnu
 • 4,112 views
0 votes
1 answer

Scala join comma delimited file as tables

Jul 9, 2019 in Apache Spark by Lohit
 • 1,245 views
0 votes
1 answer

How to create dataframe for the comma delimited file?

Jul 5, 2019 in Apache Spark by Yogi
 • 3,914 views
0 votes
1 answer

How to find the number of null contain in dataframe?

May 3, 2019 in Apache Spark by anonymous
 • 120 points • 5,562 views
+1 vote
2 answers

How do I get number of columns in each line from a delimited file??

Mar 8, 2019 in Apache Spark by Vijay Dixon
 • 190 points • 6,192 views
0 votes
1 answer

How to convert Spark data into CSV?

Nov 21, 2018 in Big Data Hadoop by digger
 • 26,740 points • 5,062 views
+1 vote
2 answers

How can I convert Spark Dataframe to Spark RDD?

Jul 4, 2018 in Apache Spark by code799
 • 21,430 views
0 votes
3 answers

Sorting rows in descending order in Spark SQL

Jul 4, 2018 in Apache Spark by code799
 • 17,225 views
+2 votes
14 answers

How to create new column with function in Spark Dataframe?

Jun 26, 2018 in Apache Spark by coldcode
 • 2,090 points • 90,928 views
0 votes
1 answer

Minimizing Data Transfers in Spark

Jun 19, 2018 in Apache Spark by shams
 • 3,670 points • 1,779 views
0 votes
1 answer

How to find the number of elements present in the array in a Spark DataFame column?

Jun 6, 2018 in Apache Spark by code799
 • 23,042 views
0 votes
3 answers

Filtering a row in Spark DataFrame based on matching values from a list

Jun 6, 2018 in Apache Spark by code799
 • 93,549 views
0 votes
1 answer

Convert the given Spar rdd object to Spark DataFrame.

Jun 6, 2018 in Apache Spark by code799
 • 1,497 views
0 votes
1 answer

Different Spark Ecosystem

Jun 4, 2018 in Apache Spark by shams
 • 3,670 points • 1,222 views
+1 vote
8 answers

How to replace null values in Spark DataFrame?

May 31, 2018 in Apache Spark by kurt_cobain
 • 9,350 points • 77,016 views
0 votes
1 answer

What is Spark Piping?

May 31, 2018 in Apache Spark by Data_Nerd
 • 2,390 points • 2,481 views
0 votes
1 answer

How to convert rdd object to dataframe in spark

May 30, 2018 in Apache Spark by Shubham
 • 13,490 points • 4,364 views
0 votes
3 answers

How to transpose Spark DataFrame?

May 24, 2018 in Apache Spark by anonymous
 • 20,784 views
0 votes
2 answers

In a Spark DataFrame how can I flatten the struct?

May 24, 2018 in Apache Spark by code799
 • 6,640 views
0 votes
3 answers

How to connect Spark to a remote Hive server?

May 17, 2018 in Big Data Hadoop by code799
 • 13,486 views
+2 votes
4 answers

use length function in substring in spark

May 3, 2018 in Apache Spark by Data_Nerd
 • 2,390 points • 44,232 views

Most popular tags

"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.