questions/apache-spark
its late but this how you can ...READ MORE
Using findspark is expected to solve the ...READ MORE
Hi @asif, Share with us please the application ...READ MORE
Hello, Your problem is here: val df_merge_final = df_merge .withColumn("version_key", ...READ MORE
How can I import zip files and ...READ MORE
val coder: (Int => String) = v ...READ MORE
You can use the function expr val data ...READ MORE
Hi@akhtar, Since Avro library is external to Spark, ...READ MORE
Hi@Amey, You can enable WebHDFS to do this ...READ MORE
Hi@Amey, It depends on your use case. Both ...READ MORE
Hi@akhtar, To convert pyspark dataframe into pandas dataframe, ...READ MORE
Hi@akhtar, Here you are trying to read a ...READ MORE
Hi, I have the input RDD as a ...READ MORE
spark do not have any concept of ...READ MORE
We do not have access to hive ...READ MORE
Try to put the kafka client for ...READ MORE
Hey @c.kothamasu You should copy your file to ...READ MORE
When you concatenate any string with a ...READ MORE
Source tags are different: { x : [ { ...READ MORE
You can do this by turning off ...READ MORE
Try df.where($"cola".isNotNull && $"cola" =!= "" && !$"colb".isin(2,3)) your ...READ MORE
What is the major use case for ...READ MORE
Try including the package while starting the ...READ MORE
Hey, you can use "contains" filter to extract ...READ MORE
Yes we can add columns to the ...READ MORE
import sqlContext.implicits._ import org.apache.spark.sql.Row import org.apache.spark.sql.types.{StructType, StructField, LongType} val df ...READ MORE
Hey, You can try this: from pyspark import SparkContext SparkContext.stop(sc) sc ...READ MORE
Well, it depends on the block of ...READ MORE
Converting text file to Orc: Using Spark, the ...READ MORE
Recently, there are two new data abstractions ...READ MORE
Check the reference code mentioned below: def main(args: ...READ MORE
Try this: val df = sc.textFile("HDFS://nameservice1/user/edureka_168049/Structure_IT/samplefile.txt") df.collect() val df = ...READ MORE
Hey, To install SBT on Ubuntu first you need ...READ MORE
Did you find any documents or example ...READ MORE
Hello, From the error I get that the ...READ MORE
Hi, You need to edit one property in ...READ MORE
Start spark shell using below line of ...READ MORE
Hi, Regarding this error, you just need to change ...READ MORE
SparkContext sets up internal services and establishes ...READ MORE
Hey, It takes a function that operates on two ...READ MORE
Please check https://kb.databricks.com/streaming/file-sink-str ...READ MORE
Instead of spliting on '\n'. You should ...READ MORE
Spark map function expresses a one-to-one transformation. ...READ MORE
I am guessing that the configuration set ...READ MORE
Preparing for an interview? We have something ...READ MORE
Comparison between Spark RDD vs DataFrame 1. Release ...READ MORE
ReduceByKey is the best for production. READ MORE
Can you share the screenshots for the ...READ MORE
Save it to a text file: line.saveAsTextFile("alicia.txt") Print contains ...READ MORE
OR
At least 1 upper-case and 1 lower-case letter
Minimum 8 characters and Maximum 50 characters
Already have an account? Sign in.