questions/apache-spark
You aren't actually overwriting anything with this ...READ MORE
Use the function as following: var notFollowingList=List(9.8,7,6,3,1) df.filter(col("uid").isin(notFollowingList:_*)) You can ...READ MORE
val coder: (Int => String) = v ...READ MORE
ReduceByKey is the best for production. READ MORE
Hi, In Spark, fill() function of DataFrameNaFunctions class is used to replace ...READ MORE
its late but this how you can ...READ MORE
Save it to a text file: line.saveAsTextFile("alicia.txt") Print contains ...READ MORE
Recently, there are two new data abstractions ...READ MORE
Comparison between Spark RDD vs DataFrame 1. Release ...READ MORE
The string Productivity has to be enclosed between single ...READ MORE
You can use the function expr val data ...READ MORE
When you concatenate any string with a ...READ MORE
Spark map function expresses a one-to-one transformation. ...READ MORE
Both 'filter' and 'where' in Spark SQL ...READ MORE
Try this: val df = sc.textFile("HDFS://nameservice1/user/edureka_168049/Structure_IT/samplefile.txt") df.collect() val df = ...READ MORE
You can select the column and apply ...READ MORE
Using findspark is expected to solve the ...READ MORE
Assuming your RDD[row] is called rdd, you ...READ MORE
Hi@akhtar, To import this module in your program, ...READ MORE
You can use filter to do this. ...READ MORE
Please check the below mentioned links for ...READ MORE
I have used a header-less csv file ...READ MORE
Hi@akhtar, Here you are trying to read a ...READ MORE
The missing driver is the JDBC one ...READ MORE
df.orderBy($"col".desc) - this works as well READ MORE
There is a difference between the two: mapValues ...READ MORE
Hi@akhtar, By default pyspark in not present in ...READ MORE
You can use the following to print ...READ MORE
Ideally, you would use snappy compression (default) ...READ MORE
Yes, you can reorder the dataframe elements. You need ...READ MORE
Spark revolves around the concept of a ...READ MORE
You can save the RDD using saveAsObjectFile and saveAsTextFile method. ...READ MORE
I am pretty sure createOrReplaceTempView just replaced ...READ MORE
spark do not have any concept of ...READ MORE
There are few reasons for keeping RDD ...READ MORE
Fold in spark Fold is a very powerful ...READ MORE
Hello, Your problem is here: val df_merge_final = df_merge .withColumn("version_key", ...READ MORE
Hi, Actions are RDD’s operation, that value returns ...READ MORE
var d=rdd2col.rdd.map(x=>x.split(",")) or val names=rd ...READ MORE
Whenever a series of transformations are performed ...READ MORE
Since the file is in HDFS so ...READ MORE
In a Spark application, when you invoke ...READ MORE
Hi, Scala provides a helper class, called App, that ...READ MORE
The cache() is used only the default storage level ...READ MORE
There seems to be a problem with ...READ MORE
Please go through the below explanation : Full ...READ MORE
val x = sc.parallelize(1 to 10, 2) // ...READ MORE
Hi, Paired RDD is a distributed collection of ...READ MORE
You can load a DAT file into ...READ MORE
Spark 2 doesn't differ much architecture-wise from ...READ MORE
OR
At least 1 upper-case and 1 lower-case letter
Minimum 8 characters and Maximum 50 characters
Already have an account? Sign in.