questions/apache-spark
val coder: (Int => String) = v ...READ MORE
its late but this how you can ...READ MORE
Save it to a text file: line.saveAsTextFile("alicia.txt") Print contains ...READ MORE
Hi, In Spark, fill() function of DataFrameNaFunctions class is used to replace ...READ MORE
ReduceByKey is the best for production. READ MORE
You aren't actually overwriting anything with this ...READ MORE
You can use the function expr val data ...READ MORE
Spark map function expresses a one-to-one transformation. ...READ MORE
Comparison between Spark RDD vs DataFrame 1. Release ...READ MORE
Fold in spark Fold is a very powerful ...READ MORE
There are few reasons for keeping RDD ...READ MORE
df.orderBy($"col".desc) - this works as well READ MORE
According to me, start with a standalone ...READ MORE
Whenever a series of transformations are performed ...READ MORE
Use the function as following: var notFollowingList=List(9.8,7,6,3,1) df.filter(col("uid").isin(notFollowingList:_*)) You can ...READ MORE
Please check the below mentioned links for ...READ MORE
Hi, @Ritu, option b for you, as Hash Partitioning ...READ MORE
Using findspark is expected to solve the ...READ MORE
When using the Java substring() method, a ...READ MORE
Hey @c.kothamasu You should copy your file to ...READ MORE
Yes we can add columns to the ...READ MORE
Recently, there are two new data abstractions ...READ MORE
var d=rdd2col.rdd.map(x=>x.split(",")) or val names=rd ...READ MORE
SparkContext sets up internal services and establishes ...READ MORE
Function Definition : def test():Unit{ var a=10 var b=20 var c=a+b } calling ...READ MORE
Instead of spliting on '\n'. You should ...READ MORE
Spark is agnostic to the underlying cluster ...READ MORE
val x = sc.parallelize(1 to 10, 2) // ...READ MORE
Assuming your RDD[row] is called rdd, you ...READ MORE
Parquet is a columnar format supported by ...READ MORE
map(): Return a new distributed dataset formed by ...READ MORE
// Collect data from input avro file ...READ MORE
I am pretty sure createOrReplaceTempView just replaced ...READ MORE
Spark 2 doesn't differ much architecture-wise from ...READ MORE
Hadoop 3 is not widely used in ...READ MORE
Hi@Prasant, If Spark Streaming is not supporting tuple, ...READ MORE
Option D) runtime error READ MORE
Option D: String class READ MORE
Hi@ritu, AWS has lots of services. For spark ...READ MORE
Hi@ritu, To start your python spark shell, you ...READ MORE
Option a) List(5,100,10) The take method returns the first n elements in an ...READ MORE
rror: expected class or object definition sc.parallelize(Array(1L,("SFO")),(2L,("ORD")),(3L,("DFW")))) ^ one error ...READ MORE
Hi@ritu, You can create a data frame from ...READ MORE
Hi@ritu, Spark's internal scheduler may truncate the lineage of the RDD graph if ...READ MORE
Hi@Edureka, Checkpointing is a process of truncating RDD ...READ MORE
Hi@Edureka, Spark's internal scheduler may truncate the lineage of the RDD graph ...READ MORE
Option c) Run time error - A READ MORE
error: expected class or object definition sc.parallelize (Array(1L, ...READ MORE
Hey, @Ritu, I am getting error in your ...READ MORE
After executing your code, there is an ...READ MORE
OR
At least 1 upper-case and 1 lower-case letter
Minimum 8 characters and Maximum 50 characters
Already have an account? Sign in.