Join in RDD using keys

Question

Hi Team,

How can I join two rdd without converting into dataframe?

rdd_x=(k1, V_x)
rdd_y=(k1, V_y)

Result should be like this: (k1(V_x, V_y)

Answer 1

Suppose you have two dataset results( id, result) and student(name, id). Now, you can join the RDD by using the below commands in Spark on the basis of the common key id.​

case class results (roll_id: Int, result: String)

case class students (name: String, roll_id: Int)

val a = sc.textFile("file:///home/edureka/Desktop/all-files/datsets/f1").map(_.split("\t" ))  // mention complete path for input dataset

val b = sc.textFile("file:///home/edureka/Desktop/all-files/datsets/f2").map(_.split("\t"))


val class_a = a.map( z => (z(0).toInt , results(z(0).toInt , z(1))))

val class_b = b.map( z => (z(1).toInt , students (z(0), z(1).toInt)))


val v_join = class_a.join(class_b) 

v_join.foreach(println)

​