How to create RDD from an external file source in scala

0 votes
Can anyone suggest how to create RDD from and external sources in scala?
Jul 4, 2019 in Apache Spark by Reena
791 views

1 answer to this question.

0 votes

Hi,

To create an RDD from external file sources, you need to do this below steps following:

scala> var test = sc . textFile("hdfs://localhost:9000/exaample/sample")

The external file here is hdfs. So, you can see the example below:

So, here you can see its created, a new RDD from external file sources that is hdfs.

Hope this helps!

To know more about Spark Scala, It's recommended to join Apache Spark training online today.

Thanks!!

answered Jul 4, 2019 by Gitika
• 65,970 points

Related Questions In Apache Spark

0 votes
1 answer

How to create RDD from existing RDD in scala?

scala> val rdd1 = sc.parallelize(List(1,2,3,4,5))                           -  Creating ...READ MORE

answered Feb 29, 2020 in Apache Spark by anonymous
520 views
+1 vote
1 answer

How can I write a text file in HDFS not from an RDD, in Spark program?

Yes, you can go ahead and write ...READ MORE

answered May 29, 2018 in Apache Spark by Shubham
• 13,480 points
5,694 views
0 votes
0 answers

How to create RDD as string file?

Can anyone suggest how to create RDD ...READ MORE

Jul 5, 2019 in Apache Spark by anand
312 views
+1 vote
2 answers
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
8,034 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
1,366 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
66,820 views
0 votes
1 answer

How to create RDD from parallelized collection in scala?

Hi, You can check this example in your ...READ MORE

answered Jul 4, 2019 in Apache Spark by Gitika
• 65,970 points
814 views
0 votes
1 answer

How can you use "for" statement in scala to print list from collection?

Hi, You can use for loop in scala using ...READ MORE

answered Jul 5, 2019 in Apache Spark by Gitika
• 65,970 points
180 views