What are Dstreams

0 votes
Hi everyone,

I am new in Spark Streaming. Can somebody tell me what is concept behind Dstreams?

Thank You
Feb 4, 2020 in Apache Spark by akhtar
• 38,230 points
305 views

1 answer to this question.

0 votes

Hi@akhtar,

Dstreams are the basic abstraction that is provided by Spark Streaming. It represents a continuous stream of data, either the input data stream received from source may be from flume, kafka  or the processed data stream generated by transforming the input stream. Simply, it is a collections of continuous series of RDDs. You can see the below figure for more understanding.

Spark Streaming

Hope this will help you.

Thank You

answered Feb 4, 2020 by MD
• 95,360 points

Related Questions In Apache Spark

0 votes
1 answer

What are the parameters in local[a,b,c] explains?

SparkContext.createTaskScheduler property parses the master parameter Local: 1 ...READ MORE

answered May 29, 2018 in Apache Spark by Shubham
• 13,490 points
319 views
0 votes
1 answer

What are the levels of parallelism in spark streaming ?

> In order to reduce the processing ...READ MORE

answered Jul 27, 2018 in Apache Spark by zombie
• 3,790 points
2,912 views
0 votes
1 answer

what are the spark real time issues ?

Some of the issues I have faced ...READ MORE

answered Mar 18, 2019 in Apache Spark by Sharman
4,082 views
0 votes
1 answer

what are the spark job and spark task and spark staging ?

In a Spark application, when you invoke ...READ MORE

answered Mar 18, 2019 in Apache Spark by Pavan
9,180 views
+1 vote
2 answers
0 votes
3 answers

How to connect Spark to a remote Hive server?

JDBC is not required here. Create a hive ...READ MORE

answered Mar 8, 2019 in Big Data Hadoop by Vijay Dixon
• 190 points
8,949 views
0 votes
3 answers

How to transpose Spark DataFrame?

Please check the below mentioned links for ...READ MORE

answered Jan 1, 2019 in Apache Spark by anonymous
17,021 views
0 votes
2 answers

In a Spark DataFrame how can I flatten the struct?

// Collect data from input avro file ...READ MORE

answered Jul 4, 2019 in Apache Spark by Dhara dhruve
4,539 views
0 votes
1 answer

What is pageRank in graphX??

Hi@akhtar, The PageRank algorithm outputs a probability distribution ...READ MORE

answered Jul 22, 2020 in Apache Spark by MD
• 95,360 points
471 views
0 votes
1 answer

What is the difference between spark streaming and spark structured streaming?

Hi@akhtar Generally, Spark streaming  is used for real time ...READ MORE

answered Feb 4, 2020 in Apache Spark by MD
• 95,360 points
2,185 views
webinar REGISTER FOR FREE WEBINAR X
Send OTP
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP