Apache Storm (6 Blogs) Become a Certified Professional

Real Time Storm Project

Last updated on Jun 26,2020 3.4K Views

Awanish
Awanish is a Sr. Research Analyst at Edureka. He has rich expertise... Awanish is a Sr. Research Analyst at Edureka. He has rich expertise in Big Data technologies like Hadoop, Spark, Storm, Kafka, Flink. Awanish also...

In this blog, we are going to execute a real-time storm project.

ICD9Data.com provides disease codes ranging from 000-999. These codes are assigned to different diseases uniquely. It is available as a mobile app also.

Storm-Real-Time-Project-2

Let us consider any mobile app used by doctors is generating sensor data. This sensor data has information such as device id, device network, user, latitude, longitude, time, disease code etc. For our scenario, we will be considering only latitude, longitude, time and disease code.

These data are processed using storm and trident topology to detect disease outbreak within any geographic location for disease code less than or equal to 322 at a particular time. If the count of a particular disease increases the threshold, in this case 2,000, then the system will send an alert message with the name of the country, disease code and time. To simplify things for this scenario, we will map every diagnosis event to the closest country.

In this scenario, we will use disease codes between 315 and 322. Refer the table below for diseases and their codes.

Storm-Real-Time-Project-1

Below is the flow of the project which is very self explanatory.

Storm-Real-Time-Project-3

Step 1: Sensor data is streamed by spout from the mobile app.

Step 2: Spout sends only latitude, longitude, time and disease code as tuple.

Step 3: The disease codes below or equal to 322 are filtered and they are sent forward.

Step 4: Depending on the latitude and longitude, a country is assigned.

Step 5: Hour is assigned to each tuple.

Step 6: GroupBy key is done where key is country, dcode, time. Count is calculated after each tuple is processed.

Step 7: Outbreak Trend State ensures that whatever is processed once should not be processed second time.

Step 8: We have kept a threshold of 2,000, if the count crosses the threshold, we consider it as an outbreak and alert message is dispatched with the details of the outbreak.

Download the storm project.

download_code

Firstly, start the storm cluster, then run the storm project using below commands.

Command: cd storm/

Command: bin/storm jar /home/edureka/SensorDataAnalysis/SDAnalysis/target/sensor-data-0.0.1-SNAPSHOT.jar in.edureka.trident.topology.DiseaseAlertTopology

The threshold of 2,000 crossed in Turkey for disease code 320 and Disease Outbreak Alert was sent.

This project is just to give you a practical view of storm’s trident topology concept. Here we have taken all the inputs randomly. You can put your business logic in the code according to your data and test it.

Got a question for us? Mention them in the comments section and we will get back to you.

Related Posts:

Introduction to Real-Time Analytics with Apache Storm

Lambda Architecture

Apache Storm Use Cases

What is Apache Storm all about?

Comments
0 Comments

Join the discussion

Browse Categories

webinar REGISTER FOR FREE WEBINAR
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP

Subscribe to our Newsletter, and get personalized recommendations.

image not found!
image not found!

Real Time Storm Project

edureka.co