How Predictive Analytics works with Big Data?

Recommended by 48 users

Nov 11, 2014
How Predictive Analytics works with Big Data?
Add to Bookmark Email this Post 2.4K    5

 Here is an incidence which took me by surprise, some time back:

About a year back, we were expecting a baby in some time and had decided to have the baby in our native place – so I was shuttling between the two cities Gurgaon, where I work and Indore – where we were planning to have the baby.

My wife was already in Indore for a few days. In order to make sure that I don’t miss out to capture any moment of the grand event, I bought a brand new, top of the line Android phone. Google had shipped a new app called “Google Now” along with this phone. When I opened the app, it looked just like a new interface to the Google Search.

I had assumed that this would be one of the apps you get with your phone, but never use! They just lie on your phone as they are shipped with the hardware. But Google had some other plans!

Exactly 7 days after I had started the app (and almost forgot about it), I received a notification on my phone that I am 30 minutes away from my home and the traffic is heavy! I was shocked, amused, scared and excited at the same time.

The normal human inside me was scared due to loss of privacy, which happened in the subtle manner and the analyst in me was excited by the feat Google had achieved!

Over the next few days, Google Now had trained itself to understand which sites I browse regularly (and updated me whenever there was a new content on these sites), which restaurants I like to visit, the bills I need to pay and my commuting pattern in both the cities. Today, Google Now would be one of the most used app on my mobile and I am in love with another product Google has created!

How does all of this relate to this article? Well, it just showcases what could be achieved by combining 2 of the hottest technologies in the world today – Big Data and Predictive Analytics. Let us understand how Big Data enabled with Predictive modelling is changing the way organizations are interacting with their customers.

How are the two technologies connected?

Traditionally, analysts used to work on samples of population and not the entire population. This was because of multiple reasons, including availability of data as well as the limitations of available data infrastructure and tools.

Now, with explosion in data generation (mobile, wearable tech, logs on internet, social media) and Big data tools and techniques, these are no longer a limitation. Analysts can now analyse information at a far more granular level, enabling personalized recommendations like never before. This is the reason why Google is able to suggest me the websites I might like based on what I have searched recently or the restaurant I might like as soon as I land in a new city!

Increased variety of data is making our predictions better

Experts estimate that more than 80% of the data which is generated is unstructured in nature. We used to almost ignore this data till a few years back, including data sources like our notes, social media activities, telephone conversations and chats over internet. Today all of this can be analysed to bring out insights in real time and customizations at a personal level. The way you like and comment on various posts on Facebook enables them to show the most relevant information and advertisement to you! There has been a dramatic increase in variety and veracity of data being analysed in the last few years.

Everyone has access to the cloud!

Even if a person (or an organization) manages to get the data about his customers, it is very difficult for them to analyse it without making a huge investment in the required infrastructure. Today, you can rent a server on the cloud and pay only on the basis of data you have analysed! This has made building predictive models on big data easier and far more accessible.

Business benefits from building predictive models on Big Data:

It is difficult to list every possible benefit here. But here are the key ones:

  • Customer delight and better experience resulting in increased sales and higher satisfaction
  • Integration of several data sources to provide single view about the customer
  • Shorter lead and waiting cycles (e.g. Amazon trying to patent predicting deliveries in areas based on the browsing pattern of the customers)
  • Conversion of dormant data  in the organization into nuggets of usable information

A few applications / examples of Predictive models built over big data:

  • Amazon delivery prediction – Amazon recently filed a patent, where it predicts consumer purchase before it actually happens, based on the consumer browsing pattern. This helps them reduce the shipping time, which they are highly obsessed about.
  • Netflix – Netflix is known for streaming movies. They are equally known for use of big data to predict customer viewing habits. Netflix today stores information from more than 30 million users, which movies did they watch? When and how many times were they paused? The ratings provided along with geo-location, device and social media data. They then use this data to come up with insights into consumer behaviour. This big data analysis led them to determine that ‘Houses Of Cards would be immensely popular before they actually launched the series.
  • Stock market predictions – Stock markets are highly volatile in nature. It has been hard to predict their movements accurately in the past. Today, there are companies which are using traditional information like accounting data along with market sentiment from Social media to predict future prices of stocks.
  • Social media-based lending – Another interesting case study is that of LendUp – a startup in the US. They use social media data along with Credit Bureau information to assess creditworthiness of their applicants. By doing so, they are able to make decisions based on bigger picture and can lend to people with bad credit history.

Hopefully, this article has piqued your interest in Big Data and predictive modelling. As I mentioned, they are two of the hottest technologies across the globe and are changing the way business decisions are happening in big organizations or start-ups alike.

Got a question for us? Mention them in the comments section and we will get back to you. 

Related Posts:

Get started with Big Data and Hadoop

Big Data Challenges

Why BI professionals need to be skilled in Hadoop

About kunal jain (1 Posts)

Kunal is the founder and CEO of Analytics Vidhya, a leading knowledge portal for analytics professionals across the globe. He has also worked with the likes of Capital One and Aviva Life Insurance.


Share on
Comments
5 Comments
  • Niranjan

    The traffic example this author has given is “Descriptive Analytics” and not “Predictive”.

    The Google example he gives, is “Prescriptive Analytics”.

    “Predictive Analytics” would have told him a day or even more ahead of time that he would encounter a traffic jam on such and such a day and to take corrective action. As an example (Transportation) Planners are trained in “Predictive Analytics” to predict what a city would look like 25 years from now (planning for you).

    Big Data still requires sample data to build models. What Big Data brings to the table is “Streaming Analytics”.

    “Analytics” does not require an computer language, but one needs to learn the packages that are common (SAS, SPSS, R, Octave, Gephi, etc – none of them require one to learn any computer language).

    Where one needs a computer language is to gather the data and clean it. Java is the go-to language, but one can easily survive in C/C++/Pythin/etc.

  • Jayanti Gala

    I am a finance and accounts professional and am not familiar with Java or other elementary languages.

  • Jayanti Gala

    Which computer languages do i need to learn to master big data and predictive analysis

    • Kunal

      Jayanti,

      The best thing would be to start with one language – either Java or Python. Python has quite a few native libraries for predictive analysis and machine learning, Java on the other hand is more evolved in Big Data ecosystem. However, you would be able to perform the tasks you want on either one.

      Regards,
      Kunal

  • Nathan Haynes Arnett

    traditionally, statistics were used to make inference about a population via experiment or observational study. Now with BigData tools, we can have the population of interest sitting on a distributed file system, pretty cool.

24 X 7 Customer Support X

  • us flag 1-800-275-9730 (Toll Free)
  • india flag +91 88808 62004