7 Ways Big Data Training Can Change Your Organization

The recent news of the United Arab Emirates making military service mandatory for all Emirati men between the age groups of 18 and 30 years has prompted me to think as to why countries regardless of their economic status ensure that citizens be prepared to defend the country.

 One could argue that limited number of citizens in a country, often force the government to make military service mandatory. But what about China? It’s the largest country by population but it also ensures citizens going for further education serve compulsory military time. In short, nations are basically preparing themselves to defend in the event of conflict and everyone must be prepared for it. Be it an electrician, a businessmen, a carpenter, they all unite for a common cause.

 Bizarre as it may sound, one can draw an uncanny parallel between such nations and today’s organizations that want to stay competitive. The current threat or rather a challenge in the form of Big Data has prompted organizations big and small to bring together its manpower across various departments to commonly address it. To go further on this, typically nations enforcing compulsory military service always have an eligibility criteria, in the same way organizations find it logical in giving big data training only to those employees who have some form of interaction with large chunks of data and are required to employ Hadoop at every touch point.

 Just as an Army General in liaison with the government decides the kind of weaponry and training to be assigned to its otherwise novice citizen-turned-new-recruit, in the same way a CTO is expected to be at the helm of IT infrastructure and legacy systems driving new technology innovation to allow his/her employees perform better. With a shared objective to tackle big data, let us try to understand in detail where big data is used and why is it important to train your comrades in it.

1.Information Technology: Improving productivity with Big Data Training 

Perhaps at the forefront of big data implementation, the IT team is the epicenter for taking the change forward. An IT training decision maker who wishes to bring big data training to the employees has to begin with the IT department. Why? Because when it comes to engagement with technology at every stage of activity, the geeks at the basement (popular slang for IT) are the closest. So how relevant is it?

Let us look at a report submitted by the popular site, CIO, which states:

“According to a recent CompTIA survey of 500 US business and IT executives, 50 percent of firms that are ahead of the curve in leveraging data, and 71 percent of firms that are average or lagging in leveraging data, feel that their staff are moderately or significantly deficient in data management and analysis skills”

Given the fact that data management and storage is a part of the core function of IT, there is a need to have a parallel approach towards big data platform implementation and strengthening the IT skills within big data. Supporting the fact is a McKinsey report stating that by 2018, there will be a shortage of over 140,000-190,0000 professionals with deep technical and analytical expertise! As more and more technical professionals require big data training, organizations are looking to train technical professionals more for quick ROI and platform specialists, admin and engineers working in the IT department are at the helm of it.

Marrying Trinity of Core IT Function with Big Data

The term Trinity often reminds me of two religious concepts: One is the Hindu mythology of the creator, preserver and destroyer and the other is the Christian concept of the father, son and the holy ghost. Both strive for the betterment of mankind. In the same way, these three functions of an IT team strive for the betterment of the entire organization with departments with different needs when it comes to information technology. Apart from security and support functions, an IT department can relate to these functions when it comes to big data implementation.

Planning- The Planning activity within an IT team focuses on ensuring the organization’s IT strategy is aligned with the business objectives. This includes working on customizing software, bringing in new platforms that meet the needs of different business departments. In other words, any new implementation will always start from IT.

Network- It involves developing networks that facilitate all forms of communication between voice, data, video and Internet traffic and there are various checkpoints for recording data be it customer interaction, sentiment analysis and traffic update, they all collect data real time! An IT department often ensures smooth integration of networks to work along with objective of processing big data.

Data- To put it simply, an IT team brings in tools to collect, store, manage, secure and distribute data to employees for various strategic decisions in the organization. All forms of data like sales record, financial records, stock details are stored in a single data center. This creates a responsibility within the IT team to implement platforms for big data that allow designated users to store and retrieve information at any data location.

In any IT team, one needs a versatile mix of members with different tasks towards big data implementation. To begin with there is a need for a specialist who ensures smooth transitioning from traditional systems to big data platforms. For that one requires a techie to focus on maintaining the platform in its entire life cycle across all departments. Then comes a need for a member who must constantly monitor whether every technological implementation is aligned with the organizational objective.

2.Product Development: Rethinking innovation across all stages of R&D

Perhaps one of the most important departments when it comes to taking the organization into the next level of innovation! One of the biggest advantages of big data is integration of data across different touch points in product development right from product design, manufacturing, quality, warranty, diagnostics, vehicle and software applications. The data generated from these touch points defines the way the product is and how successful it can be. This basically takes product developers, R&D professionals and designers to the data-driven and data-analysis approach.

Engineering Big Data into reality

When it comes to product development, one popular example would be the driver less car that Audi is developing and planning to launch by 2016. Yes, there is the product development team that has the huge task of making sure the CEO’s vision of innovation is accomplished. But along the way, there are various challenges and questions right from development to testing that only big data can answer. Let us see why.

Consider a test-ride being monitored from point A to point B. Here are the kinds of data that can be generated:

a. Sensor Data – The sensors within the car could store details about the distance it had measured between cars behind it and in front of it and the frequency of vehicles it encountered in the journey.

b. Driver Data – Multiple tests with different age groups could be carried out and the details of comfort level, performance and how many times the driver needed to override automatic driving will be compressed into large sets of rows and columns for analysis.

c. Demographic Data – A test might be carried out in India and in the US. The A.I within the automatic driving could analyze the obstructions that it encounters in driving in two different countries. Which country is more viable for automatic driving and which county is not?

d. Market Performance Data – After the product is launched and it is on the road, engineers could also monitor its success by analyzing live data with feeds being provided 24×7 by the car’s program giving insights if the the introduction of automatic driving is helping in keeping the road’s safer?

There are N number of possible data that can be churned out from product engineering. We are just beginning to explore OEM from the auto-industry. Think about the possibilities of big data across various sectors say medicine, health care, electronics and so on. Who knows?

FUN FACT: Did you know that Ford’s adopting of Big Data and Analytics saved it from a near-death experience in the 2000s when competition was stiff from European and Asian Auto-makers!

3.Finance: Training employees on big data platforms to handle financial modelling 

We might have often heard the term that money is the blood of business. Taking care of that money is a responsibility of the finance department. The business world defines the functions of the finance department as typically being involved in ‘planning, organizing, auditing, accounting and controlling its company’s finances along with producing the company’s finances.

Having said that finance department in general is often the brainchild when it comes to handling money and the role expands to various activities like generating cash flow statements, cost modelling, prize realization and compliance to name a few. A few decades ago performing all these activities with limited systems and platforms was quite feasible, but in the age of big data the two challenges every finance department face is performing regular finance functions in the changing scenario and gathering insights for the future. Let us look at it from a deeper perspective.

With the information spread across different servers, organizations often encounter the challenge of consolidating that data and perform actions as per business requirements. An important function within is internal auditing that keeps a tab on organization’s governance, risk management and management controls and conducting proactive fraud audits to identify fraudulent acts. With the rise of analytics, there is a need to integrate internal auditing as well. This has sparked new methods like audit data analytics which help assess the risk, create financial models and give an overall picture of finance within an organization.

Cost Modelling & Price Realization

 Cost modelling is an important component for effective utilization of resources. Companies must identify the activities that drive costs, the total direct materials and labor needed for task completion and so on. Cost modelling helps companies to accurately identify the overall production costs to products across all activities within the company. In the age of big data it becomes important to keep a track of every financial activity taking place at different departments within an organization that consolidates that information to build an ideal cost model. From purchase to sale, all data gets stored in finance history and the fundamental basics of developing a cost model is to fetch the large chunks of data and create a model that can apply for the future.

 Although one can debate that Price Realization efforts are directed more towards sales to improve profitability, there is a greater role played by the finance department when it comes to benefiting from price realization. To break it down to simpler terms, consider a retail outlet that plans to provide discounts to push sales. The fundamental objective is to reduce price leakage and improve pocket price.

Price leakage occurs when the price of a product is discounted so less (in a bid to make sales) that they compromise on profitability and pocket price is the selling price post discounts. To fulfill a profitable price realization effort, the sales team collaborates with the finance department to understand the structure of costs for each individual products and where discounts can be given. This in turn requires the finance department to develop a framework for price realization models for the future and define the limits within such marketing activities. The task includes processing data from procurement, warehouse cost, shelf life and then estimating the cost of goods sold (CGS).

 F-12 & Predictive Analytics

One of the important activities within financial department is to monitor the financial health of the organization. Just as a doctor uses different metrics such as pulse rate, body warmth or stimuli reaction to judge whether the patient is alive or dead, in the same way the financial world monitors the 12 metrics to know where the company is headed monetarily and what lies beyond. From Real Revenue Growth, Sustainable Revenue Growth, Pricing Policy and Pricing Index, Operating Expense Control, Comparing EBITDA versus Cash Flow, Debt Free Cash Flow, Excess Cash, Return on Assets, Working Capital, Use of Debt Financing, Net Trade Cycle and Cost of Capital form important components in financial reporting for an organization so that the upper management can take sound decision.

As a part of the challenge in the big data world, understanding these ratios requires processing large chunks of information spread across the organization to make it in a standard format for analysis. Predictive analytics comes into play when this data is processed from past history, compared with the same elements in the present such that accurate estimations are made for the future. The best part is predictive analytics platform and methods are built to process big data thereby simplifying the finance department’s task.

 FUN FACT: Did you know that the Oversea-Banking Corporation (OCBC) based in Singapore was able to use big data for customer insights which was directly responsible for 40% increase in acquiring new customer!

4.Human Resources: Redefining HR employee capabilities

Imagining Big Data in Human Resources may often urge readers to dismiss as a humbug, since an organization typically doesn’t prioritize much in implementing Big Data technology in the HR department as it would rather focus on Marketing, Operation or Finance. But in reality, the Human resources department plays a crucial role in making sure that the right talent enters the organization among other activities.

Adding more teeth to the HR

Perhaps the most ignored among all departments when it comes to Big data implementation, but in today’s fast changing world, the way that an HR department works defines the success of an organization.

According to Forbes, an average large company has more than 10 different HR applications and their core HR system is over 6 years old. This trend highlights the fact that an organization needs the correct resources to bring this data together. Training in Big Data & Analytics brings skills like data analysis, visualization and problem solving right from operational reporting to strategic analytics.

An HR department by default is expected to deliver in terms of basic HR operations, but Big Data training takes it to a whole new level. As the HR department becomes more analytical with tools, it changes their approach to engage in more strategic activity. Critical question like how to have more employee retention; factors affecting sales; quality of candidate pipeline and evaluating talent gaps is identified and strategic steps are taken through analyzing relevant data through it.

The shift will move from simple headcount to more predictive analysis.

The Oracle within Human Resources

There was a funny story that I recall of a friend who worked as an HR. She had an exhausting job of headhunting before sending the candidate to the relevant department head who would only say the magic words: “Ok, lets hire him.”

For a while, things did go well as she brought in good talent to the company. As time passed, she grew confident in her hiring skills to the extend of pushing the upper management to add more people to her team, implementing HR systems and including more third party consultancies. The tricky part was she made tall promises to the upper management with her confidence.

History has shown that the one who prepares for future event is more successful than the one riding on past glory. There was a time when she was expected to hire a large number of professionals in the domain the company was expanding in. She began filling vacancies with a compromise on hiring quality professionals. She adopted a more target-driven approach. The result? Most of the professionals she hired put down papers citing various reasons and she was questioned by the management. Often I would hear her mumble:

“I head-hunt 1000 Cvs, shortlist 100 Cvs, call 50 candidates for interview, filter 10 from my psychometric assessments, among the 10, I take 5 who are worth it, send the 5 to the management, they zero in on 1 and that one guy leaves after 2 months.”

I did chuckle at her misery apart from offering my sympathies, but it made me wonder whether human resources can make better judgment with their experience or is there a need to have a more data-driven approach to this whole hiring process? Well, we do use predictive analysis from finding which team is going to win the world cup but why not use the same techniques in the hiring process, especially when we are dealing with complex elements such as human beings?

 Now, the job of hiring is not necessarily an easy job, it involves a lot of processes and the rules of hiring often change according to the industry the HR is in; the role she is hiring for; the rules of the organization and so on.

If one observes successful organizations that use predictive analytics and have lesser attrition rates, there is a pattern of first deciding on the desired characteristics within a candidate that ensure success, consolidating it into an ‘ideal’ profile and comparing it to every candidate who is closest to it and then engaging them with customized assessments that evaluate the characteristics of these candidates.

A point to note is that the whole psychometric assessment industry with leading players such as Pearsons, Thomas Assessment & SHL sprung up due to the demand from HR professionals for analyzing candidate profile in their need to perfect hiring process!

Getting back to predictive analytics, as a part of implementing it, the HR personnel must first define who is a ‘successful candidate’ according to the organization, then she/he must define the factors that can drive effectiveness of hiring and develop and observe as to why some hires do better than the others with a hypothesis if need be. Based on that, she/he can compare it with the data of successful employees who have stayed long with the organization and thirdly use statistical techniques to measure why some people stay longer.

The approach is good for a start, but implementing predictive analytics within HR includes a lot of techniques that an HR is free to explore. The best part of this process is the reduction in the cost of replacing an employee with new ones and perhaps gaining more ROI than the old one.

At the end of the day, the combination of intuition, experience and a sound data-driven approach often refines not only an HR’s judgment but ours as well.

FUN FACT: Did you know that American giant Xerox reduced its call center turnover by 20% by applying analytics to prospective candidates with the finding that creative people were more likely to remain with the company for the 6 months necessary to recoup the $6,000 cost of their training than inquisitive people?

5.Supply Chain & Logistics: Training delivery team with big data platforms

Supply Chain & Logistics basically form an important component in organizational strategies and goals. The objective for Supply Chain & Logistics is in saving costs and improving performance, speed and agility. When it comes to logistics, they capture and track different forms of data to fundamentally improve operational efficiency, improving customer experience and new business models. These factors can often help organizations to conserve resources, build a better brand name and create a systematic process for supply chain & logistics.

Tracking Big Data across the world

Let us take an example of a e-commerce giant which uses Big Data for delivery to its customers. A product is dispatched from a location to the address of the customer. Devices within the transport vehicle such as GPS tracker, mic, sensor has structured and unstructured data that are sent back to the monitoring center for real-time updates. Along with that it also helps analyze the efficiency of delivery time, shortest path and the resources used to perform one delivery operation in the list of millions of such transactions. This gold-mine of data across different markets is consolidated by the organizations and then analyzed to bring further improvement in the process or bring a whole level of new innovation!

FUN FACT : Did you know that Big data in the form of tracking customer pages by Amazon has helped it to position its products to the warehouse nearest to customer in order to improve delivery speed and efficiency?

6.Operations, Support & Customer service: Employee training on big data at every customer interaction

The success of any product or service is based on the after-sale support that a customer receives and often the vendor takes an oath to be there for him/her at all times. This comes from the fact that when a customer takes a product or a service, he makes a ‘leap-of-faith’ in the hope that the vendor doesn’t let him/her down in the lifespan of the product/service. Delivering from this perspective is critical for organizational success.

Let us look at support at a granular level. I recently had the opportunity to watch Christopher Nolan’s ‘Interstellar’ that explored space travel to the end of space. This got me into thinking about future airlines that will offer flight services through worm-holes spanning millions of light years away! What would be the challenges then? What kind of big data is going to be generated in this almost never-ending journey? How will the on board team ensure that the passenger enjoys the ride throughout? To begin with, the service provider must focus on primary objectives like ensuring air-safety, keeping track of its flight path, delivering customer requirements and so on.

On-the-go Big Data 24×7

The idea for interstellar travel might be a distant dream for the next 100 years (being optimistic!), but it doesn’t stop us from looking at the data being generated by a similar service currently operational now which will shed more light on how customer service & support is carried out in the ‘after-sale’ scenario and how organizations can engage in improving their efforts in real-time.

Now to begin with, Southwest Airlines is one of the most celebrated airlines that took advantage of Big data in order to improve its customer experience. In its bid to improve air safety, Southwest Airlines collaborated with NASA to engage in big-data experiment for improving overall flight experience. This includes pinging NASA satellites with information on flight path, reports from pilots and other air traffic information. At the pinnacle of such innovative technique, there lies the basic big data concept called ‘text data-mining’ which converts unstructured textual information into meaningful text for insights. So you thought text data-mining ends there?

Of course it doesn’t, even a simple concept in big data such as text data-mining extends way beyond that. We all know that customer feedback is an important component in understanding where an organization goes wrong at every point of customer interaction. Text data-mining also helps customer-service by analyzing open-ended survey responses. Instead of constraining customers to common options like option A, option B, option C, open-ended questions provide more insights, but classifying them and recording the responses may be a key issue. That is where text data-mining comes into play where it groups certain set of words and consolidates them for insights!

Looking beyond that, we all must admit that no organization is perfect and that every one of them has a small set of customers who may not be happy with the service. The result? A database flooded with email, messages, tweets from customers registering complaints or ‘areas-of-improvement’ tips to put it rather softly. Text data mining goes a step ahead from traditional mail filters and can classify mails as per the priority and reroute it to the department in question.

FUN FACT : Did you know that Southwest Airlines, as part of its effort to improve customer services has deployed data analysis with the feature called ‘speech-analysis’ that records interaction between customer and personnel for insights!

7.Marketing: Training employees on a systematic marketing approach with big data

Marketing as an activity is all about numbers today. With the surge of digital marketing, we can now accurately measure the response of ads, click-through-rate, impressions, ROI and so on. For a non-marketing professionals, such metrics maybe greek, but for those in marketing this data is a gold-mine. Subsequently, along with metrics, large chunks of data is generated across at every point of customer interaction, social media & sales. It is up to the marketing professional to keep track of such data and use it to push one’s products more effectively. Training in Big Data plays an essential role here since platforms like Hadoop & R help serve the purpose.

Secondly, time-to-time marketing professionals often indulge into retrospection for their brand. Questions like :

How is my brand better than others?

What do other brands offer?

What features does my competitor have on the same product?

The study goes much deeper than this. From analyzing competitor product based on the 4Ps (Product, Price, Place, Positioning) to understanding content of which product presented in the competitor’s webpage, the amount of data generated is huge and complicated. As told before, taking advantage of text-mining can help the marketer perform competitor-analysis by simply crawling the competitor’s website. This simple function in the domain of big data can give a consolidated idea about what the competitor is doing and what products they have in place for the market, thereby giving the marketer who embraced big data an edge!

Arming the Creative

For example a social media strategist wants to know about the brand perception of his organization across social media platforms, then probably engaging in sentiment analysis in R & Hadoop will help achieve this goal. In the same way, use of Big Data tools helps marketing at various activities such as pricing, product positioning and so on.

Another example could be a marketing manager at a retail outlet looking to maximize sales. Everyone would know the example of Walmart which was able to position beer and milk side-by-side in the aisle based on past customer purchase history by retrieving large chunks of data spanning millions of customers over a timeframe!

FUN FACT: Did you know that General Motors with its yearly marketing budget of $2 Billion per year used Big Data Analytics to create detailed customer profiles and combine spatial data analytics with detailed demographics/customer information for more personalized marketing!

Why companies are shifting to Big Data platforms

Typically, organizations using old legacy systems have data spread across many systems. Due to spread of data at different locations, the processing speed goes down along with accuracy of analyzing data. This calls for consolidating data within an enterprise data hub which creates a faster access of data resulting in deeper analytics. One of the important objective of the IT department in any organization is to provide accurate data swiftly for all departments in the organization upon request.

With data being collected, it is important to unify unstructured, structured and semi-structured data sources onto one platform to perform in depth analysis and basically aid business decision making. This feature of Hadoop brings in more people to the table within the organization since there are employees who interact with data at different touch points in day-to-day operations. Also, traditional ETL and batch processes can take a long time, whereas Hadoop with its high volume batch processing speeds it up to 10 times.

The significance of Hadoop doesn’t necessarily mean that every employee within an organization needs to be trained in the Big Data platform which may not be feasible in most cases. But it would be of strategic advantage for a CTO to identify and train those professionals who are in constant interaction with data.

Having covered the storage, processing, retrieval of data through the popular Hadoop platform, another important phenomenon that is a part of the natural progression is the Big Data analytics. To put it simpler, organizations needs multiple perspective from various professionals within an organizations.

The number ‘6’ can be viewed as the number ‘9’ from the other side of the table. In other words, conclusion from observing data differs from person to person.

Organizations know this and often engage in training employees in similar platform so that people from different departments interconnected by the same activity discuss, engage and share insights for sound decision making. So, I believe it would be safe to define Big Data training as an opportunity for every employee to be on the same page and take organizations to the next level!

Got a question for us? Mention them in the comments section and we will get back to you. 

Related Posts:

Get started with Big Data and Hadoop