As the name suggests, Data Warehouses are central storehouses of data – current and historical. Data is usually collated in a data warehouse from incongruent sources and is utilized for creating analytical reports of various kinds for different levels of users and departments.
The graphic below is a comparison between what worked until now and what will work in the future for data warehousing.
(Source: Tech Target)
Given the investment of time, effort and money that goes into designing a data warehouse, companies should try and avoid certain pitfalls. We will take a look at some of them below.
Common Pitfalls of Data Warehousing
#1 Lack of scalability
The scale and nature of your business will not stay constant forever. According to a study by the Gartner Group, companies can expect users to request changes that will affect 35-50% of application functions during the first year of a data warehouse / BI implementation (Source: SQL Server Pro). So you might be in for trouble if you have not balanced hardware and software costs with the fact that you will have to scale eventually.
Another crucial aspect here is a good change-enhancement request system. Changes are unavoidable so a good system will allow you to manage them better.
#2 To buy, rent or use cloud?
If you are a company that started practicing data warehousing early on and invested heavily on data severs at that time, you may now face a dilemma once you need to scale. Should you buy, rent or use cloud? Whilst buying or renting data servers can be an expensive proposition, adopting cloud infrastructure for data warehousing seems to be the way forward.
But can companies trust the cloud for storing sensitive customer data? According to Gartner, “At year-end 2016, more than 50 percent of Global 1000 companies will have stored customer-sensitive data in the public cloud.” (Source: Wired)
#3 Reporting level priorities misjudged
Would you prefer a data warehousing system that is fast and accurate or visually very appealing at the cost of speed? Managers who realize that systems that are quick and responsive score over those that are simply full of well made charts and graphs but take forever to load will avoid this roadblock.
For example, a Manager is stepping in for a review meeting for the Board. During the meeting, he has been asked to display certain data points. In a simple report form, this information loads in 10 seconds and a chart takes upwards of 2 minutes. The Board members would definitely lose patience beyond a few seconds!
#4 Lack of business alignment
Any data warehouse design must align with the business and also receive a green signal from various departments and technical groups. Go top-down right from the topmost positions to the lowest level users and try and get everyone’s buy-in. This not only ensures their active participation and support but also ensures that the data warehouse is fulfilling its promise of providing useful information to its users.
Two companies were at the nascent stages of their data warehouse projects. One formed a multi-disciplinary committee at the very start with members from the business and technical side and the other went about the entire process in a very ad-hoc manner. No prizes for guessing who was successful – the former.
#5 Lack of data integrity
In a data warehouse, data is collated from multiple sources. It is therefore essential to check the data for any duplication and inconsistency. Inconsistent data of poor quality can defeat the very purpose of a good data warehouse. In this field, it is common to come across the phrase ‘Garbage In, Garbage Out’ which only ratifies the need for quality data.
Decentralizing data check points and making business units accountable for the quality of data that they provide is one way out. Also having audit mechanisms in place is another.
#6 Lack of analytics understanding
Analytical tools and reports, which are the powerhouse for managers to take decisions, have to be well incorporated in the design stage of creating a data warehouse. What kind of data is needed and what analysis will help improve the company’s performance is something that business users need to clearly articulate.
Being unaware of what all a data warehouse can do and perform will limit the ability of the system to be used to its full potential.
Technology parameters to overcome these issues
The graphic below is a schematic presentation of the various technology parameters that data warehousing can face and suggestive techniques to overcome them.
(Source: Technology Advice)
Managing data for an enterprise involves a concerted effort and data warehousing can prove to be the right tool. Whether the information is sought for management purposes or business intelligence efforts, the right mix of business acumen and technology can provide a foolproof solution.
Powered by Ivyclique
Got a question for us? Mention them in the comment section and we will get back to you.