Last month I wrote a blog post which proposed some reasons why the data always seems to be "f'd up" at companies with centralized data warehouses. The reasons I cited were equally divided between the business and IT sides of a company. In this blog post, I will propose some solutions to these issues.
IT Contributions to Messed Up Data
- Lack of Data Warehouse Modeling Experience: Easy solution.... get developers that have data warehouse modeling experience (theoretical and practical). If you have existing developers who don't have data warehouse modeling experience, either move them to a different (non data modeling) role or provide them with proper training. If they don't want to do either, as Alec Baldwin said in Glengarry Glen Ross, tell them to "hit the bricks", as they will be a detriment to you, and your organization, in the long run.
- Overwhelmed with New Data Requests: Implement processes and procedures to handle new data requests. This could also include implementing new technologies to aid in this process. And if necessary, lobby to get more money and hire more resources.
- Lack of Processes to Manage Source System Data Changes: Implement processes and procedures, with source system database administrators (DBA's), to alert the data warehouse DBA's that changes have been made on the source system(s) that will effect the data warehouse. Like above, there may be technologies that could be acquired or built that could aid in this process.
Business Contributions to Messed Up Data
- Lack of Basic Understanding About IT and Data: Similar point #1 in the IT section above, get business people that have a basic understanding of database principles and have worked with IT people on data previous warehouse projects. If you have existing business people who don't have this experience, either move them to a different role or provide them with proper training. If they don't want to do either, get rid of them, as they will be a detriment to you, and your organization, in the long run.
- Haphazard, Last Minute Data Requests: l know in many cases that this is unavoidable (due to current business conditions) but in some instances this behavior is unnecessary and can be avoided by a little proper prior planning.
- Not Educating IT on How Business Actually Uses the Data: In this instance a little bit of time spent, on the part of the business, can go a long way. This could include scheduling educational meetings with the IT department to demonstrate how their efforts translate into business value. Or it could be little efforts on the part of the business. I remember one time when I was talking on the phone to a DBA in India about a problem I was having with the database. To better demonstrate what I was doing with the data, I set up a web conference with the DBA to show him exactly how I was using the data (on on front end). Suffice-it-to-say, I made a friend for life!
