The Blastrac Project and Stephens Few's Tip

In the fall of 2007 I had a big problem.

As the CIO of a mid-sized manufacturing company with operations all over North America, Europe and Asia I had a challenge provided to me by the President of our company. The challenge was this:

"Get me a solution for reporting key financial data consistently for the entire global operation."

So, In early 2007 I spend months researching Global (ERP) Enterprise Reporting Systems. We looked very seriously a 5 major vendors. At the end of the process I determined that we just could not afford to wait two years for our two million plus dollar solution to be implemented. We needed solutions now!

Next I investigated what the Big BI (Business Intelligence) companies and tier-2 providers had to offer and how they could help my small team implement a data warehouse/business intelligence reporting system.

While the products and services these companies offer were interesting and demonstrated the potential of information stored in a way that was optimized for end user analysis, they all suffered some common pitfalls:

1. Still too expensive (quotes ranged from $130,000 to $580,000).
2. Poor data visualization (not interactive, required too much user skill).
3. Still required too much time to implement (6 to 9 months).

I was worried. Nothing seemed to fit our particular needs which were:

1. A quick solution that could provide immediate benefit.
2. A cost that was affordable for us (under $100,000 to implement).
3. High accessibility and flexibility for the end user. It had to be simple.

I started reading everything I could find on data warehousing, data architecture, business information systems and data visualization.

Luckily I encountered two books by Stephen Few; "Information Dashboard Design" and "Show Me the Numbers." Stephen clearly explained the importance of data visualization and the best practices in the field.

His website: www.perceptualedge.com was a goldmine of information on what to do and what NOT to do.

Ralph Kimball's books were also highly informative and totally demystified the jardon of data warehousing & data architecture. His book "The Data Warehouse Toolkit" heavily influenced my thinking on the right way to implement a data warehouse design.

You can investigate Ralph's methods here: http://www.ralphkimball.com/

What Connected the Dots?

I emailed Stephen Few in November of 2007 telling him about my challenge and asking for his help. To my surprise, Stephen responded the next day with some good news and bad news.

The bad news...he was booked for the next six months. The good news, he provided a tip which lead to an answer that met every objective I had and resulted in a fully-implemented solution (as far as the information consumers were concerned) in January of 2008.

That's right. We had a 100% solution in the hands of managers in under 3 months!

My cost to implement was under $7,000. My proof of concept amazingly required NO UPFRONT INVESTMENT. This was truly an amazing find and I owe Stephen Few a lot for the generous tip he provided in his email.

This blog will detail all the steps that I took to progress from ground zero to a fully implemented solution in weeks, not months.

Stephen Few's Tip

Stephen's tip was this: "Dan, you should look into a company called Tableau Software. They have a low cost tool that works very well."

I did and within days I knew that I had a solution which met all of our objectives:

1. Low cost (could be implemented for 10% of what Big BI players were going to charge).
2. Fast (I could implement the solutions quickly and incrementally)
3. Low Risk (the system could progress to a full-blow enterprise data warehouse but at a fraction of the cost that Big BI had to charge).

The Solution

This blog will detail every step we took to enact a highly accessible and flexible business information system using Tableau Software as the front end data access and data analysis tool.

I will detail the specific step that we took in separate articles on this blog; however, the major steps were:

1. Recognize and understand the value of light-weight data visualization tools. Tableau Software was the key enabling technology.
2. Realize that with Excel spreadsheets we could mimic a Ralph Kimball style Star Schema data architecture. (That term will be explained in detail in a separate blog post).

You can find a definition here: http://en.wikipedia.org/wiki/Star_schema

3. Immediately implement "Dimensionally-Conformed" spreadsheets with Tableau Software bolted on top. I will specifically address what a I mean by dimensionally-conformed in a separate blog post.

4. Clean your granular data with in-house resources who know the systems, internal jargon and process better than any consultant can.

5. Hired a skilled data architect to implement the data warehouse design we proved worked in steps 1 - 4.

The best part about the whole process was that we gained a very detailed understanding of the data quality issues and we took corrective action to address the data quality issues AS WE PUBLISHED reports. The end users got the data they needed and quick.

The last step was the implementation of a full enterprise-class data warehouse which was accomplished in under 5 weeks. The end users didn't notice anything when this was happening. No interruption of service or the reports they had been receiving since January of 2008.

In July of 2008 when we implemented the data warehouse back end to our Tableau Software-enabled solution the end users just gained access to a much larger amount of historical data.

By September of 2009 the solution was implemented in all of our North American and European locations with great success.

The End Result

The company I hired to implement the last phase of the project was InterWorks Inc. The enterprise data warehouse phase of the project was almost anti-climatic, but the additional historical data was highly beneficial to our user base.

I ended up coming to work at InterWorks Inc. to help other companies achieve the kind of benefit that I had on my project.

On my first week on the job I spoke to Brian Bickell. Brian was the InterWorks data architect that I worked with on my project.

I asked Brian this question:

On a scale from 1 to 10, how did my project compare to other data warehouse projects he had worked on? Brian's answer, it was an 11.

I asked him why he felt the project went so well. His outlined several areas:

1. You knew what you wanted. Very specifically.
2. You knew your data quality issues and had largely solved that issue.
3. You provided the logic I needed to audit data being loaded into the data warehouse.
4. You allowed me to architect the solution using an validated process.
5. You were flexible about suggested changes in the data architecture which allowed to adapt the schema to take full advantage of MS SQL Server.