Changing Times for Data Professionals

January 23, 2013 – Cross-posted from the PASS Business Analytics Conference Blog
By Chris Webb

One thing that’s always surprised me in the 15 years I’ve worked in the business intelligence industry is how little I’ve been involved in the analysis of the data I’ve been working with. Maybe this was because some of the business users I’ve worked with haven’t been very interested in analysing the data themselves. I deliver a nicely formatted report with a table and a few charts on it. They see whether their sales are going up or down, they get a warm fuzzy feeling (if sales are going up) or some harsh words from their boss (if sales are going down), and that’s it.

I’ve never been happy with this state of affairs, though. Many companies have ignored the potential of the data they’ve captured. What’s more, I feel I’ve been missing out as well – left out of the rewarding process of digging through terabytes of data to unearth some previously unknown insight that could transform my customer’s business.

Fortunately, times are changing. All the recent discussion around “big data” and “data science” is enticing more companies to do something useful with all the data they’ve piled up. This, in turn, presents a challenge and an opportunity to business intelligence professionals like me. The challenge is that now in addition to ensuring that the numbers are correct and the reports are pretty, I also have to be able to help my customers understand their data. I need to know about data visualisation techniques and understand why pie charts are a bad idea and why 3D graphs aren’t cool or clever. I need to know about data mining, how to calculate a forecast using linear regression, and that correlation is not the same thing as causation.

I’ll never know as much about these things as a statistician or a quant on Wall Street, but that’s OK because they’ll never know as much about the systems delivering data to them as I will. But I will need to know enough to be able to talk intelligently to people in these roles – and to be able to provide guidance to my customers when they don’t have skilled data analysts on their staff. This, of course, provides the opportunity for me to become more closely involved with how the business is run and, therefore, more valuable to it.

It’s not just BI pros who will need to move with the times. DBAs, developers, and anyone else in IT who works with data will also be affected by these changes. Having the skills to manage data or move it from place to place won’t be enough in the future; we’re all going to have to work together to add value to our data by helping people understand it. If IT as a whole isn’t willing to contribute to these business goals, it risks being relegated to a non-core function or even outsourced.

All of this is why I’m going to the PASS Business Analytics Conference in Chicago this April. Business analytics is the next stage of evolution in getting the most value from the data we collect and manage, and I want my career to encompass that full life cycle. The BA Conference is a place where traditional BI pros like me and the analysts who work with the data that we deliver can come together and learn more about each other’s responsibilities and how we can all do our jobs better. I’m excited about the opportunity to learn new skills and grow professionally, and I hope you can join me in Chicago and on that continuing journey.