Is it just me or does anyone else experience a sense of technology déjà vu from time to time? My most recent bout has been around the vaunted “rock star” status afforded the Data Scientist role emerging in the Big Data space.
I don’t want to say that having one on staff or perhaps even being a Data Scientist isn’t the greatest thing going, because it actually is. My colleague Darin Bartik recently blogged about what it takes to be a data scientist in a Big Data world. Clearly, this is one of the most sought after and highest paid positions in IT right now—but let’s not lose sight of the “right now” aspect.
Sure, Data Scientists are all the rage today, but where will they be in a decade? To me, the parallels between the early days of the commercial Internet and today’s Big Data market are eerily similar. In fact, the path—or perhaps a better term would be “Zeitgeist”—we’re experiencing feels almost identical.
Having been in the “dot com” era and then the Internet infrastructure industry before joining the Big Data software sector, I see direct parallels between now and then. In 1994, I was part of an ecommerce project that dared to change the car-buying experience from the dealer lot to the online world. Soon after, I founded a startup to help major infrastructure players secure the emerging Internet industry from newfound threats.
These past experiences prepared me for what I do now, running marketing and evangelizing the Information Management Domain for Dell Software. We build great tools like Toad, Boomi and Kitenga to manage, integrate and ultimately drive informed decision-making from data, whether it comes in “regular” size or the new Big Data “super-size.”
Just as there is great demand today for someone to guide companies through Big Data decisions, I recall when the No. 1 job was the almighty webmaster—the person who could ease the transitions to ecommerce and ensure the success of Internet infrastructure projects. This position was essential as the technology of the day was too different, complicated and fractured for the average IT organization to support rapidly. Business leaders, in a desperate attempt to gain value that was promised by connecting their organizations to the web, paid handsomely for a webmaster with experience to get them there.
Today, the same thing is occurring with the Data Scientist role. Again, a new class of technology has emerged with incredible promise and a boatload of complexities. Organizations are struggling to get on board and leverage Big Data Analytics fast. Recently we heard that Kaggle Data Scientists command up to $300 per hour.
Organizations like Walmart have put up billboards in the Bay Area to appeal to these latest IT hotshots. In fact, last week, Walmart’s Silicon Valley innovation lab, @WalmartLabs, announced the acquisition of predictive intelligence startup Inkiru to accelerate its analytics capabilities and boost data scientist bench strength.
At several Big Data Camps and related events I’ve attended, hands fly up when “who’s hiring?” is asked about Data Scientists. These days, everyone is talking about the need for more. So, it’s safe to say the run up in value of the Data Scientist and Big Data mania has begun.
But the nagging question for me is what lessons from the past can we learn from? Will today’s Data Scientist become yesteryear’s webmaster? I’ve been thinking a lot about this. If history is a guide, we will see huge early wins for organizations that find talented resources who guide them wisely.
In the medium to long term, Big Data software technology will mature and alleviate some of the complexity and fractured nature of data that Data Scientists help solve today. At the same time, IT organizations will begin to assimilate the required skill sets required, so they’ll be more capable in the future.
When that happens, will the “rock star” Data Scientist be absorbed or morphed into something else? Will we look back in a decade and think of the Data Scientist like we did the webmaster in 1995?
What do you think? Drop me a line at john_whittaker@dell.com or chat with me on Twitter at @alertsource.