By Megan Anderle, Editor and Contributing Writer
Jared Lander, founder and CEO of data science consultancy Lander Analytics,recalls when “data scientist” became the hot new job in technology.
“It started becoming popular just as I was finishing grad school in 2009, when I got sent to Burma for my very first analysis project,” Lander said. “The timing couldn’t have been better. Then in 2012, when I started teaching adjunct at Columbia University, that’s when people were really clamoring to learn this stuff.”
In the last few years, data scientists have become one of the most sought-after jobs across every industry, a fact that has been widely documented. Considering the volume of fine details businesses capture about customers and the drive to capitalize on it, the role is quite varied depending on a business’ specific needs. The modern day data scientist is a jack-of-all-trades, with knowledge of data mining, machine learning, programming and basic statistics, among other specialties.
“Some data scientists handle statistics and modeling, while others are more about computations, and others do more visualizations and reporting to tell a story,” Lander said. “And sometimes that can be one person.”
So given that the role has been around for a few years now, what’s the future of the data scientist role? With more automated systems being created to analyze data and make business predictions, will there always be a need for data scientists?
“Most of what you read these days is data is getting bigger and bigger, so the data scientist will become more important, and to a degree that’s true, but the role is also a bit of a stopgap,” John Foreman, chief data scientist at MailChimp, said. “On one hand, you have all these tools, and on the other, there’s still that expertise needed.”
While many companies use platforms such as Hadoop to assist in data analysis trends, many businesses have developed their own tools for big data. Facebook’s data team, for example, created the language Hive for programming Hadoop. Amazon, Google, LinkedIn, Twitter and Walmart have refined their own tool sets, according to Harvard Business Review.
Predictions for the future data scientist
There will always be a demand for data scientists to build the proper platforms for businesses, according to Lander.
“Maybe there won’t be as many computations, because that part will be automated, but there will always be a need for the modeling,” he said. “I don’t tell my students to memorize equations, but knowing when to use each formula and integrating it into businesses is important.”
Knowing how to program will also be an essential skill, but perhaps that programming will occur across multiple machines, Lander said.
“You might need to know how to run a few different Linux boxes,” he said. “People who know how to program well will be more efficient, and businesses will be better for it.”
Core competencies needed now and in the future
Though an explosion of big data has occurred recently, data scientists have essentially been around for years. A decade ago, someone who analyzed patterns about customers and the business was called a “business intelligence analyst” or an “applied mathematician.”
The role involves more machine learning now, but the same rules apply, according Foreman, who has been in the field for 12 years.
“The nice thing is the core principles haven’t changed,” said Foreman, who wrote a book called “Data Smart.” “You still have to be able to communicate well with engineers, executives and businesspeople to hunt down good data sources.”
While some argue that the role is overhyped with some businesses hiring data scientists unnecessarily, businesses will likely become more realistic about specific needs as entrepreneurs realize how data scientists can add value to the organization.
“I don’t think all data scientists will be successful,” as not all businesses have figured out exactly how to use them yet, Foreman said.
However, there will continue to be strong demand for naturally curious people who have expertise in a range of fields and are adept communicators. Businesses will need experts who can analyze patterns, even as more automated systems are created, according to Foreman.
“Whether it’s health care or technology, there will always be a need for smart quantitative people to solve business problems, and that’s never going to go away,” Foreman said.