By Brian T. Horowitz, Editor and Contributing Writer
Large files and data sets are fostering innovation in a number of industries as companies look to gain efficiencies and lower costs. The global market for big data and predictive analytics services is expected to grow to $125 billion in 2015, IDC reported.
Media companies such as The New York Times can use big data to see how many newspapers to put on newsstands, and organizations such as Delta Lloyd are using data to gain insight on how to process insurance claims.
Big data in the media
“Big data is having a huge impact in media,” said Chris Wiggins, chief data scientist at The New York Times and an associate professor in the department of applied physics and applied mathematics at Columbia University. “Generally we’re trying to help understand the way our readers engage with our products, particularly our digital products in order to make those products better.”
The publication uses big data analytics to determine the number of copies of the newspaper to deliver to various locations. The Times stores the data in a combination of cloud services and traditional relational databases and uses the Apache Hadoop software framework, Wiggins told Power More.
Big data aids research on population health
Health care will continue to be at the forefront of big data innovation as researchers use the data to gain an understanding of patient populations and the possibilities of new treatments.
“Health care is a space where big data is going to have a big impact, both in reporting and managing of data but also for predictive analytics and trying to advance disease treatment,” Wiggins said.
Washington state’s Medicaid program is using big data to keep better track of patients so that hospitals can reduce ER visits, according to Bloomberg Business.
Meanwhile, health information exchanges (HIEs) like the Emergency Department Information Exchange (EDIE) in Washington state and the Statewide Health Information Network of New York are using big data platforms to pool health data from multiple health systems so doctors can gain insight on the conditions of patient populations.
In addition, the data storage demands for genomic data could range from 2 to 40 exabytes and surpass the data requirements of Twitter and YouTube, according to the journal Nature.
Tracking product inventory in retail
In the retail industry, big data allows companies in retail to keep track of their operations across the enterprise. They can monitor information on delivery, fulfillment and product inventory. Beacons in brick-and-mortar stores such as GameSpot send data to shoppers’ smartphones, and then storeowners collect data in the cloud on customer preferences.
“All of e-commerce is transitioning from traditional marketing to digital marketing,” Wiggins said. “Definitely people are innovating with big data.”
Retail marketing campaigns can become more precise with the help of data.
“The field of marketing is really going to become transformed as the skills move away from conjectures of people’s behavior and move toward precision, quantification and optimization,” Wiggins said.
Retail companies are “trying to match up things better across their operation and provide [merchandise to customers] in the sizes they want,” said Jeff Cotrupe, industry director for big data and analytics at Frost & Sullivan.
Government, universities combat threats using data
Big data is also bringing innovation to government agencies and to universities.
Donald E. Brown, director of the Data Science Institute at the University of Virginia, is working with the Department of Homeland Security to determine which supervisory control and data acquisition (SCADA) systems in energy are vulnerable to attack.
SCADA software systems gather and analyze real-time data. They control conditions in oil and gas refineries, power plants and waste control facilities. Attacks on these systems can compromise these operational capabilities.
“The question is whether these SCADA systems are being attacked — is somebody trying to get in and change the way they’re supposed to operate to the way they’re not supposed to operate,” Brown said. “So that’s clearly a big data question.”
Attacks on SCADA systems increased by about four times from 163,228 in January 2013 to 675,186 in January 2014, the 2015 Dell Security Annual Threat Report revealed.
In addition the University of Virginia is using big data algorithms to predict energy usage for campus buildings. The university’s Big Data Institute collects about a terabyte of data a day on the school’s electric grid.
“People are innovating by looking at new ways to process that large amount of data, what amounts to a needle in a haystack,” Brown said.
Check out the infographic below to learn more about how big data brings innovation to businesses in various industries.