A few weeks ago, I had the pleasure of participating in a Meetup redux when the Big Data Cloud Series returned to the Bay Area. When the series began in 2011, the venue was a backroom of a small office in San Jose. Only around 20 people showed up, but the level of interest was clearly very high.
I had the honor of being the first presenter and took the opportunity to talk about the enabling technologies that powered Dell Software’s Kitenga Analytics Suite. Many of the original participants were database and data warehousing specialists, who were trying to understand Hadoop and the Big Data technology landscape as well as start to build their skills for tackling this new area of expertise.
Fast forward a year, and while Big Data continues to be a headliner at industry meetings and conferences, the underlying need for integrated information modeling and visualization is starting to gain momentum. On November 6th, I reprised my role for a larger audience at Dell Software’s Bay Area office in Santa Clara. In a nod to that original event, Paul Baclace who also presented at the first Meetup joined me. My talk had some of the same themes as the original—the more things change, the more they stay the same—focusing on intelligent data enrichment and complex search models.
I started by discussing the drive toward Big Data technologies as a fundamental industrial revolution that rivals that of steam locomotion. But it isn’t the proliferation of the data that is driving this revolution. It is the ability to infer and capitalize on the intelligent signals present in the data that is already revolutionizing information discovery, communications and disintermediating cycles of feedback.
Building on that theme, I drilled into a method for semantic understanding based on simulations of human sparse distributed memory. The approach is especially interesting in that it uses randomized algorithms to achieve intelligence, and has even been successful in approaching human-like performance on the Test of English as a Foreign Language (TOEFL). In my talk, I focused on applying these algorithms at massive scale to enrich data and bind it into organizationally-relevant information systems that enable search and discovery.
We came full-circle in more ways than just the return engagement of the Meetup, of course, because there always has been a pressing need to rapidly transform complex data into useful and re-useable information products. I suspect when we meet again next year, there will be lively discussion and debate on this topic.
The never-ending demands for better analytics capabilities show waves of innovation and thematic repetition in the history of technology, as well. In 1945, Vannevar Bush proposed a combination of microfilm and circuits to help researchers build links between scientific papers. Called memex for “memory extender,” the proposed system was intended to help Bush himself, since he felt overwhelmed by the range of scientific information he needed to do his job, which included managing the Manhattan Project.
Although it was never built, the system was one model for the invention of hypertext and the computer mouse many decades later.
Automated search and data enrichment continue with this tradition, and with the advent of Big Data, lend themselves to the next industrial revolution.
What do you think will drive the next industrial revolution? Drop me a line at Mark.Davis@Software.Dell.com for a virtual Meetup.