Before the ink has even really dried on Hadoop Summit15 in San Jose I am sitting down in a rare moment of peace to write out some reflections from my experience and what I have seen from the sessions, keynotes, partners, and users here at the show.
Hadoop Gets Real
The most lasting impression I got from the overall theme of the show and the people in attendance was that Hadoop is not an “emerging tool” anymore. The momentum, use cases, and indeed the buzz of attendees was that there is massive adoption and momentum built up in the marketplace. Behind this wave of early adoption is a lot of pent-up demand that is waiting for things to stabilize and become more enterprise ready. Once the tooling around the Hadoop ecosystem is more robust, and the platforms that it runs on are more operational, there is no limit to the demand that this ecosystem can produce.
In counterpoint to this fact, there is another countercurrent of theme that Hadoop is not “all things to all people”, and so there is a lot of discussion around the emergence of the logical successor to Hadoop as the analytics tool of record. Certainly the buzz around Spark is indicative that this is the way of the future and ties into the second theme of the show that I observed in numerous conversations and sessions.
Hadoop Gets Real Real–Time
The emphasis on real-time and near-real time capability in the Hadoop ecosystem is, without question, a top thought on people’s minds. This follows the logical analytics maturity curve of: testing – getting serious – operationalize – out-grow.
Attendees were in all 4 phases, but a lot of presentations from many of the early adopter community were definitely focused on this last phase of “we need more than what Hadoop provides, so what’s next?” Real-time is the focus of a lot of organizations as they seek to influence and engage through data-driven applications and change outcomes on the fly, rather than simply searching for insights based on historical data sets.
This change in tooling, and the increased demands on infrastructure was also a key focal point of the show overall. I spoke about this specifically with theCUBE host and SiliconAngle founder John Furrier. If you missed the video, watch it on demand here. Certainly the theme I felt really resonating the most and underpinning all the others was the last big takeaway for me.
DevOps rising
While it has a lot of connotations and nuances, the overarching point of all the sessions was that Hadoop and the emerging ecosystem of advanced analytics tools requires the full embrace of the DevOps mindset. I used the quote in my interview with theCUBE that “The Analytics space is the ‘killer app’ that DevOps has been waiting for to mature.” and this was in evidence all over the show as companies showcased how they have used the DevOps mindset and tools for rapid integration and deployment to streamline and enable advanced analytics environments.
I really enjoyed the inclusion of operations and the need for enterprise-ready platform solutions for analytics that was all over the show. I think the time is NOW for companies to begin researching and falling in with a platform solution for analytics so that they are ready when the business need catches up to them. They also need to make the shift to a data-aware, and DevOps minded culture in order to really capitalize on the momentum that analytics has and use it to harness their own data and create real value and insight.