At the recent IoT Evolution Conference & Expo, I experienced a few deja vu moments that returned me to my early days with data analytics. A decade ago, companies often struggled with a lack of data, so they were forced to make decisions less on data-driven insights and more on best practices and proven processes.
Once it became easier to accumulate data, companies began to hoard anything and everything that seemed even semi-relevant. This created a new challenge: Having too much data made it harder to extract actionable insights.
Fast forward to IoT, and it’s easy to see how quickly we can become overwhelmed with endless streams of data from connected sensors. It’s one thing to collect all the data, but what’s the best way to analyze it? My colleague John Whittaker addressed this challenge on an IoT analytics panel at the Expo and offered advice on how organizations can unlock more value from IoT-generated data.
He also blogged about edge analytics, suggesting the key is to create a path for performing real-time data analysis closer to its source. He’s dead right in asserting that edge analytics enable companies to scale IoT infrastructures while making it easier to capture, store and access data generated by gazillions of sensors and connected devices.
But getting there requires new tools, gateways and systems to support data aggregation and storing at the edge. As a passionate pragmatist, I wonder whether there is an interim step that companies can take to prepare for IoT. Is there a way to leverage existing infrastructures and look at predictive analytics through a new lens?
Instead of collecting data on everything in an IoT world, would companies be better served to keep only the data that matters most? Instead of hoarding, try selective archiving – determining which data stays by using the following criteria:
- Will it produce greater insight?
- Will it deliver ROI?
- Will it lower TCO?
We recently applied these criteria to a project for a large, global construction company and garnered some interesting results. During the proof-of-concept to stand up a new data analysis and archiving system for their OSHA required safety workflow system, the company shared concerns over a decade of unstructured safety data that had been hoarded and stored, and now is being moved into a Cloudera Hadoop cluster. As collection had been done primarily on tablets, there was no ability to look into the data and create edge analytics. And since it was expensive to retain, the company wanted to know if the data was meaningful and worth keeping.
In architecting a long-term vision, we evaluated the safety data based on the aforementioned criteria and determined there was a lot of information and possible insight. We also realized this data could be combined with other insights to provide the desired real-time analysis across the business, both inside and outside the firewall.
For instance, by combining data on shift duties with safety and compliance rules, construction planning processes and publically streaming weather reports, it was possible to flag potential safety risks. Weather data, which can change by the hour and region, is an excellent example of data that is useful for an edge analytics use case.
In this particular scenario, edge analytics could help prevent a scheduler from inadvertently sending a blow torch welder to the highest floor of a building on a windy day. As a result, the company can avoid a possible incident or the unnecessary cost of having an expensive worker sitting idle at a job site due to the high wind condition.
When it comes to getting ready for IoT, there are distinct correlations between choices you make and the consequences. You can collect data about all your things, but you’ll likely need new tools, gateways and systems to help manage it all. Or, you can focus only on collecting the things that meet the “insight, ROI and TCO” trifecta.
By using that barometer to evaluate next steps, it will become easier to see if an entirely new infrastructure is the best way to scale for IoT. Or, maybe you can start by using your existing infrastructure in a different way? You can leap in or ease in, but as long as you have an opportunity to use edge analytics, you are heading in the right direction.
Are you leaping into edge analytics or looking for a way to ease in by extracting more from your existing systems? Connect with me on Twitter at @joschloss to share your plans.