“Analyze This”: How Analytics, Pattern Recognition and Software-Defined Storage are Transforming the Datacenter…and Your Life

Pattern recognition! That’s what we carbon-based ‘humanoids’ are good at…since the very beginning when humankind first started walking upright. As a whole, we’re really adept at observing our immediate surroundings, natural environment and a sundry of other random and not so random things. We notice what happens around us in our personal biosphere, and existential universe. We notice things that touch our lives in some significant or insignificant way. From the mundane and annoyingly trivial to profoundly grave matters affecting our very existence like: Cause/effect; action/reaction; commotion/notion; fascinations/machinations; observation/correlation; discovery/recovery; monitoring/manipulating, detection/correction; measuring/tabulating; analyzing/theorizing and so on. My, how busy we are! A strong survival and pain avoidance instinct does that. And so does natural selection.

It comes down to situational awareness (SA). SA gleaned from recognizing patterns that allows us to understand interrelationships and dynamics so we can plan for; act on and hopefully accurately predict events, occurrences and outcomes. Of course, sound ‘policies’ usually evolve, improve and get codified in the process. Just think about how much you observe in your daily life. Chances are you’re constantly coming up with possible ‘situations’, course of actions (COAs) and maybe even personal policies based on probabilities or predictability for all different reasons, seasons, ligations, occasions and outcomes. And we tend to make these ‘root cause’ observations, COAs and predictions on both general abstracted assumptions and/or backed by granular data analysis. We can even ‘virtualize’ scenarios and problem solving while day dreaming….or conjuring up ‘what ifs’.

Sure. Our animal friends observe, learn and internalize various patterns too. But those are instinctual and mostly passed on genetically – or through environmental conditioning per Ivan Pavlov’s proverbial dogs. Humans, however, consciously think about observable things and draw sound, factually-based conclusions and learn…so we can have some degree of control over our lives. We analyze this and that…in an iterative loop that continually observes, associates, adjusts, plans/predicts, executes and – hopefully – maybe controls certain patterns.

This same ‘systematic’ process applies to smooth running data centers internally as well as business analytics overall and so many other vital monitoring, sensing, control and data gathering activities for resource and operational planning, predicting, forecasting and measuring results in any closed loop system or ‘function’. After all, it’s all about managing people, money, time, processes and/or limited resources and conserving, preserving, growing the same while “doing more with less”. As a result, online (and increasingly real-time) analytics are a big deal for data centers whether anticipating peak traffic/work load demands; detecting bottlenecks and impending failures to decomposing compute, network and storage usage into granular consumption based internal charge-back billing. It’s a known human trait that we all tend to use and waste less stuff when it actually costs us something. Against this backdrop of analytics in data center,, Software-Defined Storage (SDS) and Software-Defined Data Center (SDDCs) platforms are key technology enablers. Not just for ‘utility billing’ but for virtually all aspects of a data center operational status and health monitoring.

Why SDS and SDDCs? Human traits and psychology ‘observations’ aside, we technologists and purveyors of efficient data collection strive to do so for good, useful purposes…such as making our lives (and mostly mine!) easier, more efficient and consistently successful. In fact, that’s a major part of EMC’s 3rd Platform value proposition: creating accessible and actionable business intelligence for the mobile user community at large. Turns out SDS platforms provide an essential framework for online analytics, whatever their focus might be. True enough – SDS is the ‘great consolidator’ of ubiquitous and diverse data storage and processing for 3rd Platform customers everywhere.

The simple, undeniable fact is all this data (file, object/blob, HDFS, block, key) we now routinely have within our grasp needs to be selectively accessed; converted to actionable information, analyzed and refined in a timely manner. There is such a thing as ‘time value’ of data and proper execution! Hence Big Data, the Cloud, the 3rd (and soon the 4th) Platform, SDS and comprehensive analytics for monitoring and decision support are all merging. The ‘convergence’ of these current big computing trends is transforming our Information Technology industry or – more aptly — the Information Transformation industry. This “imbuement” of sorts hit me recently while I was attending IDC’s Directions 2014 conference in Boston where Big Data, Cloud Computing SDS, SDDCs and analytics were showcased. Recently, EMC World’s 2014 “Redefine” theme in Vegas only reinforced this.

REB_Blog
Source: EMC Advanced Software Competitive Research May 2014

IT has been steadily transforming not only the way people gather, store and share various data types – but also how we’re able to quickly analyze, process and convert data into useful information for meaningful execution and accurate predictions. So OLAP includes — but isn’t restricted to — in-house Data Center monitoring, event detection, actions, planning and predictive modeling. But it can also be applied to just about anything worthy of analyzing and controlling. Ironically, resource abstraction in this case brings more clarity with greater accessibility business intelligence for us masses…at our fingertips.

Indeed, technologists and solution developers have leveraged and enhanced our innate observe and analyze behavior with advanced IT hardware, software, platforms and solutions – and analytic applications — across the board and entire data center spectrum. How you may ask? We have been able to accomplish this feat through SDS’s inherent ability to aggregate, pool and even globally federate local or geographically dispersed data across underlying storage hardware resources and file, object and block data in the Cloud and various Data Spheres. And it does this with a distinct decoupling (and independence!) from the storage resource infrastructure beneath it. SDS is a key technology enabler of these advanced real-time, near-line and/or off-line analytics software applications for SDDCs and the 3rd Platform! Solutions like ScaleIO for hyper-converged/hyperscale block storage and EMC’s ViPR’s federated, holistic control and data services monitoring in mixed, heterogeneous storage hardware resource environments are leading the charge.

Yes, the 21st Century of Information Transformation is upon us…and so is the SDDC, courtesy in large part to SDS. While SDS itself is not a panacea or solution to world hunger, it’s changing the way we run our data centers and how we access and manage mobile and device level data spread across mixed storage hardware resources, data types, platforms and arrays. With the wide span of control and monitoring capabilities SDS brings to the data center and user community, applied analytics and modeling applications that will be better able to access disparate data, information and platform-centric environments from a centralized single access point – reducing (if not eliminating) the need for highly
trained data analysts and scientists! Our insatiable thirst to discover, recognize and predict patterns will be quenched at a wider, much deeper and more granular level! So we can all continue to “Analyze this!” (or “that”, or whatever inspires us).

About the Author: Rodger Burkley