You may not know it, but life on earth almost came to an end back in early May. The HL129 asteroid came within 200,000 miles of the earth, which is closer than the distance to the moon. Ok, so earth may not have had a chance to get obliterated. But that got me thinking about the idea of predictability and probability of success, especially when it comes to running ViPR Services on commodity hardware.
In September 2013, EMC announced Project Nile, which officially became the Elastic Cloud Storage (ECS) Appliance during the last EMC World in Las Vegas. One of the primary goals of the ECS Appliance is to run intelligent software on commodity hardware. This stack is designed to support running Object, HDFS, and Block services on hardware for massive global scale-out platforms. Combined with a single global namespace and an intuitive self-service portal capability, customers are able to build out a true private cloud solution within the Enterprise, or even enable public Service Providers to provide their own ECS-based cloud service.
In my last blog, I described an architecture solution with Pivotal and ViPR Object. With the upcoming release of ViPR 2.0 Geo-Protection capabilities and the ECS Appliance, we are now taking that architecture to the next level. The diagram below describes that same ViPR Object Architecture, but combined with a very dense capacity footprint, a Geo capability, and a Block-based capability, courtesy of ScaleIO.
What does this allow? Picture writing a Hadoop workload via an Object interface at Site 1, and then acting upon that workload from a Hadoop environment at a completely different geographical site. Due to the ubiquitous nature of ViPR Object, the Hadoop environment has no idea – nor does it care – where the data actually resides. Or, consider a Block service running on commodity hardware in the form of ScaleIO, all managed by ViPR.
Enter EMC ScaleIO
But first, let’s explain what ScaleIO is. Quite simply, ScaleIO is software that takes server-side storage to create scalable on-demand Block storage for clients and applications. Since it is designed as software, ScaleIO is ideal to run on ECS Appliances, and can scale to 1 petabyte or more for a single entity of storage. Further information can be found here. With ECS, we encapsulate the ScaleIO software as Linux containers that run directly on the ECS Appliance. By running ScaleIO in encapsulated containers, ViPR preserves the value and features of the ScaleIO software, while allowing a central management point. The below output shows the ScaleIO Linux containers running on an ECS Appliance, along with the ECS fabric and registry containers.
We then see the entire architecture together in the diagram below. This shows a group of ECS Appliances forming a Fabric cluster, which is in turn managed by a ViPR Controller instance.
How does this look like from a user perspective? We see a screenshot of the ViPR Dashboard below with an ECS ScaleIO cluster discovered.
We then see the ScaleIO cluster discovered under the Storage Providers tab, and launch the ScaleIO dashboard from ViPR. This shows a native ScaleIO view of the cluster.
Finally, we are going to use ViPR to create and export a ScaleIO Block volume to a Linux host. This is illustrated in the below screen, with the resulting fdisk Linux output and mounting of the volume.
Of course, we can also utilize the native ScaleIO tools to look at the cluster as well. This shows the capability of managing a large-scale ScaleIO cluster running on an ECS cluster, while at the same time being managed as a straight ScaleIO cluster.
Finally, when we talk about commodity hardware, we are not limited by locking ourselves into any one type of hardware vendor or configuration. This allows significant flexibility in terms of what hardware type and configuration can fit the right application or workload.
Looking back at the ECS announcement at EMC World, and having seen first-hand the technical capability and user experience of the ECS Appliance, I am confident that the probability of success of ECS is huge. If I were gambling in Vegas on it, I’d be all in.