Building Hyper Scale Storage on Commodity Platform with ECS Appliance Powered by ViPR

Enterprises and hosters’ alike are driven to the third platform with growing number of cloud based mobile and social applications. This has resulted in an explosion of data for customers from terabyte to petabyte and even exabyte in some scenarios. For example, an insurance company delivering their claim processing application through mobile app would result in massive data growth with images and claim forms. Building and managing a storage platform at this scale entails a few key challenges, namely:

  • Ability to be in control of the data
  • Lead times associated with storage allocation
  • Storage capacity and performance metrics across data centers
  • Cost of managing hyper-scale heterogeneous storage
  • Storage integration with cloud stacks (compute, and networking)
  • End-to-end visibility

ECS Appliance powered by ViPR addresses these challenges for building storage geared towards the 3rd Platform. At EMC World 2014, we announced the Dell EMC Elastic Cloud Storage (ECS) Appliance powered by ViPR. General availability of both ECS Appliance and ViPR 2.0 is in June 2014. In this blog, we’ll look briefly at an Elastic Cloud Storage (ECS) overview and advantages for administrators and developers.

ECS Appliance is the hardware that is built for hyper scale environments. The hardware can be purchased for serving only unstructured storage (Object, HDFS), or for serving a mixed workload (Object/HDFS + Block).

ECS_1

The software that powers ECS Appliance hardware is ViPR 2.0. In ViPR 2.0, we have invested in both the control and data plane to enable hyper-scale storage.

Control Plane: The controller has expanded in terms of 3rd party array support, and Geo Federation capabilities across sites among other enhancements.

Data Plane: Ability to do unstructured storage over commodity, Geo replication for disaster recovery, and multi-zone access are notable features among other enhancements for ViPR 2.0.

One of the principles for ECS Appliance and ViPR is to limit the churn for applications to be rewritten. Hence, the northbound APIs are unchanged for ViPR 2.0, enabling developers to bring their S3, Swift, or Atmos applications. Furthermore, ViPR abstracts bucket creation to a virtual pool. This allows storage administrators to abstract and pool underlying hardware.

All the software and hardware innovations that we have done with ECS powered by ViPR makes it an attractive platform from a TCO perspective in comparison to public cloud.  The 4 year cost to serve (CTS) is shown below comparing ECS with public cloud (one of the major cloud storage providers, but the trend is similar for most other providers we evaluated).  CTS include power, cooling, administrative, facilities, hardware, and software maintenance.  In our recent analysis, we see that the ECS is 9 % to 23 % lower in CTS than public cloud.

ECS_3EMC Internal TCO Calculations, April 2014


Why consider ECS Appliance – Storage Admin

IT lifecycle is evolving with the transition to the 3rd Platform. Scale out and efficiency are key principles which most enterprises and hosters’ are trying to operate. ECS Appliance provides a few major advantages:
• Control of data
• Protection of existing investments, while scaling out as required
• Ability to build hyper-scale storage by stacking a few ECS Appliances within a VDC
• End-to-end visibility through deep integration with monitoring tools like ViPR SRM
• Ability to pool, and automate resources as required without building silos on a per application basis
• Simple and easy two-step setup of ECS Appliance powered by ViPR

Why consider ECS Appliance – Developer / User
This enables applications to work seamlessly independent of the underlying storage hardware.

• Application lifecycle is the same with storage served by ECS
• Ability to utilize the infrastructure in a massive scale out fashion, without changing a single line of code
• Developers can get capabilities like auto-scaling similar to public cloud
• Limitless scale for applications

Stay tuned for more information on ECS Appliance and ViPR as we delve into some of these topics in detail. Don’t miss the keynote presentation from Amitabh Srivastava on ECS Appliance and ViPR from EMC World 2014.

About the Author: Kamal Srinivasan