Breakfast with ECS: Doubling Down on Docker

Welcome to another edition of Breakfast with ECS, a series where we take a look at issues related to cloud storage and ECS (Elastic Cloud Storage), EMC’s cloud-scale storage platform.

Unless you’ve been living under a rock I’m sure you’ve heard of Docker at this point. If you haven’t, it’s time to dust yourself off and understand that Docker containers will wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in. Genius right? Docker_1

The usage of containers has been around quite some time now, but the extra juice worth squeezing came from Docker’s ability to provide total isolation of resources to package and automate applications more effectively than ever before.  Docker provides system administrators and developers the ability to package any kind of software with all its dependencies into a container. Simply put, this resource efficiency standardizes each container and promotes massive scalability – this plugs in very nicely for cloud-scale, geo-distributed systems such as EMC’s Elastic Cloud Storage (ECS). In the early stages of product development, EMC took an early bet on Docker containers and it certainly has proved to payoff.

ECS is a multi-purpose, cloud storage platform that supports varied workloads (object, file, and HDFS) as well as manages your modern/next-gen and traditional applications. This 3rd generation object-based platform (successor to both EMC Centera and Atmos) from EMC is completely software-defined, fully geo-capable, and designed specifically for cloud-scale environments. Customers today are looking for a solution that provides the ease-of-use and economic benefits of the public cloud – but without the data residency and security concerns. That’s exactly what EMC provides with the ECS platform.

How does ECS use Docker? The services that run within the ECS Storage Engine are encapsulated within a Docker container and installed on each and every ECS node to provide a completely distributed and shared service. With the use of these containers, ECS is able to scale its storage and services seamlessly and without limitation to the end user.  Furthermore, with the abstraction that Docker provides, ECS is able to run on most commodity hardware and Linux operating systems. By leveraging Docker, ECS can more easily provide access to anyone, anywhere, and on any device with ensured high availability and data durability.

Read the Overview and Architecture white paper for more technical detail on Docker as well as a review of ECS hardware, software, networking and architecture.

ECS Running Docker Containers – the technical bits

ECS runs on top of the operating system as a Java application and is then encapsulated within several Docker containers. The containers are isolated but share the underlying operating system resources and hardware.  Some parts of ECS software run on all nodes and some (such as the lifecycle manager) run on one or some nodes. The components running within a Docker container include:

docker_2

  • Object-main – contains the resources and processes relating to the data services, storage engine, portal and provisioning services. Runs on every node in ECS.
  • Fabric-lifecycle – contains the processes, information and resources required for system-level monitoring, configuration management and health management. Depending on the number of nodes in the system, there will be three instances running on a four-node system and five instances for an eight-node system.
  • Fabric-zookeeper – centralized service for coordinating and synchronizing distributed processes, configuration information, groups and naming services. It is referred to as the persistence manager, which runs on three or five nodes depending on the number of nodes deployed within the ECS system.
  • Fabric-registry – location or registry of the ECS Docker images. Only one instance runs per ECS rack.

Want more on ECS? Download the latest fully containerized version of ECS for FREE for non-production use by visiting www.emc.com/getecs.

About the Author: Corey O'Connor