IDC Lab Validation: EMC Isilon Data Lake Foundation Trumps DAS for Hadoop

IDC just released this compelling Lab Validation Report on the EMC® Isilon® Data Lake. In this report, IDC concludes that an Isilon Scale-out Data Lake foundation should be core to every enterprise storage strategy and Big Data Workflow. IDC offered this opinion after validating Isilon’s multi-protocol capabilities along with its availability, security and compliance features — delivering on all these enterprise-grade service levels while performing better than direct attached storage (DAS) for Hadoop workloads.

Considering how Isilon’s scale-out architecture linearly increases performance, along with its record setting benchmarks, IDC’s findings on Isilon performance capabilities for Hadoop aren’t surprising. This IDC lab validation goes far deeper than just validating Isilon delivering improvements in Map/Reduce operations; it demonstrates the value that the Isilon Scale-Out Data Lake Foundation brings to the enterprise. The report includes the validation and articulates the significance of Isilon supporting simultaneous multi-protocol workloads, resiliency and recovery from failures including the Hadoop NameNode, secure multi-tenancy of Hadoop workloads in a shared storage environment and Isilon’s numerous enterprise security and compliance features, including SmartLock SEC17a-4 (SEC compliant) WORM.

Lab Validation

IDC Essential Guidance

IDC validated three key aspects of the Isilon Data Lake Foundation: performance, availability and security/compliance and offered this essential guidance for buyers:

  • HDFS Performance in Data Lake and NFS Performance during multi-protocol ingest: IDC validated the performance profiles of both DAS and EMC Isilon using well-known Hadoop benchmarking jobs supplied with the Hadoop Distribution. IDC concluded that the EMC Isilon Scale-out Data Lake offered excellent read and writeLab Validation 1 performance for Hadoop clusters accessing HDFS via OneFS® vis-à-vis those accessing HDFS via local (internal) storage. IDC also validated that NFS performance of EMC Isilon is significantly faster than a Hadoop DAS cluster due to optimizations on the OneFS platform. In a nutshell, via HDFS, EMC Isilon is nearly 3X faster for writes and more than 1.5X faster for reads than a Hadoop DAS cluster. So is EMC Isilon the cheaper alternative to DAS for Hadoop data analytics? Regarding cost comparison, users should note that EMC Isilon, in addition to helping store and manage multiple workloads, helps eliminate the costs associated with taking data to Hadoop by bringing Hadoop to the data and by delivering analytics in-place. Learn more about it in our white paper on EMC Isilon for In-Place Hadoop data analytics here. Also check out Data Storage for Hadoop Analytics TCO tool to help estimate costs of using EMC Isilon vs. Direct Attached Storage.
  • Data Availability: The EMC Isilon Scale-Out Data Lake provides robust data availability and protection. Component and node-level failures do not cause a noticeable drop in performance especially during or after rebuild processes; a stark contrast to the limitations posed by a standard Hadoop cluster.Lab Validation 2
  • Secure Multi-tenancy: Isilon provides a federated security fabric across the entire Data Lake. It brings enterprise-grade governance, regulatory and compliance capabilities to Big Data environments

Download the Lab Validation report here. And learn more about how to leverage EMC Isilon as your foundation for Hadoop analytics with Pivotal and Cloudera.

About the Author: Sam Grocott

Sam Grocott is the Senior Vice President of Dell Technologies Business Unit (DTBU) Marketing, which focuses on all infrastructure and client solutions as well as the associated services portfolio. DTBU Marketing also includes key GTM activities across global business-to-business (B2B) campaigns, portfolio messaging and competitive intelligence. Sam began his career with Isilon Systems in 2001 leading Product Management and was responsible for helping bring to market Isilon’s first products in 2003. After Isilon was acquired by EMC in 2010, Sam continued to lead the marketing and product management team that delivered the industry leading scale-out NAS offering. When EMC was acquired by Dell in in 2016, Sam’s role expanded to include the entire Dell EMC Infrastructure Solutions portfolio for Marketing. This set into motion what ultimately resulted in the creation of DTBU Marketing, representing the entire Dell Technologies portfolio.