Cloud-to-Cloud Messaging Has Arrived

HOW WE BUILT THE NEXT GENERATION OF INTEGRATION

As EMC puts the finishing touches on a new, SAP-based Enterprise Resource Planning system further optimizing its cloud infrastructure, we’ll soon be unveiling another innovation in the way we integrate applications and data in the cloud. It’s called the Common Integration Cloud (CIC,) an integration platform that will serve as the gateway through which all data is exchanged between the ERP platform and the rest of the enterprise.

CIC has been under development for the past two years, paralleling EMC’s ERP replacement project called PROPEL. As part of the PROPEL effort, we went through and identified all the applications across the company that needed to send data to or pull data from the new ERP platform. In that inventory, we identified more than 450 different point-to-point interfaces exchanging data with our legacy platform.

In fact, between the various technologies and vendors that often have their own proprietary systems, we had been working with just about every kind of integration middleware and platform you could possible use. That meant coping with multiple “stove-piped” skill sets, multiple costs and a system that was a major challenge to manage.

We faced a decision. Either rewrite all 450+ interfaces and continue to deal with this complex, customized and unscalable environment or come up with a new, simplified approach that would scale to meet the future needs of our business and support the cloud platforms of tomorrow.

We needed a way to accommodate all of the interfaces in a standardized and flexible way. We found the answer in VMware technology, which allowed us to create a common integration platform and framework that uses a common skillset—or a “one-skillset-any-cloud” approach to integration.

While there are commercially available, closely coupled architecture platforms, or enterprise service buses (ESBs,) we could have used to integrate, our CIC is less expensive, more flexible and better tailored to our needs for cloud integration.

Central to CIC’s architecture is VMware’s vFabric-a java-based technology that uses a uniform skillset and allows us to reuse the technology in the future, scale horizontally, and increase performance. We were able to reduce the interfaces down to approximate 120 in the new solution.

We also used SpringSource which provides for integration, batch and web services. We had been using SpringSource for other solutions in EMC IT. We recognized that we could combine it with other technologies and take it to the next level for CIC.

The CIC platform is 100 percent virtualized, operates on a VCE Vblock with Cisco, EMC and VMware technology, and supports private, public and hybrid cloud deployments.

The result is a CIC that sets the standards by which all data from the ERP platform will be managed. (See slide) We’ve achieved a holistic integration view across our entire enterprise data solution. The platform lets business units obtain data when and where they need it and minimizes data movement and data manipulation. It allows users to access information from several data sources through an integrated architecture and accommodates for multiple data center deployments.

To boil it down, think of the CIC like a clearing house, but instead of being a central hub that marshals data around the enterprise, it serves as a hub through which decentralized messages are distributed instead of being centrally managed as they would be in an ESB. The scale out is much more granular vs. a coarsely grained scale out. Messages come in from multiple places; data is collected, processed and distributed within the cloud. One system, one skillset and a manageable exchange process.

For example, with our existing integration environment, processing a sales order from customer quote to shipping would have involved multiple integration tools requiring different sets of skills to extract and deliver the needed information to multiple systems. Data on the transaction would need to be channeled to and from the customer main data base. Records would need to travel to inventory, the financial general ledger, customer service and several other locations.

Using the CIC, the information for a sales order is collected and processed once and distributed to all interested parties via a central gateway.

The CIC which we’ve created is not only more efficient but can also be leveraged as our integrations continue to grow. It also helps protect enterprise assets because it is more centralized and minimizes data movement.

About the Author: Bill Reid