Photo by Catherinette Rings Steampunk

For those who have been around IT for a while, the cloud computing wave has many of the same characteristics of any other fad: huge vendor investment, scads of new start-ups, a lot of media coverage and a few high-profile cases that you hear about over and over.  After talking this through with Diamond’s CEO Adam Gutstein and our colleague John Sviokla, I think there is one thing that makes the cloud phenomenon different. 

It directly addresses IT budget line items.

MIT’s Center for Information Systems Research periodically collect IT spend data from over 1,000 companies.  They report spending across four categories: Infrastructure, Transactional, Information and Strategic.  While it varies by industry, the infrastructure spend in their last report was around 40% of total.  They boil the world of IT spending down to four simple categories and infrastructure is one of them and it accounts for 40% of all enterprise IT spend.  One of the two compelling cloud computing stories is directly targeted at cheaper, greener and more elastic infrastructure.  Bingo!  [The other story is around business agility provided through cloud applications.]

I believe that the easier it is to see where a new technology can impact a business, the broader interest and acceptance it can have in the marketplace.  As another example of this, take a look at service oriented architecture.  SOA had a similar vendor and media buzz early but has since stalled.  Try to find “SOA” on a top-line IT budget – you can’t.  In fact, making a case for an architectural investment like SOA by itself is almost impossible.

To support the notion of an apparent fall in SOA interest, consider the responses we got from almost 600 business and IT execs on their current and planned SOA investments in our latest Diamond Digital IQ survey.  Almost 70% surveyed haven’t and aren’t planning to invest in SOA.  I believe that this has more to do with the complexity associated with the communication of its value than its overall utility.

I think that cloud computing, especially infrastructure clouds, will be more like application maintenance outsourcing in that they both have easily addressable budgets, are easy to understand and therefore, will be easy to sell organizationally.

What do you think?

The Cloud Adoption Chasm (photo by selkovjr via Flikr)

Last week, I had the privilege of presenting alongside Dr. David Reed, one of the best thinkers in the future of computing, and Steve Russell, an innovative leader at Morgan Stanley, who has led the implementation of the largest and most successful private cloud I’ve heard of – very impressive. The event was part of our on-going Diamond Exchange program.

David shared his ideas about “3 clouds.”  The first cloud is connectivity, embodied by the Internet itself.  The second cloud is made up of resources, and is really the essence of the cloud movement we are seeing now in the public and private clouds.  Reed describes the third cloud as a social cloud made up of people and relationships, not computing resources.  More on the third cloud in a future post.

Steve presented an incredible story about how he and his team at Morgan Stanley created a very high performance and low cost private cloud using a combination of off the shelf tools, open source software, custom integration glue and a lot of sweat.

I batted clean-up and shared my thoughts on why we haven’t seen faster adoption of cloud services in the enterprise.

Top Cloud Adoption Barrier is Integration

Most of us are comfortable using services in the cloud.  In fact, I bet that anyone reading this has 5+ accounts with Hotmail, GMail (and Google Docs), Flikr, Box.net, etc.  What really got me thinking about cloud adoption (or lack of) was a personal experience using some of these services combined with an experience of one of our consumer products clients.

At the end of August, I went on a salmon fishing trip to the Pacific coast of British Columbia.  During the trip, I took over 1,500 photos (no, fishing isn’t enough of a hobby for me, I had to add photography on top of it!).  To keep the photos from never seeing the light of day, I set a goal to make a photo book out of the good ones.  To make a long story shorter, I used three cloud-based applications to make this happen:

1. SmugMug – the best photo organizer out there, $30-40 per year but well worth it as there are no storage or photo size limits and the website is beautiful
2. Photoshop.com – the younger sister to the desktop photo editor Photoshop Elements line, not as functional but free
3. MyPublisher – photo book design and publishing

The interesting thing about this experience is that there isn’t a one-stop-shop for all of these services, at least with the features and functions and quality I was looking for. So, each of the photos I want to work with have to be moved from camera to computer, from computer to site 1, from site 1 to computer, computer to site 2, etc. – you get the idea.  Furthermore, the integration is done by me – I am the human integration architecture.

Compare this with a consumer products client of Diamond’s.  They have a very lean in-house infrastructure.  They use hosted or cloud based systems for everything they can, especially around interaction with customers and management of related data.  In short, they have adopted Salesforce.com and a few hosted social community applications to interact with their salesforce and customers.  With these sites and services established, they set out to tackle consolidation and analysis of all of the customer information – across several hosted systems and databases.  Unfortunately, no single vendor provided a one stop shop with the features and functions and quality they were looking for (sound familiar?).  So, they were forced to add more infrastructure, databases and software in house to serve as a customer data hub.

Until enterprise-class integration standards, tools and cloud vendors are established, we just won’t see an appreciable increase of cloud based applications in the enterprise.

by Chris Curran, Nalneesh Gaur and Rob Warren

In distilling perspectives from our clients for two upcoming events on cloud computing (Diamond Exchange, InfoWorld), we have developed an informal categorization that captures where companies are in adopting public cloud computing offerings.  “Not Interested/Not Applicable” could have been an option for our list but we don’t know anyone not at least looking into applying the cloud.

Our clients fall into one of six categories in their cloud exploration and adoption:

1. Surveying the market
2. Using an off-the-shelf cloud service or app
3. Using a customized version of off-the-shelf cloud app or service
4. Developing/developed a custom app on a cloud platform
5. Integrating cloud app with in-house app
6. Multiple cloud apps integrated with multiple in-house apps

Surveying the Market

Several companies are still learning and evaluating.  There are three example companies we are working with who fall in this category.  A financial services company who deals in very small and very high-speed transactions isn’t convinced that any cloud players can provide the processing horsepower within their SLAs and security constraints, but they continue to survey the market.  Maybe a less mission-critical application will be more appropriate?  An industrial products company we work with tends to move in a more measured way when evaluating new technologies and approaches and are, in that vein, just taking their time. Finally, one of our clients is a managed IT services vendor and is interested in moving some of their in-house help desk apps into the cloud for their clients and maybe as a more general offering – still evaluating though.

Off-the-Shelf Cloud Service

All of the users of vanilla Salesforce.com, Google’s Docs and Mail services, multi-media hosting, cloud-based backups and the host of others fall in here. (Un)fortunately, these services are so easy to buy and use, some corporate users are buying directly instead of through more formal channels (procurement, IT, etc.).  To combat this, one of our clients is establishing blacklists on their perimeter to clamp down on unapproved usage.

Customized Cloud Application

This takes the vanilla cloud app a major step further in that the organization must commit some resources to learning the cloud provider’s development tools, languages and quirks. Beware of vendor lock in due to proprietary development tools, APIs and runtime environments.  Several companies who use Salesforce.com have added custom functions and extensions to the core system using Force.com’s visualforce, application frameworks and APIs.

Custom Application on Cloud Platform

Instead of using an off-the-shelf business application, some organizations are using cloud based application development platforms like Amazon’s EC2, Google’s Python environment of 3tera.  Because Infrastructure clouds are agnostic to deployment platforms, they offer more freedom for building and deploying applications. These all have some limitations – few choices of tools and languages, limited data storage and movement options – but are great for quick and cheap prototyping, especially for web2.0 apps.

One of our transportation and logistics clients is evaluating Force.com and Google’s platform to serve as a basis for a re-write of their core legacy systems that are barely supportable and can’t scale fast enough to support aggressive geographic expansion.

Cloud – In-House Integration

A consumer products client prides itself on its efficient outsourced infrastructure.  They, too, are a Salesforce.com customer and use a few other cloud apps to gather customer information and managed on-line communities.  Their challenge is that no cloud providers seem to want to take responsibility for integrating the customer data across these apps. So, they feel forced to create an in-house data store to merge and eventually analyze their customer information.  We believe that integration services – both message passing and database integration and hosting – is a huge opportunity for cloud providers.  Amazon Simple Queue Service (SQS) offers such integration capabilities, but some of our clients have found it to be limiting in terms of number of queues, messages, and message size and have resorted to in-house application integration solutions.

Diamond has also used this approach internally to create a custom Salesforce.com instance integrated with our Peoplesoft financials and some other in-house databases.

Heterogeneous Cloud Architecture

One of our health care clients has begun the process of moving their internal and partner user identity management functions to the cloud. When they are done, all of their major applications will utilize cloud-based services to authenticate their users.  A user’s identity created in this manner becomes fungible across multiple organization as more partners participate in the identity network (which of course adds a whole new set of issues).

Conclusions

It seems like everyone is at least considering the cloud.  We think there are significant benefits in having someone else build and maintain application software and the underlying hardware and software platforms and bill it per use.  What were are still unsure about is whether anyone has a turnkey service that handles the “whole project” – transition, integration, process redesign, procurement enhancements, etc.

Interested in your thoughts and experiences.  What have we missed?

A compelling cost proposition is just the ante for providers of cloud computing services.  When considering costs for cloud services, it’s tempting to look only at the cost per drink.  However, as cloud offerings become more sophisticated and more robust enterprise and industry applications become available, the cost of the hosted service will become much less important and replaced by discussions we in the enterprise IT management realm are already very familiar with concerning people, processes and architecture.

There are 5 questions that we are hearing now from our clients as they consider taking the next steps into the cloud – whether they are already users of simple infrastructure services or have started the cloud-based application journey with GMail/GDocs for the enterprise or Salesforce.com:

1. What are the viable paths to move my application into the cloud?
2. What architectural changes are required to integrate cloud and non-cloud systems?
3. How should my infrastructure and operations processes change to take advantage of different procurement, provisioning and management models?
4. How can a private cloud give me more flexibility than current hosting approaches?
5. Who is responsible for managing identity and privacy in environments that mix cloud and non-cloud systems?

These questions follow the obvious transition from cloud applications to cloud systems.  For example, Salesforce.com is a great cloud application that tracks targets and prospects through your business development, sales and relationships cycles.  When an application like Salesforce.com really takes hold in your organization, the conversation quickly jumps to the “systems” conversation – in this example, a customer management system.  Added into the mix are questions like (replace Salesforce.com with “any cloud app” if you like):

1. How do I make sure all of the customers in Salesforce.com are synchronized with those in my customer management application, my billing application and my 6 product systems?
2. Should I add custom application logic into Salesforce to validate customer and company information against my master list or should I do it externally?
3. What kinds of skills and other organizational considerations should I make for the IT staff supporting my customer systems?

As cloud services and their underlying architectural platforms become more robust, so too will the discussions around the true costs of the “business systems” that incorporate them.  So, it’s important to quickly get beyond the usage costs for a cloud service and spend enough time to understand the total costs to migrate, implement, integrate, train, and redesign the surrounding and supporting systems and processes.

If you would be interested in joining a deeper conversation on cloud computing, consider joining me and my colleagues John Sviokla and David Reed in Chicago on November 5 at Diamond’s Seeing Thru The Clouds event.

The MIT Industrial Liaison Program holds an annual conference for its sponsors focused on technology and telecommunications.  This year’s event had excellent breadth, featuring Esther Dyson, the CTO of Blue State Digital (developers of Obama’s online social media platform), the director of MIT’s game development lab in Singapore and several others.  The one presentation that really got me thinking was by the IT Energy Coordinator for MIT’s own infrastructure, Laxmi Rao.

Her presentation centered on the consumption of electricity by computing infrastructures and what some organizations are doing to improve power efficiency.  There were two things I took away from her talk.

Power Usage Efficiency is Under-Appreciated

The average data center is very inefficient in its ability to effectively move electricity from the power grid to the computers that need it.  One measure of power efficiency is Power Usage Effectiveness (PUE) which measures the ratio of total power coming into a data center to the power that actually reaches the computing devices (there is a lot of great PUE info here).  Laxmi said that the typical data center has a PUE of 2.0, or about 50% efficiency.  Some say the average is worse.  So what?  Inefficient power usage wastes money and power.

Cloud Infrastructure Providers Are Doing Something Right

While the typical data center is inefficient, the massive data centers built recently by Google, Amazon and Microsoft, represent not only a new trend in cloud-based computing but also a view into the next generation data centers.  Specifically, each of these vendors are emphasizing power and cooling strategies and have lowered their PUEs into the 1.2 – 1.3 range, according to Laxmi and other published reports. They are achieving this through large scale application of water and air cooling techniques, hardware configurations and data center designs.

—

After all of the “easy” techniques have been applied, like virtualization and server consolidation, we will all be faced with a capacity issue that will require more strategic thought to solve.  For some, maybe adding on to an existing data center will make sense.  For others, like one of our consumer products clients in the southeast, there was no more power available from the utilities in their region.

All IT leaders should take it upon themselves to get smarter about how much power we are using  and if there are lower cost and lower impact ways to consume it.  I believe that the cloud infrastructure providers’ efforts provide at least a model for us to follow to improve our private infrastructures if not a service to seriously consider.

One of my partners was asked by the Chief Information Officer of a major financial services organization for some help thinking through his open source strategy.  Honestly, the open source conversation has not come up much lately.

Is open source in the enterprise a dead issue?

Have companies already tapped into the open source apps and tools and exhausted the options?  Or, maybe consideration of open source software is fully integrated into companies’ software selection process?

A Gartner study said that 84% of financial services companies expected to be using some form of open source by the end of 2008 and a CIO Insight survey put Red Hat at #7 in vendor value.  Given the lack of conversation with the organizations we work with, I suspect that there are more open source opportunities out there.

Furthermore, I think that the increased focus on software as a service and cloud computing will pull more open source solutions into a hosted model.  This should make it easier for enterprises (certainly small and medium sized ones) to take advantage.

Vendors have been hosting websites on Apache for years.  Why not look into other open source applications and tools provided as a service in the cloud?  Here are some examples:

This list was pulled together with some simple Google searches.  Most major hosting firms should be able to provide the basic open source apps as they tend to largely fall into the LAMP architecture – Linux, Apache, MySQL and Pearl (or PHP or Python – all scripting languages).  Of course, the major questions for these providers is if they have the management expertise for the configuration, administration and performance tuning of the open source installation, all enterprise strength computing requirements.  Otherwise, all you would get would be the open source install on someone else’s box – not much leverage.

As an IT leader in a large company, you may think that these hosted apps are not a fit for your company.  Maybe not.  But on the otherhand, a cheap, hosted LAMP+JBoss environment may be the perfect way to get a quick prototype built without the cost, procurement and administration hassles.

I recently spent a few days with several CIOs and IT leaders at an MIT Center for Information Systems Research briefing.  Prof. Wanda Orlikowski led a discussion around some early work exploring the uses and value of Web 2.o technologies in the enterprise.  For purposes of our conversation, we defined Web 2.0 technologies as those that enrich connections between people (a much simpler, but consistent take on Tim O’Reilly’s version).  Specifically, we discussed the use of things like wikis, blogs, micro-blogging, social networks and virtual worlds.

During the discussion, two kinds of corporate cases emerged:

1. IT is getting hammered by a flood of Web 2.0 requests from all over the business
2. IT has prohibited all use of Web 2.0 technologies

The first case is the one that we spent the majority of our time talking about.  What we decided is that in order to stop the flooding, some kind of prioritization is needed.  Since Web 2.0 is about improving connections and interactivity, it makes sense to think about prioritizing based on the major communication channels for an organization.

With a communication channel map, each of the links could be prioritized using factors such as the quality of the current communication for the channel and how important the channel is to the business.  This kind of analysis would form a simple Web 2.0 strategy that could be used to prioritize the incoming requests.

An additional consideration is the exposure an enterprise might have if its employees, acting in their role as “person,” expose sensitive company data on their personal Facebook, MySpace or LinkedIn pages. (Sorry, I couldn’t come up with a better name for the personal hat someone wears.  Maybe you’d like “human” better?)  While it is difficult to tell an employee what they can or cannot do at home, it may be wise to create a set of guidelines for personal Web 2.o use.

So, if you are suffering from the Web 2.0 flood, do yourself a favor and get a life raft.