2014-01-30

SDN Adoption Is Not As Easy As You Think–Take #2

Kendrick Coleman wrote a post today titled SDN on the Horizon, get ready – where amongst other things he stated:

If I were a betting man (and I am, considering I am avid horse racing fan), VMware's NSX is going to get a pretty good stronghold on its existing customer base. It's VMware for Pete's sake. If it has the brand and the label, then a majority of customers are going to buy into it for their needs because having that single vendor relationship makes sense.

This sparked a interesting change of opinions and discussion on Twitter.

I would to elaborate on the 4 points I mentioned in my tweet.

1. Silos

In most enterprise environments today – there is a separate network team, perhaps a separate storage team, and a separate server team. They work together on a daily basis – and probably get along very well with each other – they need to for their day-to-day functions.

But.. they each have their own area of responsibility. Storage admins will not have administrative access to the network switches or VMware environment, Network admins will not have administrative access to the storage array or the Vmware environment and the Server admins will be able to configure the storage arrays or the network switches. These are the separation of duties that many organizations need to have due to regulatory compliance requirements. From a security perspective these are sometimes essential.

Virtualization Admins today have to request storage resources from the storage team – and network ports, subnets and sometimes even IP addresses from the network team in order to make their virtual environment operational. Of course some of this can be offloaded – i.e. allocating subnets and IP addresses to the server team up front, and the same goes for storage space as well. But inevitably the teams are separated and each has their area of responsibility. This is cumbersome, it takes time, and is often a cause for major frustration.

In an utopian world – the Server/Network/Storage team would be one and the same – but that is not how it is today.

Each of these teams have been accustomed to working in a certain way – they have over the years, come to accept delegation of some of the responsibilities to other teams but there are certain things that are still managed solely by a single team.

It is a question of politics, ego, and trust – all of course can be overcome but that does not happen in a day (even Rome wasn’t built in a day).

2. Price

This is something that irks me (and possibly other as well). NSX went GA last year at VMworld.

  • Since then – not a single piece of software has been generally made available.
  • Not a word has been revealed about pricing – except the fact that it will probably be priced at a per VM model (with is something VMware customers DO NOT LIKE!!) (
  • An engagement with VMware PSO is mandatory (at least currently) for an NSX deployment – that could mean additional cost – training.

Of course this all depends on your environment and for some people the price of a product is negligible to the added benefit it brings to the company, your mileage may vary.

One last thing. Your company has already invested in your physical network infrastructure, paid from the budget, and continues to pay for maintenance and support every year, and equipment refresh every X amount of years. If my understanding is correct (and if not – please feel free to correct me) NSX is not a replacement for your physical Network infrastructure – it goes on top – meaning you will still need the physical network components – and in addition the new SDN components
(and I will not use the word tax)

3. Complexity

Introducing a new technology into your datacenter has a price. Be it time, be it $$$, be it training, be it changes that need to be implemented (such as an upgrade to vSphere 5.5 which is mandatory).

blackboxAnother thought I had here was the issue of the concept of a “black box” – and I would like to explain here what I mean.

VMware are looking to sell NSX to the virtualization / server teams – at least this is what I am hearing. They are not pitching (at least not yet) to the Networking teams nor to the Operations teams. (perhaps for obvious reasons)

The way I envision NSX is as follows – give the Virtualization people a segment of the network that they can manage as they like. NSX takes control of that block and does all the work internally. Which is a wonderful thing for these Admins. No more dependency on the networking guys to deploy ports, networks, configure VLANS, setup Load Balancers, Firewalls – it will all be controlled with the virtualization team. But for the Network Admins (and I will play devils advocate here for a moment) this could be a potential nightmare. A significant piece of equipment/technology (and yes it is significant – because no-one will deploy NSX for 100 VM’s) has just been placed in my topology – and I have absolutely a different level of knowledge of what is happening within this black box (I was going to use the word absolutely – but that would not totally true).

A few examples come to mind

  • QoS on the traffic in the box – how will that be implemented?
  • Traffic monitoring with current tools – will not be possible – because the insight into the network is not at the same level as current infrastructure. So now I need two tools. (single pain of glass)
  • Troubleshooting – who does the troubleshooting inside? Network team – they do not have access/control. Virtualization team – they most probably do not have enough in-depth networking knowledge to do this.
  • Who says that the internal traffic will not have any effect on the external network?
  • The blame game. Virtual admins are accustomed to hearing that it is the hypervisor’s fault, they are accustomed also to blaming the storage team for disk bottlenecks. Guess where the fingers are going to be pointed next?
  • Does NSX have any insight into the physical network – if for example there is a problem in one of my core switches – will NSX know anything about it – will it be able alert on such issues – so that I can point the networking team to the problem? Today ESXi can tell you when a physical cable is down – I am not aware if this is true or not with an NSX solution.
  • What happens if something gets compromised inside the NSX environment and proposes a security issue for the rest of the network – will the network team be able to identify the source with the same ease as they do for the physical environment – will there be an IDS/IPS solution that will work for both the physical and the virtual – or will again two different solution be needed?


4. Maturity

NSX went GA less 160 days ago (August 26th, 2013). Yes, there are some customers that have jumped feet first into NSX, even some really big ones. Not many enterprises will deploy a first version of a software product. There are many risks, many unknowns – many question marks and things that “will be fixed in the next release”. We never deploy x.0 releases at time of GA, NEVER! - why you may ask, because there are always bugs, always. And in this case replacing the complete network will is one heck of a major change. It could be that the reason that the years before 2005 were not even mentioned on this slide…

Number of VM's

… was because until version 2.5 (2005) the product was not mature enough? People did not trust it? Was it justified? I am not saying yes or no – but the numbers are there.

Today, when you say virtualization to someone VMware is seen as the leader, the visionary, and rightfully so! It took a LONG time to get there.

I would like summarize and make something clear.

SDN is the natural next step, be it ACI, NSX or whatever. To manage our networks the way we are doing today will not work in the future – just the number of devices that are connected – and will continue to climb with the IoE – it will have to change. The same goes for storage that is why the SDDC is the correct vision, the right direction and we will get there, eventually.

2014 will not be “The year of NSX”. SDN is right technology, but there are many obstacles that block technology (Self driving cars is a wonderful example). Some of these obstacles are easier to overcome than others, but there is one thing for sure.

It will take time.

Your thoughts and comments are very welcome..

2014-01-26

vCenter Orchestrator - Finally Gaining Traction

I have stated before (here), vCenter Orchestrator was the least utilized and least understood product in the VMware product suites, which is a shame – because the capabilities in Orchestrator are pretty much endless.

I started to do some number crunching and the results are quite interesting (although obvious).

Let us have a look at the number of threads in the PowerCLI community vs. the Orchestrator community over the past 4 years

Total posts per month

You can see of course that Orchestrator is far behind and that PowerCLI is the most popular over the years.

Let’s have a look at a different perspective – by the total number of threads per year

Total posts per year

Again PowerCLI seems to be the most popular.

How about how many posts were there in each of the communities on average per month?

Average posts per month

Obvious results again.

Some of my analysis on the data above.

  1. The number of threads that people posted in the forums does not necessarily correlate to the number of people who actually use the product – but I think it is safe to assume that the number would represent the actual use in the field.
  2. vCenter Orchestrator 4.0 was released in February 2008. The number of people who used it in the following 3 years was minimal – and an in my opinion that is because the documentation was horrible. There was no definitive guides, not enough information and and not enough VMware people pushing the product.
  3. That changed in March 2012 – and why was that – well I think because of the Automating vSphere: With VMware vCenter Orchestrator book by Cody Bunch. This was the first decent piece of literature that allowed people to try the product.
  4. If we would compare the ratio of amount of questions asked every month between the PowerCLI and Orchestrator communities from 2010 to 2014 you would see that it would be (approximately) 10:1 in 2010 and 4 years later it is now 2:1. That is a great leap – and if you ask me this gap will get smaller and smaller.
  5. I think the reason is – we have evolved, matured. 4 years ago most of the people who were managing VMware environments were the System Administrators, who were extremely comfortable with Windows and PowerShell – so PowerCLI was a natural and sure bet.
    Javascript?? No ways – wouldn’t touch it with a barge pole. But today – people have become more open to other tools, have found that PowerCLI cannot do everything that they want, environments are becoming more complex and the regular Powershell script just does not cut it any more.

I am extremely happy that people are finally getting better use of vCenter Orchestrator. I was suspecting this was the case – and now we finally have the numbers to back this up.

There are 3 very strong VMware employees that are monitoring the Orchestrator community -
Christophe Decanini, Burke Azbill and Jörg Lew – all of them really know their stuff! (I would advise that also follow them all on Twitter if you are not already)

Top Orchestrator Members

For the PowerCLI community – the one and only Luc Dekens (the man who eats, breathes and sleeps in the community),  Robert van den Nieuwendijk and Hal Rottenberg (all of them should be in your Twitter follow list). As opposed to the Orchestrator community – none of these three are VMware employees, which (in my honest opinion) shows their devotion and dedication – and their continued contribution to the PowerCLI community.

Top PowerCLI Members

The one thing that does come here to mind – is that the other virtualization vendors or the other solutions – do have quite a way to go until they are at the level which VMware has achieved.

I will leave you all with these final words…

Keep calm and automate

2014-01-23

VMware and the Public Cloud – Frenemies

VMWare CloudHistorically, VMware – the pioneer in virtualization – has always been a software/solutions vendor. They were never in the business of selling hardware, they left that to the hardware vendors. With the advent of the cloud, VMware established partnerships with third-party service providers, who have developed cloud-based solutions on top of VMware’s vCloud product. In this way they have continued to be leaders and have advanced cloud technology, without providing a cloud service of their own.

Approximately a year ago, rumors began to emerge that VMware would come out with a public cloud offering. Considering the billion-dollar market already enjoyed by Amazon, Rackspace and others, this wasn’t a total surprise.

VMware’s vCloud hybrid service

About six months ago, VMware announced vCloud Hybrid Service. What is this new hybrid cloud service? In essence it is a vCloud environment that lets you deploy applications onto their public cloud and also allows you to connect seamlessly between the two VMware environments. The actual environments are owned and operated by VMware themselves, but the datacenter belongs to a third party.

VMware has already deployed the service in a number of datacenters in the United States and announced the beginning of Beta operations in the UK, although expansion can be expected in the future. VMware’s entry into the public cloud market has been a source of contention for its partners, concerned about retaining their own market share. VMware partners, however, can continue to offer added benefits, in the form of contract conditions and additional functionality.

How does vCloud compare with pure cloud offerings?

Amazon is the clear leader in the public cloud market. It is the most experienced, with the largest customer base. Amazon cloud offers unique features and services, such as the DBaaS or EBS, that are not available from other providers. Different providers offer various services, including load balancing and scalability.

The cloud is about providing resources on-demand, with infinite capacity and scalability. At present, the vCloud offering does not provide full on-demand infinite capacity and allows for only limited scalability. It uses different terminology, which translates into a greater learning curve for users who are accustomed to platforms such as AWS.

Using Amazon or other cloud providers, you can set up an infinite number of machines and you can get these in matter of seconds, the exact response time will be dependent on a large number of factors. When you choose a VMware cloud offering, you reserve a certain capacity on a monthly basis. You cannot exceed that capacity. This translates into less flexibility and a need for the traditional IT capacity planning.

Overall, VMware’s approach to the cloud has a different feel to it. Whereas most cloud providers tend to treat their virtual machines like cattle – use it and replace it, VMware is treating their cloud offering as a treasured pet, carefully developing and tending it.


VMware – past and future

With the native growth of the native public clouds such as Google and Amazon, their traditional ”growing through partners” approach places them at a disadvantage in today’s cloud market. As a software company, they will expand to provide additional cloud services, including software solutions for networking, storage and more.

Today, vCloud Hybrid cloud Service is VMware’s public cloud. Although VMware started late, I believe that their solution could be a suitable one for many workloads and will evolve to become a significant market player in the future.

2014-01-22

Adopting the Cloud? Importing vs. Building Your Enterprise VMs

This is a re-post of my article, originally published on The Ravello Blog.

Importing vs. Building Your Enterprise VMsThe cloud has brought a change in how we view and manage virtualization, especially for enterprises that need to deploy their sophisticated enterprise workloads around the world. Traditionally, the preferred method was to copy and deploy a virtual machine over a WAN. This can be a difficult and time-consuming process, especially for large VMs and complex IT environments. In recent years, deployment to the cloud has rapidly increased in popularity owing in part to its relative costs and ease of implementation.

As you contemplate the potential for moving to the cloud, inevitably one of the first decisions you will need to make is if you should build your VM from scratch or import it based on a set of templates.

 

Some Background

Although there is a natural tendency to assume that it would be easier to import an existing VM using one of the migration tools delivered by the cloud provider, this is not always the case.

I first encountered the use of templates for VMs around six years ago, when I became aware of VMware’s Clone or Deploy from Template option, which creates a copy of your virtual machine while leaving the original machine untouched. This approach provides a really easy way to duplicate VMs. However, if you need to deploy your VM on different platforms, which is usually what your developers are looking for, multiple templates will be needed, one for each platform. Ideally, you would try to create a template based on the lowest common denominators, and then add the applications on top of the resulting VMs as needed with your Configuration Management tool of choice. This works pretty well for deploying VMs, but will not be the same as your physical builds. You would still need to have a separate template for physical builds.

Create your Golden Image

Sometimes, it might be easier to just create the VM from scratch. For example, the process of installing a new virtual machine on Amazon, OpenStack, and other cloud providers, is fairly straightforward. If your service installation is performed in an orderly fashion, you can simply follow the instructions and build a new clean application environment.

It is important to state that the ease of rebuilding VMs in the cloud depends on the complexity of the software stack, e.g., database or network-specific configurations. Migrating a VM as part of a whole enterprise IT environment can by very challenging. If you choose to migrate from a VMware environment to take advantage of the cost benefits offered by the cloud, most cloud providers enable you to choose between creating a VM or importing an existing VM using their respective proprietary tools. Some of these tools are not mature, therefore you might hit some bumps on the way.

It is best to always start from a clean image that reflects the most basic common denominators – this will be your “golden” image – your template. This means removing everything you don’t really need from the image. For example, you probably don’t need to include the Windows Media Player or some of the other accessories that are included in a standard Windows installation.

There are tools available to help you prepare a clean installation. For example, Microsoft’s SysPrep tool will help you generalize an installation of Windows. It is important to remove passwords, drivers for default hardware not in use, and basically anything else that you don’t need. A similar tool is also available for preparing a Linux installation, virt-sysprep. Make sure you remove any MAC addresses, passwords, IDs, etc., to provide the next user with a clean installation. Once you have created and saved your clean “golden” image, make sure to configure access control, defining specific permissions for authorized users only.

Using these tools works, but not all of the time. Learn how

Building or creating an individual VM in the cloud can be easy however as noted above, migrating a whole environment including database, storage and network is a much more challenging task for the enterprise IT. The environment components’ configurations and specific enterprise-grade appliances might require great investment and efforts in order to be migrated to the cloud “as-is” with the same on-premise capabilities and performance. Or, it might not be possible at all – and then another solution must be found.


Size does matter

Two of the key factors to consider in planning your deployment are the size of your VM’s and the potential of portability between different platforms. Of course, when you create cloned copies of your VM, the size is identical. The larger the VM, the harder it is to move or the longer it takes to import.

Today, you can choose from a wide range of virtualization options and cloud providers. Unfortunately, there is no conformity between the VMs created by the different cloud providers, not to mention your on-premises images. There are no agreed upon formats or industry-wide standards in place. For example, Amazon uses the Xen, while OpenStack, Google and others are based on variations of KVM. At present, there are no clear methods to allow portability between platforms, be it a public cloud, private cloud or on-premises.

New technology players such as Ravello Systems provide the closest approximation to such a solution. By placing a layer between the VM and the hypervisor, Ravello Systems allows you clone your multi-tiers enterprise applications and deploy it on multiple platforms, using different cloud providers without requiring further modification.

2014-01-16

SDN Adoption Is Not As Easy As You Think

Yesterday I read an article titled Martin Casado on VMware's 2014 Roadmap, the Competition, and SDN and the part on which I would like elaborate is the following

On the tactical level, meanwhile, Casado said that VMware's biggest market hurdle isn't any particular competitor or technology,
but rather "the traditional way of doing things" in IT. "Inertia in IT is surprisingly strong," he said. To overcome it, VMware and their ilk must convince enterprises that if they adopt network virtualization, it will not simply be just as secure as the traditional model, but even more secure; not just as easy to manage, but in fact easier.

"We need to convince people that flying is safer than driving," he said.

This sparked a conversation on Twitter 

image

Of course IT is the biggest obstacle to adopting SDN.

It was the same when server virtualization started out. Remember this slide?

VMworld

It took 5 years (yep 5 long years) for virtualization to become default platform on which we deploy workloads, and this is of course is growing…

Why did it take five years? well there is a number of reasons for that and to name a few, product maturity, trust and of course "the traditional way of doing things" in IT. You have to learn to trust the technology, understand it, build a proper ecosystem around it - and educate, educate, educate!!

So VMware - with NSX and Cisco with ACI are in for a long haul. SDN is the natural and logical evolution in the ever growing and changing technology landscape.

  • sdnDo you really think that Network Admins are ready to give up control of the network to their Virtual team?
  • Just give them a range of IP's and let them do what they want?
  • What about firewalls?
  • Who will ensure that what they do does not affect the rest of the network?

It will take time, I sincerely hope it does not take the same 5 years. Until that new evolution occurs, organizations change and the technology becomes well adopted and trusted, until then SDN will have a very long climb, up a very, very steep cliff.

I would also like to address the second part of the Twitter conversation.

It is not uncommon that the development part of the organization is the one that leads the way technology and their demands/requirements from IT.

  • virtual machines for testing
  • virtual machine for continuous development/integration

They do not have the patience/time/tolerance to wait for IT to provide the resources they need. That is how
Rogue IT came about.

The last time I saw someone bring up a DHCP server on the corporate network - it lit up so many red lights in the helpdesk that you might have thought it was a certain place in Amsterdam.

There are limitations (rightfully so or not, that is the question) on what you can do in the corporate network. You cannot do what you would like. Of course this brings a number of limitations to what your developers are trying to accomplish. And a good amount of these things they can get with a public cloud provider today, all they need a small piece of plastic.

A good amount of NSX is going to provide already exists in Openstack, and all in software. Routers, DHCP servers, load balancers, firewalls - it is all there, built-in.

One of the observations I had in the Openstack Israel 2013 conference - is that the adoption of Openstack is being led by developers, and IT is trying to play catch-up.

Developers are not waiting

Developers will not wait for IT to implement an SDN solution - they will go out and do it on their own.

They already are!

Please feel free to leave your thoughts and comments below.

P.S. Flying is statistically safer than driving - but only a few of us know how to fly a plane and even fewer own one.

Almost everyone owns and drives a car.

The conversation continued in an additional post.

2014-01-15

The Difference between Dev + Ops

This one has been simmering for a while, so lets see where this rolls.

DevOps (a portmanteau of development and operations) is a software development method that stresses communication, collaboration and integration between software developers and information technology (IT) professionals. DevOps is a response to the interdependence of software development and IT operations. It aims to help an organization rapidly produce software products and services.
Source Wikipedia

Devops is hot topic today, I keep on hearing around me - "In order to grow we need to change to a DevOps mindset" or, "We have to start doing DevOps.." Firstly, about those statements, I find them quite amusing because just by saying these things will not cause any change in the way things are done, and from these kind of comments - it is quite apparent that the source does not really comprehend.

I have all the respect for Developers, some of my best friends are developers, and I even dabble every now and again in writing code myself. So if there are any developers reading this, please do not take offence this is going to be exaggerated - it is based on my observations - but it is to prove a point.

Developers write code. That is what they do - and do it very well. Give them a problem and say can you write code that will make ObjectA do X,Y,Z - and provide functionality A,B and C - with the appropriate API's that can be exposed through REST - they will say, "No Problem, just give use the coderequirements of what you want it to do, and we will conjure up the code that will make it happen." Sometimes I am in complete awe of actually can be accomplished.

Most developers do just that. They are not aware of the infrastructure underneath, They have some idea about DNS, DHCP, authentication, Security, firewalls, these are all basic terms that they should know about and will probably have some knowledge of - but if you were to ask them how one of these things actually work - how they can/should be deployed, they really will not have a clue. Replication, load balancing, backup, migration, patching, these are all "dirty words" that a developer usually will not want to be hear on a cold and dark night.

I have all the respect for the the Ops guys (and girls). By this I mean the Storage guys, networking, virtualization, OS, deployment, support etc.. etc.. I have been one of these for many years. These are the people who spend countless hours trying to fix the unknown - because if they don't you will not have any email in the morning - and you won't be able to see the latest Dilbert. And when  you accidently deleted that file that was so important and cannot be retrieved from another location and will cause your boss (and you) to lose your job if you do, they will be the ones to wake up at 02:00 to get it back for you. They are the people that will work 24 hours straight to get your website back up and running - sysadminbecause it was hacked. Sometimes all of the above is one person, sometimes different people and sometimes different teams.

Ops people know how to configure your firewalls, set up a load balancing solution, proxies, clouds, switches and make magic happen when you need to have a secure connection between the support center running from your SUV in Australia - and the NOC in the Antarctica, through a 3g Modem (I am exaggerating of course). Some of them can write some scripts and have been doing so to make their lives easier and manage to control a huge amount of resources with only their iPad. But they are not developers. Ask them to create something - it is not in their comfort zone. Ask them to create a portal that will facilitate workflows that will allow you to provision resources on demand in 16 different locations, based on load and the time of day, is not something that they can do. They would rather sit in the room room and and freeze - and not have to code for a living. But if you tell them that you need a database (with redundancy), and connection between two locations, and an Apache server farm with HAProxy for load balancing - that is something they would gladly create for you - even in their sleep.

Developers are used to working with Agile methodologies, Scrum, Kanban, sprints, releases, Source control, QA, regression testing, performance tests etc.. etc..

Ops people are used to change management, maintenance windows, documented procedures, rollback plans, test labs, DR protocols.

Ask a developer to go through a change management process and he will shudder. Ask an Ops guy to deliver something in the next week as part of a sprint and he will duck for cover. Two different languages, two different cultures, meat and milk, oil and water, potato and potatoe.

Because of this - there emerges a dysfunctional process in the organization. Developers write their code - not aware of the operational aspects of what they are doing. A small example might be (and this is a real story).

Developer X devised a solution to do A, B and C, and the projected amount of data that was to accumulated per year was approximately 1TB. When approached by Admin J to ask why so much disk space is needed - because this will require an additional 3 servers with larger disks raising the price of the solution by a factor of three. Developer X then said - why so much? I can get a 1TB disk from the store next door for a 1/10th of the price and just shove it into a desktop and save the data there.

devopsSometimes developers do not understand the operational aspect. The data needs to be replicated - probably more than once - and backed up and so on and so forth - and if anyone would dare to put a desktop with a cheap SATA drive that purchased next door - they would taken outside and publicly pelted with rotten tomatoes.

Developers get their code out - and usually wash their hands of the operational problems that arise thereafter due to bad code, or bad architecture.

"Dev - We wrote the code - now the operations guys will support it."

"Ops - Why is it that we are the ones that get the SMS alerts at two in the morning when there is a problem with the component, the Dev's should be the ones to fix it"

And so back and forth.

Each of the two species will need to adapt, make changes and evolve - if they are to work together in a efficient manner - in the "Quest for DevOps". Personally I feel that the learning curve needed by the Operations people is much higher than that of the Developer's - but we all have something that we can learn from each other - and that will enable us to deliver, better products with a shorter time to market.

I could be way off - and then again maybe not - please feel free to leave your thoughts and comments below.

2014-01-06

Private Cloud – Getting Started Guide for the CIO

This is a re-post of my article, originally published on The Ravello Blog.

Private cloud - getting started guideCloud adoption is on the fast track, companies of all size are seeking ways to adopt cloud while eliminating the traditional IT project risks. The largest enterprises in the world view the private cloud as one of the most appealing ways to start, while utilizing the already made investment of their on-premises resources. CIOs and IT leaders that are thinking about creating their own cloud need to make sure to consider the current viable options and prepare a plan for the future that will suit their specific enterprise grade IT requirements. It is definitely not a decision to be taken lightly. In this post I will provide you with some guidelines and tools to help support you on your new journey.

What is a private cloud and why would you want one?

In my humble opinion, a private cloud is a way to facilitate IT resources provisioning, regardless of the infrastructure, to empower the organization’s internal users. It removes the traditional IT hassles while maintaining the business’ policy yet providing the end developer or business user great agility. Moreover, the private cloud creates better control of your resources. For example, it adds values in terms of the cost analysis, driving better efficiency hence enhancing IT management. A significant advantage of the private cloud over the public cloud, lies in its ability to enforce corporate security policies and meet traditional compliance and regulation rules.

Cloud Adoption is a Strategic Move

Before you go any further, you need determine whether or not your enterprise high level management is committed to making the move to the cloud, and whether or not the necessary resources will be made available. This will be a key element in the success of your project. It is important that you reach out to the invested parties in the company, not just the folks in IT. Try and identify the pain points and strategies for remedying them. Bottom up adoption of cloud is only natural, and if you look around carefully, don’t be surprised to find that your organization developers and testers are already using platforms such as AWS cloud. While they may benefit from agility, they might also challenge your organization’s compliance policies. Today, it is not unusual to find business units within the organization that already have a cloud-based “rogue” IT infrastructure in place.

Knowledge Acquisition and Time to Market

Do your internal teams have the necessary knowledge? Probably not. Do they have the skills to quickly learn and start to deploy? If not, do you want to hire the necessary resources or would it be best to use an integrator? These questions are crucial when an organization wants to adopt any new technology or knowledge. Here time to market is of an essence.

The process of migrating in-house legacy applications to the cloud can be very challenging, and you should expect that some things just won’t work. For example, it is a known fact that Oracle and MSSQL databases were built to rely on specific infrastructure  configurations, and due to compatibility issues, moving these databases to the cloud is not that straight forward.

What’s the best solution for your Private Cloud?

If you decide to go deploy a private cloud, you should carefully research the available options. Check out what I see as the three most critical factors: cost, support, and resource reusability.

There are two main approaches to creating a private cloud, each type of platform with its own advantages and disadvantages:

  • Open source platforms, such as Openstack or CloudStack, let you build your own cloud using your choice of hypervisors. These solutions are free, but they are complex to deploy and maintain. They also do not provide support services per se. You can go the self-support route or you can buy support from a third party. Either way, support is bound to cost you something down the road.
  • Hosted cloud platforms, such as Microsoft’s Azure cloud platform, make it easy for you to build, deploy and manage applications on your own cloud, which is hosted by them.  These platforms are not free, but ongoing support services are typically included.

If you are reselling to external customers, the ability to receive and provide ongoing support from an established vendor can be a very important consideration.

You also want to find out whether or not each potential solution will require you to deploy everything from scratch or whether you might be able to continue to utilize some of the resources already in place. What, if any, new equipment is needed, and what are the requirements and costs of that equipment?

Final Words

In the future, we are likely to see a trend where companies have more than a single cloud even if it’s a private one. The multi-cloud deployment not only enables them to compartmentalize data, but also enables them to deploy the same applications to different clouds with different providers and technologies, improving availability and achieving true cloud redundancy.

The use of the cloud will continue to expand in years to come and there will always be resources that the company wants to control for whatever the reason may be. The private cloud is a viable option to enjoy the cloud features such as cost efficiency, agility and optimal resources utilization … all within your own data center.

2014-01-02

Some Thoughts About Openstack High Availability

As of late I have been putting some thought into how the underlying infrastructure that is needed for running and OpenStack environment and the lack of built-in high availability solutions to provide a robust (and yes I might even go as far as saying - an "Enterprise-Ready") solution.

First let's go over what are the components/services that run in Openstack - the easiest would be to quote the OpenStack documentation.

openstack_havana_conceptual_arch

To simplify this I am not going to go into each and every single service that needs to be redundant but rather to go at a slightly higher level (not that much) and propose that the the following architecture could be a possible solution.

I will not go into the exact steps needed to provide the clustering mechanism for each component, I think there are more than enough resources that have done an amazing job already ( I will say though that it is not straight forward, and multiple technologies and solutions are required for the components).

  1. Introduction to OpenStack High Availability
  2. Practical Lessons from Building a Highly Available Openstack Private Cloud
  3. High Availability Update
  4. Openstack HA @Paypal

(Just as a side note - it would be a great idea to write out an article as to what operations will not be possible/what will not be available, if one of the above services/components are down)

If we were to use the basic model above - we would need at minimum 6 servers/instances/VM's - and to simplify for the sake of the article - the amount of resources allocated to each of these VM's are not really the issue.

What has been occupying me lately - is where do you should these workloads be placed? The workloads do not warrant their own physical hardware (in my humble opinion) - because they could easily run as virtual machines. This could you bring you to a solution such as the following (and by no means is this the correct/optimal way)

Openstack HA

Let's see how we can answer that $1,000,000 question.

Obviously you cannot run them under Openstack itself. Firstly the concept of building Openstack hosted on Openstack is quite problematic. In a vSphere environment this is possible because each and every host can be managed on it's own and you can install a full vCenter / vCloud environment on a standalone host - and have that managed by the VM's it is running. Is this a not always a good idea - and the VCAT documentation states that a Management Cluster is a good design practice. Separating the management layer from the actual layer that runs the resource workloads is a very good idea (if it is a viable option for your environment)

I have yet to find a way that will enable you to import instances that are already running on a nova-compute node.
Not possible.

The way I would design this environment is something like the following architecture as a management cluster.

Openstack HA1

It is not a necessity to have an underlying HA solution on the hypervisor layer (although the benefits are obvious). The components themselves will be using an application clustering solution and if one of the hosts go down (with the control nodes), the services will continue to function - albeit in a degraded state - but you should not have any major loss of functionality.

So what could you use as the host for these control nodes? vSphere, Hyper-V, or perhaps KVM? I am sure there are pros and cons for any of the above solutions, and this is again, is something that you need to take into account when deploying your infrastructure.

Some food for thought on the dawn of this new year.

I would be grateful to hear your comments and ideas on this matter. Please feel free to leave them in the comments below.