Virtualisation tips I: The Human Factor

Posted by Roel Gloudemans on 14 April 2008 | 0 Comments

Tags:

The human factor is often overlooked when implementing new technologies in the data center. Most organizations do think about training their personnel and most actually do spend budget on that, but that is it.

If companies truly want to reap the full benefits of virtualisation some other things concerning the human factor are important as well. The main reason why is because virtualisation introduces a fundamental change in the way we look at infrastructures.

Tip I.1: Boundaries between existing specialized organizational units/departments fade.
This tip involves large IT departments only. Thos usually have specialized Windows, Unix and network teams.

In a virtual infrastructure, it shouldn't matter where the virtual system is actually running. This means that the persons managing the hypervisor must work together with the persons managing the virtual machines. This means that whether you are a Windows, Unix or even BSD guy, you'll have to communicate with your peers of all specializations. Not just to change a setting, but to actually dive deep into the lower bowels of the operating system to solve e.g. performance problems.

How about the network? In an enterprise infrastructure, most systems are assigned to one VLAN or the other. If a virtual machine should be ably yo run at any place in the infrastructure, this means that the actual physical server needs to be a member of multiple, if not all, VLANs. All of a sudden not the network engineers decide which packet needs to go where, but the persons managing the hypervisor. This can only happen when departments trust each other.

Some persons will welcome these changes, but there is also a breed of technologist that absolutely abhors to have anything to do with the neighboring technologies (as proof I point to the various flame wars on usenet and public fora). One way to solve this problem is to create a virtual team/organization consisting of persons from all current units/departments and make them responsible for the infrastructure core; the hypervisors and VLAN definitions. Because this team is multi-disciplinary other units/departments will have less trouble trusting and are more willing to work with this new virtual group.

Tip I.2: Human error
People make mistakes, it is a fact of life. Most mistakes do not have a very big impact but some do lead to service interruptions. Virtualisation is also about consolidation. More services/applications will be running on one piece of hardware amd one operating system (the hypervisor). This means that the probability that a human mistake has far reaching consequences increases. Something has top be done to balance this increased risk. There are a few options, among which:

  • Follow tip I.1 and put only the persons with the best track records in the virtual team
  • Change operating procedure. In many cases, a lot can be gained by looking at the operating procedures carefully. Classify procedures into high, medium and low risk. Then think about ways to lower the risk of at least the high risk procedures.
  • Consider carefully which application runs where, or which virtual machine runs where. If a service is made up of 3 applications, running in 3 different virtual systems, it is probably best they all run together on the same piece of hardware. This limits the number of dependencies (only one hypervisor involved) and thus reduces the chance of a service interrupting mistake. Of course there are other things to consider as well. From the security point of view you might not want to run all three applications on the same system. A balance will have to be foun.
  • Cluster. Build clusters of physical machines automatic fail-over capability. This will not safeguard against all problems and mistakes, but will reduce the impact of them. An added benefit is now that all applications are high available.

Next week
Part II of the tips, appearing somewhere next week, will be about service management.