Thursday, March 29, 2012

Does Obama Watch VROOM! Videos?

I'm glad to see that after President Obama caught last week's episode of VROOM! Videos about big data and Hadoop that he has decided to put together a "Big Data" Initiative.  Here is our video.

The VROOM! video went live on Friday March 23rd and just six days later the "Big Data" initiative is announced. I do not believe this is a coincidence. I'm also glad to have him as a viewer.


Tuesday, March 27, 2012

DevOps Ninjas

As part of the ongoing video series So Say SMEs, Kong and I discussed the concept of DevOps earlier in the year.  As this is a new title or classification, we spent most of the video explaining how we understand it to be currently defined.  No doubt this will evolve over time, but the core idea will stay essentially the same.

I believe that as we move forward, more apps will be developed on or involve the use of a cloud platform in some way.  Traditional IT specialists need to expand their areas of expertise to become DevOps Ninjas!


Wednesday, March 14, 2012

Enabling HyperThreading for vSphere Servers

A question that comes up often is should hyper-threading be enabled for a vSphere (ESX) Server?

The short answer is Yes.  In all cases that I'm aware of, it has no downside and upside could be as big as a 25 percent gain in performance.

Hyper-threading allows for multiple threads to run on the same physical core or processor.  On the current generation of server chips this generally means that you get twice as many logical processors or threads as you have physical cores.  This does not however lead to a 2x increase in performance because these are just logical threads and not actual cores.  It allows for the CPUs to be more fully utilized and squeeze out another 20% or so of performance.

So why would anybody even ask if it should be enabled or not?  I think that there are two reasons.

The first is that an earlier version of Hyper-threading from about ten years ago didn't always provide a performance boost and in some cases did cause performance to get worse.  This lead to a recommendation at the time to test  performance before enabling it.  Many people remember this and are still wary of Hyper-threading.

The other reason is that there are some application vendors that recommend disabling HT even on the current chips with the better implementation.  While they probably have a valid reason for recommending this, it does not apply when the app is running in a VM on vSphere.  I would assume that the application (or OS) is in some way not HT aware and could do a very inefficient job of scheduling, using logical threads like physical cores.

With vSphere 4 and 5, the scheduler is hyper-thread aware and makes intelligent decisions on where to schedule processes.  This allows it to overcome an application or OS that is not HT aware.  The scheduler will take advantage of the extra logical threads when it needs them, but will prioritize things to be running on a core all alone if possible.

In the common scenario where there are many VMs running on a single vSphere host, all of these extra threads give the scheduler lots more options and flexibility to give the VMs what they need when they need it. This leads to better performance for all of the VMs with generally a 10 to 20 percent advantage over the same system with HT disabled.

Even if you are only going to assign a number of vCPUs that is equal to the number of physical cores, you should still enable HT.  The hypervisor will be able to use the extra CPU capacity from HT for the functions that it must do, without having to interrupt the VM as much as if HT was disabled.

From a performance engineering perspective, we sometimes run tests (or ask customers to run tests) without HT enabled for debugging purposes.  It is sometimes easier to do performance comparisons without HT enabled and sometimes we need to get a baseline in performance without it running, but I always have it enabled when trying to get the best absolute numbers.


Wednesday, March 7, 2012

VROOM! Performance Show Greenlighted for VMware TV

Something that I have been working to get going for a few months is starting this Friday.  The VROOM! Virtualization Performance Show from VMware TV will have it's first broadcast this Friday at noon CST (1 EST and 10am PST).  Here is the official announcement on the VMTN blog.

Each week will feature a video of a five to ten minute conversation between engineers about a performance topic.  It will be like you are at lunch with a VMware performance engineer, having a conversation about what they have been working on recently.

While the video will be available later on VMware TV, we will have a live chat and stream of the video on Fridays.  This will give you a chance to interact with the engineers from the video, asking questions and possibly poking fun of my performance on video.

The conversations are informal and supposed to be fun.  I record them in a single take and the questions and answers are not rehearsed.  We only decide the topic and general areas to cover before pressing record.

The first episode is with me talking to Josh Schnee (@jpschnee) about VMmark, which he has worked on for the past few years at VMware.