A Small Part in 1 Million IOPS

VMware demonstrated the impressive storage I/O capabilities of vSphere 5 recently when a single host achieved 1 MILLION IOPS.  We can do a whole bunch of IOPS - which is really cool.  This is way more IOPS than just about any workload needs and demonstrates that storage throughput is not a problem for vSphere.

I had a small part to play in this effort. I got an email in June asking if I had a large server that I could loan out for a few weeks.  It turned out that I did have a four socket server with 512GB of RAM that we had already purchased, and had shipped, but had not yet arrived.  So as soon as it arrived at our lab, I had it immediately shipped right back out to an EMC lab the same day.  They had a deadline of just a few weeks to get everything done.

Three weeks later I got an email that it was ready to ship back, I provided our address and received the 1 million IOPS server back in my lab another week later.  I'm now using this famous 1 million IOPS server for some other tests and it will reappear in future blogs and white papers.  It kinda feels like working with a rockstar.

Todd

Up to Speed with vFabric Data Director

When I read the press release announcing VMware vFabric Data Director, I wasn't sure this was what we had called project Aurora internally until I read paragraph eight.  It was at this point in the press release that we get into enough technical specifics that I could be sure.

This made me fear that the coolness of this new database from VMware would get lost.  This was most certainly was not the case.  Chad Sakac wrote  a great post giving some of the background around project Aurora and lots on exactly why this is such a powerful new capability.  Brent Ozar's post was from a completely different angle than you would expect (or maybe you would if you've seen him in his kilt at PASS) and first describes it as a new feature of Microsoft SQL Server Denali.  Scott Drummonds wrote another great post that I think puts the new VMware database into perspective for the database market.

The common thread among all of this is that the new vFabric Data Director is a new approach to using databases.  It opens up new ways to manage and use databases in a more cloud or service oriented way.  If you missed it from the initial announcement or it didn't make it onto your radar screen due to all of the other stuff going on during VMworld, a good way to get up to speed is to read these blog posts.

Todd

Getting Started with VSA

If you are going to be at VMworld this year, please come to the session I'm presenting with Edward Hsu on VSA.  We are to cover some basics and some in-depth goodies.  If you thinking about using VSA, this session will help you to understand exactly what VSA is and some details to help you understand how to deploy and manage it.  It is on Monday at 12:30 in Palazzo Ballroom G - VSP3299.

Here's a very fast overview of VSA:  It stands for vSphere Storage Appliance.  It takes the local disks on two or three vSphere 5 hosts and creates a "virtual SAN".  This virtual SAN enables all the advanced featrues of vSphere - vMotion, DRS, HA - to be possible with only local storage.

I've done lots of testing with it over the past four months and now that the product is finished, I can start talking about.  This session at VMworld is the starting point and it will be followed by a whitepaper and several blog articles.

Todd

HP TouchPad Conspiracy Theory #1

I've always wanted to be the author of a conspiracy theory and I think that HP has given me the perfect opportunity.  The whole TouchPad / WebOS apparent dismantling is just a trick.  HP has actually just pulled off one of the greatest stunts of all times.  What follows is my theory of what is really going on.

HP acquired Palm and WebOS last year and began to work on a strategy to create devices that were competitive with Apple's iPhone and iPad.  The WebOS is a technically superior OS to iOS in many ways, which is an advantage, but would not guarantee success simply due to technical superiority.  HP also had the ability to basically copy the same general tablet form factor and technical specs that all the other tablets were using.  In basic terms HP was able to create a device that was similar technically to the iPad and in some ways even better.

HP realized that they also had to compete with the Apple appstore and development community.  This is not something that can be easily solved, so a few high placed executives must of come up with the secret plan that we now see unfolding.

They planned to release the Touchpad as a high end tablet that "Worked Like Nothing Else" with celebrity spokespeople and lots of ads.  This would give consumers the impression of the high value of their new tablet.    In order to spur demand and quickly sell ALL the Touchpads they could make, they would then slash prices to $99 and announce that they were discontinuing the Touchpad.  All Touchpads would sell out in days.

This part of the secret plan has been executed already, and we have seen that it was successful.  The next steps should come in the next few days.

It is my theory that HP's secret plan next calls for HP to announce that in the wake of unprecedented demand and the need to support all of it's new cusomers - They are going to keep working on tablets and WebOS.

Maybe this is just a crazy consipiracy theory, but I thnk that it makes sense.  We'll all just have to wait and see what happens over the next couple of weeks :)

What Are You Waiting For?

I'm feeling bold today, so here it is. There are no remaining technical barriers to virtualizing your entire data-center.  The benefits are real.  What are you waiting for?

Many many customers have virtualized all the most business critical and performance intensive apps found in datacenters.  Exchange is now commonly virtualized.  SQL Server and Oracle databases are also run as VMs on vSphere in data centers all over the world.  Even bigger more complex stuff like SAP and Oracle RAC can be run virtualized with no technical issues.  There are proven examples for just about every application type out there.  It works.

The benefits are big and from what I have seen, the more that you virtualize the bigger the benefits become.  I think that everybody is aware of the clear consolidation benefit for legacy and underused servers.  Once you have completed a consolidation effort you begin to see that the operational benefits are much bigger than what you realized before starting.  In fact I would say that these benefits are much bigger than the pure consolidation gains.

The number of workloads that cannot be virtualized due to technical barriers is very small.  This leaves issues like application licensing uncertainty, organizational politics, out dated conceptions of what virtualization can do, support questions, fear of change, and others.  Some of these have simple answers and others are more difficult, but even though they are not technical reasons they are still very real reasons.

If you were to purchase a new Intel or AMD based server today with either two or four sockets, you would be hard pressed to find an application that truly needs the whole host.  The only practical way to really utilize these nice new servers is to use virtualization to be able to run multiple workloads on it at the same time.  Otherwise you are looking at a system that is probably less than 30% utilized almost all of the time.

Todd

Calculating Back of Envelope Estimates for IOPS

I recently spent some time putting together a simple spreadsheet to help me estimate how many IOPS to expect from a given config.  I thought that I would share with the group.  Some of the key factors in the equation are pretty cut and dried, but others have to estimated.  So this equation will only give you an answer that is as good as your estimates.  This formula and blog is really designed to estimate random IOPS and doesn't talk about sequential. (Previous two sentences should be read as my disclaimer and please notice the use of the word estimate in the title and throughout the blog post).

I'm first going to explain the equation, and then I'll give it to you in short form with a few examples.

Reads / Writes and RAID types - A single read results in a single read operation, but a write results in multiple operations depending on the RAID type used.  The two mirroring RAIDs of 1 and 10 have a write cost of 2.  RAID 5 has a write cost of 4.  For this reason the equation is broken up into two halves for calculating the number of reads and the number of writes.  The weight given to reads and writes is based on an estimation of the read/write ratio of the workload (this is one of the estimates that you must come up with that makes a big difference in final number).

Disk Performance and Number of Disks - Each individual disk is capable of a given number of random I/O operations per second.  The better or faster a disk is, the more IOPS it can complete.  There are lots of sources around the web with estimates for 7200, 10K , 15K, and SSD disks.  I've always used the round number of 120 IOPS as an estimate for a 10K RPM disk for example.  The other factor here is the cache on the RAID controller or SAN processor or whatever it is called by the solution that you are using.  The size of the cache will effect the overall performance with larger caches generally allowing higher performance. Estimating the IOPS per disk is the other number that can cause a big difference in the final results that must be estimated.

So you multiply the number of disks in the RAID set by the estimate for IOPS per disk to get a total IOPS for the LUN / virtual disk / RAID group / whatever you want to call it.  You then multiply this by the percentage of reads for the read half of the equation.  For the write half you do the same, but you then have to apply the write penalty and divide by the RAID cost for the RAID type you are using.  I think about now is when you realize why RAID 5 performs so poorly in write intensive environments.

Here is the formula in a simple terms:

( (Num Disks * IOPS per Disk) * pct Reads) ) + ( ( (Num Disks * IOPS per Disk) * pct Writes) / RAID Cost)

Examples:

An example for an 8 x 10K RPM disk RAID 1/0 with 60 percent Reads and 40 percent Writes

( ( 8 * 125) * .60) + ((( 8 * 125) * .40) / 2) = 800 IOPS

same but change estimate of IOPS per disk to 150

( ( 8 * 150) * .60) + ((( 8 * 150) * .40) / 2) = 960 IOPS

And now change to RAID 5 with cost of 4:

( ( 8 * 150) * .60) + ((( 8 * 150) * .40) / 4) = 840 IOPS

And finally RAID 5 with 85% writes:

( ( 8 * 150) * .15) + ((( 8 * 150) * .85) / 4) = 435 IOPS

Special thanks to Scott Lowe for his excellent blog on essentially the same topic last year.

An Announcement About An Announcement

Today there is an announcement that you can sign up to hear an announcement from VMware that will happen on July 12.  While this does seem silly, this is what everybody does these days with kinda vague invitations to hear about the next great product or thing or concept.  There really are lots of announcements about announcements. I guess that me blogging about adds even one more level.

It really reminds me of when I first realized how important it is to have small meetings before a big meeting to make sure that nobody was surprised and the big meeting went as expected.

Even though I'm having a little bit of fun about the state of affairs around these announcements, I must admit that I fall for it just about every time.  I like the anticipation of hearing about an announcement a few weeks or days before and then looking forward to what will be announced.  And that is true in this case as well, I'm looking forward to July 12th to see and hear the VMware announcement.


Todd