The Home Lab
If you are not familiar with the vSpecialist group over at EMC, we are a bunch of A.D.D nerds that love to tinker with stuff. Most, if not everyone in the group has some sort of “VMware home lab” they use to help stay current on things as well as a place to just “geek out”. In my case, I built this as part of my endeavor to get my VMware Certified Professional (VCP) Certification (just passed my VCP410 Exam YEAA!! ). A few of them have posted about their home lab (Here) (Here) and (Here). I thought I would post mine as well.
The History:
So, about 13 years ago I had a sweet home lab setup. I lived in Austin and had 5 Static IP’s from SWBELL with my DSL connection (both of these were difficult to come by back then). I hosted our family domain name on a RedHat linux box running Apache and Sendmail as well as a Linux proxy server. Then I got tired of being hacked, I also got a sweet hookup from some friends at the Austin Microsoft office with a copy of Windows 2000 Server (5 client license) and Windows Exchange 2000 with 25 Client License and moved everything over to it. I used IIS to host my site and all was right with the world ….well for a couple of years. Then I started getting calls from my wife saying “E-mail isn’t working” and I just got tired of messing with everything so I “right sourced” everything over to Network Solutions and called it good. My lab eventually went away. Fast forward to a year ago and I started to get the itch again. I decided that instead of buying hardware, I would simply use VMware Workstation and build a virtual lab on my then Xiotech Laptop. I added 8GB of RAM and also added 500GB SATA drive to it and went about building my new playground. Let me tell you, VMware Workstation ROCKS. If you want a cheap lab, that’s the first place I would start !! If you do things right, it’s amazing how far you can stretch 8GB of RAM!!! This worked really well for me. I ran 2(ea) ESXi hosts, an Active Directory server and a vCenter instance and really started getting up to speed on things. I even loaded the EMC Uber VSA Celerra system and played around with it and Unisphere !!. The only major drawback to using Workstation is it doesn’t support “nested” 64bit servers. In other words, you have a base 64bit OS like Windows 7. Then you install Workstation and you can install ESXi but you won’t be able to add a 64bit OS under the ESXi box. That wasn’t a really big deal to me at the time, not to mention 8GB of memory is not a lot for things like that.
Today:
So, that moved me to what I have today. I really wanted to start playing around with more things like VMware View 4.5 so I needed to move out of my laptop and into physical ESXi servers. I was able to source 2(ea) Dell Optiplex 755 with 8GB of memory. Don’t get me wrong, I would love to get full blown servers but at this point, these machines are PERFECT. Each one runs VMware ESXi 4.1 with one booting from an internal 2.5” SATA drive and the other boots from a USB stick. Booting to USB is the ONLY way to go and 4.1 makes it SUPER easy. Just pop in the install CD and about the 3rd or 4th question is a “where do you want to install this at” and it listed my newly inserted USB stick. BOOM, that’s it. I ran like this for a month or two, using internal HDD’s as the datastores. I setup a Windows 2008 Active Directory server, DNS and then setup another Windows 2008 for vCenter. I was then able to get my hands on an Iomega ix2 1TB NAS box (I could really use the ix4 4TB J ). This rounded out my home lab for now. So I setup the 2(ea) ESXi boxes into a cluster and then vMotioned the Windows boxes from the internal HDD Datastore over to the Iomega system I mounted with NFS. From there I’ve moved some VM’s around between the 2 system to balance things out, turned on HA & DRS, installed View 4.5 and now need a Thin or Zero client “hook up” So that I can play around with View a little. I have PocketCloud for my iPad and iPhone which is cool. I think I want to roll VDI out to my 3 kids !!
So, that’s my home lab. Everything I have can easily get picked up from EBAY for a SONG !!! If you want the big boy version check out @rootwyrm’s –“Baby Dragon” or Kendrick Coleman’s (soon to be named) system he picked up during blackfriday. Both are VERY sweet systems. For those that are interested in getting your own home lab, here are a couple of suggestions. Don’t worry about CPU’s. Focus your money on Memory and Storage !! Trust me !! I normally have 2(ea) ESXi servers booted, Server2k8 with AD, Server 2k8 with vCenter, Win7 for View and another Win7 for me to use as a playground. This sucks up way more memory and disk space then CPU’s. My other advice is spend some money on network infrastructure. My future upgrades will be a gigabit cisco switch so I can start working on my CCNA as well as shoring up my network. And MORE capacity and memory !!!
vtexan.com 
good info Tommy, nicely written. I appreciate the tip about concentrating resources on storage & memory rather than CPU.
Tommy – What kind of switch are you using for networking? Nice article!
i’m using just a simple NetGear ethernet switch at the moment – that is on the list to get replaced ASAP as well as adding a dual, or quad port ethernet card to each of the desktops. I need to brush up on my networking skills!!!!
This is a great post. I am planning on building something like the baby dragon with 16 GB of memory for a home vmwarelab.
My thought was to build one powerful box and then run ESXi 4.1 on top of itself to simulate a HA/DR environment. There are several links out there on how to do this (http://www.vcritical.com/2009/05/vmware-esx-4-can-even-virtualize-itself/)
My whole purpose is to avoid the space and noise needed for two boxes, NAS, and extra switches.
Example
[babydragon Esx 4.1 host]
[ESX 4.1 Guest 1]
[ESX 4.1 Guest 2]
[ESX 4.1 Guest 3]
[FreeNas iSCSI]
…
[Win2008…]
[Linux….
[etc….]
This way I am hoping to be able to test and play with the high-end functions of VMWare without taking over my limited space in my house.
Has anyone here done this successfully? I need vmotion to work and all the other fancy functions. Performance will not be great but my intention is to only use the esxi guests with a few dummy installs of Win2K3 server or similar for learning vmware HA/DR and load balancing.
Any thoughts?
@Roland,
You might be interested with the vTARDIS project at http://vinf.net/2010/02/25/8-node-esxi-cluster-running-60-virtual-machines-all-running-from-a-single-500gbp-physical-server/
@vTexan
You’re right about memory and storage, until your 3 kids get their virtual desktop 😉
I have also built a VMware home lab (http://deinoscloud.wordpress.com/2010/11/10/home-lab-in-a-cellar-whats-new/) and now looking at Panologic for a greater VDI experience…
Cheers,
Didier
Rolland – thanks for the comment!!
i know a few people that went that route. It’s very similar to running Workstation inside a Windows 7 Desktop – you have the same “nested” issues (you could get around this by using 32bit OS’s inside the nested ESXi 4.1’s ) but for what you are wanting to work on, it’s probably PERFECT. You should check out @Lynxbat SRM demo he did on his UBER Workstation. It’s pretty impressive and it’s spot on what you are trying to do with your server.
Link: http://nickapedia.com/2010/10/07/lights-camera-replication-uber-srm-video-guide/
Good Luck !!!
@vTexan
Hi Tommy!
Glad to see you’re all settled in at EMC. Just a wee bit different from where you left huh? I got a chuckle out of the brand of USB stick in the picture….nice! Will try to catch up with you later for a reunion happy hour.
Philip
I opted to make my lab portable. I just acquired an Alienware m17x r3 laptop with the 2ng gen i7 proc, 16gb ram and two Seagate Momentus XT hybrid drives in raid 0.
I’m using vmware workstation and run the following:
1 Server 2008 R2 for DC/DHCP/NFS
1 Virtual Center Server
2 esxi 4.1 hosts each with 4gb ram and 4vcpu
– 2 windows 7 desktops inside of esxi for demonstration of ha/drs/vmotion/svmotion and vdi
1 Panologic desktop manager (preferred zero client)
Vmware workstation is set up to provide a bridged network out the on board nic. The dhcp server provides ips to the desktops and any device connected through the little crappy 100mb switch. (be careful to disable the dhcp scope or use mac filtering if you plug into the network or you’ll end up handing out ips to everything on that vlan.)
Esx servers have host only network for management, same above bridge network for guests (no gateway), and another bridged network connected to the wireless card to provide internet access to the guests inside esx.
I’m able to use vmware’s unity feature to manage everything.
Now I can take the server out of my car.
Thanks for sharing your setup, Tommy. I am presently researching components for my own (quiet!) home lab and expect to use as its basis the Shuttle SX58H7 Pro and the 4TB or 8TB Iomega® StorCenter™ ix4-200d. The deeper into the search I venture, though, the more ominously the software aspects of this project loom.
Am I missing something about home lab VMware licensing? Almost every home lab discussion I’ve read details hardware specs and costs exclusively while avoiding even mentioning how licensing is being handled practically. All I know is that I certainly cannot afford $3,500 for an Essentials Plus kit. Purchased individually, it’s $2,900 for an Enterprise single proc license and another $1,500 for the minimal privilege of using vCenter Server Foundation. (Source: http://www.vmware.com/products/vsphere/pricing.html)
That said, are most people just repeatedly downloading and reinstalling evaluations? Are they hitting up their employers for additional production-grade pricing for home labs? Or, truth be told, are they simply using their production keys at home in violation of VMware’s licensing terms?
Have I even mentioned Microsoft licensing? 😦
Chuck – thank you for the comment !! So, the net-net is in the home VMware lab – the giant elephant in the room is the license keys you need to do what you want in your own home lab. So, Microsoft makes this pretty easy – you can get a subscription to TechNet for less than $200 and that will cover most, if not all of your license needs for Microsoft’s products. Just abide by their rules and you should be good to go.
VMware is a little more interesting. The net-net is today the best way to do it is to simply re-install vSphere every 90days. The great news is the install is quick and easy !!
Hello Sir, I need help in building a new ESXI 5.5 home lab for a VCP 5.5 exam, how can I build one? I only have a notebook for vshpere client, a internet router , nothing more!
Please help!