Case Study: VMware

VMware vSphere (see credits)
The Early History of VMware
VMware Workstation
Video: What Is VMware Workstation?
A video on VMware Workstation

More on VMware Workstation

Challenges in Bringing Virtualization to the x86

Hypervisors add a level of indirection to the domain of computer hardware. They provide the abstraction of a virtual machine: each one thinks if is "king of the hill," and has a whole machine to itself. Ideally, the VMs should be just like the emulated machine, as fast as the emulated machine, and completely isolated from each other.
VMware had these goals (general to most virtualization):

  1. Compatibility:
    Any x86 OS, and all of its applications, should be able to run on without modifications on the VM.
  2. Performance:
    The overhead of the hypervisor had to be low enough the users could use a VM as their primary machine.
    Ideally, things run as fast as on a native OS, but at least as fast as the previous chip generation.
  3. Isolation:
    The hypervisor had to ensure complete isolation of each VM, i.e., be completely in charge of the real physical resources. A VM might be infected with malicious code: this will not impact any other VM.

There was tension between the requirements. E.g., total compatibility might need to be sacrificed for performance. But the designers held isolation as paramount.
The primary challenges were:

  1. The x86 architecture did not support virtualization.
    ( Popek and Goldberg requirements for virtualization )
    Example: POPF (pop flags) would fail silently in user mode.
  2. The x86 architecture was of daunting complexity.
    Decades of "cruft" built up due to backwards compatibility goal. Four modes: real, protected, v8086, and system management.
    x86 operating modes.
  3. x86 machines had diverse peripherals.
  4. The need for a simple user experience.
    The users would be doing the installs themselves, not (e.g.) an IBM technician.
VMware Workstation: Solution Overview
Virtualizing the x86 Architecture

Possible approaches:

Trap-and-emulate

The solution:


Binary translation must be used if:

  1. The virtual machine is running in kernel mode (ring 0).
  2. The virtual machine can disable interrupts and issue I/O instructions.
  3. The virtual machine is running in real mode, a legacy 16-bit mode used by BIOS.

VMware can speed up binary translation to near-native speeds because it sets the hardware to run the code instead of translating it in software.
Runs at 80% of native speed, instead of 20%.

High-level components of the VMware virtual machine monitor.
A Guest Operating System Centric Strategy
The Virtual Hardware Platform

Two layers:

Example: the "Lance" 10-Mbps ethernet card. VMware "supported" this card long after the real thing was off the market, and eventually could run 10x faster.
The actual hardware did not have to be what the guest OS thought was there! It just talked to the VMware drivers, and they could be coupled with different back-ends.

Virtual hardware configuration of VMware workstation in 2000.
The Role of the Host Operating System

So, create three components:

  1. VMX: a user-space program the user interacts with: one per VM.
  2. VMX driver: A small kernel-mode device driver that can suspend the host OS for the...
  3. VMM: multiplexes the CPU and memory; contains trap-and-emulate, device drivers, shadow paging module, binary translator.
    Runs in kernel mode, but not "in" the host OS.
The VMware Hosted Architecture

VMX runs as an OS process. But the VMM is a peer. The VMX suspends the host OS and gives the VMM full control of the machine. This is a world switch.
The VMM and the host OS have entirely different address spaces.
Although earlier described as very time consuming, here the book says the world switch only takes 45 instructions!

The difference between a normal context switch and a world switch.
The Evolution of the VMware Workstation

The VMM / hhost OS architecture remains the same.
But today, VMware Workstation can rely on:

  1. Trap-and-emulate all the time
  2. Nested hardware page tables instead of the shadow page table
Using VMware Workstation
ESX Server: VMware's type 1 Hypervisor

Not having a host OS to rely upon means ESX has more work to do than VMware Workstation. But in a situation where IT organizations are trying to run 1000s of virtual machines, a type 1 hypervisor makes sense: it will run significantly faster.

ESX Server
  1. The CPU scheduler ensures that each virtual machine gets a fair share of the CPU: no starvation.
  2. Scalability: VMs run efficiently even when they need more memory than is actually available.
    Ballooning and transparent page sharing introduced.
  3. An optimized I/O subsystem: device drivers run directly within the ESX hypervisor, with no world switch required.
  4. ESX uses a file system (VMFS) optimized to store virtual machine images. A single ESX Server can issue over 1 million disk operations per second.
  5. The workstations were aimed at developers: one could experiment with new OS releases inside a VM.
  6. ESX Server made it easy to implement new capabilities.
  7. VMotion: live migrate a VM from one box running ESX Server to another. This required the coordination of the memory manager, the CPU scheduler, and the networking stack.
Video on Building a VMware Home Lab
VMware home lab
Quiz
  1. A key attribute of an ideal virtual machine would be
    1. it runs as fast as the real machine
    2. it runs just like the real machine
    3. it is completely isolated from other VMs
    4. all of the above.
  2. Live migrating a VM between physical devices requires coordination of
    1. the file system and the stack pointer
    2. the CPU scheduler the memory manager and the network stack
    3. the RAID array
    4. the number of applications running on each VM.
  3. A motivation for VMware was
    1. the desire to copy IBM
    2. the need for a research grant
    3. the fact that no one had ever created a VM before
    4. the difficulty in innovating in complex modern operatings systems.
  4. One factor making VMs easier to implement on mainframes than on PCs was
    1. vertical integration in the mainframe world
    2. Microsoft's opposition to virtualization
    3. the complexity of mainframe design
    4. the lack of UNIX versions on PCs
  5. Another factor making virtualization difficult on the WinTel platform was
    1. the overly simple chip architecture
    2. the amazing diversity of peripherals
    3. the competition from IBM
    4. all of the above
  6. A virtualization approach called "trap-and-emulate" involves
    1. a switch to the hypervisor when certain instructions are executed by the VM
    2. faking the guest into "thinking" is has really executed certain instructions
    3. allowing most instructions to run directly on the hardware
    4. all of the above
  7. VMware must use binary translation to handle
    1. graphics programs
    2. playing video
    3. executing privileged instructions
    4. floating point mathematics
  8. VMware manages to interact with the host OS by
    1. creating a kernel-mode device driver
    2. relying on a re-written version of the host OS
    3. using binary translation
    4. using direct execution
  9. Ballooning consists in
    1. pumping up each virtual machine to believe it is in charge of the hardware
    2. creating a process inside a virtual machine that can reclaim memory for the hypervisor
  10. On an x86 machine privileged instructions
    1. make the user who runs them the superuser
    2. are ignored by virtual machines
    3. can only be used by the rich
    4. can only be executed in kernel mode
Answers

1. d; 2. b; 3. d; 4. a; 5. b; 6. d; 7. c; 8. a; 9. b; 10. d;

Credits "VMware in the cloud" graphic by Hany R. Michael.