Automating Development and Testing Through Virtualization Anil Desai

Automating Development and Testing Through Virtualization Anil Desai Austin Code Camp (Austin. NET User’s Group) March 4 th, 2006 ADNUG: 03/04/2006 1

Outline I. III. IV. V. VII. Virtual Server Architecture Managing VMs Managing Virtual Hard Disks Managing Virtual Networks Managing Removable Media Monitoring Performance Virtualization Best Practices ADNUG: 03/04/2006 2

Virtualization Features l Benefits l l l l Simplified administration Hardware independence/portability Increased hardware utilization Server consolidation Decreased provisioning times Improved security Scenarios l l l Software Development Testing / Quality Assurance Product evaluations / demonstrations Training Disaster Recovery ADNUG: 03/04/2006 3

Development/Testing-Related Scenarios l Setting up a complex test lab l l Rapid setup and tear-down of test environments l l Testing multi-tier applications Isolated networks Simulating multiple clients Consistent configurations Ability to easily roll-back changes Automation of VM setup and software installation Other Benefits l l l Portability of VMs Duplication of errors Simplified administration ADNUG: 03/04/2006 4

Virtual Server Architecture An Overview of Virtual Server 2005 ADNUG: 03/04/2006 5

Virtual Server Overview ADNUG: 03/04/2006 6

Virtual Server Components l l l Virtual Server Service (vssrvc. exe) Virtual Machine Helper (vmh. exe) Virtual Server Administration Web Site Virtual Machine Remote Console (VMRC) New Features in Virtual Server 2005 R 2 l l 64 -bit Host OS Support Improved Guest OS Support l l l Updated Virtual Machine Additions Improved performance Other Features: l l l Virtual SCSI Driver Virtual Disk Precompactor Support for PXE booting ADNUG: 03/04/2006 7

Virtual Server vs. Virtual PC l Virtual Server Features: l Improved performance (multi-threaded) l Web-Based Administration l Scripting and automation support (COM API) l Improved security l Advanced networking l Manageability l SCSI Device Support l Reliability/availability features (clustering) ADNUG: 03/04/2006 l Virtual PC Features: l Optimized for one or a few concurrent VMs (singlethreaded) l Simplified administration l Sound card emulation l Easier file sharing l Automatic client desktop resizing 8

VM Compatibility (VPC vs. Virtual Server) l Compatible: l l l Virtual machine hardware platform VM configuration files (. vmc) Virtual hard disk files (. vhd) Differencing disks Undo disks (. vud) Incompatible: l l l Saved-state files (. vsv) SCSI-mounted virtual hard disks CD/DVD-ROM devices (Virtual. PC = 1; Virtual Server = 4) Sound card support (not available in Virtual. PC) Network files ADNUG: 03/04/2006 9

Installing Virtual Server l Pre-Installation: l Admin Web Site Requires IIS (Application Server Role) l l Warning: Network connections will be temporarily dropped during installation Administration Web Site: l l On Windows XP, new Virtual Directory will be created On Windows Server, new web site will be created (port 1024) Default URL: http: //Server. Name: 1024/Virtual. Server/VSWeb. App. exe Basic Virtual Server Settings l l l Search Paths Enabling the VMRC Server Viewing the Event Log ADNUG: 03/04/2006 10

Emulated Hardware Platform Component Specifications Chipset Intel 440 BX CPU Based on the host CPU (single CPU only) BIOS AMI BIOS Video S 3 Trio 32/64 with 4 MB of video memory Memory Up to 3. 6 GB Input Devices PS/2 Mouse and Keyboard Floppy Drives Up to two 1. 44 MB 3. 5” floppy drives Communications Ports 2 Serial Port 1 Parallel Port IDE Controllers Two channels, each of which supports up to two devices each (up to four IDE total devices) SCSI Controllers Up to four Adaptec 2940 SCSI Controllers (each supports up to 7 devices; 28 devices total) Network Interfaces Up to four Intel 21140 10/100 Mb network interface cards ADNUG: 03/04/2006 11

Creating a New VM l VM Hardware Configuration (demo) l OS Installation Options: l l l Installing the Virtual Machine Additions l l l 1) Physical Media 2) ISO Files 3) Network-Based Installations 4) Automated deployment Improved general performance Improved mouse and keyboard support Time synchronization with the host VM “heartbeats” Enabling Remote Management in the Guest ADNUG: 03/04/2006 12

Virtual Server Files File Type Filename Extension Virtual Machine Configuration File . vmc An XML-based file that specifies disk, memory, and other settings for the virtual machine. Virtual Hard Disk File . vhd This includes fixed-size hard disks, dynamically expanding hard disks and differencing disks. Saved State File . vsv Stores the contents of memory for a virtual machine that has been placed in the “Saved” state By default, these files will be stored in the same location as the. vmc file. Virtual Server configuration file (options. xml) Options. xml Stores information about the configuration of Virtual Server, including search paths, VMRC settings, and other options. Virtual network configuration . vnc Undo disk files . vud ADNUG: 03/04/2006 / Description By default, these files will be stored in the same location as the. vmc file. 13

Automating Virtual Server Accessing the Virtual Server COM API ADNUG: 03/04/2006 14

Introduction to the Object Model l Setup installs the “Virtual Server 2005 Type Library” l l l Virtual. Server object l l l Can be accessed directly using VBScript Can be added as a COM reference in Visual Studio 6. 0 / Visual Studio. NET Connect to a local or remote instance of Virtual Server Add/Remove virtual machines. Virtual. Machines collection provides list of defined VMs. Find. Virtual. Machine method used for getting a VM object by name Create “global” objects: virtual networks, virtual hard disks, etc. Virtual. Machine object l l Represents an existing virtual machine Control VM power states (turn on, turn off, etc. ) Contains properties and collections for the virtual hardware configuration. Accountant can be used to get performance statistics ADNUG: 03/04/2006 15

Setup: Visual Studio. NET l Add the COM reference for “Virtual Server 2005 R 2 Type Library” l l l COM Security: l l Virtual Server requires “Impersonation” (or higher) level “Identification” is the default, therefore must call Co. Intialize. Ex(), then Co. Initialize. Security() before accessing the Virtual Server Interfaces Alternatively, use DCom. Cnfg. exe (or the Component Services Administrative Tool) and modify the Default Properties to set the Default Impersonation Level to “Impersonate” Visual Studio 2005: l l l Add the namespace of Microsoft. Virtual. Server. Interop Should use the multi-threaded apartment (MTA) model (STA may cause performance problems) Disable debugging under the Visual Studio host process to prevent COM security errors In Visual Studio, click on the Properties of a project, then uncheck “Enable the Visual Studio hosting process. ” More information: l l Virtual Server Programmer’s Guide (full sample code for C#) Blog: “Programming Virtual Server from a managed application” (Ben Armstrong) ADNUG: 03/04/2006 16

VM Power States l Virtual Machine power operations: l l l Guest OS Power Management l l Turn On / Turn Off Reset Shut Down Guest OS Save State Pause Standby Suspend Hibernate By default, VMs are placed in Saved State when the Virtual Server service is stopped l VMs are not automatically started, by default ADNUG: 03/04/2006 17

Virtual Machines: Tips l Whenever possible, store files for each VM in a single folder l Moving/Copying VMs: 1. 2. 3. 4. 5. Shut down (or save state of) the virtual machine. Move or copy all required files to the destination server. Add the virtual machine on the destination server (using the Virtual Server Administration web site) If the path to any physical VHD file has changed, edit the configuration of the virtual machine to point to the new physical location (Optional) Remove the VM from the source server (to prevent conflicts) ADNUG: 03/04/2006 18

Disk Architecture: Technical Overview ADNUG: 03/04/2006 19

Base Virtual Hard Disk Types l Base Disk Types: l Fixed-Size VHDs l l Dynamically-Expanding VHDs l l l Overhead due to expansion and fragmentation Can be compacted to reclaim space Linked Disks l l l Best performance, lowest space utilization Links to an entire physical disk (not just a volume) Used primarily for physical-to-virtual (P 2 V) conversions Virtual Attachment Options: l IDE l l l 2 channels * 2 devices each Up to 127 GB disk size SCSI l l Up to 4 controllers * 7 devices each Up to 2. 0 TB disk size ADNUG: 03/04/2006 20

State-Related Disks l Undo Disks (. vud) l l Applies to an entire virtual machine Options: l l Differencing Disks (. vhd) l l Keep Commit Discard Can be based on fixed-size, dynamically-expanding, or linked hard disks Allows the creation of a disk hierarchy Parent disks must be read-only Saved-state files (. vsv) l l Contains contents of memory Virtual Server automatically reserves disk space based on memory allocation ADNUG: 03/04/2006 21

Undo Disks: Example ADNUG: 03/04/2006 22

Differencing Disks: Example ADNUG: 03/04/2006 23

Merging Differencing Disks ADNUG: 03/04/2006 24

Managing Virtual Hard Disks l Converting VHDs l l l Compacting VHDs l l Fixed-Size Dynamically-Expanding Linked disk to Fixed-size or dynamically-expanding Disk space requirements Use the Virtual Disk Precompactor Can be very resource intensive Disk Defragmentation l l Host: Optimizes VHD file placement Guest: Reduces IO within the VM ADNUG: 03/04/2006 25

VHD Management: Objects ADNUG: 03/04/2006 26

Disk Management: Scripting l Enumerating virtual hard disk files l obj. Virtual. Machine. Hard. Disk. Connections l l obj. HDConnection. Hard. Disk. File l l Phyiscal filename/path of the VHD file Creating virtual hard disks l l Collection of Hard Disk Connection objects obj. Virtual. Machine. Create. Dynamic. Virtual. Hard. Disk(path, size) obj. Virtual. Machine. Create. Fixed. Virtual. Hard. Disk(path, size) obj. Virtual. Server. Create. Differencing. Virtual. Hard. Disk (path, Parent. Path) Adding a virtual hard disk: l obj. Virtual. Machine. Add. Hard. Disk. Connection (path, Bus. Type, Bus. Number, Device. Number) ADNUG: 03/04/2006 27

Disk Management Tips: l Monitor Disk Space l l l Perf. Mon: Logical Disk % Free Space Use Virtual Server / Virtual Machine events to send alerts Host Interface Types: l l l IDE Serial ATA (SATA) SCSI RAID Controllers Performance Tips l l l Separate files across multiple spindles Use SCSI disks within the virtual machine Monitor disk resource usage to better distribute virtual machines ADNUG: 03/04/2006 28

Managing Media: Overview l Changing media: l l l Hardware: l l l Media can be mounted/changed while the VM is running Media devices can only be added or removed when the VM is not running Can have up to four virtual CD/DVD devices and one floppy device Can capture a physical device on the host or capture image files Image File Types: l ISO Images: l l l Can represent CD or DVD media Read-only Floppy Disk Images l l Can be created within Virtual Server Read/Write ADNUG: 03/04/2006 29

Managing Media: Scripts l Adding/Removing DVD ROM drives: l l Capturing Media: l l obj. Virtual. Machine. DVDROMDrives: Collection of VMDVDDrive objects obj. Virtual. Machine. Add. DVDROMDrive(Bus. Type, Bus. Number, Device. Number) obj. Virtual. Machine. Remove. DVDROMDrive(VMDVDDrive object) obj. DVDRom. Drive. Attach. Host. Drive(string Host. Drive. Letter) obj. DVDRom. Drive. Attach. Image(string path) obj. DVDRom. Drive. Release. Image(string path) Tips: l l Mounting media files is useful for software deployment/updates Use startup Registry keys to automatically install software ADNUG: 03/04/2006 30

Network Architecture: Overview ADNUG: 03/04/2006 31

Network Automation ADNUG: 03/04/2006 32

Network Architecture: Details l Virtual network options: l l Guest-only Host Network Guest-to-host only (through Microsoft Loopback Adapter) Network Adapters l l All Ethernet NIC types are supported The Virtual Machine Networking Service must be enabled for Virtual Server to use the adapter Each VM can have up to four virtual NICs A dedicated NIC can be used for Virtual Server l l Only the Virtual Machine Networking Service must be bound More Features l l Virtual DHCP Server for each virtual network Integration with Internet Connection Firewall (ICF) and Internet Connection Sharing (ICS) ADNUG: 03/04/2006 33

Networking: Scripting l Managing Virtual Networks: l obj. Virtual. Server. Virtual. Networks l l Collection of virtual networks obj. Virtual. Server. Create. Virtual. Network (name, path) obj. Virtual. Server. Register. Virtual. Network (name, path) Managing Virtual Network Adapters l obj. Virtual. Machine. Network. Adapters l l Collection of network adapter objects obj. Network. Adapter. Attach. To. Virtual. Network() obj. Network. Adapter. Attach. To. Virtual. Network(Network. Name) obj. Network. Adapter. Detach. From. Virtual. Network() ADNUG: 03/04/2006 34

Network-Based Storage l Improving Performance: l Use NIC Teaming l l l Dedicated “appliances” that provide file-level I/O Storage Area Networks (SANs) l l l CIFS / SMB (UNC Shares) Multiple VMs can access the same files Network-Attached Storage (NAS) l l Use Jumbo Frames Remote Storage l l Fail-over Load-balancing Fibre Channel-based, block-level I/O High performance, but highest cost i. SCSI l l l Block-level I/O over Ethernet Performance benefits of SANs with much lower costs (uses copper) Supported on the Host OS as well as from within Guest OS’s ADNUG: 03/04/2006 35

Networking: ICF and ICW ADNUG: 03/04/2006 36

Networking: Firewalls TCP Port Number Purpose Notes 1024 Virtual Server Administration Web Site. The port can be changed during the installation of Virtual Server, or afterwards (see Chapter #9 for details). Administrators can also enable SSL within IIS. 5900 VMRC Server By default, the VMRC server is disabled. The port number can be modified using the Virtual Server Administration Web Site (see Chapter #9). 88, 137, 138 VMRC Kerberos Authentication These ports are used only when Kerberos is being used (for example, when Active Directory-based users are attempting to authenticate to the VMRC server). 137 – 139 Net. BIOS over TCP/IP This is only necessary if Net. BIOS is being used for file transfers across the firewall. ADNUG: 03/04/2006 37

CPU Management l CPU Resource Allocation l Weight-Based Resource Allocation l l l Constraint-Based Resource Allocation l l Reserved capacity Maximum capacity More complicated, but finer-grained control Host CPU Options: l l Hyper-Threading Dual-Core / Multi-Core 64 -Bit (Large Memory Support) Virtualization Support l l l Simpler method (single number to adjust) Ensures full resource usage AMD’s “Pacifica” Intel’s VT Heat-Related Issues l CPUs will automatically step-down when heat thresholds are exceeded ADNUG: 03/04/2006 38

CPU Management: Example ADNUG: 03/04/2006 39

CPU Management: Scripting l CPU Performance History l obj. Virtual. Machine. Account. CPUUtilization l l obj. Virtual. Machine. Account. CPUUtilization. History l l Array of CPU util. stats Viewing CPU Settings l obj. Virtual. Machine. Accountant. l l l Last CPU utilization statistic . relative. Weight. reserved. System. Capacity. allowable. Maximum. System. Capacity. allowable. Reserved. System. Capacity . NET Apps can also easily use system performance counters ADNUG: 03/04/2006 40

Managing Memory l Memory Architecture l Virtual Server can use all of the memory avaialble to the host OS l l l l For 32 -bit, large memory support, enable PAE For large installations, 64 -bit is better Up to 3. 6 GB of physical memory per VM Memory settings can be changed only when VM is turned off Be sure to leave physical memory for the host OS Allow an additional ~32 MB for each virtual machine (virtualization overhead) Physical memory cannot be over-committed Scripting Memory Settings l obj. Virtual. Machine. Memory property l Amount of Memory (in bytes) (read-write property) ADNUG: 03/04/2006 41

VMRC: Overview l Configuring the VMRC Server l VMRC Server is disabled by l l Potential security issues with default configuration: l l Unencrypted communications NTLM security Can customize TCP address and ports Can enable SSL/TLS security l l l Enabled in Server Properties (VS Admin Web Site) Encryption Certificates VMRC Client l l Active. X control that can be hosted in IE or within a client app COM Reference: l l l Virtual Machine Remote Control (VMRC) Active. X client type library Added to. NET namespace VMRCClient. Control. Interop In Visual Studio, add the Active. X control using Tools Choose Toolbox Items. Under “COM Components”, enable “Microsoft Virtual Server VMRC Control” ADNUG: 03/04/2006 42

Virtualization Best Practices Tips and techniques for making the most of virtualization ADNUG: 03/04/2006 43

Managing Performance l Profile your applications l Windows System Monitor l l l l Counter Logs Event Viewer Task Manager Windows Management Instrumentation (WMI) Microsoft Operations Manager (MOM) Systems Management Server (SMS) Monitoring Approach: l l Host Server: Aggregate performance data Guest OS: Detailed resource usage ADNUG: 03/04/2006 44

VM Maintenance l Establish a base library of images l l l Ensures consistency Helps in troubleshooting Prepare VMs for duplication (Sys. Prep) Create a Library of ISO files Manage virtual machines as if they were physical machines l Security updates l l l Patches Corporate security standards Configure anti-spyware / anti-malware exceptions l l Automatic Updates (if supported in Guest OS) Microsoft Baseline Security Analyzer (MBSA) Windows Server Update Services (SUS) *. vhd, *. vud, *. vsv, *. iso Automate common tasks ADNUG: 03/04/2006 45

VM Backup and Recovery l Option 1: l l Option 2: l l Shut down/save state of the VM Copy all required files. Resume the VM Use Backup Agents within the Guest OS. Automating backups: l Build scripts/applications to automate virtual machine backup and recovery: l l l 1) Save state of the virtual machine 2) Copy all related files 3) Resume the virtual machine ADNUG: 03/04/2006 46

More Best Practices l “Virtualization on a stick!” l l Treat virtual machines like physical ones l l Use removable media for creating portable VMs Standard security and maintenance practices Consider performing Physical to Virtual (p 2 v) migrations l l Virtual Server Migration Toolkit (VSMT) Third-party utilities ADNUG: 03/04/2006 47

The Future of Virtualization l Other Virtualization Platforms: l VMWare: l l l XEN Open Source Virtualization Platform CPUs: l l l Workstation GSX Server ESX Server Intel AMD Sun Third-Party Vendors Windows Vista / Longhorn Server l Virtualization will use a “hypervisor” approach and will be included in the platform ADNUG: 03/04/2006 48

References l Microsoft Resources l Microsoft Virtual Server Web Site l l l Microsoft Virtual Server Script Repository l l l Downloadable White Papers and updates Pricing and licensing information Hundreds of sample scripts (VBScript format) “Program Customized Testing Environments Without Trashing Your Machine” (Ben Waldron, MSDN Magazine) Community Resources l l l http: //Anil. Desai. net Virtualization. info (blog) Virtual PC Guy’s Web. Log (Ben Armstrong) ADNUG: 03/04/2006 49

For More Information l Do More With Less: The Power of Virtualization l l l The Rational Guide to Managing Microsoft Virtual Server 2005 (Anil Desai) l l Q 1, 2006 The Rational Guide to Scripting Virtual Server 2005 (Anil Desai) l l Microsoft Tech. Net Magazine (March, 2006) http: //www. microsoft. com/technetmag/ Q 2, 2006 Contact Anil Desai l l E-Mail: Anil@Anil. Desai. net Web Site: http: //Anil. Desai. net l Copies of the slides and source code ADNUG: 03/04/2006 50