Скачать презентацию Virtual Machines in Condor Jaime Frey Computer Sciences Скачать презентацию Virtual Machines in Condor Jaime Frey Computer Sciences

4c6241d3991d4f1dce258da2e42b47d1.ppt

  • Количество слайдов: 56

Virtual Machines in Condor Jaime Frey Computer Sciences Department University of Wisconsin-Madison jfrey@cs. wisc. Virtual Machines in Condor Jaime Frey Computer Sciences Department University of Wisconsin-Madison jfrey@cs. wisc. edu http: //www. cs. wisc. edu/condor

Virtual Machines › Simulated hardware › Software in the VM thinks it’s running on Virtual Machines › Simulated hardware › Software in the VM thinks it’s running on a normal machine www. cs. wisc. edu/condor

Virtual Machines Real Machine Virtual Machines www. cs. wisc. edu/condor Virtual Machines Real Machine Virtual Machines www. cs. wisc. edu/condor

Benefits of Virtual Machines › › Job sandboxing Checkpoint and migration Jobs with elevated Benefits of Virtual Machines › › Job sandboxing Checkpoint and migration Jobs with elevated privileges Platform independence www. cs. wisc. edu/condor

Job Sandboxing › Protect machines from jobs h. Both accidental and malicious damage › Job Sandboxing › Protect machines from jobs h. Both accidental and malicious damage › Machine owners more willing to run unfamiliar jobs www. cs. wisc. edu/condor

Checkpoint and Migration › State of entire VM (OS › and all) is recorded Checkpoint and Migration › State of entire VM (OS › and all) is recorded VM can be checkpointed for… h. Failure recovery h. Migration to other machines www. cs. wisc. edu/condor

Jobs with Elevated Privileges › Run as root or › › administrator user Alter Jobs with Elevated Privileges › Run as root or › › administrator user Alter OS installation Useful for automated testing of software like Condor www. cs. wisc. edu/condor

Platform Independence › Jobs can run on more › › machines Machines can run Platform Independence › Jobs can run on more › › machines Machines can run more jobs Linux jobs on Windows machines h. And vice versa www. cs. wisc. edu/condor

VM Image Provided By… › Machine Owner h. Condor runs inside a VM h. VM Image Provided By… › Machine Owner h. Condor runs inside a VM h. VM becomes a node in your Condor pool › Job Owner h. VM universe h. Condor runs a user-provided VM image www. cs. wisc. edu/condor

Condor in a VM › › Run Condor in a VM VM joins your Condor in a VM › › Run Condor in a VM VM joins your pool VM acts like any other node Condor in VM can gather information from host machine h. E. g. load average, keyboard idle time www. cs. wisc. edu/condor

Condor in a VM Submit Machine Execute Machine Startd Schedd VM Startd Job www. Condor in a VM Submit Machine Execute Machine Startd Schedd VM Startd Job www. cs. wisc. edu/condor

Config Settings › Host config file h VMP_VM_LIST = vm 1. bar. edu, vm Config Settings › Host config file h VMP_VM_LIST = vm 1. bar. edu, vm 2. bar. edu h HOSTALLOW_WRITE = $(HOSTALLOW_WRITE), $(VMP_VM_LSIT) › VM config file h VMP_HOST_MACHINE = foo. bar. edu h START = (Keyboard. Idle > 150) && (HOST_Keyboard. Idle > 150) www. cs. wisc. edu/condor

Dueling Startds › No intelligent sharing of machine between host and VM startds www. Dueling Startds › No intelligent sharing of machine between host and VM startds www. cs. wisc. edu/condor

If Startds Know About Each Other › Host startd always defers to VM › If Startds Know About Each Other › Host startd always defers to VM › startd (never runs jobs) Possible solution h. Write script to start or stop VM if active startd has no work www. cs. wisc. edu/condor

If Startds Don’t Know About Each Other › Both startds may run jobs, contending If Startds Don’t Know About Each Other › Both startds may run jobs, contending › for machine Possible solution h. Use Hawkeye to cross-publish startd status www. cs. wisc. edu/condor

VM Universe › The VM image is the job › Job output is the VM Universe › The VM image is the job › Job output is the modified VM image › VMWare and Xen are supported www. cs. wisc. edu/condor

VM GAHP › Program used by condor_starter to › interact with VMWare and Xen VM GAHP › Program used by condor_starter to › interact with VMWare and Xen Has own configuration file www. cs. wisc. edu/condor

VM Universe Example Submit Machine Schedd Execute Machine Startd www. cs. wisc. edu/condor VM Universe Example Submit Machine Schedd Execute Machine Startd www. cs. wisc. edu/condor

VM Universe Example Submit Machine Schedd Execute Machine Startd www. cs. wisc. edu/condor VM Universe Example Submit Machine Schedd Execute Machine Startd www. cs. wisc. edu/condor

VM Universe Example Submit Machine Schedd Execute Machine Startd VM GAHP www. cs. wisc. VM Universe Example Submit Machine Schedd Execute Machine Startd VM GAHP www. cs. wisc. edu/condor

VM Universe Example Submit Machine Execute Machine Startd Schedd VM GAHP VM Job www. VM Universe Example Submit Machine Execute Machine Startd Schedd VM GAHP VM Job www. cs. wisc. edu/condor

VM Universe Example Submit Machine Execute Machine Startd Schedd VM GAHP VM Job www. VM Universe Example Submit Machine Execute Machine Startd Schedd VM GAHP VM Job www. cs. wisc. edu/condor

VM Universe Example Submit Machine Schedd Execute Machine Startd VM GAHP www. cs. wisc. VM Universe Example Submit Machine Schedd Execute Machine Startd VM GAHP www. cs. wisc. edu/condor

VM Universe Example Submit Machine Schedd Execute Machine Startd www. cs. wisc. edu/condor VM Universe Example Submit Machine Schedd Execute Machine Startd www. cs. wisc. edu/condor

VM Config Parameters › Condor config file h. Additional parameters › VM GAHP config VM Config Parameters › Condor config file h. Additional parameters › VM GAHP config file h. Config file used by the VM GAHP h. Can’t use macros from Condor config file www. cs. wisc. edu/condor

Condor Config File VM_GAHP_SERVER = $(SBIN)/condor_vm-gahp VM_GAHP_CONFIG = /path/to/vmgahp. config VM_GAHP_LOG = /tmp/VMGAHPLog. $(USERNAME) Condor Config File VM_GAHP_SERVER = $(SBIN)/condor_vm-gahp VM_GAHP_CONFIG = /path/to/vmgahp. config VM_GAHP_LOG = /tmp/VMGAHPLog. $(USERNAME) MAX_VM_GAHP_LOG = 1000000 VM_TYPE = VM_MEMORY = 128 VM_NETWORKING = TRUE VM_MAX_NUMBER = 2 VM_SOFT_SUSPEND = TRUE VM_UNIV_NOBODY_USER = ALWAYS_VM_UNIV_USE_NOBODY = FALSE www. cs. wisc. edu/condor

VM GAHP Config File Common ALLOW_USERS = condor VM_TYPE = <vmware|xen> VM_VERSION = server VM GAHP Config File Common ALLOW_USERS = condor VM_TYPE = VM_VERSION = server 1. 0 VM_MAX_MEMORY = 128 VM_NETWORKING = True VM_NETWORKING_TYPE = nat, bridge VM_DEFAULT_NETWORKING_TYPE = nat www. cs. wisc. edu/condor

VM GAHP Condor File VMWare VMWARE_PERL = perl VMWARE_SCRIPT = <condor_sbin> /condor_vm_vmware. pl VMWARE_NETWORKING_TYPE VM GAHP Condor File VMWare VMWARE_PERL = perl VMWARE_SCRIPT = /condor_vm_vmware. pl VMWARE_NETWORKING_TYPE = www. cs. wisc. edu/condor

VM GAHP Condor File Xen XEN_CONTROLLER = <xm|virsh> XEN_SCRIPT = <condor_sbin> /condor_vm_xen. sh XEN_DEFAULT_KERNEL VM GAHP Condor File Xen XEN_CONTROLLER = XEN_SCRIPT = /condor_vm_xen. sh XEN_DEFAULT_KERNEL = /path/to/kernel XEN_DEFAULT_INITRD = /path/to/image XEN_BOOTLOADER = /usr/bin/pygrub XEN_VIF_PARAMETER = [‘’] XEN_BRIDGE_SCRIPT = vif-bridge=xenbr 0 XEN_IMAGE_IO_TYPE = file: www. cs. wisc. edu/condor

Machine Class. Ad Has. VM = True VM_Avail. Num = 2 VM_Memory = 256 Machine Class. Ad Has. VM = True VM_Avail. Num = 2 VM_Memory = 256 VM_Networking = True VM_Networking_Types = "nat, bridge” VM_GAHP_VERSION = "$VMGahp. Version…" VM_Type = "vmware" VM_Version = "server-1. 0" www. cs. wisc. edu/condor

Build a Submit File › universe = vm › executable = My. Job 1 Build a Submit File › universe = vm › executable = My. Job 1 h. Executable only used for naming in condor_q display › vm_type = www. cs. wisc. edu/condor

Build a Submit File › vm_memory = 256 h. Units are megabytes www. cs. Build a Submit File › vm_memory = 256 h. Units are megabytes www. cs. wisc. edu/condor

Build a Submit File › vm_networking = <True|False> h. Does VM require a network Build a Submit File › vm_networking = h. Does VM require a network interface? h. Some machines may not provide one › vm_networking_type = h. Does VM require a specific type of network interface? h. Some machines may not provide both types www. cs. wisc. edu/condor

Build a Submit File › vm_no_output_vm = <True|False> h. Should modified VM image be Build a Submit File › vm_no_output_vm = h. Should modified VM image be returned to user? h. Some VM jobs may send results over the network www. cs. wisc. edu/condor

Build a Submit File › vm_cdrom_files = a. txt, b. txt h. Files are Build a Submit File › vm_cdrom_files = a. txt, b. txt h. Files are mounted in VM as a CD-ROM image h. Allows you to use a VM image for many different jobs h. You can replace the list of files with a single ISO image www. cs. wisc. edu/condor

Build a Submit File › vm_should_transfer_cdrom_files = <True|False> h. If True, files for CD-ROM Build a Submit File › vm_should_transfer_cdrom_files = h. If True, files for CD-ROM image are transferred from submit machine to execute machine h. If False, files are read from a shared filesystem on execute machine www. cs. wisc. edu/condor

Build a Submit File › vm_checkpoint = <True|False> h. If True, Condor will checkpoint Build a Submit File › vm_checkpoint = h. If True, Condor will checkpoint VM periodically and on eviction from execute machine www. cs. wisc. edu/condor

VMWare Parameters › vmware_dir = <path> h. Directory containing the VMWare VM image to VMWare Parameters › vmware_dir = h. Directory containing the VMWare VM image to be run www. cs. wisc. edu/condor

VMWare Parameters › vmware_snapshot_disk = <True|False> h. A snapshot disk records only the changes VMWare Parameters › vmware_snapshot_disk = h. A snapshot disk records only the changes from the original VM image h. Saves network bandwidth and disk space on submit machine www. cs. wisc. edu/condor

VMWare Parameters › vmware_should_transfer_files = <True|False> h. If True, files in vmware_dir are transferred VMWare Parameters › vmware_should_transfer_files = h. If True, files in vmware_dir are transferred from submit machine to execute machine h. If False, files are read from a shared filesystem on execute machine www. cs. wisc. edu/condor

Xen Parameters › xen_disk = file 1: dev 1: perm 1, file 2: dev Xen Parameters › xen_disk = file 1: dev 1: perm 1, file 2: dev 2: perm 2 h. The Xen VM image is a list of disk image files, along with the devices they should be mapped to in the VM and the permissions they should have h. The image files can be whole disks or disk partitions www. cs. wisc. edu/condor

Xen Parameters › xen_kernel = included h. The kernel is in the disk image Xen Parameters › xen_kernel = included h. The kernel is in the disk image file › xen_kernel = any h. Use the default kernel provided on the execute machine › xen_kernel = /path/to/kernel h. Use the indicated kernel www. cs. wisc. edu/condor

Xen Parameters › xen_kernel_params = <params> h. Append <params> to Xen kernel command line Xen Parameters › xen_kernel_params = h. Append to Xen kernel command line › xen_root = h. Indicates root disk when kernel not included in disk image www. cs. wisc. edu/condor

Xen Parameters › xen_initrd = <path> h. Path to initrd image to be used Xen Parameters › xen_initrd = h. Path to initrd image to be used › xen_cdrom_device = h. When using vm_cdrom_files, you must specify what device the CD-ROM image will be mapped to www. cs. wisc. edu/condor

Xen Parameters › xen_transfer_files = file 1, file 2 h. Xen-related files to be Xen Parameters › xen_transfer_files = file 1, file 2 h. Xen-related files to be transferred from the submit machine to the execute machine h. Any Xen-related image files not listed are assumed to accessible on the execute machine www. cs. wisc. edu/condor

Checkpointing and Networking › VM’s MAC and IP address are saved across › checkpoint Checkpointing and Networking › VM’s MAC and IP address are saved across › checkpoint and restart Network connections may be lost h. If NAT networking is used and job changes machines h. If job is idle for too long before restart › VMWare provides a tool to maintain DHCP leases across checkpoint and restart www. cs. wisc. edu/condor

VM Checkpointing vs. Standard Universe › No relinking › Works with more types of VM Checkpointing vs. Standard Universe › No relinking › Works with more types of jobs h. Multiple processes and threads h. Networking (but migration problematic) › No Remote IO h. Must specify input files www. cs. wisc. edu/condor

Suspending VM Jobs › Hard suspend h. Memory being used by a VM will Suspending VM Jobs › Hard suspend h. Memory being used by a VM will be released and the memory will be saved into a file › Soft suspend h. Memory being used by a VM will not be released and the VM will be just paused like with SIGSTOP www. cs. wisc. edu/condor

Creating a VM Image › Configure OS to… h. Run your application on boot-up Creating a VM Image › Configure OS to… h. Run your application on boot-up h. Shut down when your application exits › Input files can be read from CD-ROM image h. Input files can include application binary www. cs. wisc. edu/condor

How to Create VM images › VMware Server h. Using VMware Server Console www. How to Create VM images › VMware Server h. Using VMware Server Console www. cs. wisc. edu/condor

How to Create VM images › VMware Server h. Can download pre-created VMs from How to Create VM images › VMware Server h. Can download pre-created VMs from http: //www. vmware. com/appliances/ h. Many Linux distributions: Ubuntu, Fedora, Red Hat Enterprise, open. SUSE, Cent. OS www. cs. wisc. edu/condor

How to Create VM images › Xen h. Several Linux distributions have GUI or How to Create VM images › Xen h. Several Linux distributions have GUI or command line tool to create a Xen VM • On Fedora Core, virt-install and virt-manager • On Open. Suse, through Ya. ST h. Can create a VM from a scratch by using dd, mke 2 fs, and mount –o loop e. g. http: //code. google. com/p/seedlinux/wiki/How. To. Create. AXen. Image www. cs. wisc. edu/condor

Small VM Images › Damn Small Linux hwww. damnsmalllinux. org h. As small as Small VM Images › Damn Small Linux hwww. damnsmalllinux. org h. As small as 6 MB › Lite. PC hwww. litepc. com h. Windows 2000 in 150 MB h. Windows 9 x in 40 MB www. cs. wisc. edu/condor

Thank You › Any questions? › Anything missing? h. Write your local Condor developer Thank You › Any questions? › Anything missing? h. Write your local Condor developer › Virtualization in Condor Bo. F h. Friday, 1: 30 -2: 30 www. cs. wisc. edu/condor

VM Universe Submit Machine Execute Machine Startd Schedd VM GAHP VM Job www. cs. VM Universe Submit Machine Execute Machine Startd Schedd VM GAHP VM Job www. cs. wisc. edu/condor

Checkpointing and Networking › Checkpoint and networking should be mixed › with caution Network Checkpointing and Networking › Checkpoint and networking should be mixed › with caution Network connections may be lost on restart h. Especially if IP and MAC addresses change › Other networking problems may occur on restart www. cs. wisc. edu/condor