Server virtualization

This presentation covers the server virtualization with KVM as the hypervisor

1. Server Virtualization Concepts Kingston Smiler . S ( [email protected] )

2. About Me

3. Levels of Server Virtualization

4. Server Virtualization Wikipedia Definition Partitioning of physical server into number of small virtual servers Uses Virtualization Software and hardware virtualization techniques In most cases the servers are not utilized fully which results in inefficient usage

5. Why Server Virtualization

6. History of Server Virtualization 1970s 1990s 2000s

7. IBM Mainframes IBM CP-67 (Mid 1960s) First Virtualized IBM Mainframe Server Split the computer into multiple virtual machines Different “tasks” can be run separately and independently on the same mainframe If one virtual machine or “task” has a problem, other virtual machines are unaffected

8. Personal Computer Evolution Introduction of Intel/AMD servers (x86 architecture) Each PC/server runs its own OS (Windows / Unix) and runs application. Best approach for personal computers. For server and IT infrastructure brings overhead while going for massive scale. Typically in IT and infrastructure deployment one server runs one application (Email Server, Application Server, DB Server etc ) File Server Web Server File Server Web Server File Server Domain Server App Server DNS Server Each Server Running 1 Application

9. Server Virtualization

10. Physical Server vs Virtual Server

11. Components of Server Virtualization Virtual Machine Software Implementation of a machine that executes program like a physical machine. Host Operating System Operating System actually running in the server / hardware Guest Operating System The Operating System running in the simulated virtual machine HyperVisor or VMM The Operating System running in the simulated virtual machine Host Operating System Hardware -- “real machine ” Virtual Machine Guest Operating System APP APP Virtual Machine Guest Operating System APP APP Hypervisor / VMM

12. Virtual Machine System Virtual Machine Full Virtualization VMs run directly on the server infrastructure without any general purpose OS Hardware -- “real machine” Virtual Machine Monitor (VMM) Simulated Machine Operating System APP APP Simulated Machine Operating System APP APP Hardware -- “real machine” Host Operating Syste m Simulated Machine Operating System APP APP Simulated Machine Operating System APP APP Process Virtual Machine? Needs host operating syste m Virtual Machine running as a process in the host OS

13. HyperVisor Hypervisor Hypervisor is a variant of supervisor and is the “Supervisor of Supervisor ” Piece of computer software, firmware or hardware that creates and runs VM Presents the guest operating system a virtual operating platform and manages the execution of guest OS Multiple instances of a variety of operating systems may share the virtualized hardware resources

14. Type 1 Hypervisors Also known as bare-metal implementation They sit directly on the top of hardware, without needing any operating system. Since they can directly communicate with hardware resources, they are much faster than type 2 hypervisor Pro’s Single virtual machine crash does not affect rest of guest system. More Secure than type2. Generate less overhead and fast. Example Vmware Esx , Microsoft Hyper-V

15. Type 2 Hypervisors Type 2 hypervisor resides on top of the operating system. They cannot directly communicate with the hardware, they are less efficient than the type 1 Pro’s Install is easier. OS takes care of all the hardware, that’s why can support wide range of hardware . Con’s More points of failure, anything that affects the stability of base os can also affect the guest os . Examples KVM, VirtualBox

16. Server V irtualization Components

17. CPU Virtualization

18. CPU Virtualization What is CPU Virtualization? Single CPU acting as if there are more than 1 CPU in the system. Most instructions are executed directly on the physical CPU The Hypervisor / VMM safely performs privileged instructions

19. X86 Architecture Privileges Four Levels of Privileges as Ring 0, 1, 2, 3 for both kernel space (OS) and user space applications These privileges are meant for accessing the hardware resources by the OS and user space applications User Apps run in Ring 3 and OS runs in Ring 0 The AMD-V and Intel-VT CPUs use a new privilege level called Ring -1 for the VMM

20. Server V irtualization Components

21. Full Virtualization User Apps Guest OS VMM System Hardware Ring 3 Ring 2 Ring 1 Ring 0 Full Virtualization Requires no hardware assist or OS assist for virtualization support Guest OS is not aware of whether it is running in VM or physical server . The Guest OS runs in Ring 1 privilege Hypervisor runs in Ring 0 privilege

22. Bare Metal Execution Methodology Guest user apps can’t tamper with the guest OS due to ring protections. However user level code can be directly executed on the processor for high performance virtualization. Guest user apps and guest OS can’t tamper with VMM due to ring protections When the guest OS executes a privileged instruction , it will trap into the VMM When a guest app generates a system call or exception, the app will trap into the VMM VMM’s trap handler uses a policy to decide what to do (e.g., emulate the instruction) User Apps Guest OS VMM System Hardware Ring 3 Ring 2 Ring 1 Ring 0 Direct Execution of virtualizable instructions Binary Translation of non- virtualizable instructions Bare Metal

23. Full Virtualization – Execution Methodology Guest user apps can’t tamper with the guest OS due to ring protections. However user level code can be directly executed on the processor for high performance virtualization. Guest user apps and guest OS can’t tamper with VMM due to ring protections Any instruction either privileged or non-privileged will trap into the VMM When a guest app generates a system call or exception, the app will trap into the VMM VMM’s trap handler uses a policy to decide what to do (e.g., emulate the instruction) User Apps Guest OS Host OS System Hardware Ring 3 Ring 2 Ring 1 Ring 0 Binary Translation of All Instructions Host Based VM

24. Full Virtualization – Drawbacks

25. Para Virtualization Para Virtualization Requires communication between the guest OS and hypervisor to improve performance and efficiency Guest OS should be aware of whether it is running in VM or physical server Guest OS kernel needs to be modified to replace non virtualizable instructions with hypercalls that communicate directly with the hypervisor The Guest OS runs in Ring 1 privilege. Hypervisor runs in Ring 0 privilege User Apps Guest OS Instrumented Host S System Hardware Ring 3 Ring 2 Ring 1 Ring 0 Virtualization Layer

26. Para Virtualization – Execution Methodology Guest user apps can’t tamper with the guest OS due to ring protections. However user level code can be directly executed on the processor for high performance virtualization. Guest user apps and guest OS can’t tamper with VMM due to ring protections As the guest OS kernel is modified to replace non virtualizable instructions with hypercalls that communicate directly with the hypervisor it will issue a hypercall to the VMM VMM’s trap handler uses a policy to decide what to do (e.g., execute the instruction)

27. Hardware Assisted Virtualization Hardware Assisted Virtualization Techniques like Intel VT-X, AMD-V facilitates the hardware assisted virtualization Privileged and sensitive calls are set to automatically trap to the hypervisor and handled by hardware The Guest OS runs in Ring 1 privilege Hypervisor runs in Ring 0 privilege User Apps Guest OS System Hardware Ring 3 Ring 2 Ring 1 Ring 0 VMM Ring -1

28. Multithread Vs Multicore Vs CPU Virtualization

29. Memory Virtualization What is Memory Virtualization? Abstracts the physical memory resources in a virtualized server. Handles the physical memory, virtual memory of both guest and host OS and maintains mapping between these entities.

30. I/O Virtualization

31. Emulated I/O Guest VM Guest OS Host OS (Linux) KVM (kernel module) QEMU vCPU vCPU User space Kernel space I/O emulation I/O operation MMIO or PIO Trap Native drivers Interrupt Software emulates real hardware device VMs run same driver for the emulated hardware device Trap  hypervisor  I/O emulator (e.g., QEMU) Every I/O operation generates trap and emulation Poor performance

32. Para-Virtualized I/O Split driver model Front-end driver in a guest VM Virtual driver to forward an I/O request to its back-end driver Back-end driver in a host OS Request a forwarded I/O to HW via native driver Guest VM Guest OS Host OS (Linux) KVM (kernel module) QEMU vCPU vCPU User space Kernel space VirtIO Backend I/O operation Native drivers VirtIO Frontend Shared descriptor ring: Optimization by batching I/O requests Reducing VMM intervention cost

33. Direct I/O Guest VM Guest OS Host OS (Linux) KVM (kernel module) QEMU vCPU vCPU User space Kernel space I/O emulation I/O operation MMIO or PIO Native drivers 33 /32 Directly assign device to Guest High performance and low CPU utilization

34. KVM, QEMU, LibVirt

35. Agenda

36. KVM (Kernel Virtual Machine) By itself, KVM does not perform any emulation. It exposes / dev / kvm interface for virtualization KVM (Kernel VM) Hardware Assisted Virtualization infra for linux on X86 Requires hardware containing virtualization extensions (Intel VT or AMD-V) Open Source Software under GPL license Two Components: Kernel Component part of mainline linux for 2.6.20. User Component (part of Qemu )

37. Why KVM

38. KVM

39. QEMU (Quick Emulator) QEMU (Quick EMUlator ) Generic and open source machine emulator and virtualizer QEMU can run programs made for one machine ( for example: an ARM board) on a different machine Hosted hypervisor/VMM that performs hardware virtualization Work along with KVM for providing Hardware Assisted Virtualization

40. Emulation Vs Virtualization Both can act as a hypervisor Virtualization Slice the underlying hardware into multiple disjoint virtual devices The ISA of host and guest typically are same Guest operations can be directly dispatched to hardware Emulation Host and Guest may have different ISA Guest ISA is managed in software Guest operations needs translation

41. Usage of QEMU

42. KVM vs QEMU Both are hypervisors Stand alone QEMU uses paravirtualization where as KVM provides infra for hardware virtualization KVM virtualizes only CPU and memory whereas Qemu virtualizes all other devices When QEMU and KVM are working together we get hardware assisted virtualization

43. KVM – Components KVM.ko Core KVM kernel module Implements vCPU and MMU Emulates few virtual devices in kernel Provides ioctls for communications Kvm-intel.ko / kvm-amd.ko Provide support for Intel VMX and AMDs SVM support qemu-kvm Based on Qemu emulator Implements much of virtual devices Provides user interface to KVM

44. KVM – High Level Overview

45. KVM – High Level Overview

46. libvirt Libvirt Provides Open Source API for managing the various hypervisors The management task includes provision, creation, modification, monitoring, control, migration etc Various cloud orchestration platforms uses libvirt libraries for instantiating the VMs Any libvirt client can remotely connect to the libvirt running in a host server

47. Libvirt and QEMU

48. KVM Virtualization

49. KVM Concepts KVM converts the Linux Kernel to provide Hypervisor support However there is no change introduced in the Linux kernel to provide the Hypervisor support. Instead new kernel modules are introduced. Linux kernel already does all the process, memory and i/o management for the process running in the host OS KVM introduces a new mode called guest mode along with the User Mode and Kernel Mode

50. KVM Virtualization

51. Intel Hardware Support For Virtualization Intel Virtualization Support For processor  virtualization, Intel offers the  VT-x  . VT-X adds the guest mode or ring –1 privilege for KVM For memory virtualization, Intel offers the EPT, which translates  the virtual address to the machine's physical addresses to improve   performance For I/O virtualization, Intel implements VT-d and  VT-c

52. CPU Virtualization Every VM is a normal application process in the host machine. The VM process in the host machine can be scheduled to run on the CPU by the host OS (like normal process). However KVM allows the quest code / OS to run directly into the host CPU (although the quest OS / code runs as an application process) using the guest mode execution For scheduling, memory management, and other services KVM uses the linux kernel however it helps the user program to make use of the hardware virtualization

53. CPU Modes

54. Mode Transitions

55. Virtual Memory Virtualization Virtual Memory Virtualization is similar to Virtual Memory concept. In virtual memory concept the operating system maintains mappings of virtual memory to machine memory using page tables. However in VM environment, virtual memory virtualization involves mapping between virtual memory of guest operating system to physical memory of guest operating system along with the mapping of physical memory of guest operating system to the machine memory The first mapping will be maintained by guest OS and the second mapping will be maintained by the hypervisor

56. KVM Device Virtualization

57. KVM Emulated Device Virtualization QEMU in user space provides the emulated device which is a software implementation of the hardware. The guest VM is not aware that it is an emulated device and interacts with the device When there is any I/O operation by the guest VM, the hypervisor intercepts all I/O requests from the guest operating system and emulates the operation of the real I/O hardware

58. KVM Para Device Virtualization / VirtIO Para-virtualized devices are software implementations of hardware devices Para-virtualization requires that the guest operating system be modified to communicate with the hypervisor. It’s a split driver implementation with front-end driver sitting in VM and backend driver sitting in KVM The frontend and backend drivers communicate through a path that is nothing but virtio The frontend driver manages the I/O requests of the guest OSes and the backend driver is responsible for managing the real I/O devices and multiplexing the I/O data of different VMs

59. KVM Para Device Virtualization / VirtIO

60. KVM Network Virtualization

61. vNIC , TUN/TAP Interface vNIC A VNIC is a virtualized Network Interface Card, used by a Virtual Machine as its network interface. Linux Bridge The Linux bridge code implements a subset of the ANSI/IEEE 802.1d standard. Bridge can be created inside a system and can be used to provide connectivity across the VM TAP Interface TAP simulates a link layer device and it operates with layer 2 packets like Ethernet frames . Packets sent by an operating system via a TAP device are delivered to a user-space program which attaches itself to the device OVS open-source implementation of a distributed virtual multilayer switch. Provides a switching stack for hardware virtualization environments

62. KVM Network Virtualization

63. User Networking When ? When the virtual machine is required to access to the host, to internet or to resources available on your local network. T here is no need to access your guest from the network or from another guest. Procedure Simply run guest without specifying network parameters, which by default will create user-level ( a.k.a slirp ) networking

64. Private Virtual Bridge When ? When setting up a private network between 2 or more virtual machines. This network won't be seen from the other virtual machines nor from the real network. Procedure Create a bridge Create a tap device Attach the tap device to the bridge

65. Public Bridge When ? You want to assign IP addresses to your virtual machines and make them accessible from your local network Procedure Create a bridge Create a tap device Attach the tap device to the bridge Attach the host machine interface to the bridge

66. Routing With IP Tables When ? You want to pass through all the traffic from VM to IP tables of Host Procedure Create a bridge Create a tap device Attach the tap device to the bridge Attach the host machine interface to the bridge

67. VirtIO Example (Packet IO)

68. Handson

69. KVM Installation Check your CPU has hardware virtualization support – either Intel VT-x or AMD-V . egrep -c ‘( svm|vmx )’ / proc / cpuinfo should return a value 1 or more Install KVM, l ib- virt , linux bridge utilities and GUI based virt -manager sudo apt-get install qemu-kvm libvirt -bin bridge- utils virt -manager Run the command to add your user account to the libvirt group sudo adduser   name   libvirtd

70. Basic QEMU-KVM CLI Commands qemu-kvm options disk_img qemu-kvm -name “Telsites" -M pc-0.12 -m 512 - cpu kvm64 - smp 2 / images/telsitessp1.raw qemu-kvm - hda / images/selsitessp1.raw -net nic -net user qemu-kvm - hda /images/imagei1.raw - hdb /images/image2.raw - hdc \ /images/image3.raw - hdd /images/image4.raw

71. Thank you [email protected]