QEMU
Creating virtual nodes using the QEMU
In this paper, we are going to learn how to setup a vm instance using the QEMU. Based on a single OS (ubuntu) image, multiple virtual machine with their own identification will be run simultaneously. Like AWS and GCloud, we can setup instances to build up the cloud infra.
QEMU
Prerequisite
Required packages
You have to install qemu, qemu-utils packages.
lstlisting caption= Install packages ,style=bash
sudo apt install qemu qemu-utils
lstlisting
lstlisting caption= Install packages for MacOS ,style=bash
brew install qemu
lstlisting
Create a virtual strogae device
lstlisting caption= Create a qcow2 image ,style=bash
sudo qemu-img create -f qcow2 ubuntu.img 20G
lstlisting
Mount the qcow2 image on your host machine
lstlisting caption= Load the NBD kernel module ,style=bash
modprobe nbd max_part=8
lstlisting
Connect to the qcow2 as a network block device
lstlisting caption= Connect to the NBD ,style=bash
qemu-nbd --connect=/dev/nbd0 /var/lib/vz/images/100/vm-100-disk-1.qcow2
lstlisting
Mount the network block device as a type of the loop device
lstlisting caption= Mount the NBD ,style=bash
mount /dev/nbd0p2 /mnt/loop
lstlisting
Clean up the NBD configurations
lstlisting caption= Clean up the NBD ,style=bash
umount /mnt/loop
qemu-nbd --disconnect /dev/nbd0
rmmod nbd
lstlisting
Shrink the qcow2 image file size
After creating the image file, we can shrink the image size for saving storage space. shrinkqemuimg
lstlisting caption= Shrink the qcow2 image file ,style=bash
qemu-img convert -O qcow2 source.qcow2 shrunk.qcow2
lstlisting
Download an image file of the Ubuntu server ISO
Find an image file of ubuntu server from https://releases.ubuntu.com
(example, http://releases.ubuntu.com/18.04.3/ubuntu-18.04.3-live-server-amd64.iso? _ga=2.209385489.209752045.1576807893-1215086369.1576225824 -O ubuntu-18.04.3-live-server-amd64.iso)
Boot up VM instance with downloaded Ubuntu image file
lstlisting caption= Boot a VM instance from the CD ,style=bash
sudo qemu-system-x86_64 -hda ubuntu.img -enable-kvm -boot d -cdrom ubuntu-18.04.3-live-server-amd64.iso -m 1g
lstlisting
lstlisting caption= Boot a VM instance from the CD - macOS ,style=bash
sudo qemu-system-x86_64 -hda ubuntu.img -accel hvf -boot d -cdrom ubuntu-18.04.3-live-server-amd64.iso -m 1g
lstlisting
Boot up VM instance
In order to install the kubernetes mastr node, a VM instance should be created with at least 2 CPUs.
lstlisting caption= Boot a VM instance ,style=bash
qemu-system-x86_64 -hda ubuntu.img -device e1000,netdev=net0,mac=de:ad:be:ef:00:00 -netdev tap,id=net0 -enable-kvm -m 1g -cpu host -serial mon:stdio -smp 2 -nographic
lstlisting
Setting up a console and a monitor path for the VM instance
In order to manage the VM instance efficiently, we should prepare a virtual (serial) console and a QEMU monitor path.
The QEMU qemudoc provide several methods for them. qemusocket
First, the kernel image and initramfs image should be accessible from the host system. The QEMU will start boot from those files.
After the kernel boot is done, the VM OS will mount a root device using a qcow2 image file given with -hda option of the qemu-system-x86 _64 command. (/home/vm/link-local.img)
lstlisting caption= Unix Domain Socket ,style=bash
qemu-system-x86_64 -hda /home/vm/link-local.img -kernel "/home/vm/boot/vmlinuz" -initrd "/home/vm/boot/initrd.img" -append "root=UUID=d1fe778d-f6f2-4f03-88e4-047359b532ed ro maybe-ubiquity console=ttyS0,115200n8" -device e1000,netdev=net0,mac=be:ee:be:ee:00:00 -netdev tap,id=net0 -enable-kvm -m 1g -cpu host -smp 2 -nographic -chardev socket,id=monitor,path=/home/vm/run/link-local.monitor.sock,server,nowait -monitor chardev:monitor -chardev socket,id=serial0,path=/home/vm/run/link-local.console.sock,server,nowait -serial chardev:serial0
lstlisting
lstlisting caption= Network setting for MacOS ,style=bash
sudo qemu-system-x86_64 -hda ubuntu.img -nic vmnet-bridged,ifname=en0 -accel hvf -m 1g
lstlisting
After the VM instance is activated, you can attach minicom to it.
lstlisting caption= Minicom to socket ,style=bash
minicom -D unix #/tmp/.qemu_link-local.console.sock
lstlisting
Network configuration
When you launch a new VM instance, the script /etc/qmeu-ifup will be invoked automatically.
Therefore, we can configure network interfaces for the new VM instance.
caption= /etc/qemu-ifup ,style=bash examples/qemu/qemu-ifup
Also the /etc/qemu-ifdown script will be invoked when the VM instance is terminated.
caption= /etc/qemu-ifdown ,style=bash examples/qemu/qemu-ifdown
We will create a bridge interface first and then create a new tap device.
After creating a tap device, let the tap join into the created bridge interface.
First of all, we should set the name of the bridge interface.
Here, in this project, we will use a "br _" as a prefix of the name of the bridge device.
And we will get the last name from parsing a given name of a tap device.
The script will receive the name of the tap device with a numbered suffix. Therefore we can parse the prefix of the name of a tap device.
You would need to know interface type to make decision for whether add a new bridge interface or not netbridge .
TUN/TAP
Q. How to know if a network interface is tap, tun, bridge or physical?
A. tuntap: /sys/class/net/ NAME /tun _flags
bridge: /sys/class/net/ NAME /bridge
bridges and lo: /sys/class/net/ NAME /address is 00:00:00:00:00:00
physical: /sys/class/net/ NAME /device symlink
findiftype
In order to give a permission to create tun/tap device to qemu, you should set valid permission for it. getcap
caption= Change permissions ,style=bash examples/qemu/setcap.cmd
Register the VM as a systemd service
First you should create a systemd service script.
caption= Service file ,style=bash examples/qemu/qemu@.service
The important thing is suffix of a filename AT('@') sign.
The systemd service gets option from command line after the AT('@') sign.
begin lstlisting caption= Command Line ,style=bash
systemctl start qemu@master
lstlisting
The "master" string following after AT('@') sign would replace a " %i" string in the service file.
You are able to set the hostname of each VM instance via kernel command line (boot parameters).
It is supported by systemd v246.
You are able to set the hostname using "systemd.hostname" option.
In order to send the MAC address as for each VM instance's dhcp-client-id,
The /etc/dhcp/dhclient.conf file should be updated.
lstlisting caption= DHCP Client Id ,style=bash
send dhcp-client-identifier hardware;
lstlisting
In the most case, we use the systemd-networkd.
Then we should change the /etc/systemd/networkd.conf file.
lstlisting caption= networkd.conf ,style=bash
...
DHCP
ClientIdentifier=mac
DUIDType=link-layer-time
...
lstlisting
kvmnetworking
netplanbr
tunctl
ipcmd
netbridge