Uploading a custom KVM/Qemu image

We are closing down - please check your email for details.

Qemu can be run on just about any PC so it is very straight forward to create and test a custom image, even if Qemu is somewhat slow. Some distributions now include KVM by default so you can also use that to work with images and enjoy the significant speed up. This tutorial explains how you can then copy your image and run it directly on your VPS.

It is very simple to upload an image to the VPS if it is the right size. The easiest way to make sure of this is to start from scratch, but we will also outline how to resize an image at the end of the tutorial.

The size of the disk

If your VPS is not currently running Linux please follow the instructions on booting the Linux repair CD found at Repairing a broken KVM install.

Your hard disk is found either at /dev/hda or /dev/sda depending on your distribution. To find out the exact size of the hard disk use the blockdev tool to report the size in bytes. 

  root@vps $ blockdev --getsize64 /dev/hda
  root@vps $ blockdev --getsize64 /dev/sda

One of the two devices will not be present, and the size of the other one will be displayed. For the rest of this tutorial we will use an example size of 10007265280.

If you have not yet ordered your VPS please note that we use SI units for disk space (as used by all disk manufacturers) so a 5GB drive will be (at least) 5000000000 bytes.

Creating a local KVM/Qemu image.

Now that you know the exact size of the disk, use qemu-img to create a local image that is the same size. The final image will need to have a raw format, so we will start off by using the raw format so no conversion is necessary. On some file systems it will be necessary to have enough free space to create this image.

Unfortunately qemu-img does not work in bytes, the smallest unit it accepts is kilobytes. It rather incorrectly uses 1024 byte kilobytes, so we must divide by 1024 and round down to the nearest integer. For our example value of 10007265280 this means we have to pass 9772720 to qmeu-img. 

  desktop $ qemu-img create -f raw vps-img.raw 9772720

Your image is now ready to be used locally to do any installation and customization that you require. To make the transition as seemless as possible we recommend that you use the same virtual hardware that is used on your VPS:

  • IDE disks
  • E1000 network card

Uploading the image

When you are satisfied that the image is ready to be uploaded you will need to boot the Linux repair CD as this has full access to your disk.

Since there is nowhere to store a temporary copy of your image on the VPS you cannot simply SCP it. Instead we use pipes to do the following steps all in one go:

  • Read the local image.
  • Compress it locally.
  • Send it to the VPS.
  • Decompress it on the VPS.
  • Write it directly to the disk.

This is fairly straight forward really, the only consideration is that pipes used on the VPS side must be quoted so they are not interpreted locally. It is highly recommended to run this as an unprivileged local user, so a mistake made at the command line cannot overwrite the contents of your local disks! 

 desktop $ cat vps-img.raw | bzip2 -9 | ssh root@vps bunzip2 \| dd -of /dev/hda

Final steps

While your VPS is still running the repair CD, take a look at /tmp/info/networks.xml and make a record of all the network settings. Then flush all data to disk using the "sync" command. Finally change the CD to "No CD" and reset the VPS.

Your VPS should now boot your custom image, however, it will still have your local network settings. Use the remote console to reconfigure the networking appropriately and you should be all set.

Resizing images

If you already have an image you want to use but it is not the correct size you have several options. If the image is smaller than the virtual disk and you do not need the extra space you can just upload it as above.

How you go about shrinking or growing an image is entirely dependent on the OS installed on that image. Typically you will need to resize filesystems and partitions, always remembering to grow the underlying storage before growing filesystems, and conversely shrinking filesystems before shrinking storage.

One method that can work well is to create a new image of the correct size, and then run KVM/Qemu with both images and a disk utility boot CD. The disk utility (e.g. GNU parted) will then see both the old image and the new image as separate disks and might be able to transfer the partitions from one to another.