Hands On with Cloud-init

6 minutes read
cloud-init • cloud • automation • virtual machines • VM • AWS • IaaS
cloud init

I remember talking to a friend about AWS a few years after we had both finished our undergrad degrees. He described it to me like this:

Imagine you could take a Linux distro, install it, and set someone up with an SSH shell as soon as their credit card cleared. That’s AWS.

— Mike

At this point, mid 2000s, we had both had plenty of experience with SSH and I used to jokingly say that it didn’t matter what computer I used because I was just using it to get to something better anyway. So you can imagine my scepticism in employing new tools when I could typically get away with a dumb terminal and shell scripts to set up anything I want.

Well I found myself in exactly that situation while working on a build system project. I needed to configure virtual machines.

Surely it’s not that hard to automatically configure a VM, right?

Well, configuring VMs provides some very unique challenges, many of which only apply to the first boot:

  • What if you need to resize the root partition?

  • What if you want to install extra packages?

  • How are you going to install SSH keys?

  • What additional services do you need?

  • What order do you perform these operations in?

Sure, you could script these things in BASH, and be very careful about what order they happen in, and set a flag so it only runs on first boot, and account for various differences between distros, and undoubtedly have to pipe some BS to stdin on a tool that is meant to be interactive, and…​ well you get the picture. You also have to figure out how you get the script running on the VM initially. You could make a custom VM to build custom VMs off of I suppose, but it seems like it would be double the work every time your base distro has a major change.

On top of that, you’re usually not just configuring one VM, but hundreds that may be going up or coming down in response to some sort of scaling factor: traffic, demand, time-of-day, etc. Each of those may have slightly different requirements, services, mount points, etc.

It’s a headache. If you don’t believe me, try it without any tools. You’ll quickly find yourself either building a tool or reaching for something like Ansible or…​

Cloud-init. What does it do?

At its most basic, cloud-init solves the problem of how do you get your scripts onto a new, fresh VM. If the VM is cloud-init enabled, it will look for a cloud-config from a few different sources the first time it boots up. Your data source options can be seen here and it suffices to say there are a lot of them.

I know what you’re thinking, "Maybe someday when my friends and family leave me and I find myself living in a sleeping bag in the cold aisle of a datacenter (it isn’t that far fetched) I’ll have the time to configure and run yet another service. Until then I don’t want to run yet another service for cloud-init data." Well fortunately for us, the Nocloud data source allows us to utilize cloud-init without running any extra services.

A practical example

For a build system that I’m creating I needed Debian QEMU images that automatically login root on ttyS0, have a few extra packages, and have enough room to actually build something. All this can be accomplished with two files, packaged in an ISO (with the volume label cidata), and passed as a CD drive to QEMU. Here are those files:

meta-data
instance-id: rob-run
local-hostname: rob-run

It’s about as simple as it gets there. The meta-data file contains information about how we name our instance.

user-data
#cloud-config
bootcmd:
  - apt-get install -y gnupg
growpart:
  mode: auto
  devices: ['/']
  ignore_growroot_disabled: false
mounts:
  - [ /dev/sr0, /mnt ]
apt:
  sources:
    docker.list:
      source: deb [arch=amd64] https://download.docker.com/linux/debian $RELEASE stable
      keyid: 9DC858229FC7DD38854AE2D88D81803C0EBFCD88
packages:
  - docker-ce
  - docker-ce-cli
  - docker-compose
  - containerd.io
  - git
  - curl
write_files:
  - path: /etc/systemd/system/serial-getty@ttyS0.service.d/override.conf
    permissions: '0644'
    owner: root
    content: |
      [Service]
      ExecStart=
      ExecStart=-/sbin/agetty -o '-p -- \\u' --keep-baud 115200,38400,9600 --noclear --autologin root ttyS0 $TERM
  - path: /etc/rootshelltty
    permissions: '0644'
    owner: root
    content: |
      /dev/ttyS0
runcmd:
  - sed -i '4s:^:\nauth sufficient pam_listfile.so item=tty sense=allow file=/etc/rootshelltty onerr=fail apply=root\n:' /etc/pam.d/login
  - systemctl daemon-reload
  - systemctl restart serial-getty@ttyS0
  - apt-get purge -y openssh-server openssh-sftp-server
  - apt-get clean -y
  - fstrim -av
power_state:
  mode: poweroff
  timeout: 30
  condition: True

It is worth noting that I’ve listed steps in the order they are run which is pretty important when configuring a system. Also, you have a lot of modules to choose from. These are just the ones I used to meet my needs:

bootcmd

This is a list of commands that are run before most others. In my situation I had to install gnupg because the image I was using with cloud-init on it didn’t actually have gnupg installed. Meaning when I went to install a new apt source later it wouldn’t be able to import the key and would fail.

growpart

This will make your partition fill all of the available space. It means I can resize a copy of my base image with qemu-img and automatically have the partition table use the space on first boot. resize_fs is also triggered automatically that make the filesystem use the available space.

mounts

This adds an entry in fstab to mount the CD drive to /mnt (with the default settings) on boot. Ultimately this is used by the build system to pass test scripts.

apt

We use this to install a new repository to pull Docker from. My students often use Docker so we’ll need a build image that mimics their environment.

packages

Packages installs a few additional packages are required for our VM including Docker, curl, and git.

write_files

The systemd ttyS0 service is overridden to automatically login root on ttyS0. A list of users for PAM to automatically authenticate is also written to /etc/rootshelltty.

runcmd

These are commands that are run one at a time to update /etc/pam.d/login to automatically login the root user on ttyS0, restart the ttyS0 service, and clean the filesystem a bit.

power_state

Lastly the machine is shut down so our build script exits automatically.

Putting it all together

So with cloud-init in our toolbelt, we can now easily automate the creation of a VM for a build system. Here’s the actual script I provide for mine:

#!/bin/bash

BASE_IMAGE="debian-10-openstack-amd64.qcow2"
SIZE="10G"
OUTPUT_IMAGE="../debian-10.qcow2"

if [ ! -f ${BASE_IMAGE} ]; then
  echo "Downloading Debian 10 OpenStack image"
  wget https://cloud.debian.org/cdimage/openstack/current-10/${BASE_IMAGE}
fi

cp ${BASE_IMAGE} ${OUTPUT_IMAGE}
qemu-img resize ${OUTPUT_IMAGE} ${SIZE}

genisoimage -o cloud-init.iso -V cidata -r -J meta-data user-data
qemu-system-x86_64 -cdrom cloud-init.iso -hda ${OUTPUT_IMAGE} -m 1G -netdev "user,id=n1" \
  -device "virtio-net-pci,netdev=n1" -nographic

cat <<EOF
You now have an image in ${OUTPUT_IMAGE} that can be used with Rob the Builder.
The default prompt is 'root@rob-run:~#'.

If you want to build a memsnapshot so your image can boot 'instantly', run your
newly created image with the 'images/run.sh' command. Once it has booted type
'ctrl-a c' to get the the Qemu monitor and run
'migrate "exec: gzip -c > memsnapshot.gz"'. You can then exit with the 'quit'
command.
EOF

Ultimately cloud-init lets me make it easier to create build images. This benefits people who may want to use the build system including myself who often can’t remember how the heck I built that VM I was using last week.