One fundamental development practice is to have a bullet-proof, repeatable process for building, upgrading and maintaining development environments. My current development practice relies on developing inside a VM using Visual Studio Code’s Remote Development plugin. I treat the development VMs as ‘disposable’, i.e. at any moment in time I ought to be able to commit my work in progress to Github, destroy the VM, build a new one and pick up right where I left off. I should also be able to move seamlessly from one virtualized environment to another – for example, I should be able to develop in the Proxmox VM at home and a VirtualBox VM on my laptop when I travel without any friction between the two.
I use Hashicorp’s Packer tool to automate the VM build and configuration process. I usually maintain two target platforms: 1) a Proxmox server with an NFS backend I maintain at home and 2) VirtualBox which is installed on my laptops. Packer is declarative and supports multiple ‘builders’ for different backends. Both Proxmox and VirtualBox are supported and there is minimal difference in the builder specifications between the two.
Practical Packer
Packer and its builders and provisioners do most of the heavy lifting for you in terms of getting a ‘vanilla’ VM built. Perhaps the most complicated bit is figuring out the correct ‘boot command prefix’ to get past the bootloader. Honestly, getting a prefix that works involves a bit of trial and error and is somewhat cryptic. For the projects I have in Github, the prefix ‘works for me’ but if you are running on either a very fast or very slow machine, then your mileage may vary.
With a ‘vanilla’ VM in hand, the next step is to tailor it to your development needs. Packer is not inherently modular but I have managed to introduce some modularity by providing a collection of scripts that will be run inside the VM after it boots to customize the environment. The Packer specification will invoke these scripts which will either execute or return immediately based on environment variables set from Packer variables which can be set in a HCL file or set on the Packer command line.
The main challenge when executing the scripts is determining if you want the scripted commands to run as root, which is how the provisioner executes shell scripts, or as the ‘development user’ created early in the provisioning process. Essentially, the ‘development user’ is the username you will want to use when logging into the VM for development. The scripts will automatically create this user and assign a single-use password that will have to be changed on first login. The ‘change on login’ feature was not straightforward – so if you want a similar capability, just lift it from my code.
Github Projects
In the https://github.com/stephanfr/Packer repository you will find the Packer specifications for building Ubuntu development VMs in either a Proxmox or VirtualBox environment as well as a project which will allow you to build a VM which can then be used to build bootable, customized RPi images in QEMU. This is a very nice capability and is all due to the work of Mateusz Kaczanowski’s in his PackerBuilderArm Project.
One VM build option sets up an AArch64 bare-metal build environment inside the VM with a directory structure for my RPi Bare Metal OS project. In general though, if you are looking for an easy ARM toolchain setup – you can lift that code as well and modify it for your purposes.
Maintenance
Since I use these specs myself, I will track Ubuntu releases and tooling updates – but the timing is likely to be a bit erratic. I do not generally update tools immediately, I tend to value a stable environment over ‘latest and greatest’ but once a year or so I will update. Mostly updates *should* be limited to tweaking config values but sometimes, stuff just breaks. For example, I have not had success automating deployment of the very most recent Ubuntu VMs.