In early July, Edison Su introduced the Apache CloudStack (incubating) development community to a project he had been working on: DevCloud. Edison had created a single VM image that can be downloaded to run within a developer's workstation. I won't go into the architecture of DevCloud itself here. You can go read about it on the cloudstack wiki.
Almost immediately, developers started downloading the image and working through setup / usage issues. Edison did a great job dealing with the problems and suggestions, and pretty soon a decent body of knowledge emerged for how to use it to hack on CloudStack. That included instructions for how to build your own Devcloud image from scratch. It's a great tool for everyone to use, especially with the help of the rdeploy, rdeploydb and rdebug ant targets in the CloudStack build files.
Concurrent with all that, a few of us were discussing how to make it easy to update the devcloud image. After getting some consensus on the CloudStack dev list, we headed down a path of using Vagrant to manage the image within VirtualBox, and to get the image build process fully automated via VeeWee and Puppet.
We ended up with a three phase process:
- VeeWee is used to install Ubuntu on a new VM image (plus a few post install scripted steps, so that Vagrant can manage the VM)
- Vagrant / Puppet are used to take that basic Ubuntu OS installation, and configure XCP, Xen, networking on the VM and the grub boot loader.
- The resulting VM is then packaged as a .box file, and can be used in conjunction with another Vagrantfile and puppet module for daily use.
We left a bunch of the configuration steps out of this initial packaging process, mainly because we wanted to be able to consistently reconfigure a Devcloud VM after a developer may have "broken it" (through the magic of puppet's declaritive nature). That choice also affords us some flexibility in how the devcloud.box base image is configured. For example, we could setup a simple configuration where CloudStack itself isn't running on the VM, but Xen and the other required configuration is.
Issues we ran into:
Vagrant requires the use of the Virtual Box Guest Additions for many of it's features. Unfortunately, we were unable to get the Guest Additions kernel module to load correctly when the VM booted into the Xen Server configuration. The puppet provisioner module relies on VirtualBox shared folders, which don't function without that properly loaded Guest Additions kernel module. Since we were trying to use the Vagrant Puppet provisioner, we were stuck. To get around that, Edison modified the Vagrant core code itself, so that it could use a "with_ssh" option for it's puppet provisioner to SCP files onto the VM.
We also had an issue with guest OS identification within Vagrant. Even though we had added a new OS type (xen), there were problems getting Vagrant to skip it's attempt to mount shared folders. You can see the changes that were required in Vagrant if you check out the hacked version of Vagrant on Github.
Using the new image
Sebastien Goasguen, another member of the community, started testing the build process last week, and took the time to get an initial cut of the build process documentation up on the ASF wiki. I know there are a couple other members of the community that have been trying it out, and I'm looking forward to seeing their feedback and / or patches.
To use the newly packaged box, a developer needs to get the box loaded into Vagrant ("# vagrant box add devcloud devcloud.box"). Then get a local copy of the tools/devcloud/devcloudbox directory from the main git repo. You also have to install the hacked version of Vagrant.
From there, it's as simple as cd'ing into the devcloudbox directory and issuing the vagrant up command.
If you feel like giving it a try, the scripts and configuration files are in the apache git repo, and can be found under tools/devcloud if you do a git clone of https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git. The community is accepting patch submissions through the ASF Review Board instance (be sure to select the "cloudstack" group for your review request, and you will probably want to make sure you are on the cloudstack dev email list to discuss your submission).
Warning - Mileage may vary
This image creation process isn't considered "stable", and I would still point people to Edison's original download location to get your hands on a Devcloud VM.
So what's left to do?
- Testing!
- We're still working out a location from which to create and publish the devcloud.box base image, but expect that to get sorted out in time.
- The changes that were made to Vagrant need to be pulled out of the "hacked" version, and established as proper Vagrant plugins.
- The puppet modules for the basebox and devcloudbox could be cleaned up a bunch. Ideally, they would be more modular in and of themselves, so that we could create various configurations like:
- Xen host only devcloud, for developers that want to run the mangement server in a host OS tomcat instance, but connect to the devcloud VM as the Xen hypervisor
- Multiple VMs acting as Xen hosts (it would require a ton of RAM, but I could see this being used by a developers with big servers to spare)
- Devcloud with CloudStack running in it's agent simulator mode only (no need for Xen)
- Seed the required CloudStack configuration data (using the Marvin test tool) <-- This is already in progress (thanks Edison / Sebastien)
If you want to get involved in the Apache CloudStack (Incubating) project, check out the How to Contribute to Apache Cloudstack page on cloudstack.org, and be sure to join the developer mailing list.
If you'd like to help me pull the Vagrant hacks out of Vagrant core, and into proper plugins, get the hacked code from me on Github and feel free to send me a patch or reference to your repo. Of course, I'd love it if someone were to do the work, and then give it directly to the Apache CloudStack project!