As we have a basic build infrastructure with Git, Jenkins, FitNesse, CppCheck and Valgrind, now it is time to turn attention to leash development system itself. Not all developers like to prepare and manage their build systems. However, it is also a necessity to have a common, portable and consistent development environment. In fact, preparing and handling this system is a non value added essential part in Cost of Software Quality harnessing. It is better to let people that is expert and willing to handle system dependencies prepare the system and do the work once, then be sure that everybody else use this prescribed recipe. This abstraction may be handled with vagrant & puppet. Vagrant (https://www.vagrantup.com/), provides a common development environment using virtual machines, regardless of the operating system behind. Hypervizor may be VmWare, KVM, Zen but default is VirtualBox.
To start with, install Vagrant (https://www.vagrantup.com/downloads.html) and VirtualBox (https://www.virtualbox.org/wiki/Downloads). Create an initial configuration file with selecting a base virtual machine. Some templates may be found at https://atlas.hashicorp.com/boxes/search. Here, I will select a basic debian environment with puppet.
Initialize vagrant with selected template,
vagrant init puppetlabs/debian-7.8-64-puppet
This will create a minimum
VagrantFile which will specify machine definitions. Then perform a
vagrant up. Since this is a brand new up, vagrant will search for template and download it, which may take some time.
At the end of verbose, there is a couple interesting information. One is about ssh connection, which states that private key ssh session available through
127.0.0.1:2222. Normally vagrant machines may be accessed through
vagrant/vagrant username password pair. In this specific puppet template, username private key pair is preferred. The other information is about a shared folder between host and guest systems, which will be handy. The shared folder
D:\capturer\vagrant\debian in host will be mounted as
/vagrant in guest.
We may try ssh connection through
In my case, vagrant complains about finding ssh executable, which may easily be fixed through setting any path (I preferred using git path, and it is better to add this to system path
C:\Program Files (x86)\Git\bin for persistence purpose). The verbose also gives location of private key, that may be used through other ssh clients. For example, we may use this key at putty. Putty does not accept the somewhat standard format and needs slight modification. First convert the original key with puttygen as;
And, save as it as a private key (
vagrant_debian.ppk in this specific example). Then set this key as authorization mechanism in putty ssh session setup.
Now, ssh through putty should also be successful;
As basics are over, it is time to make customization that will provide real benefit of vagrant. First add a local interface, define a hostname, and set virtual machine memory to 1024 MB. These may be set through
Provisioning is only done at initial setup to decrease boot up time. For this reason, remember to re provision so that changes of
Vagrantfile becomes effective. This may be achieved by
vagrant provision fur running machine,
vagrant reload --provision for running machine after a reload and finally
vagrant up --provision for a halted previously provisioned system. Result of configutaion change may be checked from console output;
Provisioners represent the core of Vagrant. Usually shell provisioner may be used for simple tasks, like setting environment variables, followed by automation tools like puppet for complex jobs. Power of automation tools comes from their state handling schema, and they know what it was done before, to perform only brand new tasks. Therefore there becomes a good level of abstraction and you only define higher level policies, and underlying steps are performed by those provisioners.
Puppet is robust well established IT automation tool, that is capable of orchestrating tasks such as installing packages and configuring systems.
Puppet apply (standalone version of Puppet) does not require any installations and may instantly be used in Vagrant. In Puppet, tasks are not executed in the order of appearance, therefore dependencies between tasks should be explicitly defined. Puppet scripts are called manifests and these may be grouped into modules.
Vagrantfile modification below, manifest folder with
default.pp manifest file inside, will be assumed to be inside directory
puppet at same level of
Vagrantfile. Besides, any modules are expected to be in
puppet/modules folder. The directory structure will be as;
manifests are collection of
resources that perform tasks.
Resources may be thought of system resource state, such as existing of a file, or ensuring presence of a package. For example, in the upper default manifest, we want to ensure that some system packages should be installed after an aptitude update. After re provisioning, state changes to corresponding packages may be seen below;
Now, let’s make necessary modification in order to clone repository. Addition to the packages introduced above, we should also be sure that
git is installed. Since aim is to automate trivial stuff as much as possible, we will have a small script that will be used in getting the signature of repository and write it to
.ssh/known_hosts file. Modifications in
Vagrantfile will be as;
In order to connect repository without password, ssh keys may be used in login. The key pair of host may be used. Then we either may copy the private key during provision, or a more elegant solution we may use ssh agent to forward these keys.
Shh-agent may be used in transit ssh cases, so that we are able to eliminate unnecessary in the middle keys.
ssh-agent by itself starts agent and displays port and process id but not make these available by default. In Linux, this may be handled by
eval $(ssh-agent -s) / eval 'ssh-agent -s' / exec ssh-agent bash. Then private keys should be added using
ssh-add. Result may be seen through
ssh-add -L. Host forwarding should also be enabled in
~/.shh/config through addition of
ForwardAgent yes line at corresponding host section. In windows, this will be a bit more tricky. First capture output to a temporary
.bat file, then modify this file to replace
set commands, followed by a little tidying as in the figure below,
This setting will do our job but be working only on the open terminal, and should be enhanced for global persistence operation. Anyway, in order to check agent,
In windows, there is one more problem. Although the upper modification will enable shh forwarding for
vagrant ssh which uses
OpenSsh in backstage, it will not do the desired job in
vagrant provision which uses
net-ssh in backstage. For provisioning, the keys should be converted and presented in the format that
net-ssh will understand. First convert the desired keys to putty style keys *.ppk with
puttygen executable described above. Then make
pageant present these keys through importing.
We see that we have desired development environment ready with source code to be worked on.