Tags

, ,

As we have a basic build infrastructure with Git, Jenkins, FitNesse, CppCheck and Valgrind, now it is time to turn attention to leash development system itself. Not all developers like to prepare and manage their build systems. However, it is also a necessity to have a common, portable and consistent development environment. In fact, preparing and handling this system is a non value added essential part in Cost of Software Quality harnessing. It is better to let people that is expert and willing to handle system dependencies prepare the system and do the work once, then be sure that everybody else use this prescribed recipe. This abstraction may be handled with vagrant & puppet. Vagrant (https://www.vagrantup.com/), provides a common development environment using virtual machines, regardless of the operating system behind. Hypervizor may be VmWare, KVM, Zen but default is VirtualBox.

vagrant setup - vagrant logo

To start with, install Vagrant (https://www.vagrantup.com/downloads.html) and VirtualBox (https://www.virtualbox.org/wiki/Downloads). Create an initial configuration file with selecting a base virtual machine. Some templates may be found at https://atlas.hashicorp.com/boxes/search. Here, I will select a basic debian environment with puppet.

vagrant setup - selecting a base system

Initialize vagrant with selected template, vagrant init puppetlabs/debian-7.8-64-puppet

vagrant setup - creating a vagrant file

This will create a minimum VagrantFile which will specify machine definitions. Then perform a vagrant up. Since this is a brand new up, vagrant will search for template and download it, which may take some time.

vagrant setup - initial vagrant up

At the end of verbose, there is a couple interesting information. One is about ssh connection, which states that private key ssh session available through 127.0.0.1:2222. Normally vagrant machines may be accessed through vagrant/vagrant username password pair. In this specific puppet template, username private key pair is preferred. The other information is about a shared folder between host and guest systems, which will be handy. The shared folder D:\capturer\vagrant\debian in host will be mounted as /vagrant in guest.

vagrant setup - initial vagrant up end

We may try ssh connection through vagrant ssh.

vagrant setup - vagrant ssh

In my case, vagrant complains about finding ssh executable, which may easily be fixed through setting any path (I preferred using git path, and it is better to add this to system path C:\Program Files (x86)\Git\bin for persistence purpose). The verbose also gives location of private key, that may be used through other ssh clients. For example, we may use this key at putty. Putty does not accept the somewhat standard format and needs slight modification. First convert the original key with puttygen as;

vagrant setup - using puttygen to convert ssh key

And, save as it as a private key (vagrant_debian.ppk in this specific example). Then set this key as authorization mechanism in putty ssh session setup.

vagrant setup - using putty to ssh

Now, ssh through putty should also be successful;

vagrant setup - putty ssh result

As basics are over, it is time to make customization that will provide real benefit of vagrant. First add a local interface, define a hostname, and set virtual machine memory to 1024 MB. These may be set through Vagrantfile as;

vagrant setup - changing hostname and ip

Provisioning is only done at initial setup to decrease boot up time. For this reason, remember to re provision so that changes of Vagrantfile becomes effective. This may be achieved by vagrant provision fur running machine, vagrant reload --provision for running machine after a reload and finally vagrant up --provision for a halted previously provisioned system. Result of configutaion change may be checked from console output;

vagrant setup - changing hostname and ip result

Provisioners represent the core of Vagrant. Usually shell provisioner may be used for simple tasks, like setting environment variables, followed by automation tools like puppet for complex jobs. Power of automation tools comes from their state handling schema, and they know what it was done before, to perform only brand new tasks. Therefore there becomes a good level of abstraction and you only define higher level policies, and underlying steps are performed by those provisioners.

Puppet is robust well established IT automation tool, that is capable of orchestrating tasks such as installing packages and configuring systems. Puppet apply (standalone version of Puppet) does not require any installations and may instantly be used in Vagrant. In Puppet, tasks are not executed in the order of appearance, therefore dependencies between tasks should be explicitly defined. Puppet scripts are called manifests and these may be grouped into modules.

For the Vagrantfile modification below, manifest folder with default.pp manifest file inside, will be assumed to be inside directory puppet at same level of Vagrantfile. Besides, any modules are expected to be in puppet/modules folder. The directory structure will be as;

vagrant setup - puppet configuration

Puppet manifests are collection of resources that perform tasks. Resources may be thought of system resource state, such as existing of a file, or ensuring presence of a package. For example, in the upper default manifest, we want to ensure that some system packages should be installed after an aptitude update. After re provisioning, state changes to corresponding packages may be seen below;

vagrant setup - package state changes

Now, let’s make necessary modification in order to clone repository. Addition to the packages introduced above, we should also be sure that git is installed. Since aim is to automate trivial stuff as much as possible, we will have a small script that will be used in getting the signature of repository and write it to .ssh/known_hosts file. Modifications in default.pp and Vagrantfile will be as;

vagrant setup - configuration modification for repository cloning

In order to connect repository without password, ssh keys may be used in login. The key pair of host may be used. Then we either may copy the private key during provision, or a more elegant solution we may use ssh agent to forward these keys.

Shh-agent may be used in transit ssh cases, so that we are able to eliminate unnecessary in the middle keys. ssh-agent by itself starts agent and displays port and process id but not make these available by default. In Linux, this may be handled by eval $(ssh-agent -s) / eval 'ssh-agent -s' / exec ssh-agent bash. Then private keys should be added using ssh-add. Result may be seen through ssh-add -L. Host forwarding should also be enabled in ~/.shh/config through addition of ForwardAgent yes line at corresponding host section. In windows, this will be a bit more tricky. First capture output to a temporary .bat file, then modify this file to replace export with set commands, followed by a little tidying as in the figure below,

vagrant setup - setting up ssh agent

This setting will do our job but be working only on the open terminal, and should be enhanced for global persistence operation. Anyway, in order to check agent,

vagrant setup - checking ssh agent

In windows, there is one more problem. Although the upper modification will enable shh forwarding for vagrant ssh which uses OpenSsh in backstage, it will not do the desired job in vagrant provision which uses net-ssh in backstage. For provisioning, the keys should be converted and presented in the format that net-ssh will understand. First convert the desired keys to putty style keys *.ppk with puttygen executable described above. Then make pageant present these keys through importing.

vagrant setup - setting up pageant

We see that we have desired development environment ready with source code to be worked on.

vagrant setup - building project

Advertisements