Recently I was working on a simple continuous deployment process for puppet code. The process steps are as follows:
Although the last step looks trivial, but actually, because of puppet’s communication module, it isn’t so easy to do. By default, puppet master can’t initiate a direct connection to managed agents, so in order to achieve that I decided to give MCollective a try.
Mcollective is a framework for building service orchestration on top of puppet regular infrastructure.
Without going into too much details, MCollective consists of 3 main components:
An easy way to start playing around with MCollective is by using a vagrant ready repository
[vagrant@middleware ~]$ git clone https://github.com/ripienaar/mcollective-vagrant.git [vagrant@middleware ~]$ cd mcollective-vagrant
Edit the Vagrantfile (pay attention to the amount of instances which will be previsioned in addition to the middleware. By default it is 5, and it can be a little bit intensive on some laptops)
[vagrant@middleware ~]$ vi Vagrantfile [vagrant@middleware ~]$ vagrant up
After all the VM’s are up (and it can take a while), connect to the middleware server (since by default the client is installed there), and check that all the nodes are configured correctly
[vagrant@middleware ~]$ mco ping node0.example.net time=85.38 ms middleware.example.net time=90.15 ms
To get all the available information on a specific node run:
[vagrant@middleware ~]$ mco inventory node0.example.net Inventory for node0.example.net: Agents: discovery filemgr integration nettest nrpe package process puppet rpcutil service urltest Data Plugins: agent fstat nettest nrpe process puppet resource service Configuration Management Classes: default mcollective mcollective::agent::filemgr mcollective::agent::integration mcollective::agent::nettest mcollective::agent::nrpe mcollective::agent::package mcollective::agent::process mcollective::agent::puppet mcollective::agent::service mcollective::agent::urltest mcollective::config mcollective::install mcollective::service motd nagios nrpe nrpe::params puppet repos roles::node settings Facts: architecture => x86_64 augeasversion => 0.9.0 caller_module_name => mcollective clientcert => node0.example.net clientversion => 2.7.17
The output here is actually very helpful:
Till now, we were addressing all the managed nodes or a specific server. However usually you would like to communicate with a subset of your environment. For that reason MCollective provides an easy way to define command filters.
For example, we can see a summary of all the node’s rubysitedir fact
[vagrant@middleware ~]$ mco facts rubysitedir Report for fact: rubysitedir /usr/lib/ruby/site_ruby/1.8 found 2 times /usr/lib/ruby/site_ruby/1.9.3 found 1 times Finished processing 3 / 3 hosts in 60.00 ms
… and then we can easily filter the nodes with ruby version 1.8
[vagrant@middleware ~]$ mco ping -F rubysitedir=/usr/lib/ruby/site_ruby/1.8 node0.example.net time=53.12 ms middleware.example.net time=54.14 ms
… or we can query by applied classes (which nodes has nagios class applied to them?)
[vagrant@middleware ~]$ mco ping -C nagios node0.example.net time=49.30 ms middleware.example.net time=52.98 ms
… also we can combine the filters for a more complex query expression
[vagrant@middleware ~]$ mco ping -S "operatingsystem=CentOS and roles::node" node0.example.net time=38.22 ms
Since MCollective is a framework, by performing the default installation you actually will get very limit set of commands. The vagrant repository we were working with till now, already installed some plugins for you, but remember that in production you will need to take care of the plugins installation in addition to the basic MCollective components configuration. You can check out the available community plugins here.
For the continuous deployment process I mentioned earlier, I needed the puppet plugin, which provides an easy way to enable, disable and run puppet daemons. For example, to rerun all puppet agents, you can just execute the following command
[vagrant@middleware ~]$ mco puppet runonce
MCollective isn’t so intuitive for a beginner, probably because of all the different components terminology and a over complicated documentation. I hope this post will ease a bit the first steps, and make this solid tool more available for a wider audience.
For advanced MCollective topics, you should review the following sources: