Recently I was working on a simple continuous deployment process for puppet code. The process steps are as follows:
Although the last step looks trivial, but actually, because of puppet’s communication module, it isn’t so easy to do. By default, puppet master can’t initiate a direct connection to managed agents, so in order to achieve that I decided to give MCollective a try.
Mcollective is a framework for building service orchestration on top of puppet regular infrastructure.
Without going into too much details, MCollective consists of 3 main components:
An easy way to start playing around with MCollective is by using a vagrant ready repository
[vagrant@middleware ~]$ git clone https://github.com/ripienaar/mcollective-vagrant.git
[vagrant@middleware ~]$ cd mcollective-vagrant
Edit the Vagrantfile (pay attention to the amount of instances which will be previsioned in addition to the middleware. By default it is 5, and it can be a little bit intensive on some laptops)
[vagrant@middleware ~]$ vi Vagrantfile
[vagrant@middleware ~]$ vagrant up
After all the VM’s are up (and it can take a while), connect to the middleware server (since by default the client is installed there), and check that all the nodes are configured correctly
[vagrant@middleware ~]$ mco ping
node0.example.net time=85.38 ms
middleware.example.net time=90.15 ms
To get all the available information on a specific node run:
[vagrant@middleware ~]$ mco inventory node0.example.net
Inventory for node0.example.net:
Agents:
discovery filemgr integration
nettest nrpe package
process puppet rpcutil
service urltest
Data Plugins:
agent fstat nettest
nrpe process puppet
resource service
Configuration Management Classes:
default mcollective
mcollective::agent::filemgr mcollective::agent::integration
mcollective::agent::nettest mcollective::agent::nrpe
mcollective::agent::package mcollective::agent::process
mcollective::agent::puppet mcollective::agent::service
mcollective::agent::urltest mcollective::config
mcollective::install mcollective::service
motd nagios
nrpe nrpe::params
puppet repos
roles::node settings
Facts:
architecture => x86_64
augeasversion => 0.9.0
caller_module_name => mcollective
clientcert => node0.example.net
clientversion => 2.7.17
The output here is actually very helpful:
Till now, we were addressing all the managed nodes or a specific server. However usually you would like to communicate with a subset of your environment. For that reason MCollective provides an easy way to define command filters.
For example, we can see a summary of all the node’s rubysitedir fact
[vagrant@middleware ~]$ mco facts rubysitedir
Report for fact: rubysitedir
/usr/lib/ruby/site_ruby/1.8 found 2 times
/usr/lib/ruby/site_ruby/1.9.3 found 1 times
Finished processing 3 / 3 hosts in 60.00 ms
… and then we can easily filter the nodes with ruby version 1.8
[vagrant@middleware ~]$ mco ping -F rubysitedir=/usr/lib/ruby/site_ruby/1.8
node0.example.net time=53.12 ms
middleware.example.net time=54.14 ms
… or we can query by applied classes (which nodes has nagios class applied to them?)
[vagrant@middleware ~]$ mco ping -C nagios
node0.example.net time=49.30 ms
middleware.example.net time=52.98 ms
… also we can combine the filters for a more complex query expression
[vagrant@middleware ~]$ mco ping -S "operatingsystem=CentOS and roles::node"
node0.example.net time=38.22 ms
Since MCollective is a framework, by performing the default installation you actually will get very limit set of commands. The vagrant repository we were working with till now, already installed some plugins for you, but remember that in production you will need to take care of the plugins installation in addition to the basic MCollective components configuration. You can check out the available community plugins here.
For the continuous deployment process I mentioned earlier, I needed the puppet plugin, which provides an easy way to enable, disable and run puppet daemons. For example, to rerun all puppet agents, you can just execute the following command
[vagrant@middleware ~]$ mco puppet runonce
MCollective isn’t so intuitive for a beginner, probably because of all the different components terminology and a over complicated documentation. I hope this post will ease a bit the first steps, and make this solid tool more available for a wider audience.
For advanced MCollective topics, you should review the following sources: