I am incompetent and I can’t make Vagrant work.
At least, that’s the excuse I’ve been making for not joining in with the
rest of the floor in using that for Puppet (among other things) development. Instead, my dev-rig is a VM running as a
puppetmaster that’s tracking the changes I make to a given branch in our Git repo via the magic of post-commit hooks and
another VM in which I can run ‘puppet agent –debug –blah –server (first VM)’. Once in a while I remember to blow away
the second VM so I can make sure everything builds in the right order. However, even with snapshots it’s just slightly
too painful to happen regularly.
Meanwhile, quite a lot of the recent developments at Future have involved rigs of between eight and twelve boxes.
Generating a worthwhile test/dev version of one of those is rather tiresome because even if you’ve got the spare
horsepower lying about, you have to spend yea-long wiring it all together, sanitising the static and/or test data and
it all quickly gets completely oh god why did i even get out of bed i should have been a farmer like my dad mind you if
i had done that i’d have been out of bed at six to go feed the sheep on long barrow bank perhaps not after all…
So when a mildly broken Dell R620 arrived back from one of the DCs coincidentally with me wanting to have a play with
this Docker business, it all seemed a bit convenient.
I am incompetent and I can’t make Docker work. On Debian.
LXC, on the other hand, was slightly simpler than falling off a log.
Given that it’s simple to build a puppetmaster that’s the same as one of the live ones, and that all the machine config
I currently care about is in a manifest, it should be pretty easy to generate a container and have it puppet itself up
This indeed turned out to be the case. However, having to hand-allocate IP addresses and fiddle about with container
naming such that they picked up the configs in use on the live rig was all a bit too hands-on and really not what I
DNSMasq fixed the first problem. It is a surprisingly useful tool.
A rakefile which read a list of made-up machine names, generated softlinks to the actual hiera node configs and then
instantiated the relevant containers fixed the second.
I also spent quite some time building a Wheezy ‘image’ that minimised apt-get as much as possible.
Result - fully puppeted containers come up in circa a minute. Somewhat longer if you have to install PHP.
If I didn’t have quite such a rational hatred of golden images and all who sail in them, it would likely be faster
The next part is a bit fiddly.
The example problem I now have is that some parts of my collection of yea-many VMs want to connect to other parts. For
instance if I have a redis slave, I need to know what the master’s IP address might be during the puppet run. At Future,
we generate a location fact and use that in our hiera, er, hierarchy to configure things like message brokers,
smarthosts and DNS ordering. I could just add yet another location - testbox, or something - allocate a block of IP
space and then add some extra indirection. And then I could do that again for each person and/or project that wanted to
run up a test-rig. At which point one has just run into a behaviour pattern that should probably be named ‘It’s OK, I
can fix that for you.’
I first came across this in, er, 1991 when doing some NHS-related coding. One of the other chaps had written a thing
which had to deal with, oh I don’t know, ten items or something. Because he was a forward-thinking sort, he allocated
sixteen slots in his array and beetled off all smug for a coffee and a corned beef sandwich. As you might expect, a few
months later one site or other had a list of seventeen items and a bug report. ‘It’s OK, I can fix that for you!’ went
our chap and expanded the array to the clearly ludicrous value of some twenty-three slots…
There’s scope for an Eric Berne knockoff book of tiresome technical behaviour antipatterns, isn’t there?
Anyway, I’m using DHCP, and I wanted the entire edifice to work with little or no extra typing.
CoreOS’s ETCD looked like a good fit. Emit salient facts to etcd database when bringing up (say) redis master, then
query same via Garethr’s hiera-etcd when bringing up the slave. Profit!
That bit did take a little tinkering to get right.
It seems to me that the notion of a reactive puppet configuration is really rather interesting. Other people may well be
screaming in terror and jabbering about things like ‘deadly embrace’ and ‘terrible feedback loops are fine for the Jesus
and Mary Chain (Or A Place to Bury Strangers if that’s too retro for you) but have no place in a theoretically stable
configuration.’ However, just as a top-down decision process enforced by rigid hierarchy is a hateful idea for a
workplace environment, so it is for a machine environment.
TL;DR - code in Github, patches welcome.