I have been hacking on stomp-git over the past couple of days. Mostly because there may or may not be a lurking nasty with the way it sometimes stops listening on its, er listen topic, but also because it has needed a deal of de-Futuring and general fettling so it doesn’t make proper coders claw their own faces off in horror.
And because I had forgotten how I’d made it work when it came to the svn->git rollout of one of our major sites.
Thus it seems as good a time as any to explain how I think it should work and indeed why it works like that.
The elevator pitch (of sorts) is that if you’ve got a message-bus that connects all your servers together…
Actually there wasn’t an elevator pitch. I think it was a combination of wondering about how one might make deploys faster (and thus less painful), implementing Hunter Haugen’s dynamic git-branching for puppet and making the AMQ rig more reliable because we’d sold one of the dev teams on the idea of using it for inter-server comms.
What the stomp-git daemon does is subscribe to a topic, in our case named ‘future.git.commits’ and if it spots a commit message for a repository it cares about, issues a ‘git fetch’ command in order to update its local copy of that repo. Thus when it comes time to deploy a site, the relevant tag or branch is already lurking in the git tree and the actual ‘git checkout epic-bugfix-4.77’ happens tolerably quickly.
This does involve a selection of scripts and daemons flying in close formation, so I shall attempt to describe the flow of events.
An assumption I made is that there’s something like gitolite where most of your repositories live and it’s been configured to emit post-commit messages.
If you’re using GitHub, they have post-commit hooks that’ll talk to all sorts of things, and their example Sinatra code is trivially hackable. I’ll cover that in a later post.
A commit message looks a lot like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | |
… And if that looks an awful lot like the output from the example post-commit mail, that’s because I cribbed it wholesale.
The only section that the stomp-git daemon is looking for is the ‘repo: spog’ line. The rest is syntactic sugar on the off-chance that someone wants to subscribe to that topic and (say) pass the output to an IRC channel.
Here’s the YAML config for the stomp-git daemon:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 | |
… Where ‘rserver’ is a connection URL you can feed to the Ruby stomp library, listen- and report- topics should be moderately self-evident and the interesting bit is at the bottom - the repository names and locations on the target machine. So that turns into ‘cd repodir && git fetch’. Sorted.
Puppetmaster specials?
The dynamic git-branch stuff above does its stuff with ssh pubkeys and a post-commit shellscript. This works fine for our one puppetmaster that’s on the same VLAN as our gitolite box. However, our other puppetmaster is on the inside of the corporate firewall where inbound connections are verboten. Like don’t even ask because a chorus of derision from the rest of the ops team can sometimes offend the sorts of people who become wedded to single ways of doing things.
However, the AMQ rig is Allowed, so the stomp-git daemon has a puppetmaster mode. (Which would probably be better being split off into a different program, but I wrote it in a temper and and…)
In puppetmaster mode, the daemon will also checkout the update it has just fetched. You really don’t want want this sort of behaviour under most circumstances.
Referencing the above config, ‘puppet-environments’ is the location of the cloned copy of your puppetmaster tree and ‘environments’ is the checked-out version that features in the linked description of dynamic branching even further above.