* Steve Kemp <skx@debian.org> [2008-06-13 04:40-0400]:
> On Fri Jun 13, 2008 at 12:06:50 +0100, Keith Edmunds wrote:
>
> > How do others approach the problem of security updates? Up until now,
> > we've done this manually with some help from 'cssh' for some servers;
> > however, that solution doesn't scale as the number of servers increases.
>
> > We're reluctant to have servers automatically install updates. We're
> > looking at CfEngine and Puppet, but I'd be interested in hearing of other
> > approaches.
I think puppet is the way to go, however it does still mean there will
be challenges in figuring out the best way to handle these things.
> I think you need to choose; either you have automatic updates or
> you do it manually, though there is a middle-ground where you could
> apply automatically to machines A, B, and C. Then after you observe
> no breakage for a period of time you could instruct machines D, E, F...,
> to update themselves too.
>
> I personally use cron-apt to auto-install security updates, at the
> (small) risk of suffering breakages if there is a borked security update.
> So far that hasn't been a problem, but I accept it is only a matter
> of time & bad luck until I get a borked upgrade requiring manual
> intervention on 200+ machines!
I thought about cron-apt to auto-install security updates, but I didn't
want to take the risk of suffering breakages. Specifically kernel
related reboots scare me, and there have been some issues with certain
packages that sometimes require specific things to be done after an
update.
I went with a compromise with puppet. Every system has scheduled apt-get
updates run on it, every system has apticron and apt-show-versions
installed on it. This results in me getting an email once a day to give
me an up-to-date list of packages that are currently pending an upgrade.
If the package is something I am comfortable with upgrading on all the
machines that have it installed, I go ahead and add it to my puppet
manifest as an upgrade_package definition. This definition is something
I created in puppet which will upgrade the package to the specified
version if it is installed, otherwise it wont (you can also specify
'latest' as the version). Puppet runs every 15 minutes or so on all the
systems, so they query this and determine if they should run this
upgrade, and do it if they need to[0].
For packages that I am less comfortable with blowing out there, I will
decide what to do with them, sometimes I can do that work in puppet (in
the case of the clamav upgrades, I could write specific things in puppet
that had to be done to manage the upgrade), or do them manually (I have
a few systems that I need to schedule outages for kernel security
upgrades, and they need to have fail-overs initiated before I reboot
them).
micah
0. I have the following puppet definition, which allows me to do the
following:
1. in site.pp:
node somesystem
include etch_security_upgrades
2. in etch_upgrades.pp:
class etch_security_upgrades {
upgrade_package { "perl":
version => 5.8.8-7etch1;
"syslog-ng":
version => latest;
"perl-modules":
}
}
3. Then in components/upgrades.pp:
define upgrade_package ($version = "") {
case $version {
'': {
exec { "aptitude -y install $name":
onlyif => [ "grep-status -F Status installed -a -P $name -q", "apt-show-versions -u $name | grep -q upgradeable" ],
}
}
'latest': {
exec { "aptitude -y install $name":
onlyif => [ "grep-status -F Status installed -a -P $name -q", "apt-show-versions -u $name | grep -q upgradeable" ],
}
}
default: {
exec { "aptitude -y install $name=$version":
onlyif => [ "grep-status -F Status installed -a -P $name -q", "apt-show-versions -u $name | grep -q upgradeable" ],
}
}
}
}
Attachment:
signature.asc
Description: Digital signature