Re: Finding a Debian consultancy for help with large-scale platform upgrade (Was Re: Systems upgrading)

[Dan Ritter <dsr@randomstring.org>]

Andy Smith wrote: 
> Just as some free advice though…
> 
> 1. I find it hard to believe you have more than 2000 Debian installs
>    without some sort of existing automation / configuration
>    management
> 
> 2. Given (1), I would approach the task by learning your config
>    management and modifying it to deploy a Debian 12 version of each
>    kind of Debian 11 server you already have.
> 
> 3. I'd then do a rolling deploy that slowly takes Debian 11 servers
>    out of service and re-provisions them as Debian 12. I would not
>    try to upgrade anything in place. Although Debian supports that,
>    at scale I find it harder to account for all variables than with
>    a clean install, and if you already have automation to deploy and
>    configure a host then an in-lace upgrade also takes longer in my
>    experience.

We do hundreds rather than thousands, but we do them:

- with an existing configuration automation system (chef/cinc)

- in-place upgrades

- in tiers, where a given function (e.g. web servers) will have
  representative machines in each tier, starting with a very
  small proof-of-concept upgrade, followed by corrections; then
  a somewhat larger upgrade group, followed by all the rest of
  the machines.

Automated monitoring, too.

-dsr-