Re: Debian, updates and a working system: regression test
On Thu, Feb 24, 2005 at 04:30:04PM +0100, Bob Alexander wrote:
> Dear Friends,
> I have beem tinkering on and off with Debian at least since 1996 (Buzz,
> 1.1) :) My use has mainly been on my personal machines be it home system
> or laptops.
>
> My current incarnation is a Debian sid system on my beloved IBM T40
> Thinkpad. What a sweet assortment.
>
> I quite regularly update/upgrade with apt-listbugs giving some warnings
> from times to times.
>
> So far I never had some major outage striking me.
>
> But here comes the consideration/question.
>
> My system has slightly over 700 packages installed.
>
> Running a roughly weekly upgrade shows 10-20 packages that can be upgraded.
>
> This morning going from memory I remember some alsa stuff,
> kernel-package, xserver-xfree86, mondo, gimp ...
>
> Now despite what apt-listbugs says it could happen that one of these
> packages could break something on my system.
>
> Specific to the list I would imagine that I could run the following
> related battery of tests:
>
> alsa upgraded -> sound works, even after a reboot ?
> kernel-package -> see if my splendid make-kpkg shell works aok
> xserver -> is the desktop showing; even after a reboot ? (this one is
> obvious)
> mondo -> try a full backup and if a real man also a restore
> gimp -> obvious
>
> Sometimes things are less obvious. For instance an hotplug upgrade with
> a new developers config removed a blacklisted device
>
> If something in samba or cups changes I could suddendly find myself
> printing ok on network printers but not on the windows shared ones.
>
> So either I run a series of tests each time (hardly feasible) or shen I
> try something and it does not work I cannot be sure which upgrade of
> which component (as opposed to my changes to configs) broke the function.
>
> Is there some better way out of this maintenance problem ? Of course I
> know I could run a "stable" release system but that is not the point I
> want to make.
>
> Thank you for reading and I hope to get some interesting debate on this.
>
> Bob
>
This response is not nearly as interesting as you might hope, but the
people, like me, who run Sarge and update fairly frequently, _are_ the
regression test. In so far as you also update frequently, you _are_
also part of the regression test. I imagine that serious sysadmins of
shops running a lot of boxes have most of them running Woody, but a
few running Sarge. They do this both to help with the regression testing
and to be current with what 'stable' will be, once Sarge becomes stable.
Although some kind of automated regression testing would probably be
nice to have, there are many other things, like the newly improved
install process, that soak up the talent. If you have an idea as to how
to implement it _with_limited_resources_, go for it!
By the way, I'm not nearly as adventuresome you might suppose from my
claim to be doing testing. Regression tests are doing again, some basic
function which has always worked before. For me that means just doing
my thing. So far I haven't found much bad to report, which is just as
it should be in regression testing.
--
Paul E Condon
pecondon@mesanetworks.net
Reply to: