Re: Debbugs: The Next Generation
On Wed, Aug 08, 2001 at 02:07:10PM +1000, Anthony Towns wrote:
> On Tue, Aug 07, 2001 at 11:11:37PM -0400, Matt Zimmerman wrote:
> > Me: "It's long past time that the BTS had a real database backend. Is
> > there any way I can help?"
>
> You do understand how a jump from "perl + known text backend + working and
> tested" to "C++ + new SQL backend + important parts yet unimplemented" by
> someone who hasn't done much with the BTS as it stands isn't going to inspire
> confidence straight off, right?
I'm not worried about inspiring confidence just yet. I even specifically
referred to it as a prototype, so that nobody would feel threatened and think
that I was trying to take over, but I guess some do anyway. I tried.
> Personally, eg, I find C++ a revolting language, and not one that's
> particularly well suited to debbugs (where speed isn't particularly critical,
> and the slowness is due to algorithmic inefficiencies (like reindexing
> regularly rather than updating them as the database changes; and like having
> large numbers of files in a signle ext2 directory). Switching from an
> arch-independent language, to one whose ABI changes incompatibly quite
> regularly is also something I wouldn't do.
I chose C++ because it's relatively easy to use C and C++ code from interpreted
languages, but not to share code written in, say, Perl, with other languages.
With the current debbugs, there seems to be a choice between using the Perl
stuff or parsing the files yourself. While munging the text files by hand
isn't that unappealing, futzing with the database by hand is, so I would prefer
to share that code.
If you look at the little command-line programs in cmdline/ (see below), they
could (apart from the exception handling and ostream formatting magic) be
translated almost line-for-line into Perl or Python once the necessary wrappers
were generated. That is what I hope to do with SWIG.
I won't argue the general advantages and disadvantages of various programming
languages, but for this particular project, C++ was a pretty natural fit, and
the initial implementation went smoothly. I consider that to be a good sign.
> But if you or someone else who actually likes C++ is going to maintain the
> BTS and make sure it works on a day-to-day basis, that could be okay.
At this point, it's an experiment, and if it doesn't work out, nobody needs to
worry about maintaining it. In the event that it were to be used in a
production environment, of course, I would absolutely step forward to maintain
it.
> Breaking the RC bug list scanners (ie, bugscan) and requiring them to be
> rewritten in some major way during the freeze would probably be bad.
I wouldn't even consider disabling the current system for a long time. If
things go well, the next logical step would be to copy @bugs email to both
systems and run them in parallel for a while, with the new one being used only
for testing and evaluation.
> I'm impressed if you've managed to duplicate all the behaviour of the mail
> interface (assuming you have only just been working on it recently, rather
> than the last seven months). Actually, even then, I'm impressed, it could
> easily take seven months just to understand wtf some of the perl in there's
> doing... That's definitely the first step in reimplementing it, though.
I started thinking about it back in January, but the first code was written on
30 July. It's amazing how quickly these things get done when you're
unemployed.
There is some hairy Perl in there, I agree. I've been doing the implementation
"from the outside in", without worrying too much about debbugs internals apart
from the file formats.
> Personally, I'd be inclined towards changing the .log format to be something
> akin to batched-SMTP and stored in the filesystem (so that it can be
> "replayed" if the database crashes; and so that the way the bugs logs are
> displayed can be changed in future).
Jason suggested something similar, due to a bad experience with postgres.
Personally, I think regular database dumps and backups would avoid most of
these worries. Looking over them, all of the bug operations (except for
submission, which can be worked around) should be idempotent, so if the system
kept a log of commands executed, that log could be used as a replay script as
well.
What do we do if the current debbugs data gets corrupted (if we notice)?
For the bug logs, I track bug history and message logs separately, and (where
appropriate) key history events to logged messages. This should make it easy
to present the data in lots of useful ways.
The code and schema are up at
http://alcor.ddts.net/cgi-bin/viewcvs.cgi/debian/debbugs-ng/ for whoever wants
to look at them. It's a bit rough, and there is absolutely no documentation
yet, but it has passed the basic functionality tests I've given it.
--
- mdz
Reply to: