[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Debbugs: The Next Generation



On Wed, Aug 08, 2001 at 01:31:48AM -0400, Matt Zimmerman wrote:
> > Personally, eg, I find C++ a revolting language, and not one that's
> > particularly well suited to debbugs (where speed isn't particularly critical,
> > and the slowness is due to algorithmic inefficiencies (like reindexing
> > regularly rather than updating them as the database changes; and like having
> > large numbers of files in a signle ext2 directory). Switching from an
> > arch-independent language, to one whose ABI changes incompatibly quite
> > regularly is also something I wouldn't do.
> I chose C++ because it's relatively easy to use C and C++ code from interpreted
> languages, but not to share code written in, say, Perl, with other languages.

<SorePoint>It's perfectly easy to share code written in arbitrary
languages if you just write a simple utility and put it in /usr/bin,
or similar...</SorePoint>

> If you look at the little command-line programs in cmdline/ (see below), they
> could (apart from the exception handling and ostream formatting magic) be
> translated almost line-for-line into Perl or Python once the necessary wrappers
> were generated.  That is what I hope to do with SWIG.

If you're going to write the exact same thing, the perl's the same;
if you're going to write something subtly different (like, say, making
pkgreport implement all the features of the pkgreport.cgi, such as by
submitter and by maintainer reports) you have to start digging around
in the C++ libraries.

In particular, efficiently getting a list of bugs by some particular
selection mechanism "all RC bugs", "all bugs related to source package
<foo>" or whatever, pretty much requires you to hack around the C++
or SQL stuff.

You're currently missing:
	* HTML generation
	* bug submission and pseudo-header parsing
	* "receive"

You're also missing any way to do really quick hacks about working out
bug stuff. The CGI scripts have a /var/lib/debbugs/index.db which I find
pretty handy, eg.

> I started thinking about it back in January, but the first code was written on
> 30 July.  It's amazing how quickly these things get done when you're
> unemployed.

Your "thanks" handling seems wrong too.

> > Personally, I'd be inclined towards changing the .log format to be something
> > akin to batched-SMTP and stored in the filesystem (so that it can be
> > "replayed" if the database crashes; and so that the way the bugs logs are
> > displayed can be changed in future).
> Jason suggested something similar, due to a bad experience with postgres.
> Personally, I think regular database dumps and backups would avoid most of
> these worries.

Doing regular database dumps seems likely to be pretty expensive. The
debbugs archive is currently 2GB, while the active database is just under
1GB. By contrast, the twice-daily backups of the package pool database
on ftp-master is 16MB of SQL. debbugs gets a lot more activity than the
pool database too, so probably has a lot more chances to trip over any
bugs in postgresql.

> Looking over them, all of the bug operations (except for
> submission, which can be worked around) should be idempotent, so if the system
> kept a log of commands executed, that log could be used as a replay script as
> well.

Erm. None of those operations are idempotent in that they change what's
in the bug log.

> What do we do if the current debbugs data gets corrupted (if we notice)?

It doesn't get corrupted.

Well, not by the underlying storage technology, anyway. Corruption
due to bugs in debbugs itself is detected by verifying the db
independently. Mostly, it just means we need to reprocess a messsage
in /var/lib/debbugs/spool/incoming; otherwise we just expect people to
refile the bug.

But in the context of Jason's message, it just doesn't get corrupted.

On Tue, Aug 07, 2001 at 03:11:54PM -0500, Adam Heath wrote:
> On Wed, 8 Aug 2001, Matt Zimmerman wrote:
> > On Wed, Aug 08, 2001 at 02:07:10PM +1000, Anthony Towns wrote:
> > > Personally, eg, I find C++ a revolting language, and not one that's
> > > particularly well suited to debbugs [...]
> Those who work on debbugs want an easily maintainable system.  C/C++ do not
> come to mind as being in that category.

Nor does the perl debbugs is currently written in, though.

> The proper way to do this, is to have a low-level(read c/c++) library that
> does the absolute bare minimum to store the data in some kind of back end.
> Then, the logic can be implemented in whatever language is desired.

And to get the data out of that backend efficiently. "Unfortunately",
you have to know basically everything about your original query in order
to optimise that at all well. You can't write a "backend interface" and
assume you'll be able to treat it independtly of the frontend for debbugs.

> > > But if you or someone else who actually likes C++ is going to maintain the
> > > BTS and make sure it works on a day-to-day basis, that could be okay.
> Could this be read as you(aj) not wanting to deal with a C++ version of
> debbugs?  If so, I agree.

I don't want to deal with C++ at all. I'd be quite happy if someone
else was handling debbugs to my satisfaction and I didn't have to have
anything to do with it, though.

Cheers,
aj

-- 
Anthony Towns <aj@humbug.org.au> <http://azure.humbug.org.au/~aj/>
I don't speak for anyone save myself. GPG signed mail preferred.

``_Any_ increase in interface difficulty, in exchange for a benefit you
  do not understand, cannot perceive, or don't care about, is too much.''
                      -- John S. Novak, III (The Humblest Man on the Net)

Attachment: pgpAzAEtYJBTc.pgp
Description: PGP signature


Reply to: