Redesigning the autopkgtest controller/workers CI system for Ubuntu and Debian

[mpitt@debian.org (Martin Pitt)]

Hey Andy,

Andy Doan [2014-03-13 18:31 -0500]:
> I'm a little confused but the rabbit messages do include some
> attributes that might help with this:

The rabbit queues are fine. The problem is with storing the results in
swift. We have multiple writers and want to create a directory
structure which can be traversed reasonably effectively, i. e. we need
sortable file names. Either by having a counter at the end similar to
jenkins (trusty/amd64/foo/37) or by having a timestamp
(trusty/amd64/foo/20140314-150135) with extra .<number> suffixes on
collisions. But I see no documented way of doing this kind of "lock
this file name", "get me an increased global counter", or "start a
file system transaction" on swift, which makes swift fairly hard to
use for concurrent writes.

How do other systems deal with that? I mean, this is *the* fundamental
problem with multiple writers to a file system, ther must be some
solution?

Thanks,

Martin

-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)