[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Support for source formats 3.0 (quilt) and 3.0 (native)


I have prepared a set of patches to add support of the new source formats
to dak. It's in the pu/deb-src-3.0 branch in my git repo:

I have succesfully uploaded packages using source formats 3.0 (quilt) and
3.0 (native). I also tested with a package using multiple .orig tarballs
and with other compressions schemes than gzip. If you want to create such
test packages, just do this with normal package:
$ mkdir -p debian/source
$ echo "3.0 (quilt)" >debian/source/format
$ debuild

I also tested the move of packages between two locations/component
(main->non-free). I tested uploads that required new processing
and some that did not.

The support of multiple .orig tarballs has consequences on the .dak format but
I added support to read both the old and the new format so that an upgraded dak
can still deal with old .dak lying around in various queues.

I added a configuration parameter to define the set of formats accepted.

Please review the patches and share your comments. Let me know if I missed
some updates.

It's not as big a change as I feared:
$ git diff --stat master..
 config/debian/dak.conf   |    5 ++
 dak/check_archive.py     |   27 ++++------
 dak/import_archive.py    |   16 +++---
 dak/process_accepted.py  |   58 ++++++++++++----------
 dak/process_unchecked.py |   59 ++++++++---------------
 dak/show_new.py          |    6 ++-
 daklib/queue.py          |  120 +++++++++++++++++++++++++++-------------------
 daklib/regexes.py        |    6 ++-
 daklib/utils.py          |  101 +++++++++++++++++++++++++++++++++++++-
 9 files changed, 252 insertions(+), 146 deletions(-)

That said, if you have refactoring going on in parallel, it would be nice
to have it finalized and merged in tho not-too-distant future to avoid problems
to keep it up-to-date.


$ git log --stat --reverse master..
commit bbb79e472bfb0985a1dda37ab98290eb80e203b2
Author: Raphael Hertzog <hertzog@debian.org>
Date:   Sat Feb 28 17:39:31 2009 +0100

    daklib/regexes.py: updates to support new source files
    The regex re_issource is modified to recognize the new compression scheme
    supported by the 3.0 source package formats. Also add the
    the debian.tar file used by 3.0 (quilt) and 2.0 and the
    .orig-component.tar for additional upstream tarballs.
    Add a new regex re_is_orig_source to know if a file correspond
    to some upstream source tarballs (where the version should not have a
    debian revision).
    Add new regexes re_source_ext and re_orig_source_ext to match only the
    extension of source files. And reuse those in the definition of
    re_is_orig_source and re_issource to avoid duplication.
    Signed-off-by: Raphael Hertzog <hertzog@debian.org>

 daklib/regexes.py |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)

commit 6aa3d6016c935d35bb95b028b2c271253dbc809c
Author: Raphael Hertzog <hertzog@debian.org>
Date:   Sat Feb 28 19:26:52 2009 +0100

    dak process-unchecked: new parameter for accepted source formats
    Add a new parameter Dinstall::AllowedSourceFormats listing the set of
    source package formats accepted in the archive.
    check_dsc() in p-u enhanced to only allow source packages with the allowed
    formats. Also add lots of sanity checks ensuring that the right files are
    there for each format. Put those sanity checks into utils so that they can
    be reused by check-archive too.
    Signed-off-by: Raphael Hertzog <hertzog@debian.org>

 config/debian/dak.conf   |    5 ++
 dak/process_unchecked.py |   28 +++++--------
 daklib/utils.py          |   97 +++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 111 insertions(+), 19 deletions(-)

commit 78f9a6c92746e975b369ef2e9ccd716803291ba1
Author: Raphael Hertzog <hertzog@debian.org>
Date:   Sat Feb 28 23:38:34 2009 +0100

    Fix utils.build_file_list() for 3.x source packages
    Let utils.build_file_list() know that we understand newer
    formats like 3.0 (quilt) or 3.0 (native).
    Signed-off-by: Raphael Hertzog <hertzog@debian.org>

 daklib/utils.py |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

commit 6a17e55e0e01d25b97fc87f3b4e1f8d32e423a64
Author: Raphael Hertzog <hertzog@debian.org>
Date:   Sat Feb 28 21:13:01 2009 +0100

    queue.Upload: add support for multiple orig tarballs
    Structure of Upload.pkg is adapted to handle multiple upstream tarballs.
    This also means that the .dak format is modified.
    Since the .dak format has been modified, improve the update_vars()
    funtion to be able to read both the new and the old format. When
    reading the old format, it auto-converts the data to the new structure.
    It uses python's type() introspection to know what format is used.
    This allows in-place upgrade even when old .dak are still stored
    in various queues.
    Signed-off-by: Raphael Hertzog <hertzog@debian.org>

 daklib/queue.py |   93 +++++++++++++++++++++++++++++++++++-------------------
 1 files changed, 60 insertions(+), 33 deletions(-)

commit e91e0581bb65d50e29f90b5051eb05f1c7a34d99
Author: Raphael Hertzog <hertzog@debian.org>
Date:   Sat Feb 28 22:17:25 2009 +0100

    Fix queue.get_type() to handle all source files extensions

 daklib/queue.py |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

commit 7ff0df024c3045a2dfa7a4f1f41a631fe300aeac
Author: Raphael Hertzog <hertzog@debian.org>
Date:   Sat Feb 28 21:41:33 2009 +0100

    Remove dead code in p-u and queue.Upload
    Upload.check_source_against_db() never returns a second parameter
    that is not None. The only code path that returned a value there
    was disabled with "and False". Thus remove all the dead code
    in queue.Upload and p-u.
    Signed-off-by: Raphael Hertzog <hertzog@debian.org>

 dak/process_unchecked.py |   14 --------------
 daklib/queue.py          |   30 +++++++++++-------------------
 2 files changed, 11 insertions(+), 33 deletions(-)

commit f148a535c9980dc00a779e6d945b8c50aca7744c
Author: Raphael Hertzog <hertzog@debian.org>
Date:   Sat Feb 28 21:47:38 2009 +0100

    dak process-unchecked: add support for multiple orig tarballs
    Signed-off-by: Raphael Hertzog <hertzog@debian.org>

 dak/process_unchecked.py |   17 ++++++++++-------
 1 files changed, 10 insertions(+), 7 deletions(-)

commit c6797133fe0c4c0c2195a369bf0bb6b8ccc90481
Author: Raphael Hertzog <hertzog@debian.org>
Date:   Sat Feb 28 22:50:01 2009 +0100

    dak process-accepted: add support for multiple orig tarballs
    Signed-off-by: Raphael Hertzog <hertzog@debian.org>

 dak/process_accepted.py |   58 +++++++++++++++++++++++++---------------------
 1 files changed, 31 insertions(+), 27 deletions(-)

commit 204c0bc329e978a2aef712f135d846fd6c959f41
Author: Raphael Hertzog <hertzog@debian.org>
Date:   Sat Feb 28 23:11:14 2009 +0100

    dak show-new: use new regex to cope with all source files extensions
    Signed-off-by: Raphael Hertzog <hertzog@debian.org>

 dak/show_new.py |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)

commit 48ece99c0b982486cb130fed3ca8f5977d0a730f
Author: Raphael Hertzog <hertzog@debian.org>
Date:   Sat Feb 28 23:12:40 2009 +0100

    dak import-archive: add support for multiple orig tarballs
    Also renamed the variable used for the cache for consistency.
    Signed-off-by: Raphael Hertzog <hertzog@debian.org>

 dak/import_archive.py |   16 ++++++++--------
 1 files changed, 8 insertions(+), 8 deletions(-)

commit 56156516f98c0447413ea654e66a60c8e97aa4df
Author: Raphael Hertzog <hertzog@debian.org>
Date:   Sat Feb 28 23:42:58 2009 +0100

    dak check-archive: replace check tar-gz-in-dsc
    The check tar-gz-in-dsc is replaced by a more meaningful
    test similar to what is done is process-unchecked. The new test
    is called files-in-dsc and verifies that the files listed
    in Files are correct for the format announced in the Format field.
    Signed-off-by: Raphael Hertzog <hertzog@debian.org>

 dak/check_archive.py |   27 +++++++++++----------------
 1 files changed, 11 insertions(+), 16 deletions(-)

Raphaël Hertzog

Le best-seller français mis à jour pour Debian Etch :

Reply to: