Re: Utnubu Team Founded - Merging Ubuntu changes to Debian

[Goswin von Brederlow <brederlo@informatik.uni-tuebingen.de>]

Petter Reinholdtsen <pere@hungry.com> writes:

> [Goswin von Brederlow]
>> If anyone is intrested in adapting this drop me a note.
>
> Please provide URL and repository for this source?

Unfortunately it isn't anywhere public. Maybe we should put all the
amd64.debian.net scriptlets and conffiles into a public
cvs/svn/arch/whatever.

The script rsyncs the Packages/Sources files from REMOTE_BASE and
copies the local (amd64) ones from LOCAL_BASE and then generates the
differences between the two in STATSDIR.

*.current-*.txt are files that match name AND version
*.missing-*.txt are files in REMOTE but not LOCAL
*.removed-*.txt are files in LOCAL but no REMOTE

Note that a version change shows up in both missing and removed.

I plan to add black/white listing for non-free packages. You probably
want the same to filter out all known Debian packages Ubuntu doesn't
have.

Anyway, here it comes:
----------------------------------------------------------------------
#!/bin/sh
#
# report differences in sources and arch:all packages
#
# Copyright (C) 2005 Goswin von Brederlow <brederlo@informatik.uni-tuebingen.de>
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License as
# published by the Free Software Foundation; either version 2 of the
# License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

set -e

REMOTE_BASE=sinclair::debian
LOCAL_BASE=/org/amd64.debian.net/ftp/debian/

REMOTE_ARCH="i386" # dummy arch as we only need arch:all
LOCAL_ARCH="amd64" # dummy arch as we only need arch:all
DISTS="testing testing-proposed-updates unstable experimental"
SECTIONS="main contrib"

STATSDIR="`pwd`/stats"

TMPDIR="/tmp/report-source.$$"
TMPDIR="`pwd`/work2"

RSYNC="rsync -q"

mkdir -p "$STATSDIR"
mkdir -p "$TMPDIR"
cd "$TMPDIR"

for DIST in $DISTS; do
  echo $DIST: fetching meta files
  case "$DIST" in
    experimental) DIST_DIR="../project/experimental"; HAVE_DI="false";;
    *) DIST_DIR="$DIST"; HAVE_DI="true";;
  esac
  rm -f $DIST.Packages.new $DIST.Sources.new $DIST.Packages.old $DIST.Sources.old
  for SECTION in $SECTIONS; do
    $RSYNC "$REMOTE_BASE/dists/$DIST_DIR/$SECTION/binary-$REMOTE_ARCH/Packages.gz" "$TMPDIR/$DIST.$SECTION.Packages.gz"
    gunzip < "$TMPDIR/$DIST.$SECTION.Packages.gz" >> $DIST.Packages.new || exit 1
    if [ "$HAVE_DI" = "true" -a "$SECTION" = "main" ]; then
      $RSYNC "$REMOTE_BASE/dists/$DIST_DIR/$SECTION/debian-installer/binary-$REMOTE_ARCH/Packages.gz" "$TMPDIR/$DIST.$SECTION.di.Packages.gz"
      gunzip < "$TMPDIR/$DIST.$SECTION.di.Packages.gz" >> $DIST.Packages.new || exit 1
    fi
    $RSYNC "$REMOTE_BASE/dists/$DIST_DIR/$SECTION/source/Sources.gz" "$TMPDIR/$DIST.$SECTION.Sources.gz"
    gunzip < "$TMPDIR/$DIST.$SECTION.Sources.gz" >> $DIST.Sources.new || exit 1

    gunzip < "$LOCAL_BASE/dists/$DIST_DIR/$SECTION/binary-$LOCAL_ARCH/Packages.gz" >> $DIST.Packages.old || exit 1
    if [ "$HAVE_DI" = "true" -a "$SECTION" = "main" ]; then
      gunzip < "$LOCAL_BASE/dists/$DIST_DIR/$SECTION/debian-installer/binary-$LOCAL_ARCH/Packages.gz" >> $DIST.Packages.old || exit 1
    fi
    gunzip < "$LOCAL_BASE/dists/$DIST_DIR/$SECTION/source/Sources.gz" >> $DIST.Sources.old || exit 1
  done

  echo $DIST: diff sources
  grep-dctrl "" -s Package,Version -n < $DIST.Sources.new | paste -sd "  \n" > $DIST.src-in.new
  grep-dctrl "" -s Package,Version -n < $DIST.Sources.old | paste -sd "  \n" > $DIST.src-in.old
  cat $DIST.src-in.new $DIST.src-in.old $DIST.src-in.old | sort | uniq -c > $DIST.src-out.count

  grep "^      1" $DIST.src-out.count | cut -b9- | grep -v "^$" > "$STATSDIR/$DIST.missing-sources.txt"
  grep "^      2" $DIST.src-out.count | cut -b9- | grep -v "^$" > "$STATSDIR/$DIST.removed-sources.txt"
  grep "^      3" $DIST.src-out.count | cut -b9- | grep -v "^$" > "$STATSDIR/$DIST.current-sources.txt"

  echo $DIST: diff packages
  grep-dctrl -XF Architecture all -s Package,Version -n < $DIST.Packages.new | paste -sd "  \n" | sort -u > $DIST.pkg-in.new
  grep-dctrl -XF Architecture all -s Package,Version -n < $DIST.Packages.old | paste -sd "  \n" | sort -u > $DIST.pkg-in.old
  cat $DIST.pkg-in.new $DIST.pkg-in.old $DIST.pkg-in.old | sort | uniq -c > $DIST.pkg-out.count

  grep "^      1" $DIST.pkg-out.count | cut -b9- | grep -v "^$" > "$STATSDIR/$DIST.missing-packages.txt"
  grep "^      2" $DIST.pkg-out.count | cut -b9- | grep -v "^$" > "$STATSDIR/$DIST.removed-packages.txt"
  grep "^      3" $DIST.pkg-out.count | cut -b9- | grep -v "^$" > "$STATSDIR/$DIST.current-packages.txt"
done || exit 1

cd
#rm -rf "$TMPDIR"