Re: packages becoming virtual during jessie->stretch upgrades
Hi Andreas,
> if anybody has some spare time, I would be interested in packages that
> are real packages in jessie but virtual packages in stretch, and the
> packages in stretch that depend (recommend) on the virtual package.
There appears to be 121 packages that were real packages in jessie and are
now purely virtual packages in stretch. The list of newly virtual packages
is attached as 'real-to-virtual.txt' is in the form "virtual-name (list, of,
providers)".
Many of them have some level of dependency relationship declared on them.
Filtering this list of newly virtual packages and looking for
reverse-{depends,recommends,suggests} in stretch generates the attached list
in deb822 format 'newly-virtual-deps.txt'. There are 51 stanzas from the
Provides of 20 real packages.
Some crude python to generate these data attached for general amusement but
also to perhaps help you change the output format etc if desired. (Feel free
to ask for further filtering or different output if that would help.)
Would it be useful to have a regular cron job generating this sort of
analysis for stable→testing? Does piuparts already spot these problems or
does it need to be given suitable pairs of packages to provoke them?
(I've only looked at the main archive here; contrib and non-free could be
easily concatenated to the Packages file if desired)
cheers
Stuart
--
Stuart Prescott http://www.nanonanonano.net/ stuart@nanonanonano.net
Debian Developer http://www.debian.org/ stuart@debian.org
GPG fingerprint 90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7acpidump (acpica-tools)
android-system-dev (android-platform-system-core-headers)
apache2-suexec (apache2-suexec-custom, apache2-suexec-pristine)
camlp4-extra (camlp4)
ceve (dose-extra)
citadel-mta (citadel-server)
doctrine (php-doctrine-orm)
ftplib-dev (libftp-dev)
gaim-librvp (pidgin-librvp)
ghc-dynamic (ghc)
ghc-haddock (ghc)
gir1.2-gpaste-2.0 (gir1.2-gpaste-6.0)
gmt-gshhs-full (gmt-gshhg-full)
gmt-gshhs-high (gmt-gshhg-high)
gmt-gshhs-low (gmt-gshhg-low)
golang-bindata-dev (golang-github-jteeuwen-go-bindata-dev)
golang-blackfriday-dev (golang-github-russross-blackfriday-dev)
golang-go-md2man-dev (golang-github-cpuguy83-go-md2man-dev)
golang-go-systemd-dev (golang-github-coreos-go-systemd-dev)
golang-testify-dev (golang-github-stretchr-testify-dev)
guacamole-tomcat (guacamole)
iasl (acpica-tools)
icebox (zeroc-icebox)
iceweasel-downthemall (xul-ext-downthemall)
kde-style-qtcurve (kde-style-qtcurve-qt4)
kradio (kradio4)
latex-beamer (texlive-latex-recommended)
latex-xcolor (texlive-latex-recommended)
lib32readline6-dev (lib32readline-dev)
libdcmtk2-dev (libdcmtk-dev)
libdigest-perl (perl-modules-5.24)
libestools2.1-dev (libestools-dev)
libextutils-cbuilder-perl (perl-modules-5.24)
libextutils-parsexs-perl (perl-modules-5.24)
libgd2-noxpm-dev (libgd-dev)
libgd2-xpm-dev (libgd-dev)
libgeos-ruby1.8 (ruby-geos)
libghc-binary-dev (ghc)
libghc-binary-doc (ghc-doc)
libghc-binary-prof (ghc-prof)
libghc-haskeline-dev (ghc)
libghc-haskeline-doc (ghc-doc)
libghc-haskeline-prof (ghc-prof)
libghc-terminfo-dev (ghc)
libghc-terminfo-doc (ghc-doc)
libghc-terminfo-prof (ghc-prof)
libghc-transformers-dev (ghc)
libghc-transformers-doc (ghc-doc)
libghc-transformers-prof (ghc-prof)
libghc-xhtml-dev (ghc)
libghc-xhtml-doc (ghc-doc)
libghc-xhtml-prof (ghc-prof)
libgpaste2 (libgpaste6)
libgpaste2-dev (libgpaste6-dev)
libgsl0-dbg (libgsl-dbg)
libgsl0-dev (libgsl-dev)
libio-compress-perl (libperl5.24)
libjortho-freeplane-java (freeplane)
liblog-any-adapter-perl (liblog-any-perl)
libmagickcore-6.q16-2-extra (libmagickcore-6.q16-3-extra)
libmapscript-ruby (ruby-mapscript)
libmapscript-ruby1.8 (ruby-mapscript)
libmapscript-ruby1.9.1 (ruby-mapscript)
libmariadb-client-lgpl-dev (libmariadb-dev)
libmariadb-client-lgpl-dev-compat (libmariadb-dev-compat)
libmath-complex-perl (perl-modules-5.24)
libmathcomp-coq (libssreflect-coq)
libobexftp-ruby (ruby-obexftp)
libosmium-dev (libosmium2-dev)
libparse-cpan-meta-perl (perl-modules-5.24)
libpg-java (libpostgresql-jdbc-java)
libphobos-dev (libgphobos-dev)
libpolybori-dev (libbrial-dev)
libpolybori-groebner-dev (libbrial-groebner-dev)
libqgis-customwidgets2.4.0 (libqgis-customwidgets)
libqt4-ruby1.8 (ruby-qt4)
libquazip-qt5-1 (libquazip5-1)
libreadline6-dev (libreadline-dev)
libsdformat-dev (libsdformat4-dev)
libsys-syslog-perl (libperl5.24)
libtachyon-dev (libtachyon-mpi-dev, libtachyon-mpich-0-dev, libtachyon-mt-0-dev, libtachyon-openmpi-0-dev, libtachyon-serial-0-dev)
libtcltk-ruby (ruby-tcltk)
libtest-harness-perl (perl-modules-5.24)
libtest-tester-perl (libtest-simple-perl, perl-modules-5.24)
libtest-use-ok-perl (libtest-simple-perl, perl-modules-5.24)
libtxc-dxtn-s2tc0 (libtxc-dxtn-s2tc)
luatex (texlive-binaries)
mediawiki-extensions-geshi (mediawiki)
mendexk (texlive-binaries)
motif-clients (mwm)
mr (myrepos)
mscore (musescore)
mscore-common (musescore-common)
ndiswrapper-common (ndiswrapper)
otf-stix (fonts-stix)
pdfjam (texlive-extra-utils)
perl-modules (perl-modules-5.24)
pgf (texlive-pictures)
php-calendar (php7.0-common)
plt-scheme (racket)
plt-scheme-doc (racket-doc)
python-icinga2 (icinga2-common)
python-imaging-tk (python-pil.imagetk)
python-ufc (python-ffc)
qgis-mapserver (qgis-server)
r-other-bio3d (r-cran-bio3d)
redshift-plasmoid (plasma-applet-redshift-control)
resource-agents-dev (resource-agents)
ruby-amq-client (ruby-amqp)
snd-gtk (snd-gtk-jack, snd-gtk-pulse)
snd-nox-alsa (snd-nox)
squid3-common (squid-common)
stun (stun-server, stuntman-server)
texlive-latex3 (texlive-latex-recommended)
texlive-math-extra (texlive-science)
thin1.8 (thin)
ttf-fifthhorseman-dkg-handwriting (fonts-dkg-handwriting)
ufc (python-ffc)
update-notifier (gnome-packagekit)
xserver-xorg-video-modesetting (xserver-xorg-core)
yorick-yeti-gsl (yorick-ygsl)
== Depends ==
Virtual-Package: apache2-suexec
Providers: apache2-suexec-custom, apache2-suexec-pristine
Reverse-Depends: ikiwiki-hosting-web
Virtual-Package: camlp4-extra
Providers: camlp4
Reverse-Depends: otags
Virtual-Package: citadel-mta
Providers: citadel-server
Reverse-Depends: citadel-suite
Virtual-Package: ghc-haddock
Providers: ghc
Reverse-Depends: haskell-platform, haskell-devscripts-minimal
Virtual-Package: golang-testify-dev
Providers: golang-github-stretchr-testify-dev
Reverse-Depends: golang-github-aws-aws-sdk-go-dev
Virtual-Package: latex-xcolor
Providers: texlive-latex-recommended
Reverse-Depends: auto-multiple-choice-common, doxygen-latex
Virtual-Package: libdigest-perl
Providers: perl-modules-5.24
Reverse-Depends: cil
Virtual-Package: libextutils-parsexs-perl
Providers: perl-modules-5.24
Reverse-Depends: libmodule-build-withxspp-perl, libextutils-typemap-perl, libmodule-install-xsutil-perl, libextutils-xspp-perl, libextutils-typemaps-default-perl
Virtual-Package: libgd2-noxpm-dev
Providers: libgd-dev
Reverse-Depends: libgdchart-gd2-xpm-dev, libgdchart-gd2-noxpm-dev
Virtual-Package: libghc-transformers-dev
Providers: ghc
Reverse-Depends: haskell-platform, propellor
Virtual-Package: libghc-transformers-doc
Providers: ghc-doc
Reverse-Depends: haskell-platform-doc
Virtual-Package: libghc-transformers-prof
Providers: ghc-prof
Reverse-Depends: haskell-platform-prof
Virtual-Package: libghc-xhtml-dev
Providers: ghc
Reverse-Depends: haskell-platform
Virtual-Package: libghc-xhtml-doc
Providers: ghc-doc
Reverse-Depends: haskell-platform-doc
Virtual-Package: libghc-xhtml-prof
Providers: ghc-prof
Reverse-Depends: haskell-platform-prof
Virtual-Package: libgsl0-dev
Providers: libgsl-dev
Reverse-Depends: liblorene-dev, pktools-dev, libhkl-dev
Virtual-Package: libio-compress-perl
Providers: libperl5.24
Reverse-Depends: backuppc, collectl, libterm-ttyrec-plus-perl, jigit, libconvert-ascii-armour-perl, auto-multiple-choice-common, awit-dbackup, libio-compress-lzma-perl, xmltv-util, vdradmin-am, libhttp-message-perl, checkgmail
Virtual-Package: liblog-any-adapter-perl
Providers: liblog-any-perl
Reverse-Depends: liblog-any-adapter-filehandle-perl, liblog-any-adapter-dispatch-perl, liblog-any-adapter-callback-perl, liblog-any-adapter-screen-perl, libdata-hal-perl, libpoet-perl
Virtual-Package: libmath-complex-perl
Providers: perl-modules-5.24
Reverse-Depends: rel2gpx
Virtual-Package: libparse-cpan-meta-perl
Providers: perl-modules-5.24
Reverse-Depends: libmodule-faker-perl, cpanminus, libmodule-install-perl
Virtual-Package: libreadline6-dev
Providers: libreadline-dev
Reverse-Depends: ruby-build, libopendht-dev, liboctave-dev, guile-2.0-dev, libghc-readline-dev
Virtual-Package: libsdformat-dev
Providers: libsdformat4-dev
Reverse-Depends: libgazebo7-dev
Virtual-Package: libsys-syslog-perl
Providers: libperl5.24
Reverse-Depends: dyndns, ldap-account-manager-lamdaemon, liblogger-syslog-perl, liblog-dispatch-perl, liblog-fast-perl
Virtual-Package: libtest-harness-perl
Providers: perl-modules-5.24
Reverse-Depends: libmodule-build-tiny-perl, libmodule-build-perl, libtap-formatter-junit-perl, libtest-most-perl
Virtual-Package: libtest-tester-perl
Providers: libtest-simple-perl, perl-modules-5.24
Reverse-Depends: libtest-trap-perl, libtest-json-perl
Virtual-Package: perl-modules
Providers: perl-modules-5.24
Reverse-Depends: dirvish, fig2ps, cd-circleprint, po4a, libfinance-yahooquote-perl, conmux, ecaccess, rcconf, yample, oar-restful-api, debian-builder, dl10n, libtk-tablematrix-perl, wakeonlan, pod2pdf, chronicle
Virtual-Package: pgf
Providers: texlive-pictures
Reverse-Depends: logidee-tools, qtikz, ktikz
Virtual-Package: python-imaging-tk
Providers: python-pil.imagetk
Reverse-Depends: fgo, keysync, forg
Virtual-Package: texlive-math-extra
Providers: texlive-science
Reverse-Depends: dblatex, cadabra, lhs2tex, sisu-pdf
== Recommends ==
Virtual-Package: apache2-suexec
Providers: apache2-suexec-custom, apache2-suexec-pristine
Reverse-Recommends: ikiwiki-hosting-web
Virtual-Package: latex-xcolor
Providers: texlive-latex-recommended
Reverse-Recommends: auto-multiple-choice-common
Virtual-Package: libghc-transformers-doc
Providers: ghc-doc
Reverse-Recommends: haskell-platform-doc
Virtual-Package: libghc-xhtml-doc
Providers: ghc-doc
Reverse-Recommends: haskell-platform-doc
Virtual-Package: libgsl0-dev
Providers: libgsl-dev
Reverse-Recommends: liblorene-dev
Virtual-Package: libio-compress-perl
Providers: libperl5.24
Reverse-Recommends: backuppc, collectl, jigit, auto-multiple-choice-common, awit-dbackup, xmltv-util, checkgmail
Virtual-Package: libparse-cpan-meta-perl
Providers: perl-modules-5.24
Reverse-Recommends: libmodule-install-perl
Virtual-Package: libreadline6-dev
Providers: libreadline-dev
Reverse-Recommends: ruby-build
Virtual-Package: libsys-syslog-perl
Providers: libperl5.24
Reverse-Recommends: liblog-dispatch-perl
Virtual-Package: libtest-harness-perl
Providers: perl-modules-5.24
Reverse-Recommends: libmodule-build-perl
Virtual-Package: perl-modules
Providers: perl-modules-5.24
Reverse-Recommends: dirvish, fig2ps, po4a, conmux, oar-restful-api, pod2pdf
Virtual-Package: pgf
Providers: texlive-pictures
Reverse-Recommends: qtikz, ktikz
Virtual-Package: python-imaging-tk
Providers: python-pil.imagetk
Reverse-Recommends: fgo
Virtual-Package: texlive-math-extra
Providers: texlive-science
Reverse-Recommends: dblatex
== Suggests ==
Virtual-Package: ghc-haddock
Providers: ghc
Reverse-Suggests: haskell-platform, haskell-devscripts-minimal
Virtual-Package: libghc-transformers-dev
Providers: ghc
Reverse-Suggests: haskell-platform
Virtual-Package: libghc-xhtml-dev
Providers: ghc
Reverse-Suggests: haskell-platform
Virtual-Package: libgsl0-dev
Providers: libgsl-dev
Reverse-Suggests: liblorene-dev
Virtual-Package: libio-compress-perl
Providers: libperl5.24
Reverse-Suggests: backuppc, xmltv-util, vdradmin-am
Virtual-Package: libreadline6-dev
Providers: libreadline-dev
Reverse-Suggests: ruby-build, libghc-readline-dev
Virtual-Package: perl-modules
Providers: perl-modules-5.24
Reverse-Suggests: fig2ps, cd-circleprint, yample, oar-restful-api, pod2pdf, chronicle
Virtual-Package: texlive-math-extra
Providers: texlive-science
Reverse-Suggests: dblatex, sisu-pdf
#!/usr/bin/python3
# Look for packages that switch between being real packages in one release
# and virtual packages in the next release. For each of these packages,
# report what packages have Provides entries and what packages have dependency
# relationships with that virtual package.
#
# Copyright (c) 2017 Stuart Prescott
# Freely available under the terms of the MIT License
# https://opensource.org/licenses/MIT
testing_packages_file = 'Packages-stretch'
stable_packages_file = 'Packages-jessie'
from collections import defaultdict
from debian import deb822
def read_packages(filename):
packages = defaultdict(list)
provides = defaultdict(list)
with open(filename) as fh:
for pkg in deb822.Packages.iter_paragraphs(fh):
packages[pkg['Package']] = pkg
if 'Provides' in pkg:
for p in pkg['Provides'].split(', '):
provides[p].append(pkg)
return packages, provides
def virtual_packages(packages, provides):
return [p for p in provides if p not in packages]
testing_packages, testing_provides = read_packages(testing_packages_file)
testing_virtual = virtual_packages(testing_packages, testing_provides)
print("Testing: %d packages, %d provided packages, %d pure virtual" %
(len(testing_packages), len(testing_provides), len(testing_virtual)))
stable_packages, stable_provides = read_packages(stable_packages_file)
stable_virtual = virtual_packages(stable_packages, stable_provides)
print("Stable: %d packages, %d provided packages, %d pure virtual" %
(len(stable_packages), len(stable_provides), len(stable_virtual)))
# Look for new virtual packages in testing
new_virtual = sorted(set(p for p in testing_virtual if p not in stable_virtual))
print("New virtual packages: %d" % len(new_virtual))
with open('new-virtual.txt', 'w') as fh:
for p in new_virtual:
l = "%-40s (%s)\n" % (p, ", ".join(v['Package'] for v in testing_provides[p]))
fh.write(l)
# Look for purely virtual packages in testing that are real packages in stable
real_to_virtual = sorted(set(p for p in testing_virtual if p in stable_packages))
print("Real â?? virtual packages: %d" % len(real_to_virtual))
with open('real-to-virtual.txt', 'w') as fh:
for p in real_to_virtual:
l = "%-40s (%s)\n" % (p, ", ".join(v['Package'] for v in testing_provides[p]))
fh.write(l)
# Make an r-deps cache
rels = ['Depends', 'Recommends', 'Suggests']
depcache = {r: defaultdict(list) for r in rels}
rdepcache = {r: defaultdict(list) for r in rels}
for name, pkg in testing_packages.items():
for reltype in ['Depends', 'Recommends', 'Suggests']:
if reltype in pkg:
deps = [r['name'] for gp in pkg.relations['depends'] for r in gp]
depcache[reltype][name].extend(deps)
for depname in deps:
rdepcache[reltype][depname].append(name)
# Look for purely virtual packages in testing that are real packages in stable
# AND that have dependency relationships on them in testing.
with open('newly-virtual-deps.txt', 'w') as fh:
for reltype in rels:
fh.write('== %s ==\n\n' % reltype)
for name in real_to_virtual:
if rdepcache[reltype][name]:
s = [
"Virtual-Package: %s" % name,
"Providers: %s" % (", ".join(v['Package'] for v in testing_provides[name])),
"Reverse-%s: %s" % (reltype, ", ".join(rdepcache[reltype][name])),
"",
"",
]
fh.write("\n".join(s))
fh.write("\n")
Reply to: