Bug#572596: python-apt: TagSection corruption when iterating through a TagFile
Package: python-apt
Version: 0.7.93.3
Severity: important
Hello,
While trying to convert deb822 in python-debian to use the TagParser
iterator API (rather than an ugly hack to avoid using shared storage), I
ran into this little bug. It seems that after somewhere between 30 or
40 paragraphs (I assume this depends on the size of the paragraphs), old
section instances become corrupt.
Here's a reproducer:
#!/usr/bin/python
import apt_pkg; apt_pkg.init()
from debian_bundle import deb822
deb822_objs = list(deb822.Deb822.iter_paragraphs(open("/tmp/test_Packages"),
use_apt_pkg=False))
def test_keys(iter1, iter2):
iter1 = iter(iter1)
for section in iter2:
d = iter1.next()
if section.keys() != d.keys():
print "key mismatch:", section.keys(), "!=", d.keys()
# This prints nothing
print "Testing while iterating"
test_keys(deb822_objs, apt_pkg.TagFile(open("/tmp/test_Packages")))
# This has errors, apt_pkg data is corrupt
print "Testing after making a list from the iterator"
test_keys(deb822_objs, list(apt_pkg.TagFile(open("/tmp/test_Packages"))))
# This doesn't have an error for me
print "Testing after making a list out of only the first 30 entries"
shorter = []
sections = iter(apt_pkg.TagFile(open("/tmp/test_Packages")))
for i in range(30):
shorter.append(sections.next())
test_keys(deb822_objs, shorter)
# This doesn't have an error for me
print "Testing after making a list out of only the first 35 entries"
shorter = []
sections = iter(apt_pkg.TagFile(open("/tmp/test_Packages")))
for i in range(35):
shorter.append(sections.next())
test_keys(deb822_objs, shorter)
If I run it on test_Packages from #571470 [1], I get output like:
Testing while iterating
Testing after making a list from the iterator
key mismatch: ['Package', 'pliance-checker\nPri', 'ority', '\nSection', 'alled-Size', '@gmail.com>\nArchit', 'ecture', 'sion', ' pool/main/a', '/abi-compliance-checker/abi-compliance-ch', 'ecker_1.6-1_all.deb\nSize', '4a3436442d147c3af44f98a66\nSHA1', 'SHA256', ' by examining the shared objects\n (.so) files themselves,'] != ['Package', 'Priority', 'Section', 'Installed-Size', 'Maintainer', 'Architecture', 'Version', 'Filename', 'Size', 'MD5sum', 'SHA1', 'SHA256', 'Description', 'Tag']
<lots of similar messages, up to...>
key mismatch: ['le with a variety of plugi', 'ns.\n .\n This packag', 'e includes many of', ' the available impor', 't/export plugins allowing\n AbiWord to interact with ODT, WordPerfect, and other formats. It also\n inclu', 'des tools plugins, o', 'ffering live co', 'llaboration with A', 'biWord users\n on Linux and Windows (using', ' TCP or Jabber/XMPP), web translation and', '\n dictionary support, and more.\n .\n Additional plugin', 's that require significant amounts of extr', 'a software to\n function are in the various abiword-plugin-* packa', 'ges.\nHomepag', 'e', 'ce', 'oolkit', 'with-format', 'Homepage', 'Tag'] != ['Package', 'Priority', 'Section', 'Installed-Size', 'Maintainer', 'Architecture', 'Source', 'Version', 'Replaces', 'Provides', 'Depends', 'Conflicts', 'Filename', 'Size', 'MD5sum', 'SHA1', 'SHA256', 'Description', 'Homepage', 'Tag']
Testing after making a list out of only the first 30 entries
Testing after making a list out of only the first 35 entries
key mismatch: ['Package', 'pliance-checker\nPri', 'ority', '\nSection', 'alled-Size', '@gmail.com>\nArchit', 'ecture', 'sion', ' pool/main/a', '/abi-compliance-checker/abi-compliance-ch', 'ecker_1.6-1_all.deb\nSize', '4a3436442d147c3af44f98a66\nSHA1', 'SHA256', ' by examining the shared objects\n (.so) files themselves,'] != ['Package', 'Priority', 'Section', 'Installed-Size', 'Maintainer', 'Architecture', 'Version', 'Filename', 'Size', 'MD5sum', 'SHA1', 'SHA256', 'Description', 'Tag']
<lots of similar messages, up to...>
key mismatch: ['le with a variety of plugi', 'ns.\n .\n This packag', 'e includes many of', ' the available impor', 't/export plugins allowing\n AbiWord to interact with ODT, WordPerfect, and other formats. It also\n inclu', 'des tools plugins, o', 'ffering live co', 'llaboration with A', 'biWord users\n on Linux and Windows (using', ' TCP or Jabber/XMPP), web translation and', '\n dictionary support, and more.\n .\n Additional plugin', 's that require significant amounts of extr', 'a software to\n function are in the various abiword-plugin-* packa', 'ges.\nHomepag', 'e', 'ce', 'oolkit', 'with-format', 'Homepage', 'Tag'] != ['Package', 'Priority', 'Section', 'Installed-Size', 'Maintainer', 'Architecture', 'Source', 'Version', 'Replaces', 'Provides', 'Depends', 'Conflicts', 'Filename', 'Size', 'MD5sum', 'SHA1', 'SHA256', 'Description', 'Homepage', 'Tag']
[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=5;filename=test_Packages;att=1;bug=571470
-- System Information:
Debian Release: squeeze/sid
APT prefers oldstable
APT policy: (500, 'oldstable'), (500, 'unstable'), (1, 'experimental')
Architecture: i386 (i686)
Kernel: Linux 2.6.32-trunk-686 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash
Versions of packages python-apt depends on:
ii apt [libapt-pkg-libc6.9-6-4. 0.7.25 Advanced front-end for dpkg
ii apt-utils [libapt-inst-libc6 0.7.25 APT utility programs
ii libc6 2.10.2-4 Embedded GNU C Library: Shared lib
ii libgcc1 1:4.4.2-8 GCC support library
ii libstdc++6 4.4.2-8 The GNU Standard C++ Library v3
ii python 2.5.4-5 An interactive high-level object-o
ii python-central 0.6.14+nmu2 register and build utility for Pyt
ii python2.6 2.6.2-2 An interactive high-level object-o
Versions of packages python-apt recommends:
ii iso-codes 3.12.1-1 ISO language, territory, currency,
ii libjs-jquery 1.3.3-2 JavaScript library for dynamic web
ii lsb-release 3.2-23 Linux Standard Base version report
Versions of packages python-apt suggests:
pn python-apt-dbg <none> (no description available)
ii python-gtk2 2.16.0-1 Python bindings for the GTK+ widge
ii python-vte 1:0.22.5-1 Python bindings for the VTE widget
-- no debconf information
Reply to: