--- Begin Message ---
Package: apt
Version: 0.8.10.3+squeeze1
Severity: important
Tags: lfs
I have created a package for a very large piece of software and am trying to distribute it using an in-house repository. The package works fine when installed using "dpkg -i", but fails when I try to install it through apt-get or aptitude. The package in question is 4.2 GB in size.
Apt gives me the error message "Bizarre Error - File size is not what the server reported", then proceeds to list the correct file size followed by a bogus filesize (generally deviating from the correct value by about 1 MB). However, the downloaded file, as observed in /var/cache/apt, is not corrupted (correct filesize and md5sum). The server and repository both appear to be configured correctly as well.
There are also minor issues with the progress indicator during the package download, but I am unsure if these are related or a separate issue.
I have tested the versions of apt in stable, testing, and experimental with similar results.
.... the following is a detailed copy/paste description of the problem. The package in question is ansys_13.0sp2-1_amd64.deb and has a size of 4489033432 bytes.
Attempting to install the large package looks like this:
# apt-get install ansys
[...]
Get:1 http://internal_server/debian/ puc-ems/main ansys amd64 13.0sp2-1 [4,489 MB
2% [1 ansys 97.2 MB/194 MB 50%] 11.3 MB/s 6min 28s
[...later...]
41% [1 ansys 1,879 MB/1,879 MB 100%] 11.4 MB/s 3min 48s
[...later...]
Fetched 194 MB in 6min 46s (477 kB/s)
Failed to fetch http://emssupport.purduecal.edu/debian/pool/main/a/ansys/ansys_13.0sp2-1_amd64.deb Size mismatch
W: Bizarre Error - File size is not what the server reported 4489033432 4485863888
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Note that the progress indicator seemed to indicate a total filesize of 194 MB on one line, yet the full 4,489 MB size was listed on the line above it. Once the download passed the 194 MB mark, the amount downloaded and total size increased together until the download was complete. (at which point the incorrect size was displayed)
However, the downloaded file (located at /var/cache/apt/archives/partial/ansys_13.0sp2-1_amd64.deb) matches the filesize and md5sum observed on the server... and it installs properly with "dpkg -i".
# ls -l /var/cache/apt/archives/partial/ansys_13.0sp2-1_amd64.deb
-rw-r--r-- 1 root root 4489033432 Jun 30 03:39 /var/cache/apt/archives/partial/ansys_13.0sp2-1_amd64.deb
The output from apt-get is identical with the version of apt in squeeze/stable (0.8.10.3+squeeze1) and wheezy/testing (0.8.14.1).
The version in experimental also seems to have the bug, but it behaves slightly differently. Its error message reads:
W: Size of file /var/cache/apt/archives/partial/ansys_13.0sp2-1_amd64.deb is not what the server reported 4489033432 4487664456
....and it seems to disregard the error and install the package anyway. The downloaded file (this time located at /var/cache/apt/archives/ansys_13.0sp2-1_amd64.deb) is the correct size and has the correct md5sum.
I don't think this is a problem with the server. To verify this, I checked the headers it was generating:
# wget -S http://internal_server/debian/pool/main/a/ansys/ansys_13.0sp2-1_amd64.deb
[...]
HTTP request sent, awaiting response...
HTTP/1.0 200 OK
Content-Type: application/x-debian-package
Accept-Ranges: bytes
ETag: "3495314447"
Last-Modified: Thu, 30 Jun 2011 08:39:56 GMT
Content-Length: 4489033432
Connection: keep-alive
Date: Fri, 01 Jul 2011 01:47:48 GMT
Server: lighttpd/1.4.28
Length: 4489033432 (4.2G) [application/x-debian-package]
[...]
Similarly, I see the correct size and md5sum in the binary-amd64/Packages file on the server.
Any ideas on what I should check next?
-- Package-specific info:
-- apt-config dump --
APT "";
APT::Architecture "amd64";
APT::Build-Essential "";
APT::Build-Essential:: "build-essential";
APT::Install-Recommends "1";
APT::Install-Suggests "0";
APT::Acquire "";
APT::Acquire::Translation "environment";
APT::Authentication "";
APT::Authentication::TrustCDROM "true";
APT::NeverAutoRemove "";
APT::NeverAutoRemove:: "^firmware-linux.*";
APT::NeverAutoRemove:: "^linux-firmware$";
APT::NeverAutoRemove:: "^linux-image.*";
APT::NeverAutoRemove:: "^kfreebsd-image.*";
APT::NeverAutoRemove:: "^linux-restricted-modules.*";
APT::NeverAutoRemove:: "^linux-ubuntu-modules-.*";
APT::Never-MarkAuto-Sections "";
APT::Never-MarkAuto-Sections:: "metapackages";
APT::Never-MarkAuto-Sections:: "restricted/metapackages";
APT::Never-MarkAuto-Sections:: "universe/metapackages";
APT::Never-MarkAuto-Sections:: "multiverse/metapackages";
APT::Never-MarkAuto-Sections:: "oldlibs";
APT::Never-MarkAuto-Sections:: "restricted/oldlibs";
APT::Never-MarkAuto-Sections:: "universe/oldlibs";
APT::Never-MarkAuto-Sections:: "multiverse/oldlibs";
APT::Periodic "";
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "0";
APT::Periodic::AutocleanInterval "0";
APT::Update "";
APT::Update::Post-Invoke "";
APT::Update::Post-Invoke:: "touch /var/lib/apt/periodic/update-success-stamp 2>/dev/null || true";
APT::Update::Post-Invoke-Success "";
APT::Update::Post-Invoke-Success:: "[ ! -f /var/run/dbus/system_bus_socket ] || /usr/bin/dbus-send --system --dest=org.debian.apt --type=signal /org/debian/apt org.debian.apt.CacheChanged || true";
APT::Archives "";
APT::Archives::MaxAge "30";
APT::Archives::MinAge "2";
APT::Archives::MaxSize "500";
Dir "/";
Dir::State "var/lib/apt/";
Dir::State::lists "lists/";
Dir::State::cdroms "cdroms.list";
Dir::State::mirrors "mirrors/";
Dir::State::extended_states "extended_states";
Dir::State::status "/var/lib/dpkg/status";
Dir::Cache "var/cache/apt/";
Dir::Cache::archives "archives/";
Dir::Cache::srcpkgcache "srcpkgcache.bin";
Dir::Cache::pkgcache "pkgcache.bin";
Dir::Etc "etc/apt/";
Dir::Etc::sourcelist "sources.list";
Dir::Etc::sourceparts "sources.list.d";
Dir::Etc::vendorlist "vendors.list";
Dir::Etc::vendorparts "vendors.list.d";
Dir::Etc::main "apt.conf";
Dir::Etc::netrc "auth.conf";
Dir::Etc::parts "apt.conf.d";
Dir::Etc::preferences "preferences";
Dir::Etc::preferencesparts "preferences.d";
Dir::Etc::trusted "trusted.gpg";
Dir::Etc::trustedparts "trusted.gpg.d";
Dir::Bin "";
Dir::Bin::methods "/usr/lib/apt/methods";
Dir::Bin::dpkg "/usr/bin/dpkg";
Dir::Media "";
Dir::Media::MountPath "/media/cdrom";
Dir::Log "var/log/apt";
Dir::Log::Terminal "term.log";
Dir::Log::History "history.log";
Dir::Ignore-Files-Silently "";
Dir::Ignore-Files-Silently:: "~$";
Dir::Ignore-Files-Silently:: "\.disabled$";
Dir::Ignore-Files-Silently:: "\.bak$";
Dir::Ignore-Files-Silently:: "\.dpkg-[a-z]+$";
Acquire "";
Acquire::cdrom "";
Acquire::cdrom::mount "/media/cdrom";
DPkg "";
DPkg::Pre-Install-Pkgs "";
DPkg::Pre-Install-Pkgs:: "/usr/bin/apt-listchanges --apt || test $? -ne 10";
DPkg::Pre-Install-Pkgs:: "/usr/sbin/dpkg-preconfigure --apt || true";
DPkg::Tools "";
DPkg::Tools::Options "";
DPkg::Tools::Options::/usr/bin/apt-listchanges "";
DPkg::Tools::Options::/usr/bin/apt-listchanges::Version "2";
DPkg::Post-Invoke "";
DPkg::Post-Invoke:: "if [ -d /var/lib/update-notifier ]; then touch /var/lib/update-notifier/dpkg-run-stamp; fi; if [ -e /var/lib/update-notifier/updates-available ]; then echo > /var/lib/update-notifier/updates-available; fi ";
Unattended-Upgrade "";
Unattended-Upgrade::Allowed-Origins "";
Unattended-Upgrade::Allowed-Origins:: "${distro_id} stable";
Unattended-Upgrade::Allowed-Origins:: "${distro_id} ${distro_codename}-security";
CommandLine "";
CommandLine::AsString "apt-config dump";
-- /etc/apt/preferences --
Package: *
Pin: release a=testing
Pin-Priority: 99
-- /etc/apt/sources.list --
#
# deb cdrom:[Debian GNU/Linux 6.0.1a _Squeeze_ - Official Multi-architecture amd64/i386 NETINST #1 20110320-15:14]/ squeeze main
#deb cdrom:[Debian GNU/Linux 6.0.1a _Squeeze_ - Official Multi-architecture amd64/i386 NETINST #1 20110320-15:14]/ squeeze main
deb http://debian.osuosl.org/debian/ squeeze main
deb-src http://debian.osuosl.org/debian/ squeeze main
deb http://security.debian.org/ squeeze/updates main
deb-src http://security.debian.org/ squeeze/updates main
# squeeze-updates, previously known as 'volatile'
deb http://debian.osuosl.org/debian/ squeeze-updates main
deb-src http://debian.osuosl.org/debian/ squeeze-updates main
deb http://internal_server/debian/ puc-ems main
deb http://ftp.us.debian.org/debian/ testing main
deb-src http://ftp.us.debian.org/debian/ testing main
-- System Information:
Debian Release: 6.0.2
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable'), (99, 'testing')
Architecture: amd64 (x86_64)
Kernel: Linux 2.6.32-5-amd64 (SMP w/1 CPU core)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Versions of packages apt depends on:
ii debian-archive-keyring 2010.08.28 GnuPG archive keys of the Debian a
ii gnupg 1.4.10-4 GNU privacy guard - a free PGP rep
ii libc6 2.11.2-10 Embedded GNU C Library: Shared lib
ii libgcc1 1:4.4.5-8 GCC support library
ii libstdc++6 4.6.0-10 The GNU Standard C++ Library v3
ii zlib1g 1:1.2.3.4.dfsg-3 compression library - runtime
apt recommends no packages.
Versions of packages apt suggests:
pn apt-doc <none> (no description available)
ii aptitude 0.6.3-3.2 terminal-based package manager (te
ii bzip2 1.0.5-6 high-quality block-sorting file co
pn dpkg-dev <none> (no description available)
pn lzma <none> (no description available)
ii python-apt 0.7.100.1+squeeze1 Python interface to libapt-pkg
ii synaptic 0.70~pre1+b1 Graphical package manager
-- no debconf information
--- End Message ---
--- Begin Message ---
Source: apt
Source-Version: 0.8.16~exp3
We believe that the bug you reported is fixed in the latest version of
apt, which is due to be installed in the Debian FTP archive:
apt-doc_0.8.16~exp3_all.deb
to main/a/apt/apt-doc_0.8.16~exp3_all.deb
apt-transport-https_0.8.16~exp3_amd64.deb
to main/a/apt/apt-transport-https_0.8.16~exp3_amd64.deb
apt-utils_0.8.16~exp3_amd64.deb
to main/a/apt/apt-utils_0.8.16~exp3_amd64.deb
apt_0.8.16~exp3.dsc
to main/a/apt/apt_0.8.16~exp3.dsc
apt_0.8.16~exp3.tar.gz
to main/a/apt/apt_0.8.16~exp3.tar.gz
apt_0.8.16~exp3_amd64.deb
to main/a/apt/apt_0.8.16~exp3_amd64.deb
libapt-inst1.3_0.8.16~exp3_amd64.deb
to main/a/apt/libapt-inst1.3_0.8.16~exp3_amd64.deb
libapt-pkg-dev_0.8.16~exp3_amd64.deb
to main/a/apt/libapt-pkg-dev_0.8.16~exp3_amd64.deb
libapt-pkg-doc_0.8.16~exp3_all.deb
to main/a/apt/libapt-pkg-doc_0.8.16~exp3_all.deb
libapt-pkg4.11_0.8.16~exp3_amd64.deb
to main/a/apt/libapt-pkg4.11_0.8.16~exp3_amd64.deb
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 632271@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Michael Vogt <mvo@debian.org> (supplier of updated apt package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Format: 1.8
Date: Fri, 15 Jul 2011 09:56:17 +0200
Source: apt
Binary: apt libapt-pkg4.11 libapt-inst1.3 apt-doc libapt-pkg-dev libapt-pkg-doc apt-utils apt-transport-https
Architecture: source all amd64
Version: 0.8.16~exp3
Distribution: experimental
Urgency: low
Maintainer: APT Development Team <deity@lists.debian.org>
Changed-By: Michael Vogt <mvo@debian.org>
Description:
apt - APT's commandline package manager
apt-doc - Documentation for APT
apt-transport-https - https download transport for APT
apt-utils - APT utility programs
libapt-inst1.3 - APT's deb package format runtime library
libapt-pkg-dev - Development files for APT's libapt-pkg and libapt-inst
libapt-pkg-doc - Documentation for APT development
libapt-pkg4.11 - APT's package managment runtime library
Closes: 596498 632271 632520
Changes:
apt (0.8.16~exp3) experimental; urgency=low
.
[ David Kalnischkies ]
* apt-pkg/pkgcache.h:
- readd All{Foreign,Allowed} as suggested by Julian to
remain strictly API compatible
* apt-pkg/acquire*.{cc,h}:
- try even harder to support really big files in the fetcher by
converting (hopefully) everything to 'long long' (Closes: #632271)
* ftparchive/writer.cc:
- generate all checksums in one run over the file for Release
* cmdline/apt-get.cc:
- add an --assume-no option for testing to say 'no' to everything
* apt-pkg/deb/debmetaindex.cc:
- add trusted=yes option to mark unsigned (local) repository as trusted
based on a patch from Ansgar Burchardt, thanks a lot! (Closes: #596498)
.
[ Michael Vogt ]
* merge fixes from the debian/unstable upload
* merge lp:~mvo/apt/sha512-template to get fixes for the
sha1/md5 verifiation (closes: #632520)
Checksums-Sha1:
48f6fa39fe1ae599777ff06f8e1bd889edfc2721 1402 apt_0.8.16~exp3.dsc
f55cfeebfdc12478d20ff2f6da8e1fbf051b3228 3466665 apt_0.8.16~exp3.tar.gz
0076bef7059f98cbffa416fb83047cc1e5904aff 246198 apt-doc_0.8.16~exp3_all.deb
571fc6734f2e406969f9129debb50df6ad1d8a24 777704 libapt-pkg-doc_0.8.16~exp3_all.deb
d9832d73602d6fe7570fb81b8c7129507dfb8446 994384 libapt-pkg4.11_0.8.16~exp3_amd64.deb
3c638b72abf4543a8975463ff9e64e7c72ccfc12 179982 libapt-inst1.3_0.8.16~exp3_amd64.deb
91eef1725c35377f83bd8fab1bd9d269a2cc59b4 1130402 apt_0.8.16~exp3_amd64.deb
63c033087e8c28120ddbcfab34e6bbbdb3277586 166978 libapt-pkg-dev_0.8.16~exp3_amd64.deb
274c465a1e5c378be49fe8b082bf0016b390c842 263480 apt-utils_0.8.16~exp3_amd64.deb
77c305a3eb626adf6a570c9edb62de68dac88eea 91994 apt-transport-https_0.8.16~exp3_amd64.deb
Checksums-Sha256:
8be5a926f4c163b59e8a9fac509bc0addd5cc119354be86165cbfbbf2d1f75c5 1402 apt_0.8.16~exp3.dsc
100f449a0843ee557f7f1a16db9df7c6b4c5636ce86ce303547e9d6aaf082ef1 3466665 apt_0.8.16~exp3.tar.gz
e36b78ec95d91868b0a876eea6cd06c3acdf3be799b345f1cd9f8591a1e03922 246198 apt-doc_0.8.16~exp3_all.deb
55e08fff724566a327129189e9b83af352b038b0c09198827168385be1060c70 777704 libapt-pkg-doc_0.8.16~exp3_all.deb
3bb0dc4c190600b4d9106001447757b5e3fdf4328185792d807a65e0ef180acb 994384 libapt-pkg4.11_0.8.16~exp3_amd64.deb
cea4eb8f41ed29e19d27d69acbf5256030baf6a9b063fec8d86825cc2b87af48 179982 libapt-inst1.3_0.8.16~exp3_amd64.deb
e7fc241b9dcf67901f26f40191c8bfd2be20207317879f7a447ff0462d6e99fb 1130402 apt_0.8.16~exp3_amd64.deb
d03a81bd11f54072466eb4fc2385a5ba557e36f7e7838b32e113950fae0e0275 166978 libapt-pkg-dev_0.8.16~exp3_amd64.deb
ff0c2bade637fac62775ef9053ac778ae4bd2f38e036b12c67619ba29da0d8f7 263480 apt-utils_0.8.16~exp3_amd64.deb
b83932e9a641acea34233b25598e39628e6da265de88bc32c037af3b286e165d 91994 apt-transport-https_0.8.16~exp3_amd64.deb
Files:
d9796b41e1f3091fa65e069f4a2fea9e 1402 admin important apt_0.8.16~exp3.dsc
9a350280597a986f74f347b5b94f5fbf 3466665 admin important apt_0.8.16~exp3.tar.gz
443866ec0e0b65f57afc925c39f64462 246198 doc optional apt-doc_0.8.16~exp3_all.deb
3cbc9865aef981568f382843a19d48c4 777704 doc optional libapt-pkg-doc_0.8.16~exp3_all.deb
02ca6b1bb235db0b59768f9d0a04f980 994384 admin important libapt-pkg4.11_0.8.16~exp3_amd64.deb
e2d84ed7ddb2f7413eda12513ae52d84 179982 admin important libapt-inst1.3_0.8.16~exp3_amd64.deb
beb8730b3985f7bdcd36aa48bc51389d 1130402 admin important apt_0.8.16~exp3_amd64.deb
df87b32747bd299aa8226af2aa4ebb05 166978 libdevel optional libapt-pkg-dev_0.8.16~exp3_amd64.deb
e3ecc2aabc8896042d28edd73a5a2088 263480 admin important apt-utils_0.8.16~exp3_amd64.deb
d94f19ce5c36552b11654189d2fd29b0 91994 admin optional apt-transport-https_0.8.16~exp3_amd64.deb
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iEYEARECAAYFAk4f9IQACgkQliSD4VZixzTGkACeKi7hCW2KNIomcSBzSgPnkzoR
uhcAoI7cuQ9omWp+ifmz/uw5URUSfWdD
=iF2G
-----END PGP SIGNATURE-----
--- End Message ---