[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#632271: apt fails to install very large packages: file size overflow?

Package: apt
Severity: important
Tags: lfs

I have created a package for a very large piece of software and am trying to distribute it using an in-house repository.  The package works fine when installed using "dpkg -i", but fails when I try to install it through apt-get or aptitude.  The package in question is 4.2 GB in size.

Apt gives me the error message "Bizarre Error - File size is not what the server reported", then proceeds to list the correct file size followed by a bogus filesize (generally deviating from the correct value by about 1 MB).  However, the downloaded file, as observed in /var/cache/apt, is not corrupted (correct filesize and md5sum).  The server and repository both appear to be configured correctly as well.

There are also minor issues with the progress indicator during the package download, but I am unsure if these are related or a separate issue.

I have tested the versions of apt in stable, testing, and experimental with similar results.

.... the following is a detailed copy/paste description of the problem.  The package in question is ansys_13.0sp2-1_amd64.deb and has a size of 4489033432 bytes.

Attempting to install the large package looks like this:
# apt-get install ansys
Get:1 http://internal_server/debian/ puc-ems/main ansys amd64 13.0sp2-1 [4,489 MB
2% [1 ansys 97.2 MB/194 MB 50%]                              11.3 MB/s 6min 28s
41% [1 ansys 1,879 MB/1,879 MB 100%]                         11.4 MB/s 3min 48s
Fetched 194 MB in 6min 46s (477 kB/s)                                          
Failed to fetch http://emssupport.purduecal.edu/debian/pool/main/a/ansys/ansys_13.0sp2-1_amd64.deb  Size mismatch
W: Bizarre Error - File size is not what the server reported 4489033432 4485863888
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

Note that the progress indicator seemed to indicate a total filesize of 194 MB on one line, yet the full 4,489 MB size was listed on the line above it.  Once the download passed the 194 MB mark, the amount downloaded and total size increased together until the download was complete.  (at which point the incorrect size was displayed)

However, the downloaded file (located at /var/cache/apt/archives/partial/ansys_13.0sp2-1_amd64.deb) matches the filesize and md5sum observed on the server... and it installs properly with "dpkg -i".

# ls -l /var/cache/apt/archives/partial/ansys_13.0sp2-1_amd64.deb 
-rw-r--r-- 1 root root 4489033432 Jun 30 03:39 /var/cache/apt/archives/partial/ansys_13.0sp2-1_amd64.deb

The output from apt-get is identical with the version of apt in squeeze/stable ( and wheezy/testing (

The version in experimental also seems to have the bug, but it behaves slightly differently.  Its error message reads:
W: Size of file /var/cache/apt/archives/partial/ansys_13.0sp2-1_amd64.deb is not what the server reported 4489033432 4487664456

....and it seems to disregard the error and install the package anyway.  The downloaded file (this time located at /var/cache/apt/archives/ansys_13.0sp2-1_amd64.deb) is the correct size and has the correct md5sum.

I don't think this is a problem with the server.  To verify this, I checked the headers it was generating:
# wget -S http://internal_server/debian/pool/main/a/ansys/ansys_13.0sp2-1_amd64.deb
HTTP request sent, awaiting response... 
  HTTP/1.0 200 OK
  Content-Type: application/x-debian-package
  Accept-Ranges: bytes
  ETag: "3495314447"
  Last-Modified: Thu, 30 Jun 2011 08:39:56 GMT
  Content-Length: 4489033432
  Connection: keep-alive
  Date: Fri, 01 Jul 2011 01:47:48 GMT
  Server: lighttpd/1.4.28
Length: 4489033432 (4.2G) [application/x-debian-package]

Similarly, I see the correct size and md5sum in the binary-amd64/Packages file on the server.

Any ideas on what I should check next?

-- Package-specific info:

-- apt-config dump --

APT "";
APT::Architecture "amd64";
APT::Build-Essential "";
APT::Build-Essential:: "build-essential";
APT::Install-Recommends "1";
APT::Install-Suggests "0";
APT::Acquire "";
APT::Acquire::Translation "environment";
APT::Authentication "";
APT::Authentication::TrustCDROM "true";
APT::NeverAutoRemove "";
APT::NeverAutoRemove:: "^firmware-linux.*";
APT::NeverAutoRemove:: "^linux-firmware$";
APT::NeverAutoRemove:: "^linux-image.*";
APT::NeverAutoRemove:: "^kfreebsd-image.*";
APT::NeverAutoRemove:: "^linux-restricted-modules.*";
APT::NeverAutoRemove:: "^linux-ubuntu-modules-.*";
APT::Never-MarkAuto-Sections "";
APT::Never-MarkAuto-Sections:: "metapackages";
APT::Never-MarkAuto-Sections:: "restricted/metapackages";
APT::Never-MarkAuto-Sections:: "universe/metapackages";
APT::Never-MarkAuto-Sections:: "multiverse/metapackages";
APT::Never-MarkAuto-Sections:: "oldlibs";
APT::Never-MarkAuto-Sections:: "restricted/oldlibs";
APT::Never-MarkAuto-Sections:: "universe/oldlibs";
APT::Never-MarkAuto-Sections:: "multiverse/oldlibs";
APT::Periodic "";
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "0";
APT::Periodic::AutocleanInterval "0";
APT::Update "";
APT::Update::Post-Invoke "";
APT::Update::Post-Invoke:: "touch /var/lib/apt/periodic/update-success-stamp 2>/dev/null || true";
APT::Update::Post-Invoke-Success "";
APT::Update::Post-Invoke-Success:: "[ ! -f /var/run/dbus/system_bus_socket ] || /usr/bin/dbus-send --system --dest=org.debian.apt --type=signal /org/debian/apt org.debian.apt.CacheChanged || true";
APT::Archives "";
APT::Archives::MaxAge "30";
APT::Archives::MinAge "2";
APT::Archives::MaxSize "500";
Dir "/";
Dir::State "var/lib/apt/";
Dir::State::lists "lists/";
Dir::State::cdroms "cdroms.list";
Dir::State::mirrors "mirrors/";
Dir::State::extended_states "extended_states";
Dir::State::status "/var/lib/dpkg/status";
Dir::Cache "var/cache/apt/";
Dir::Cache::archives "archives/";
Dir::Cache::srcpkgcache "srcpkgcache.bin";
Dir::Cache::pkgcache "pkgcache.bin";
Dir::Etc "etc/apt/";
Dir::Etc::sourcelist "sources.list";
Dir::Etc::sourceparts "sources.list.d";
Dir::Etc::vendorlist "vendors.list";
Dir::Etc::vendorparts "vendors.list.d";
Dir::Etc::main "apt.conf";
Dir::Etc::netrc "auth.conf";
Dir::Etc::parts "apt.conf.d";
Dir::Etc::preferences "preferences";
Dir::Etc::preferencesparts "preferences.d";
Dir::Etc::trusted "trusted.gpg";
Dir::Etc::trustedparts "trusted.gpg.d";
Dir::Bin "";
Dir::Bin::methods "/usr/lib/apt/methods";
Dir::Bin::dpkg "/usr/bin/dpkg";
Dir::Media "";
Dir::Media::MountPath "/media/cdrom";
Dir::Log "var/log/apt";
Dir::Log::Terminal "term.log";
Dir::Log::History "history.log";
Dir::Ignore-Files-Silently "";
Dir::Ignore-Files-Silently:: "~$";
Dir::Ignore-Files-Silently:: "\.disabled$";
Dir::Ignore-Files-Silently:: "\.bak$";
Dir::Ignore-Files-Silently:: "\.dpkg-[a-z]+$";
Acquire "";
Acquire::cdrom "";
Acquire::cdrom::mount "/media/cdrom";
DPkg "";
DPkg::Pre-Install-Pkgs "";
DPkg::Pre-Install-Pkgs:: "/usr/bin/apt-listchanges --apt || test $? -ne 10";
DPkg::Pre-Install-Pkgs:: "/usr/sbin/dpkg-preconfigure --apt || true";
DPkg::Tools "";
DPkg::Tools::Options "";
DPkg::Tools::Options::/usr/bin/apt-listchanges "";
DPkg::Tools::Options::/usr/bin/apt-listchanges::Version "2";
DPkg::Post-Invoke "";
DPkg::Post-Invoke:: "if [ -d /var/lib/update-notifier ]; then touch /var/lib/update-notifier/dpkg-run-stamp; fi; if [ -e /var/lib/update-notifier/updates-available ]; then echo > /var/lib/update-notifier/updates-available; fi ";
Unattended-Upgrade "";
Unattended-Upgrade::Allowed-Origins "";
Unattended-Upgrade::Allowed-Origins:: "${distro_id} stable";
Unattended-Upgrade::Allowed-Origins:: "${distro_id} ${distro_codename}-security";
CommandLine "";
CommandLine::AsString "apt-config dump";

-- /etc/apt/preferences --

Package: *
Pin: release a=testing
Pin-Priority: 99

-- /etc/apt/sources.list --


# deb cdrom:[Debian GNU/Linux 6.0.1a _Squeeze_ - Official Multi-architecture amd64/i386 NETINST #1 20110320-15:14]/ squeeze main

#deb cdrom:[Debian GNU/Linux 6.0.1a _Squeeze_ - Official Multi-architecture amd64/i386 NETINST #1 20110320-15:14]/ squeeze main

deb http://debian.osuosl.org/debian/ squeeze main
deb-src http://debian.osuosl.org/debian/ squeeze main

deb http://security.debian.org/ squeeze/updates main
deb-src http://security.debian.org/ squeeze/updates main

# squeeze-updates, previously known as 'volatile'
deb http://debian.osuosl.org/debian/ squeeze-updates main
deb-src http://debian.osuosl.org/debian/ squeeze-updates main

deb http://internal_server/debian/ puc-ems main

deb http://ftp.us.debian.org/debian/ testing main
deb-src http://ftp.us.debian.org/debian/ testing main

-- System Information:
Debian Release: 6.0.2
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable'), (99, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.32-5-amd64 (SMP w/1 CPU core)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages apt depends on:
ii  debian-archive-keyring  2010.08.28       GnuPG archive keys of the Debian a
ii  gnupg                   1.4.10-4         GNU privacy guard - a free PGP rep
ii  libc6                   2.11.2-10        Embedded GNU C Library: Shared lib
ii  libgcc1                 1:4.4.5-8        GCC support library
ii  libstdc++6              4.6.0-10         The GNU Standard C++ Library v3
ii  zlib1g                  1: compression library - runtime

apt recommends no packages.

Versions of packages apt suggests:
pn  apt-doc               <none>             (no description available)
ii  aptitude              0.6.3-3.2          terminal-based package manager (te
ii  bzip2                 1.0.5-6            high-quality block-sorting file co
pn  dpkg-dev              <none>             (no description available)
pn  lzma                  <none>             (no description available)
ii  python-apt   Python interface to libapt-pkg
ii  synaptic              0.70~pre1+b1       Graphical package manager

-- no debconf information

Reply to: