WG: Snapshot.debian.org: "503 Response object too large" issue prevents retrieval of files exceeding 2GB
Dear Debian maintainers,
may I try again here to get help with the (Debian snapshot related) issue described below? No one seems to answer on the most appropriate mailing list for this issue, i.e. debian-snapshot@lists.debian.org.
Many thanks for your help and your valuable attention,
Christof Warlich
Dear Debian Snapshot maintainers,
I encountered a severe usability limitation with snapshot.debian.org concerning packages whose source archives exceed ~2GB in size. Such files systematically fail with the error "503 Response object too large," rendering the affected files effectively impossible to download.
Explicit attempts include:
- Direct wget or curl downloading (with and without HTTP Range headers).
- apt-get source using Acquire::http::Dl-Limit "1000" (range requests).
All methods tested result consistently in "503 Response object too large" errors returned by your Varnish-based HTTP caching infrastructure. After careful troubleshooting and research, it's clear this limitation is deliberately configured on the snapshot infrastructure.
Given this explicit limitation and its impact on users relying on snapshot.d.o for reproducibility purposes (e.g., using aptly, which internally depends on plain HTTP), may I ask:
- Is the Debian Snapshot Team aware of this 2 GB limitation as an explicit design choice?
- Are there recommended ways to mitigate or overcome this limitation officially that maybe I overlooked?
- Would you consider revising this limitation-or at least explicitly documenting this prominent limitation and explaining recommended alternative methods or infrastructure?
Since snapshot.d.o explicitly archives Debian historical data for reproducibility and long-term access, not allowing retrieval of archived files explicitly hosted there seems unexpected. Users reasonably expect any file listed by snapshot.d.o to be available for download.
Thank you for clarifying and considering my feedback.
Best regards,
Christof Warlich
P.S.: Attached is a log that reproduces the issue in a (slightly adapted, see log) debian:bookworm Docker container:
root@a41c9c7d7fad:/# cat /etc/apt/apt.conf.d/80snapshot Acquire::http::Dl-Limit "1000"; root@a41c9c7d7fad:/# cat /etc/apt/sources.list.d/debian.sources
Types: deb deb-src
# http://snapshot.debian.org/archive/debian/20250407T000000Z
URIs: http://snapshot.debian.org/archive/debian/20250407T000000Z
#URIs: http://deb.debian.org/debian
Suites: bookworm bookworm-updates
Components: main
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
Types: deb
# http://snapshot.debian.org/archive/debian-security/20250407T000000Z
URIs: http://deb.debian.org/debian-security
Suites: bookworm-security
Components: main
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
root@a41c9c7d7fad:/# apt update -o Acquire::Check-Valid-Until=false
Hit:1 http://snapshot.debian.org/archive/debian/20250407T000000Z bookworm InRelease
Hit:3 http://deb.debian.org/debian-security bookworm-security InRelease
Hit:2 http://snapshot.debian.org/archive/debian/20250407T000000Z bookworm-updates InRelease Reading package lists... Done Building dependency tree... Done Reading state information... Done
3 packages can be upgraded. Run 'apt list --upgradable' to see them.
root@a41c9c7d7fad:/# apt-get source texlive-extra Reading package lists... Done
NOTICE: 'texlive-extra' packaging is maintained in the 'Git' version control system at:
https://github.com/debian-tex/texlive-nonbin.git [texlive-extra] Please use:
git clone https://github.com/debian-tex/texlive-nonbin.git [texlive-extra] to retrieve the latest (possibly unreleased) updates to the package.
Skipping already downloaded file 'texlive-extra_2022.20230122-4.dsc'
Skipping already downloaded file 'texlive-extra_2022.20230122.orig-tex4ht.tar.xz'
Skipping already downloaded file 'texlive-extra_2022.20230122-4.debian.tar.xz'
Need to get 2286 MB of source archives.
Ign:1 http://snapshot.debian.org/archive/debian/20250407T000000Z bookworm/main texlive-extra 2022.20230122-4 (tar)
Ign:1 http://snapshot.debian.org/archive/debian/20250407T000000Z bookworm/main texlive-extra 2022.20230122-4 (tar)
Ign:1 http://snapshot.debian.org/archive/debian/20250407T000000Z bookworm/main texlive-extra 2022.20230122-4 (tar)
Err:1 http://snapshot.debian.org/archive/debian/20250407T000000Z bookworm/main texlive-extra 2022.20230122-4 (tar)
503 Response object too large [IP: 199.232.30.132 80]
W: Download is performed unsandboxed as root as file 'texlive-extra_2022.20230122.orig.tar.xz' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)
E: Failed to fetch http://snapshot.debian.org/file/0a6a9424fc7a1812a533fcc4c9b779d535057d7a/texlive-extra_2022.20230122.orig.tar.xz 503 Response object too large [IP: 199.232.30.132 80]
E: Failed to fetch some archives.
root@a41c9c7d7fad:/#
Reply to: