Bug#760473: marked as done (apt method http uses 100% cpu with high transfer rate)
Your message dated Tue, 18 Aug 2015 08:56:12 +0000
with message-id <83e8d67cc94c025ce7dc578080b58571@rainloop>
and subject line Re: Bug#760473: Further analysis and a patch
has caused the Debian Bug report #760473,
regarding apt method http uses 100% cpu with high transfer rate
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@bugs.debian.org
immediately.)
--
760473: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=760473
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems
--- Begin Message ---
- To: Debian Bug Tracking System <submit@bugs.debian.org>
- Subject: apt method http uses 100% cpu with high transfer rate
- From: Lennart Weller <weller@bfw-online.de>
- Date: Thu, 04 Sep 2014 13:53:35 +0200
- Message-id: <20140904115335.31328.34716.reportbug@lar.bfw.de>
Package: apt
Version: 1.0.7
Severity: normal
Dear Maintainer,
The current implementation of the http method puts a much higher load
on the CPU than a similar call done in wget or curl.
As an example I downloaded a large file from the deb mirror (0ad-data)
which is then cached by local squid. Which in turn then provides higher transfer
rates in case I download it again and causing this bug to appear:
cli:
% apt-get download 0ad-data
Get:1 http://ftp.de.debian.org/debian/ jessie/main 0ad-data all 0.0.16-1 [530 MB]
46% [1 0ad-data 247 MB/530 MB 46%] 24.6 MB/s 11s
top:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
8516 weller 20 0 5516 4200 3944 R 99.4 0.0 0:04.60 http
For comparison the same download with wget:
cli:
% wget http://ftp.de.debian.org/debian/pool/main/0/0ad-data/0ad-data_0.0.16-1_all.deb
(58.5 MB/s) - '0ad-data_0.0.16-1_all.deb' saved [530253138/530253138]
top:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
12578 weller 20 0 5992 4104 3712 R 18.2 0.0 0:00.57 wget
As you can see from these lines the download is capped by the http
method to almost 40% of the transfer rate due to the single core CPU usage.
My guess is that this problem stems from the circular buffer design used
in the http method which might've provided a speed boost back in the day
but hinders it now.
/proc/cpuinfo:
processor : 31
model name : Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz
cpu MHz : 2328.042
-- Package-specific info:
-- (/etc/apt/preferences present, but not submitted) --
-- (/etc/apt/sources.list present, but not submitted) --
-- System Information:
Debian Release: jessie/sid
APT prefers testing
APT policy: (550, 'testing'), (500, 'stable'), (210, 'unstable'), (1, 'experimental')
Architecture: i386 (x86_64)
Kernel: Linux 3.15.3-64+ (SMP w/32 CPU cores; PREEMPT)
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)
Shell: /bin/sh linked to /bin/dash
Versions of packages apt depends on:
ii debian-archive-keyring 2012.4
ii gnupg 1.4.18-2
ii libapt-pkg4.12 1.0.7
ii libc6 2.19-9bfw1
ii libgcc1 1:4.9.1-4bfw1
ii libstdc++6 4.9.1-4bfw1
apt recommends no packages.
Versions of packages apt suggests:
pn apt-doc <none>
ii aptitude 0.6.11-1bfw1
ii dpkg-dev 1.17.13
ii python-apt 0.9.3.9
-- Configuration Files:
/etc/logrotate.d/apt changed [not included]
-- no debconf information
--- End Message ---
--- Begin Message ---
August 18 2015 10:15 AM, "Julian Andres Klode" <jak@debian.org> wrote:
> Also, the existing code works and is reasonably fast, so replacing
> it is IMHO not a good idea. If we can reach 30 or 23 MB/s does
> not really matter, there is no reasonable gain here (1 GB takes
> 33 seconds at 30 MB/s, or 44 seconds at 23 MB/s, and updates
> are usually *much* smaller).
>
> So don't waste your time, we're not going to merge any patch adding
> a dependency.
I won't.
--- End Message ---
Reply to: