--- Begin Message ---
- To: Debian Bug Tracking System <submit@bugs.debian.org>
- Subject: bittorrent: Patch to add a global maxium upload cap
- From: Daniel Dickinson <cshore@fionavar.ca>
- Date: Sat, 24 May 2008 13:30:50 -0400
- Message-id: <20080524173050.21215.46325.reportbug@brennin.fionavar.dd>
Package: bittorrent
Version: 3.4.2-11
Severity: wishlist
Tags: patch
Here is a patch that adds a global maxium upload rate cap. This ensures that bittorrent never uses more than --global_max_upload_rate kB/s regardless of how many torrents are active. It is also adaptive, so if three torrents are upload, the cap is divided among them, and if increases or decreases, the cap is modified to reflect the number of uploading torrents. In addition, if there are active torrents that would use less than the cap they would be given, the 'extra' is divided among the torrents that will exceed their simple division caps.
I have also updated the man page for btlaunchmany to reflect this change.
-- System Information:
Debian Release: lenny/sid
APT prefers testing
APT policy: (500, 'testing'), (500, 'stable'), (1, 'experimental'), (1, 'unstable'), (1, 'testing'), (1, 'stable')
Architecture: i386 (i686)
Kernel: Linux 2.6.24-1-686 (SMP w/1 CPU core)
Locale: LANG=en_CA.UTF-8, LC_CTYPE=en_CA.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash
Versions of packages bittorrent depends on:
ii lsb-base 3.2-11 Linux Standard Base 3.2 init scrip
ii python 2.5.2-1 An interactive high-level object-o
ii python-support 0.7.7 automated rebuilding support for P
Versions of packages bittorrent recommends:
ii mime-support 3.40-1.1 MIME files 'mime.types' & 'mailcap
diff -Naur bittorrent-3.4.2-11-pristine/BitTorrent/Connecter.py bittorrent-3.4.2/BitTorrent/Connecter.py
--- bittorrent-3.4.2-11-pristine/BitTorrent/Connecter.py 2004-03-30 00:41:36.000000000 -0500
+++ bittorrent-3.4.2/BitTorrent/Connecter.py 2008-05-23 20:49:10.000000000 -0400
@@ -103,6 +103,20 @@
def _update_upload_rate(self, amount):
self.totalup.update_rate(amount)
+
+ # Only do global rate throttling if the global maxium and the current
+ # global rate are not None
+ if global_max_upload_rate != None and global_rate != None:
+ # If we need to throttle
+ if global_max_upload_rate > 0:
+ if global_max_upload_rate <= global_rate:
+ # Set the per-torrent maxium as calculated by btlaunchmany
+ self.max_upload_rate = global_slice
+ else:
+ self.max_upload_rate = global_max_upload_rate
+ else:
+ self.max_upload_rate = 0
+
if self.max_upload_rate > 0 and self.totalup.get_rate_noupdate() > self.max_upload_rate:
self.rate_capped = True
self.sched(self._uncap, self.totalup.time_until_rate(self.max_upload_rate))
diff -Naur bittorrent-3.4.2-11-pristine/btlaunchmany.py bittorrent-3.4.2/btlaunchmany.py
--- bittorrent-3.4.2-11-pristine/btlaunchmany.py 2008-05-22 21:58:42.000000000 -0400
+++ bittorrent-3.4.2/btlaunchmany.py 2008-05-23 08:58:43.000000000 -0400
@@ -15,6 +15,13 @@
from sys import argv, stdout, exit
from time import sleep
import traceback
+import getopt
+from BitTorrent import Connecter
+import locale
+
+Connecter.global_max_upload_rate = 0
+Connecter.global_rate = 1
+Connecter.global_slice = Connecter.global_max_upload_rate
def dummy(*args, **kwargs):
pass
@@ -80,6 +87,81 @@
deadfiles.remove(file)
sleep(1)
+def calc_slice(calckiller):
+ """Calculate bandwitch cap for each active torrent"""
+ global threads
+
+ # Keep going until main program terminates this thread
+ while not calckiller.isSet():
+ # Base calculation only on torrents that actually have traffic
+ active_uploads = 0
+ # Total of global_max_rate that would be unused if we simply
+ # divided global_max_rate by the number of active torrents (because
+ # some torrents aren't using the full rate that they would be allowed)
+ global_under_slice = 0
+ # Total upload rate
+ totalup = 0
+
+ # For each torrent
+ for file, threadinfo in threads.items():
+ uprate = threadinfo.get('uprate', 0)
+ # add torrents upload rate to the total
+ totalup += uprate
+
+ # If the torrent is uploading it is considered active, otherwise
+ # ignored for per-torrent rate cap calculation
+ if uprate > 0:
+ active_uploads += 1
+
+ # Connecter is where the throttling happens, so it needs to know if
+ # we are exceeding the global maximum upload rate. The maximum rate
+ # is defined elsewhere. This tells Connecter the current total upload
+ # rate
+ Connecter.global_rate = totalup
+
+ # Calculate how much we are over the global max rate
+ global_over_max = Connecter.global_max_upload_rate - totalup
+
+ # Calculate the total bandwith that would be unused by simple division
+ global_under_slice = 0
+
+ # Number of active torrents that are under the simple division slice
+ num_under_slice = 0
+
+ # Only cap if we are exceeding the global max rate
+ if active_uploads > 0:
+ # Simple division slice
+ max_slice = Connecter.global_max_upload_rate / active_uploads
+
+ # If we're not exceeding global maximum rate, per-torrent max rate
+ # becomes global max rate (until excess happens)
+ else:
+ max_slice = Connecter.global_max_upload_rate
+
+ # For each torrent
+ for file, threadinfo in threads.items():
+ uprate = threadinfo.get('uprate', 0)
+
+ # If the simple division slice is greater than bandwidth we're using
+ if max_slice > uprate:
+ # add the amount we're not using the total we're not using
+ global_under_slice += max_slice - uprate
+ # count number of torrents that are under simple division slice
+ num_under_slice += 1
+
+ # If there is a torrent under the simple division slice
+ if num_under_slice > 0 and active_uploads > num_under_slice:
+ # Increase the maximum slice by the bandwith under slice divided by
+ # the number of downloads that are over or equal to the slice
+ # Basically every torrent that is over slice gets an equal share of
+ # what would be unused otherwise
+ Connecter.global_slice = max_slice + global_under_slice / (active_uploads - num_under_slice)
+ else:
+ # If there are no torrents under slice, slice is simple division
+ # slice
+ Connecter.global_slice = max_slice
+ sleep(1)
+
def display_thread(displaykiller):
interval = 1.0
global threads, status
@@ -210,13 +292,44 @@
print """Usage: btlaunchmany.py <directory> <global options>
<directory> - directory to look for .torrent files (non-recursive)
<global options> - options to be applied to all torrents (see btdownloadheadless.py)
+ --global_max_upload_rate <rate> - combined maximum upload rate in kB/s (optional)
"""
exit(-1)
+
+ # We need to determine the global_upload_rate
+ # So parse the command line after the torrent directory URL
+
+ # If we have global maximum upload rate
+ if len(argv) > 3:
+ if argv[2] == "--global_max_upload_rate":
+ # convert the rate to an integer representing KB/s
+ Connecter.global_max_upload_rate = locale.atoi(argv[3]) * 1024
+ # Remove --global_max_upload_rate rate from the argument list
+ # so that download.py (the bittorrent downloader, which is used by
+ # btlaunchmany to do the actual downloads) doesn't get this
+ # parameter, which it doesn't understand
+ del argv[3]
+ del argv[2]
+
try:
+ # Add event that allows us to cancel updating display when
+ # we want to exit this program
displaykiller = Event()
+ # Create updating display thread and start it
displaythread = Thread(target = display_thread, name = 'display', args = [displaykiller])
displaythread.start()
+ # Add event that allows us to cancel global slice calculation when
+ # we want to exit this program
+ calckiller = Event()
+ # Create global slice calculation thread ands start it
+ calcslicethread = Thread(target = calc_slice, name='calc_slice', args = [calckiller])
+ calcslicethread.start()
+ # Execute the main action loop, which checks for new torrent files and
+ # checks what has already been downloaded, downloads and uploads the
+ # torrent
dropdir_mainloop(argv[1], argv[2:])
+
+ # Until interrupted by Ctrl-C or kill -INT
except KeyboardInterrupt:
print '^C caught! Killing torrents..'
for file, threadinfo in threads.items():
@@ -224,7 +337,15 @@
threadinfo['kill'].set()
threadinfo['thread'].join()
del threads[file]
+ # Kill display thread
displaykiller.set()
displaythread.join()
+ # Kill slice calcuation thread
+ calckiller.set()
+ calcslicethread.join()
+ # and exit
+
+ # if there was an uncaught exception
except:
+ # print a traceback and exit
traceback.print_exc()
diff -Naur bittorrent-3.4.2-11-pristine/debian/bittorrent-multi-downloader.bittorrent.1 bittorrent-3.4.2/debian/bittorrent-multi-downloader.bittorrent.1
--- bittorrent-3.4.2-11-pristine/debian/bittorrent-multi-downloader.bittorrent.1 2008-05-22 21:58:42.000000000 -0400
+++ bittorrent-3.4.2/debian/bittorrent-multi-downloader.bittorrent.1 2008-05-23 07:19:25.000000000 -0400
@@ -20,6 +20,12 @@
These programs have the exact same options as the normal
downloaders, which are documented in \fBbittorrent-downloader\fP(1).
+.PP
+Except that \fBbtlaunchmany\fP(1) also as the following option:
+.TP
+.B \-\-global_max_upload_rate \fIrate\fP
+Cap the upload rate for the combined total of all active torrents to
+\fIrate\fP (in KB/s).
.SH SEE ALSO
.BR bittorrent-downloader (1),
diff -Naur bittorrent-3.4.2-11-pristine/debian/changelog bittorrent-3.4.2/debian/changelog
--- bittorrent-3.4.2-11-pristine/debian/changelog 2008-05-22 21:58:42.000000000 -0400
+++ bittorrent-3.4.2/debian/changelog 2008-05-23 07:07:08.000000000 -0400
@@ -1,3 +1,9 @@
+bittorrent (3.4.2-11.1~dfd3) unstable; urgency=low
+
+ * Add global upload rate throttle.
+
+ -- Daniel Dickinson <cshore@fionavar.ca> Fri, 23 May 2008 07:07:10 -0500
+
bittorrent (3.4.2-11) unstable; urgency=low
* Add LSB logging functionality. (thanks David!) Closes: #384724
--- End Message ---