[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#482478: marked as done (bittorrent: btlaunchmany checks all files at once which hits the system hard)



Your message dated Fri, 18 Oct 2019 04:23:38 +0000
with message-id <E1iLJne-000Co3-DG@fasolo.debian.org>
and subject line Bug#936210: Removed package(s) from unstable
has caused the Debian Bug report #482478,
regarding bittorrent: btlaunchmany checks all files at once which hits the system hard
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@bugs.debian.org
immediately.)


-- 
482478: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=482478
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems
--- Begin Message ---
Package: bittorrent
Version: 3.4.2-11
Severity: normal
Tags: patch


btlaunchmany does parallel checks of all the files in the directory at once, which if you have many files results in a huge hit to the system, with 100% CPU and max disk, plus the load average steadily climbs to a high level.  The attached patch limits checking to one file at a time, which is still max CPU and large disk, but the system is not overloaded.

The important line is line #199 in the patched file:

        if self.activity != 'checking existing file' and self.activity != 'disk check' and self.checking:

The rest of the patch are the deadfile removal and checking locking fixes I submitted a short while ago.  I don't think they are actually needed for this bug, but I haven't tested without the full patch.


-- System Information:
Debian Release: lenny/sid
  APT prefers testing
  APT policy: (500, 'testing'), (500, 'stable'), (1, 'experimental'), (1, 'unstable'), (1, 'testing'), (1, 'stable')
Architecture: i386 (i686)

Kernel: Linux 2.6.24-1-686 (SMP w/1 CPU core)
Locale: LANG=en_CA.UTF-8, LC_CTYPE=en_CA.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages bittorrent depends on:
ii  lsb-base                      3.2-11     Linux Standard Base 3.2 init scrip
ii  python                        2.5.2-1    An interactive high-level object-o
ii  python-support                0.7.7      automated rebuilding support for P

Versions of packages bittorrent recommends:
ii  mime-support                  3.40-1.1   MIME files 'mime.types' & 'mailcap
--- bittorrent-3.4.2/btlaunchmany.py	2008-05-22 23:49:35.000000000 -0400
+++ bittorrent-3.4.2-11-pristine/btlaunchmany.py	2008-05-22 21:58:42.000000000 -0400
@@ -25,23 +25,19 @@
 filecheck = Lock()
 
 def dropdir_mainloop(d, params):
-    global filecheck
     deadfiles = []
     global threads, status
     while 1:
         files = listdir(d)
         # new files
         for file in files: 
-            if file[-len(ext):] == ext:                
+            if file[-len(ext):] == ext:
                 if file not in threads.keys() + deadfiles:
-                    if filecheck.acquire(0):
-                        threads[file] = {'kill': Event(), 'try': 1}
-                        print 'New torrent: %s' % file
-                        stdout.flush()
-                        status_updater = StatusUpdater(join(d, file), params, file)
-                        status_updater.checking = 1
-                        threads[file]['thread'] = Thread(target = status_updater.download, name = file)
-                        threads[file]['thread'].start()
+                    threads[file] = {'kill': Event(), 'try': 1}
+                    print 'New torrent: %s' % file
+                    stdout.flush()
+                    threads[file]['thread'] = Thread(target = StatusUpdater(join(d, file), params, file).download, name = file)
+                    threads[file]['thread'].start()
         # files with multiple tries
         for file, threadinfo in threads.items():
             if threadinfo.get('timeout') == 0:
@@ -58,15 +54,15 @@
                 # if it was checking the file, it isn't anymore.
                 if threadinfo.get('checking', None):
                     filecheck.release()
-                    if threadinfo.get('try') == 6: 
-                        # Died on the sixth try? You're dead.
-                        deadfiles.append(file)
-                        print '%s died 6 times, added to dead list' % fil
-                        stdout.flush()
-                        del threads[file]
-                    else:
-                        del threadinfo['thread']
-                        threadinfo['timeout'] = 10
+                if threadinfo.get('try') == 6: 
+                    # Died on the sixth try? You're dead.
+                    deadfiles.append(file)
+                    print '%s died 6 times, added to dead list' % fil
+                    stdout.flush()
+                    del threads[file]
+                else:
+                    del threadinfo['thread']
+                    threadinfo['timeout'] = 10
             # dealing with files that dissapear
             if file not in files:
                 print 'Torrent file dissapeared, killing %s' % file
@@ -77,11 +73,11 @@
                 # if this thread was filechecking, open it up
                 if threadinfo.get('checking', None): 
                     filecheck.release()
-                    del threads[file]
-                for file in deadfiles:
-                    # if the file dissapears, remove it from our dead list
-                    if file not in files: 
-                        deadfiles.remove(file)
+                del threads[file]
+        for file in deadfiles:
+            # if the file dissapears, remove it from our dead list
+            if file not in files: 
+                deadfiles.remove(file)
         sleep(1)
 
 def display_thread(displaykiller):
@@ -166,15 +162,14 @@
         if saveas == '': 
             saveas = default
         # it asks me where I want to save it before checking the file.. 
-#        if exists(self.file[:-len(ext)]) and (getsize(self.file[:-len(ext)]) > 0):
-#            filecheck.release(0)
-#            # file will get checked
-#            while (not filecheck.acquire(0) and not self.myinfo['kill'].isSet()):
-#                self.myinfo['status'] = 'disk wait'
-#                sleep(0.1)
-#            if not self.myinfo['kill'].isSet():
-#                self.checking = 1
-#                self.myinfo['checking'] = 1
+        if exists(self.file[:-len(ext)]) and (getsize(self.file[:-len(ext)]) > 0):
+            # file will get checked
+            while (not filecheck.acquire(0) and not self.myinfo['kill'].isSet()):
+                self.myinfo['status'] = 'disk wait'
+                sleep(0.1)
+            if not self.myinfo['kill'].isSet():
+                self.checking = 1
+                self.myinfo['checking'] = 1
         self.myinfo['savefile'] = self.file[:-len(ext)]
         return self.file[:-len(ext)]
     
@@ -196,7 +191,7 @@
             self.myinfo['status'] = '%s %.0f%%' % (self.activity, fractionDone * 100)
         else:
             self.myinfo['status'] = self.activity
-        if self.activity != 'checking existing file' and self.activity != 'disk check' and self.checking:
+        if self.activity != 'checking existing file' and self.checking:
             # we finished checking our files. 
             filecheck.release()
             self.checking = 0

--- End Message ---
--- Begin Message ---
Version: 3.4.2-12+rm

Dear submitter,

as the package bittorrent has just been removed from the Debian archive
unstable we hereby close the associated bug reports.  We are sorry
that we couldn't deal with your issue properly.

For details on the removal, please see https://bugs.debian.org/936210

The version of this package that was in Debian prior to this removal
can still be found using http://snapshot.debian.org/.

This message was generated automatically; if you believe that there is
a problem with it please contact the archive administrators by mailing
ftpmaster@ftp-master.debian.org.

Debian distribution maintenance software
pp.
Scott Kitterman (the ftpmaster behind the curtain)

--- End Message ---

Reply to: