Re: Delete 4 million files

[Ron Johnson <ron.l.johnson@cox.net>]

On 2009-03-25 18:50, Paul E Condon wrote:
> Top post because it is OT, but related. Yesterday I had occasion to
> remove some old data from my system. I had a backup copy, and the data
> was all on a single partition. I could have simply rewritten the file
> system with mke2fs, but I decided, instead, to use rm -rf * at the top
> level of the file structure.  There were about 25million files on an
> 7 GB partition on a 60 GB IDE drive. It took rm a little over an hour
> to unlink it all.
>
> (Yes, I know that unlinking doesn't get rid of the data. I wiped it
> because I didn't want to leave it in place and forget that it is the
> old data of which I have a perfectly good archival copy. Not security,
> but good housekeeping.)

http://www.cs.auckland.ac.nz/~pgut001/pubs/secure_del.html

After reading Gutmann's epilogues of his 1996 paper, I wrote a 
simple freespace wiper that runs a lot faster than sfill from 
secure-delete.

$ cat /home/me/sfill.py
#!/usr/bin/python

import sys, statvfs, os
import time
import traceback
import math

__buff = ''

def write_buff(f, buff_size):
    global __buff
    stats = os.statvfs(loc)
    free_bytes = stats[statvfs.F_BSIZE] * stats[statvfs.F_BFREE]
    if free_bytes < buff_size:
        __buff = chr(0xFF) * free_bytes
    try:
        f.write(__buff)
        f.flush()
    except:
        print 'write error'
        traceback.print_exc(file=sys.stdout)
        sys.exit()
    return free_bytes - buff_size

try:
    loc = sys.argv[1]
except:
    print 'need a mount point!!!'
    sys.exit(1)

try:
    stats = os.statvfs(loc)
except:
    print loc + ' is an invalid mount point'
free_bytes = stats[statvfs.F_BSIZE] * stats[statvfs.F_BFREE]
print 'free bytes = ', free_bytes

try:
    buff_size = int(sys.argv[2]) * 2**20
except:
    buff_size = 16 * 2**20
__buff = chr(0xFF) * buff_size

f = open(loc+'/oooooooo.ooo', 'wb')

start_tm = time.time()
start_st = time.localtime(start_tm)
print 'Starting at ', time.strftime('%Y-%m-%d %H:%M:%S', start_st)

c = 0
bytes_free = free_bytes
try:
    while bytes_free >= 0:
        bytes_free = write_buff(f, buff_size)
        c += 1
        if int(math.fmod(c, 500)) == 0:
            print \
                 '%s %8d %16d' % \
                 (time.strftime('%Y-%m-%d %H:%M:%S'), \
                 c, free_bytes - bytes_free)
except:
    print 'Terminating prematurely'
    traceback.print_exc(file=sys.stdout)

f.close()

os.remove(loc+'/oooooooo.ooo')

end_tm = time.time()
end_st = time.localtime(end_tm)
print 'Stopping at ', time.strftime('%Y-%m-%d %H:%M:%S', end_st)
elapsed = end_tm - start_tm
print 'Elapsed minutes: %10.2f' % (elapsed*1.0/60,)


--
Ron Johnson, Jr.
Jefferson LA  USA

"Freedom is not a license for anarchy."