[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: A question about deleting a big file structure from a big disk in Jessie: Why does this work? I'm really worried.

Paul E Condon wrote:
> For several years I have been making daily backups of my four Debian
> computers using Rsync and a small script of my own devising. The data
> has been accumulating on an external USB drive in a partition with the
> I'm worried about what I found. I want to interest someone who has far
> more knowledge about how the kernel actually works internally to look
> into this. I done other experiments more complicated to report, I can't
> find anything comforting about this situation. If you think it's OK,
> you probably don't understand, IMHO.

I often have problems with USB mounted file systems.  I believe the
cheap nature of the USB hardware all around to be the major
contributor.  I do use USB for "a big floppy" all of the time.  But
whenever I keep a USB disk mounted for a long time it has always
failed after a while.  A month.  Six months.  I find the USB file
system subsystem unreliable.  I would never trust it for critical data
such as backups.  I think you are seeing the same unreliable mounted
USB disk problems that I have seen for a long time.

If you remove the disk from your USB container and mount it directly
with its native SATA (or IDE) connector then you will find that it is
as reliable as the rest of the native storage.  I blame the cheap USB
controller electronics.  Although perhaps the kernel driver is also to
blame in there too.

[On the converse I find USB network adapters and USB sound cards to
have been rock solid.  Meaning that while I avoid USB disks I actively
use USB networking on several machines to add additional NICs.  I am
planning another site using additional USB NICs.  It is probably
hardware dependent but they have been working great for me regardless
of seeing the opposite for disks.  And I have three sites using USB
sound cards very robustly.  Since I disparaged USB disk I felt I
needed to clarify that it was only disk and not other USB.]

> I found two other ways to delay the crash:
> 1) using nice as in: ' nice -n 19 find -depth -print -delete'
>    (this, I think, slows down the main running job in relation to the
>    running of the kernel.)

Read the man page for "ionice".  You might consider using it instead
of nice.  Nice works with cpu usage.  But ionice works with I/O usage
and is directly what you are fighting.  You might try:

  ionice -c 3 find -depth -print -delete

If I am deleting an entire file system I will usually simply mkfs on
top and reset it to empty that way.  On a file system with millions of
files that will be much faster than deleting them individually.
Obviously only works if it is the entire file system.


Attachment: signature.asc
Description: Digital signature

Reply to: