Paul E Condon wrote: > For several years I have been making daily backups of my four Debian > computers using Rsync and a small script of my own devising. The data > has been accumulating on an external USB drive in a partition with the >... > I'm worried about what I found. I want to interest someone who has far > more knowledge about how the kernel actually works internally to look > into this. I done other experiments more complicated to report, I can't > find anything comforting about this situation. If you think it's OK, > you probably don't understand, IMHO. I often have problems with USB mounted file systems. I believe the cheap nature of the USB hardware all around to be the major contributor. I do use USB for "a big floppy" all of the time. But whenever I keep a USB disk mounted for a long time it has always failed after a while. A month. Six months. I find the USB file system subsystem unreliable. I would never trust it for critical data such as backups. I think you are seeing the same unreliable mounted USB disk problems that I have seen for a long time. If you remove the disk from your USB container and mount it directly with its native SATA (or IDE) connector then you will find that it is as reliable as the rest of the native storage. I blame the cheap USB controller electronics. Although perhaps the kernel driver is also to blame in there too. [On the converse I find USB network adapters and USB sound cards to have been rock solid. Meaning that while I avoid USB disks I actively use USB networking on several machines to add additional NICs. I am planning another site using additional USB NICs. It is probably hardware dependent but they have been working great for me regardless of seeing the opposite for disks. And I have three sites using USB sound cards very robustly. Since I disparaged USB disk I felt I needed to clarify that it was only disk and not other USB.] > I found two other ways to delay the crash: > 1) using nice as in: ' nice -n 19 find -depth -print -delete' > (this, I think, slows down the main running job in relation to the > running of the kernel.) Read the man page for "ionice". You might consider using it instead of nice. Nice works with cpu usage. But ionice works with I/O usage and is directly what you are fighting. You might try: ionice -c 3 find -depth -print -delete If I am deleting an entire file system I will usually simply mkfs on top and reset it to empty that way. On a file system with millions of files that will be much faster than deleting them individually. Obviously only works if it is the entire file system. Bob
Attachment:
signature.asc
Description: Digital signature