Re: A question about deleting a big file structure from a big disk in Jessie: Why does this work? I'm really worried.
On 20150402_2135-0500, David Wright wrote:
> Quoting Paul E Condon (firstname.lastname@example.org):
> > Some time ago I decided to a make a copy of these data,
> > so I would have more than one copy.
> Is the copying between a USB disk and an internal, or between two
> partitions on the same USB disk, or between two USB disks? (Ranked in
> decreasing reliability in my own experience.)
> > When I tried, the job would always crash well before completion.
> What are the symptoms of a crash? (Hang, segfault, write-failure
> as readonly, etc)
Switch of target disk to read-only mount.
> > But in both cases the
> > deletion failed because 'gfx2' has been remounted read-only, which
> > makes it impossible to update the target directory tree.
> Do you watch /var/log/kern.log which this is going on. I find that
> quite useful. For example, messages like
> usb 1-8: reset high-speed USB device number 5 using ehci_hcd
> are accompanied by a pause of anything up to a minute in file
> transfer. I get these quite frequently if I do massive copies between
> two USB disks, so I now stage such copies through the internal disk.
Thanks. I do watch my own capture of the file descriptors 1 and 2 into
a file in /var/pec/ (sub-dir name, pec, is my initials). This will be
a useful addition, I'm sure.
In my system, most of my hypothesizing is from observing coincident
changes in two or more of the nine (soon to be ten) windows that I
monitor on system 'big'
> I'm not so unlucky as Bob appears to be (he says, touching wood), but
I think Bob came to his conclusion during a previous period of
instability in Debian, but rather than start an argument that can only
degenerate trying to score debating point, I want to gather more
date. Bob has already helped me by making a truly useful suggestion,
for which I thank him.
It's getting late here. I won't get anything useful done tonight.
I'll just start making mistakes, if I start something new now.
May you both have a good night,
> I do get occasional I/O errors on USB transfers, which can make the
> disk readonly, but sometimes make it disappear altogether (ie it
> gets unmounted, not remounted).
All of my file systems are journaled. Did you notice a delay in remount
as the journal was replayed?
> > I have not tried it, but from my investigation I'm sure that a
> > massive delete of some obsolete file structure from the HD that
> > was /dev/sda1 during Debian install would trigger a remount-ro,
> > which surely would lead to a system crash in short order.
> You get streams of error messages (like when the disk fills up)
> but it shouldn't actually crash.
> (OTOH when you get a kernel error, there can be circumstances where
> the system will panic and *not* sync/write to the disk because to do
> so could cause corruption.)
So it helps to know about data that can be ignored for a legitimate
> > I'm worried about what I found. I want to interest someone who has far
> > more knowledge about how the kernel actually works internally to look
> > into this. I done other experiments more complicated to report, I can't
> > find anything comforting about this situation. If you think it's OK,
> > you probably don't understand, IMHO.
> My prejudices, based on no more than observations of my system, make me,
> like Bob, suspect the interface rather than the kernel. My wife,
> running windows, sees similar external symptoms (pauses, errors),
> though neither of us would know how to observe them in like manner to
I like to play the scientist in my old age. All theories begin with curiosity
about a random observation. A classic case is the observation of a pocket
watch lieing on the path during a walk in a park. In our case of hypotheses
about the design of the kernel, it is unacceptable to me to invoke the idea
of a universal creator, and perfectly OK to contemplate the possibility of
a flaw in the design.
> Just in passing, if clamav wakes up and spots the USB drive, file
clamav is terra incognita to me.
> transfers can stop for 10 or 15 minutes; the USB disk heads will still
> be very active. I keep an xterm running top so I can spot that (and
> other cpu-guzzlers like xulrunner).
> Oh, and to David C, this happens irrespective of wheezy or jessie
> (for me).
Paul E Condon