[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: rm -rf is too slow on large files and directory structure(Around 30000)



Bilal mk wrote:
> I tried to remove 5GB directory. In that directory around 30000 files and
> directory. It will take more than 30 min to complete.

A large number of files consuming a large number of blocks will take a
significant amount of time to process.  That is all there is to it.

Some filesystems are faster than others.  What filesystem are you
using?  On what type of cpu?

If you happen to be destroying an entire filesystem then you could
simply destroy the entire filesystem by unmounting it and then making
a new filesystem on top of it..

> There is no other cpu intensive process running. After sometime it goes to
> D state and unbale to kii that process.

If you have processes stuck in the D state (uninterruptible sleep)
then something bad has happened.  This would indicate a bug.

It sounds like you are having kernel bugs.  You may need to fsck your
filesystems.  I would double check that dma is enabled to your drives.

> I have also tried find with xargs method to remove. It will also take long
> time to complete
> find /directory | xargs rm -rf

I doubt the problem is in rm since it has already been optimized to be
quite fast.  The newer versions have even more optimization.  But it
isn't worth the trouble to do anything other than wait.  Most of the
time will be spent in the kernel organizing the now free blocks.

If you want to experiment you could try find.

  find /directory -depth -delete

That is basically the same as rm -rf but using find only.

Bob

Attachment: signature.asc
Description: Digital signature


Reply to: