Re: door bell like sound effect

[Karl Vogel <vogelke@pobox.com>]

On Wed, Aug 30, 2023 at 07:55:14AM -0400, songbird wrote:
> Karl Vogel wrote:
> ...
> > If nothing else, it's faster to run "locate" and look for file extensions;
> > running "file" on that much crap took nearly 9 hours.
> 
> do you have SSDs or spinning rust?

  I have a 256-Gb SSD and two mirrored Western Digital Blue 1.8-Tb drives.
  About 2 million files are on SSD and the rest are on rust.

  I used "file" v5.45 built from source, which does a nice job but is IO-
  and CPU-intensive.

> when i just did this:
>     # find / -type f | wc -l
> it took all of 24 seconds for the 2.4 million files found.

  Generating hashes for SSD files is faster than getting the filetype;
  it takes about 17 minutes for 3.6 million files (153 Gbytes).  I like
  the Blake-2 hash cuz it's fast as hell, among other things:

    #!/bin/ksh
    #<zroot-hash: run Blake hash on all zroot dataset files

    export PATH=/usr/local/bin:/bin:/usr/bin
    tag=${0##*/}
    set -o nounset
    umask 022

    logmsg () { logger -t "$tag" "$@"; }
    die ()    { logmsg "FATAL: $@"; exit 1; }

    work=$(mktemp -q "/tmp/$tag.work.XXXXXX")
    case "$?" in
        0)  test -f "$work" || die "$work: tmp list file not found" ;;
        *)  die "can't create work file" ;;
    esac

    # Get a list of all regular files on SSD.

    mount | grep '^zroot' | awk '{print $3}' |
      while read dataset
      do
          logmsg "listing $dataset"
          find "$dataset" -xdev -type f -print0 >> $work
      done

    # Store hashes for SSD datasets.
    # The hash file is sorted by filename to make comparisons easier.

    logmsg "running b2sum"
    fdbdir=$(date '+/var/fdb/%Y/%m%d')
    sort -z $work | xargs -0r b2sum -l 128 > "$fdbdir/zroot.sum"
    rm $work
    exit 0

  Useful for finding changed files -- security, backups, etc.

> what script did you use?

    #!/bin/ksh
    #<ftype: get a sampling of filetypes for all SSD filesystems, /src.

    export PATH=/usr/local/bin:/bin:/usr/bin
    set -o nounset
    tag=${0##*/}
    umask 022

    logmsg () { logger -t "$tag" "$@"; }

    work="/tmp/$tag.$$"
    fsys="/ /doc /home /usr/local /search /usr/src /dist /src"

    logmsg start
    find $fsys -xdev -print0 | xargs -0 file -N --mime-type > $work
    logmsg finish

    mv $work filetypes
    exit 0

-- 
Karl Vogel                      I don't speak for the USAF or my company

Assisted in daily preparation of large quantities of consumable items
in a fast-paced setting.  (Translation: Short-order cook)
                                        --from a list of resume' blunders