[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: bash, grep, and regular expressions

Mike wrote:



Pigeon wrote:


FWIW I've just tried out every possibility that has been suggested by
everyone so far. Some of them work as expected, some of them don't.
There does appear to be a certain amount of weirdness going on, though
less than Freddy is getting.

/tmp/m was a file containing all the suggestions, one per line
/tmp/n was the output file
/tmp/x was a directory containing some example files and directories

Bash version: GNU bash, version 2.05a.0(1)-release (i386-pc-linux-gnu)
             Copyright 2001 Free Software Foundation, Inc.
Kernel: Linux schnellbox 2.6.6 #1 Sun Dec 12 01:10:00 GMT 2004 i686 GNU/Linux 

To run all the suggestions without running into quoting hassles, I
did, from within /tmp/x:
nn=`cat ../m | wc -l`; for x in `seq 1 $nn`; do head -$x ../m | tail -1 > ../cmd; chmod a+x ../cmd; cat ../cmd >> ../n; bash --norc --noprofile -c ../cmd >> ../n; echo ====================== >> ../n; done 

I added a plain 'ls -al' to the start of the suggestions file to show
the full listing of /tmp/x.

The suggestions:
ls -al
ls -al | grep -v ' \.\<[a-zA-Z0-9].*\>' # returns everything
ls -al | grep -e '\<[^.][[:alnum:]]'  # returns everything
ls -al | grep -e '\<[.][[:alnum:]]'  # returns an empty set
ls -al | grep -v ' \.'
find -maxdepth 1 |grep -v "^\./\."
ls -a | grep -v '^\.'
ls -al | egrep "\B\."
ls -a1 | egrep -v '^\.'
ls -la | egrep ':[[:digit:]]{2} [^.]'
ls -la | awk '$8 ~ /^[^.]/{print}' ls -la | awk '{ if (substr($9,0,1) != ".") {print $9}}' 
ls -al | grep -e ^d | grep -e '[.][a-z]'
ls -al | grep -e ^d | grep -e '[^.][a-z]'
ls -al | grep -e ^d | grep -e ':[0-9][0-9] [^\.][a-zA-Z]'
ls -al | grep -e ^d | grep -e '\<[^.][a-z]'
ls -la | grep -v -w "\..*"
I did the same tests on my box, But instead of using them on a fabricated directory, I used them on /root. This gave me enough variability in file names, lengths any types. ls -la | wc -l provides 152 entries, ls -l | wc -l provides 35 viewable entries, the rest being hidden. 


The results:
ls -al
total 32
drwxr-xr-x    6 pigeon   pigeon       4096 Feb 18 19:05 .
drwxrwxrwt    3 root     root        12288 Feb 18 19:27 ..
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 .hiddendir1
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:05 .hiddendir2.dir
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 .hiddenfile1
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 .hiddenfile2.txt
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir1
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir2.dir
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 file1
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 file2.txt
======================
ls -al | grep -v ' \.\<[a-zA-Z0-9].*\>' # returns everything
total 32
drwxr-xr-x    6 pigeon   pigeon       4096 Feb 18 19:05 .
drwxrwxrwt    3 root     root        12288 Feb 18 19:27 ..
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir1
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir2.dir
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 file1
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 file2.txt
This one almost preformed as advertised. I gave all the visible files plus the dot and double dot. 


======================
ls -al | grep -e '\<[^.][[:alnum:]]'  # returns everything
total 32
drwxr-xr-x    6 pigeon   pigeon       4096 Feb 18 19:05 .
drwxrwxrwt    3 root     root        12288 Feb 18 19:27 ..
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 .hiddendir1
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:05 .hiddendir2.dir
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 .hiddenfile1
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 .hiddenfile2.txt
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir1
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir2.dir
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 file1
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 file2.txt

I agree, this one returned everything, all 152 entries.


======================
ls -al | grep -e '\<[.][[:alnum:]]'  # returns an empty set

This on returned nothing.


======================
ls -al | grep -v ' \.'
total 32
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir1
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir2.dir
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 file1
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 file2.txt

This one worked as advertised.


======================
find -maxdepth 1 |grep -v "^\./\."
.
./file1
./file2.txt
./dir1
./dir2.dir

This one worked as advertised, but preappended a './' to each entry.


======================
ls -a | grep -v '^\.'
dir1
dir2.dir
file1
file2.txt
This on couldn't tell the difference between the two directories mail and Mail, so only listed mail 


======================
ls -al | egrep "\B\."
drwxr-xr-x    6 pigeon   pigeon       4096 Feb 18 19:05 .
drwxrwxrwt    3 root     root        12288 Feb 18 19:27 ..
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 .hiddendir1
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:05 .hiddendir2.dir
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 .hiddenfile1
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 .hiddenfile2.txt

This one listed all the hidden files, all 117 of them.


======================
ls -a1 | egrep -v '^\.'
dir1
dir2.dir
file1
file2.txt

As advertised


======================
ls -la | egrep ':[[:digit:]]{2} [^.]'
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir1
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir2.dir
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 file1
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 file2.txt

This one found only 12 of the visible files.


======================
ls -la | awk '$8 ~ /^[^.]/{print}' drwxr-xr-x 6 pigeon pigeon 4096 Feb 18 19:05 . 
drwxrwxrwt    3 root     root        12288 Feb 18 19:27 ..
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 .hiddendir1
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:05 .hiddendir2.dir
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 .hiddenfile1
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 .hiddenfile2.txt
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir1
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir2.dir
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 file1
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 file2.txt

This one found 151 of my total 152 files


======================
ls -la | awk '{ if (substr($9,0,1) != ".") {print $9}}'

.
..
.hiddendir1
.hiddendir2.dir
.hiddenfile1
.hiddenfile2.txt
dir1
dir2.dir
file1
file2.txt

Worked as advertised.


======================
ls -al | grep -e ^d | grep -e '[.][a-z]'
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 .hiddendir1
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:05 .hiddendir2.dir
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir2.dir

This one found only hidden directories.


======================
ls -al | grep -e ^d | grep -e '[^.][a-z]'
drwxr-xr-x    6 pigeon   pigeon       4096 Feb 18 19:05 .
drwxrwxrwt    3 root     root        12288 Feb 18 19:27 ..
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 .hiddendir1
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:05 .hiddendir2.dir
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir1
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir2.dir

This one only found directories, both hidden and visible,


======================
ls -al | grep -e ^d | grep -e ':[0-9][0-9] [^\.][a-zA-Z]'
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir1
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir2.dir
I don't know what this one is doing, it found only 4 of 7 visible directories. 


======================
ls -al | grep -e ^d | grep -e '\<[^.][a-z]'
drwxr-xr-x    6 pigeon   pigeon       4096 Feb 18 19:05 .
drwxrwxrwt    3 root     root        12288 Feb 18 19:27 ..
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 .hiddendir1
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:05 .hiddendir2.dir
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir1
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir2.dir

Again, this one found only hidden and visible directories.


======================
ls -la | grep -v -w "\..*" total 32
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir1
drwxr-xr-x    2 pigeon   pigeon       4096 Feb 18 19:04 dir2.dir
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 file1
-rw-r--r--    1 pigeon   pigeon          0 Feb 18 19:04 file2.txt

And this one worked as advertised.


======================

ls -al | grep -e '[0-9][0-9]:[0-9][0-9] [.]'
This one found only 42 of my 117 hidden files.
My system is running kernel-source-2.6.10-5, built manually. Upgraded last Saturday from the unstable tree. ls, grep and awk are from the following packages: 
grep_2.5.1.ds1-4_i386.deb
gawk_1%3a3.1.4-2_i386.deb
coreutils_5.2.1-2_i386.deb
By the way, the entry that I submitted "ls -la | awk '{ if (substr($9,0,1) != ".") {print $9}}'" works not only on my Linux system, but also, unmodified, on AIX 5.1, HPUX 11.0 and HPUX 11.11. So if it didn't work for you, you must not be using the same versions of ls and awk that I am. 
-mike
OK. After getting rid of the alias there are several that work correctly for me. 

 ls -al | grep -v ' \.\<[a-zA-Z0-9].*\>'
This returns all files that aren't hidden. Changing the v to an e returns all hidden files.
ls -al | grep -e ' \.' This now works as advertised too. And changing the e to a v returns what it should too.
ls -la | awk '$8 ~ /^[^.]/{print}' This one now works as advertised too.
To all of you who particated and taught me today I say thank you. I appreciate it. I don't know why I didn't start debugging earlier as I'd been pretty much convinced in my own mind that I had a bug someplace when everyone else had expressions work and non of them worked for me, I guess it just never occurred to me that an alias could do what it did. The reinforcement of that thought from one of you was enough to get me going I guess. I'm just such a noob at regular expressions that I was reluctant to blame anything but my sketchy knowledge. 

BTW, back references now work as they should too.
Reply to: