Re: [OT] - Bash question: get output as a variable?
On Fri, 5 Feb 2010 12:42:47 -0500 (EST), Bob McGowan wrote:
> Stephen Powell wrote:
>> This is off topic from the OP's question, but one of the things that I
>> miss in the Linux environment that I used to use a lot in the CMS
>> environment is CMS Pipelines. The shell supports pipelines, but they
>> are *single-stream* pipelines. CMS pipelines supports *multi-stream*
>> pipelines. That one feature alone makes CMS Pipelines so much more
>> powerful than shell pipelines. I wish the shell supported multi-stream
>> pipelines.
>
> As I'm not familiar with CMS, I have no idea if the following matches or
> is completely off target ...
>
> The default pipeline in shells always operates with stdout/stdin, which
> is a bit limiting.
>
> But, you can work around it, a bit, like this:
>
> cat abc_does_not_exist 2>&1 | wc
>
> The 'cat' error message gets sent to wc. The normal cat output is still
> to stdout, so it also would be processed by wc, when the file exists.
>
> The shell also allows you to open your own descriptors to files, using:
>
> exec 5>abc
> date >&5
> cat abc
> Fri Feb 5 09:15:15 PST 2010
>
> Or, you can set the descriptor to point to another one:
>
> exec 5>&1
> date >&5
> Fri Feb 5 09:15:15 PST 2010
>
> You can also create "named pipes" using 'mknod a_name p'. For example:
>
> mknod fifo1 p
> mknod fifo2 p
> tr '[A-Z]' '[a-z]' < fifo1 >fifo2 &
> [1] 25269
> echo ABCDEFG > fifo1
> cat fifo2
> abcdefg
> [1]+ Done tr '[A-Z]' '[a-z]' <fifo1 >fifo2
>
> Perhaps one or another of the above will do what you want?
These are all helpful tips and tricks, and I'll keep them
in mind. I really won't know if I can do the kind of things
I want with these until I try, I suppose. I don't have a
specific application in mind right now. Let me give you a
simple example of a CMS pipeline for illustrative purposes,
so that you will get some idea of what I'm talking about.
This is a REXX EXEC (i.e. a script written in the REXX
interpretive language) designed to run on the CMS operating
system running in a virtual machine under z/VM:
----------
'PIPE (ENDCHAR ?)', /* Declare pipeline endchar. */
' < USER DIRECT|', /* Read directory file. */
' NFIND *|', /* Discard comments. */
'LOCM: LOCATE /MDISK/|', /* Select MDISK records */
'FIN: FANINANY|', /* Collect MDISKs and LINKs. */
' > OUTPUT FILE A', /* Write them to a file. */
'?', /* End of stream. */
'LOCM: |', /* Non-MDISK cards to here */
' LOCATE /LINK/|', /* Select LINK cards. */
'FIN:' /* Send them to faninany. */
/* End of stream and pipe */
----------
This is all one command, as viewed by the REXX interpreter.
The stuff between /* and */ are comments. They are ignored
by the interpreter. Note that all lines end in a comma,
which is the continuation character, except the last. The
vertical bar, as with shell pipelines, is the stage separator.
The apostrophes serve as quote characters, just as in the shell.
PIPE is the name of the command to invoke. The rest of the
stuff is just arguments to the PIPE command. In other words,
the REXX interpreter itself has no concept of a pipeline.
It is just passing a string of characters to a CMS command
called PIPE. It is the PIPE command that understands what
a pipeline is. The question mark is defined as an
end-of-stream character. It marks the end of a section of
pipe, if you will.
The pipe consists of a number of stages. The stage numbers
and their names are as follows:
1. <
2. NFIND
3. LOCATE
4. FANINANY
5. >
6. LOCATE
Don't be confused by "<" and ">". They are not redirection
operators. There's no such thing as a redirection operator
in REXX or in CMS Pipelines. "<" and ">" are the names of
stages, just like the others. Stage 2 and stage 6 are two
different instances of the LOCATE stage. They have separate
inputs and outputs. LOCM and FIN are labels. They are
names given to some of the stages so that they can be
referenced later. You can tell they are labels because they
end with a colon.
Stage 1, "<", reads records (lines) from a file, the name
of which is USER DIRECT *. (The asterisk, meaning "on the
first accessed disk you can find it on" is implied by default.)
Input stream 1 is unconnected
and not used, since it is the first stage. Records from
the file are written to output stream 1.
Stage 2, NFIND, has its input stream 1 connected to output stream
1 of stage 1. Records which do not begin with an asterisk
in column 1 are written to output stream 1. Records which
do begin with an asterisk are written to output stream 2,
if it is connected. Otherwise, they are discarded.
Since output stream 2 is not connected, they are discarded.
Stage 3, LOCATE, has its input stream 1 connected to output
stream 1 of stage 2. Records containing the character string
"MDISK" are written to output stream 1. Records which do not
contain the character string "MDISK" are written to output
stream 2, if it is connected. Otherwise, they are discarded.
As we will later see, output stream 2 is connected; so these
records are not discarded. This stage is given the label
LOCM so that it can be referred to later.
Stage 4, FANINANY, has two connected input streams. Input
stream 1 is connected to output stream 1 from stage 3.
We will see later what is connected to input stream 2.
FANINANY reads from whichever input stream has a record
ready and writes it to output stream 1.
Stage 5, ">", has its input stream 1 connected to output
stream 1 of stage 4. ">" writes these records to a file,
whose name is OUTPUT FILE on the "A" disk. Output
stream 1 is unconnected, since it is followed by the
end-of-stream character. It is not used.
Stage 6, the second LOCATE stage, has its input stream 1
connected to output stream 2 of stage 3, since it is
preceded by "LOCM: |". Therefore input records from
stage 3 which do not contain the character string
"MDISK" end up coming in on input stream 1 of stage 6.
If any of these records contain the character string
"LINK", they are written to output stream 1. Records
which do not contain the character string "LINK" are
written to output stream 2, if it is connected. Otherwise,
they are discarded. Since output stream 2 is not
connected, these records are discarded. The trailing
string "|FIN:" indicates that ouput stream 1 is to be
connected to input stream 2 (since it is the second
reference to the label) of the FANINANY stage, stage 4.
This pipeline can be illustrated graphically as follows:
+---+ +-------+ +--------+ +----------+ +---+
|-| < |-->| NFIND |-->| LOCATE |------>| FANINANY |-->| > |-|
+---+ +-------+ +--------+ 1 1 +----------+ +---+
| 2 | 2
V A
| +--------+ |
LOCM:-->| LOCATE |-->-FIN:
+--------+
(Obviously, you must be viewing this using a non-proportional
font or it will look terrible.)
The net result is that, using a single pass through the
input file, records which contain the character string
MDISK or LINK, not including records that have been commented
out, are extracted from the input file and written to the
output file. This is a very simple example of a multi-stream
pipeline. Much more complex pipeline topologies can be
defined, as you can imagine. And there are lots of input
device stages, output device stages, filter stages, and
stages which do various other kinds of processing and
transformations.
Perhaps this gives you a flavor for CMS Pipelines. I don't
see anything like that available in shell pipelines.
Sorry for the long post, but I thought a specific example
would help.
Reply to: