Re: xorriso: extracting files to stdout
> Date: Wed, 16 Apr 2014 09:20:04 +0200
> From: Thomas Schmitt <scdbackup@gmx.net>
> Subject: Re: xorriso: extracting files to stdout
> > Is it possible with xorriso to extract one file (or even cating
> > several
> > files) from an iso image to stdout (or to a given filename that would
> > not be unlinked first like a fifo or /dev/fd/1 or to any file
> > descriptor
> > or maybe to a pipe to a supplied command to exec)?
>
> Currently not. But it would be doable.
Good. I was wondering whether you had resorted to seek into the
extracted files to write the data in several passes because of potential
complexities.
> One could lift that precaution for named pipes, block devices,
> and character devices.
> /dev/fd/1 is supposed to be a symbolic link pointing to some
> /dev/pts/NNN. So the ban would also have to be lifted for
> symbolic links with appropriate target file types.
/dev/fd/<n> may probably be a non symlink, and it points to /dev/pts/
when the corresponding FD is a pseudo-terminal (and the system uses UNIX
98 pseudo-terminals). On Linux, a pipe is a symbolic link to nowhere
and it is unlikely that all UNICES would output something similar:
$ ls -l /dev/fd/ |( cat; ls -l /dev/fd/ )
total 0
lrwx------ 1 graud graud 64 Apr 17 0707:28 0 -> /dev/pts/15
l-wx------ 1 graud graud 64 Apr 17 0707:28 1 -> pipe:[17009189]
lrwx------ 1 graud graud 64 Apr 17 0707:28 2 -> /dev/pts/15
lr-x------ 1 graud graud 64 Apr 17 0707:28 3 -> /proc/11471/fd/
total 0
lr-x------ 1 graud graud 64 Apr 17 0707:28 0 -> pipe:[17009189]
lrwx------ 1 graud graud 64 Apr 17 0707:28 1 -> /dev/pts/15
lrwx------ 1 graud graud 64 Apr 17 0707:28 2 -> /dev/pts/15
lr-x------ 1 graud graud 64 Apr 17 0707:28 3 -> /proc/11474/fd/
I don't think you should try to guess what a user might want to do, even
if it is legitimate to prevent misuse, because that is often what breaks
the versatility of (an interface of) a piece of software. For example
in our case, the chasing of symlinks inside /dev/fd/ (which is
/proc/<pid>/fd/ in fact) does not work as in a regular filesystem:
$ cat </dev/null |
( set -ex; /usr/bin/test -p /dev/fd/0;
/usr/bin/test -p /dev/fd/$(readlink /dev/fd/0); echo true )
+ /usr/bin/test -p /dev/fd/0
++ readlink /dev/fd/0
+ /usr/bin/test -p '/dev/fd/pipe:[17061066]'
# on a regular storage fs, this should have succeeded or failed
# early because the link does not contain any path component
$ cat </dev/null |chase /dev/fd/0
chase: /proc/31147/fd/pipe:[17103574]: No such file or directory
$ cat </dev/null |/usr/bin/test -e /dev/fd/0 && echo true
true
> So you see any use case where a regular data file should be
> overwritten rather than being replaced ?
> (Other than the existing xorriso command -paste_in, which
> composes a disk file from several data files in the ISO.)
1. Files that are deliberately not regular: fifo (named pipe), pipe, pty,
/dev/null (could be used to test the readability of a file or to
simply discard it), /dev/full (to test the error detection capacity
of xorriso?), symbolic link to a regular file or to a non existing
file (some users might like to use those to dispatch the writing
across filesystems, to redirect unneeded extracted data to /dev/null,
to concat several files to the same regular file, ...), block device
(to restore a backup of a USB key for example), special device
corresponding to a non standard driver (assume that the user knows
what he is doing).
2. Files that cannot be removed/recreated/chased (1. also matches this
category):
- filesystems under the control of special drivers, like a FUSE
filesystem to access a network resource or the previously mentionned
/proc filesystem;
- if the directory permissions does not allow the user to remove/create
files but a specific file was created for his use with the
appropriate permissions to write to it.
3. Concatenating data streams by appending to a file (example:
/VIDEO_TS/VTS_01_*.VOB on a DVD, similar to -paste_in I suppose).
4. Avoiding race conditions: an independent program might test that a
file does not exist just after xorriso removed it and use the name;
letting xorriso overwrite a file would put the responsibility of
avoiding such race conditions in the hand of the user (if he so
desires).
5. Anything else that I could not guess.
> There remains the problem, that xorriso might also write text messages
> to stdout, which would be hard to distinguish from file content.
This is easy to avoid if one can write to any FD and not just to stdout.
For example to wrap xorriso so that stdout is sent to stderr (here the
terminal) and FD 4 is redirected to stdout (here the pipe):
$ sh -c 'exec 4>&1 1>&2 "$0" "$@"' xorriso other args -cat "$name" /dev/fd/4 |
cmd of my own that no one else could have thought of |
to the network without touching the filesystem
> A similar situation is given with ISO image output to stdout. Probably
> i can derive safety precautions from that occasion.
I do not understand what you mean by those sentences.
> How urgent is this feature to you ?
Not urgent. You should take the time to decide which way is the best to
allow to do use the full potential of the UNIX file abstraction.
Particularly if you care for backward compatibility as you would not
revise your choice.
I would say the simplest way is to add a new command -cat that would be
similar -cpx but that would disable any smartness and simply append
to the given file and not try to set any metadata (thus skip special
files or fail on them). Either let it take 2 args: the iso filename and
the output or one, the iso filename, and add a global option, for
example -catfile, to redirect the output of the command -cat to
something else than stdout. -cpx would be used to try to recreate a
file system object and -cat to retrieve the file data.
I would also suggest allowing to create pipes directly in xorriso for the
convenience of users, particularly if stdout is not usable directly as
input to a pipe. For example with the following syntax '|shell command
string'. And maybe accept FD numbers with '|<n>' for the unlikely case
that the system does not have /dev/fd/<n>.
> The permission to follow symbolic links would have to be separated
> from the permission to keep a target file of suitable type linked.
> (If my above corrected question yields "yes" then there would be the
> need for a third permission: "do not unlink regular files".)
I think having options to change the behaviour of -cpx, -extract etc. so
that they accomodate what they find on the filesystem instead of adding
-cat_r etc. would be another sensible choice. -cat would still be
useful be useful for concatenating some files in the order specified on
the command line.
According to my previous experiments with the chasing of pipes (or
probably anything that is not on a special filesystem like a FUSE
filesystem), you should first get the file type then try to follow the
link if you really think there is a need to do so yourself, but
should'nt open() be sufficient (when the file pointed to does exist)?
In the case that symlinks are followed (or that they are not treated
especially), why still unlink the destination (except possibly for non
regular files on the iso)? Isn't choosing whether to append or truncate
and whether to set FS attributes sufficient?
> Further the extraction source might be no regular data file. In this
> case it will not be possible to write its content into existing
> file objects on hard disk.
> If in "do not unlink" mode, shall this lead to refusal or shall
> the target object be unlinked and replaced by a copy of the source ?
The less surprising would be to report something. Another option would
be to always skip special files when the user requested only file data
without metadata. This choice should match the spirit of xorriso
however, and I do not know it very well. In any case the user should
have options to detect any possible discrepancy between the iso and what
is extracted if that is not the default.
Regards
--
G.raud Meyer
Reply to: