[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: mkisofs gives 'file too large for defined data type' error on 4.3GB file



scdbackup@gmx.net wrote:
Hi,


I do not know wether tar needs a neat EOF from the media
[...]

Actually, if reading from a named pipe instead of the device, it works OK. Then just put in disc 1, 'cat /dev/dvdrw > named-pipe', and on another terminal 'tar -t -f /named-pipe --multi-volume'. When the output ends from the cat command, tar asks for the next volume, so the next disc goes in and cat sends the disc data to the pipe to continue the multi-volume archive! It works pretty good!


This astounds me a bit.
Whatever, if it does work for you and stands all
your tests: Congrats.


To my own experience the CD-ROM drivers of Linux return
a more or less arbitrary number of extra bytes at the
end of the media. Not much, but still enough to spoil
a neat seamless volume change.
With afio or uncompressed tar the reading archiver might
get back on track after a few extra kB. With gzipped tar
you will not get happy any more afterwards.

With DVR[+-]RW it is even worse since most read drivers
return a substantial amount of extra bytes. Usually up to
the highest byte count that was ever written to that media.
Sometimes even up to the full (unwritten) capacity of the
media.


Yes, you're right. When I wrote a smaller tar file to the DVD over an existing larger one, tar started reading the remnants of the second one after finishing on the first one. Whoops! Below I detail how I got over this little obstacle...



I'm getting tar to output to a named pipe. I just do a 'growisofs -Z /dev/dvdrw=/bak/dvd-bak.pipe' and it works fine. when tar asks for the next 'tape', I just put in a new disk and execute the same command and it works
great.


Can it be you rely on growisofs not to read more bytes
from the pipe than it can write to the media ? Given
the new ring buffer in 6.0, i would say this is a
daring assumption.

man tar is so sparse ... google ...
http://www.gnu.org/software/tar/manual/tar.html#Using-Multiple-Tapes
Aha, there is a companion for --multi-volume named
--tape-length= .
That might be a cleaner way to deal with the problem.
Did you already use that option ?

That's exactly what I do. I use the --tape-length= option with a value of 4580000 - that's 10MB under the max capacity of a DVD+RW - I'm just paranoid like that.

Then I read from the disks into the named pipe using 'dd if=/dev/dvdrw of=named-pipe bs=1k count=4580000' - exactly the same number of kb - and read via tar with '--multi-volume -f named-pipe' and voila! When tar asks for the next volume, I just swap discs and run 'dd' again outputting into the pipe, and it vorks wery vell! No extra bytes here!

Obviously, you may have guessed that I use two separate terminals for this little jobby - actually I use 2 split GNU Screen virtual terminals, one on top of the other.


I read there:
"Each volume of a multi-volume archive is an independent tar
 archive, complete in itself."
That might explain why you have no problems with the extra bytes when reading.

So i believe it is not like:

When the output ends from
the cat command, tar asks for the next volume,

but rather that tar ends reading the volume after
encountering its volume/archive end mark and opens
a new communication channel via the named pipe
object for the next volume. The old cat command then
probably dies from broken pipe.

It used to - that's why I now use dd with a 'count=' flag :-)


Nevertheless, a nice stunt.

LOL - thanks :-)



Have a nice day :)

Thomas





Reply to: