Re: backup archive format saved to disk

[Douglas Tutty <dtutty@porchlight.ca>]

On Tue, Dec 05, 2006 at 05:47:23PM -0600, Mike McCarty wrote:
> Johannes Wiedersich wrote:
> >Douglas Tutty wrote:
> >
> >>I'm going to be backing up to a portable ruggedized hard drive.
> >>Currently, my backups end up in tar.bz2 format.
> >>
> >>It would be nice if there was some redundancy in the data stream to
> >>handle blocks that go bad while the drive is in storage (e.g. archive).
> >>
> >>How is this handled on tape?  Is it built-into the hardware
> >>compression?
> >>
> >>Do I need to put a file system on a disk partition if I'm only saving
> >>one archive file or can I just write the archive to the partition
> >>directly (and read it back) as if it was a scsi tape?
> >>
> >>Is there an archive or compression format that includes the ability to
> >>not only detect errors but to correct them? (e.g. store ECC data
> >>elsewhere in the file)  If there was, and I could write it directly to
> >>the disk, then that would solve the blocks-failing-while-drive-stored
> >>issue.
> >
> >
> >Now, to something completely different....
> >If data integrity is your concern, than maybe a better solution than
> >compression is to copy all your data with rsync or another backup tool
> >that 'mirrors' your files instead of packing them all together in one
> >large file. If something goes wrong with this large file you might loose
> >the backup of all your files. If something goes wrong with the
> 
> [snip]
> 
> My understanding of the BZ2 format is that it compresses individual
> blocks independently, and that the loss of a block will not compromize
> the entire archive, only those files which are contained in a given
> block.

Yes. But I don't want to loose any data at all.

I've looked at par2.  It looks interesting.  For me, the question is how
to implement it for archiving onto a drive since the ECC data are
separate files rather than being included within one data stream.

Separate files suggests that it be on a file system, and we're back to
where we started since I haven't found a parfs.

I suppose I could use par2 to create the ECC files, then feed the ECC
files one at a time, followed by the main data file, followed by the ECC
files again.

I'll check out with my zip drive if I can write a tar file directly to
disk without a fs (unless someone knows the answer).

Doug.