Re: backup archive format saved to disk
On Thu, Dec 07, 2006 at 09:16:11AM -0500, Douglas Tutty wrote:
> On Wed, Dec 06, 2006 at 09:02:37PM -0600, Reid Priedhorsky wrote:
> > No, you _should_ compress it and then use some of the space you saved to
> > add some carefully chosen redundancy which will allow you to reconstruct
> > everything, not just some things, in case of failure. (E.g., using par2.)
> > Scenario C: Compression plus redundancy
> > Suppose you have 100 megabytes of files, uncompressed. You create a tar
> > archive and compress it down to 75M. You then create 10M of redundancy
> > using (e.g.) par2, for a total of 85M. A failure occurs, and 2M of data
> > is lost. You use par2 to reconstruct the archive, and nothing is lost.
> > (You can do this regardless of whether data, redundancy, or both are
> > destroyed.) You are happy.
> Hi Reid,
> I've been looking at par2. The question remains how to apply it to data
> stored on media where the potential failure is one of media not
> transmittion. If I only protect the tar.bz2 file and a media failure
> occurs, how could I have set up the par2 redundancy files to allow me to
> recover the data.
> Apparently, hard disks use FEC themselves so that they either can fix
> the data or there is too much damage and the drive is inaccessible. It
> seems to be an all-or-nothing propositition. If someone has experience
> of FEC drive failures that refutes this I'd be very interested.
> The only disk failures I have experienced are on older drives without
> FEC that for a given sector return an error about bad CRC but one can
> carry on and read the rest of the disk. It was from this perspective
> that I proposed the question that led to this thread.
> If drives are atomic in this way, it seems that the only way to achieve
> redundancy is through multiple copies (either manually done or via
> I'm still hoping that someone who knows how linux software raid work can
> tell me how it decides that a drive has failed. This question was posed
> in a thread about raid1 internals.
I quite agree. But in the absence of error-correction codes,
uncompressed is batter.
And if your error-correction software ahould happen to be unusable in several
years, your errors will not be easy to corrected.
Did you ever write any code in the 1970's that can't be run any more?