Re: Some help with dd backing up into an iso
I'd like to thank in advance ALL that responded, I think this is
valuable for an archive of a manual for the nearly illiterate.
Thomas Schmitt:
> GiaThnYgeia wrote:
>>> I used dd if=/dev/sdb of=usbfilename.iso
>>> The resulting image was the full size of the disk.
>
> That's the job of dd: Copying block by block.
>
> As David stated, the file usbfilename.iso will not be an ISO 9660 filesystem
> but rather a disk image.
I have 3 encyclopedic questions based on this. I noticed the difference
as I saved my first try. With other "factory" iso images I can see most
of their contents with an archive program, this indicated a different
beast. Although my restoration seemed to have worked fine but took for
ever.
1 So an img file does not matter what extension it has, it can be zzz
and it would be recognized as an image. Correct? The iso image is a
specific imaging system of cd/dvd format.
2.
2.1 Block by block, and as it is mentioned in other responses about
zero-ing empty blocks, am I to understand correctly that erased data on
an empty block can be recovered because they are not zero. Correct?
2.2 If a block is zeroed it can't be unzeroed? Is this what they
mean by deep cleaning/erasing? That simple? I remember in dos all it
took to make a block available was to take the first letter off of the
file name and it would vanish, unless someone put it back. UNDELETE!
3 If an encrypted partition is included in the image, empty or not, it
would be treated as a chunk of blocks that can't be altered without
being corrupted. Correct? So 4gb full of data in 1 partition with 4gb
of empty but encrypted partition is treated the same as 4gb full of
data, Correct?
> David Christensen wrote:
>> dd if=/dev/sda | gzip > myimage.img
>
> The name suffix .img is more appropriate than .iso, indeed.
> If your USB stick contains a lot of blocks with zeros, you may get even
> better compression by using bzip2 instead of gzip:
>
> dd if=/dev/sdb | bzip2 >usbfilename.img
So the of=usbfilename is replaced by the | bzip2?
> Zeroing the unused blocks before running dd will normally improve the
> compression ratio and thus yield an even smaller .img file.
>
> When you put it back on the USB stick, you need to uncompress:
>
> bunzip2 <usbfilename.img | dd of=/dev/sdb
I will report back ... I'm willing to try this on my 1.8gb system on the
8gb stick.
> This will recreate your two partitions and the filesystems which reside in
> them.
You must be reading my mind, this is a bootable system, with a swap area
partition and the file system. I have yet to see anything been written
in the swap area. Is debian carrying this from an old functionality and
maybe due to systemd it is no longer being used? Or it is getting used
only by certain applications. I made it half the default size in case I
see it get used and be insufficient. I'm going to shrink it some more :)
> By default dd copies chunks of 512 bytes.
> Changing to 1 MiB chunks by dd option
> bs=1M
> might speed up copying substantially.
1M used to be big! I remember the excitement over the huge and mostly
unusable Double Density 1,4MB disks that replaced the paper 512KB double
sided monsters. I wonder if kids would laugh at the sight of one. I am
willing to bet that this dd goes back to backing up hard drives one
5.25" disk at the time. And now we have 4mB empty files. I just created
out of curiosity an empty file and called it x.odt. It was 20bytes. I
opened it with LibrOffice and saved it. It became an empty 10kB file.
>> I looked at the disk and
>> it seemed complete with all files in tact, so maybe I killed it
>> somewhere in the verification process.
>
> There is no verification process with plain dd.
I assumed there might be because there was nothing messed up by my
premature interruption. Etcher (https://etcher.io/) after finishing
went back and "verified" the whole thing, I guess the empty blocks as
well. I suspect it is a fork of xorriso which I have downloaded but had
too many gaps in knowledge to understand how to use its options.
Etcher-electron is a tool for restoring images not for making them.
usbootin was too flaky and was dumped long ago.
> Probably you successfully copied the first partition and the directory
> tree of the second one. It has to be expected that not all files in the
> second partition bear their original content, because it was not copied.
True, if the data was less than 25% of the whole it was probably halfway
through copying the empty blocks!
>> Is there someway one can avoid creating such a large iso for no reason,
>> when the filesize is a fraction of the whole disk.
>
> That's called backup and archiving.
> While the filesystem is mounted but fewly busy, you let a program read
> the files and pack them up in that program's archive format.
I may be unclear on this, my system was running on sda backing up a
different system on sdb, it wasn't archiving itself. I thought that
this can't be done or shouldn't as the active system is not the same as
the shutdown-ed system. I suspect this is not what you meant, you are
talking about used resources on sda trying to backup sdb?
> The classic archiver is called "tar". Caution: wrong arguments or wrong
> sequence of arguments can easily shoot your foot.
Never again! I have the barrel pointing straight at the cpu! Go ahead
punk!
> Assumed you have your filesystems mounted as
> /mnt/usb_part1
> /mnt/usb_part2
> you may pack them up gzip compressed by
>
> tar cvzf usb_part1.tar.gz /mnt/usb_part1
> tar cvzf usb_part2.tar.gz /mnt/usb_part2
But there is this gray area called usb_part(no number) which seems to
have some goodies that make parts 1 and 2 sing. My formal training on
file systems goes back to stack of punched cards as input to huge
machines with huge tapes. There were "computer offices" called RJE with
slaves working at the R(oman)emote Job Entry who handed out the output.
They were the monitors at the time and if they didn't cut off the one
output from the other some other student walked away with a box of your
work. Past DOS6.22 I don't remember reading much about modern file
systems :) Let alone ext4. And it took a while to comprehend the boot
sector that I couldn't copy (I did eventually).
EFI and UEFI and BIOS-boot ... are still a hazy gib-rish symbolism that
I will not understand until I can mess with them enough and break them.
Theory was never good enough for me, I had to get the carburator off and
fix it to achieve perfect fuel mixture. That resulted in wet plugs and
holes on pistons. Now I know :)
> The archive files usb_part1.tar.gz and usb_part2.tar.gz can be unpacked into
> some directory by:
>
> cd /some/directory/for/part1
> tar xzf /where/it/is/usb_part1.tar.gz
> cd /some/directory/for/part2
> tar xzf /where/it/is/usb_part2.tar.gz
That makes great sense, especially if you just want to back-up one
partition and not the whole thing.
> If the archive shall be an ISO 9660 filesystem, you may create it by
>
> xorriso -for_backup \
> -outdev usb_part1_and_2.iso \
> -map /mnt/usb_part1 /part1 \
> -map /mnt/usb_part2 /part2 \
I gave up trying to understand all the tsorriso options, my brain turned
to jello, and just went on with dd (Default for Dummies) tool which took
my sleep away. Just to come up with a valid command like what you say
one needs to spend a weekend studying Xausage or xorriso.
> The superuser will be able to mount the result for random access:
>
> mkdir /mnt/usb_par1_and_2
> mount -o loop /where/it/is/usb_part1_and2.iso /mnt/usb_par1_and_2
>
> The normal user will then see and be able to read two file trees
> /mnt/usb_par1_and_2/part1
> /mnt/usb_par1_and_2/part2
And that is how a network of dummy terminals that always boot up a fresh
installation of linux works. No magic?
> To unmount the .iso, the superuser later does:
> umount /mnt/usb_par1_and_2
Your time is up ... please insert 4 quarters for 15' more.
> Have a nice day :)
> Thomas
You have a teaching talent.... unlike the person who writes the
descriptions in the deb package archives who speaks in machine language
Most of them say: ZZZ is a tool for extracting YYY out of XXX in zyxxyz
system files so they can be translated to common xuzyzysyzyy archives
and be used by zyzyzyuxuxux systems!
WTFAYTA?
Imagine if English wasn't your native language!
Thanks again and to all that responded, there is material to read for a
weekend.
PS There must be apple-msoft people out there laughing at all this for
just trying to make a backup of a usb stick. Then they copy all this
free code, lock it in a fancy gui and sell it for billions.
--
"The most violent element in society is ignorance" rEG
Reply to: