Re: Intresting dd fsck grub uuid fstab action
On Fri 26 May 2017 at 17:52:33 (-0400), Fungi4All wrote:
> From: deblis@lionunicorn.co.uk
>> On Thu 25 May 2017 at 16:41:37 (-0400), Fungi4All wrote:
>> > Le 25/05/2017 à 05:11, Fungi4All a écrit :
>> > > I experimented in switching a clone of my sid installation to an experimental, that was the plan.
>> > > Since I was doing other things I thought I'll let the cloning take place unattended.
>> > > Let's say sda5/6/7/8/9 were to be cloned to sdb5-9 (5 / 6 var 7 sw 8 tmp 9 home) all b partitions were slightly larger.
>> > > I used dd bs=1M for each one and though all I had to do is log in sda5 edit the sdb5 fstab and then update-grub.
>>
>> … and I get the impression that that's exactly what you did:
>> 1 dd
>> 2a "log in sda5" (is that mount, or boot?)
>>
> Boot
>>
>> 2b edit an fstab file (to what purpose, at this point?)
>>
> Why would I edit the fstab in the original installation that I copied from?
How would I know which fstab you edited? You have this situation:
fsUUIDs Disk A Disk B
↓
MBR points to any fsUUID ABCD unknown MBR
some other partitions some other partitions
ABCD sda5 identical with sdb5
EF23 sda6 identical with sdb6
4567 sda7 identical with sdb7
89AB sda8 identical with sdb8
CDEF sda9 identical with sdb9
or do you? Why not:
fsUUIDs Disk A Disk B
↓
MBR points to any fsUUID ABCD unknown MBR
some other partitions some other partitions
ABCD sdb5 identical with sda5
EF23 sdb6 identical with sda6
4567 sdb7 identical with sda7
89AB sdb8 identical with sda8
CDEF sdb9 identical with sda9
IOW how did you decide which disk the kernel had decided to call sda?
How did you tell which was the original and which was the identical copy?
> From sda5 I edited the new copy on sdb5 but sda5 did not boot my first
> time around.
I don't know what "my first time around" means. I've struggled to
reconstruct what you did from a list of numbered questions/comments
in your OP (not a strict list of actions) and some responses to
comments in your follow-up.
I asked "to what purpose do you edit fstab" because at this
stage I see no sense in it. The original disk has an MBR that
allegedly boots partition 5. Presumably it then mounts the
partitions of interest, 6-9 using it's fstab.
The "other" disk has an unknown MBR but is otherwise the same.
Were you to disconnect the original disk's data cable, you
could boot a rescue/live stick and install the correct MBR/grub
on the "other" disk and be able to boot *its" partition 5.
Its fstab would remain the same. All the UUIDs would still tie
up. Everything would be hunky dory so long as the disks never
"met" each other.
> The fsck messages I could only think were pertaining to
> something wrong in the new partitions but even IT did not make sense.
I have no evidence to suggest that problems could be caused by
the trailing garbage in the copied partitions (which you said
were bigger), but it would spur me into finding out.
>>
>> 3 run update-grub
>>
> Yes, thinking that at least this would fix the original booting up.
> Either source or target wouldn't matter.
But I think this would screw up the (which?) grub.cfg royally.
And, unless you're actually a forensic scientist prepared to
carry out experiments and examine the code (because there's
some seriously valuable deductions to be made), picking over
the messed up file is probably a waste of time.
>>
>> Assuming that your copies are good, and that the apparent
>> "alterations" to sda… arise from misunderstanding or misinterpreting
>> UUIDs (or that they actually post-date your running update-grub), then
>> your problems arise from one or both of:
>>
> I had no intention to alter anything in sda, why would I. It was just a
> source to copy from an already functional system. The only change
> would have been an additional entry to grub.cfg in sda5 for booting to
> sdb5.
Computers don't take notice of intentions, only actions.
>>
>> Booting into sda5 with both devices sda and sdb connected,
>> Running update-grub with both devices sda and sdb connected.
>>
> That only happened after I had already copied the "new" uuids on sdb5
> to the fstab there.
That's no help. You've already booted a system with duplicate UUIDs
using a version of grub that finding things by UUID (choosing between
original and copied partitions at random?, who knows?) and a kernel
that guesses at random which disk to label /dev/sda and which to
label /dev/sdb.
> Simple copy-paste of 5 uuids from /dev/disk/by-uuid
I don't understand this. You seem to be fiddling round with the
UUIDs in the fstab(s) without being concerned about the actual
duplicate fsUUIDs embedded in the filesystems.
>>
>> > > 1 - Does dd leave uuid targets as they are, does it create new ones, or does it copy uuid from source to target?
>>
>> I assume that's your reporting of step 1.
>>
> ???
As I said, your OP didn't give a strict timeline of actions and
consequences, but just a numbered list of questions. Thus I wrote:
… and I get the impression that that's exactly what you did:
1 dd
If you hadn't asked this question, I might have written something
like:
… and I get the impression that this might be what you did:
1a dd
1b tune2fs -U … to create new UUIDs on the target disk
IOW in the absence of a list of actions, I was reconstructing
one in order to get a handle on what might have happened.
>>
>> > > 2 - When I rebooted sda5 was not booting up properly. Eventually I checked
>> > its fstab and corrected it but at this poing the system run an fsck and I got
>> > some errors and hit y,y,y,y, for fixing. I don't know if the errors came from
>> > uuids existing in the system twice.
>>
> Something about partition inconsistencies but the print was too small for me to read
> in detail, never mind remembering.
>>
>> Is that "log in"? What did you correct in fstab? What was wrong?
>>
> The only thing I did to fstab up to this point was at the target installation, not the original.
How do you know that? You haven't presented any evidence that you knew
which disk was which when you first started making changes. All your
actions are predicated on knowing which disk was which. When I was
cloning systems 15 years ago, that was relatively easy. /dev/hdaX
corresponded with the cables and jumpers, masters and slaves.
Now, you can't make those assumptions.
> It took a few reboots trying to figure out why the source was so affected. At this point
> I think I got the target to boot but what it was doing as the entry of Debian on sdb5 it
> would boot sda5 while using sdb6,7,8,9. The reason I guess is because sda5 and sdb5
> still had the same uuid. At some point when I used gparted to check all 10 partitions and
> create new uuids and labels, and redid both fstab to the new uuids I went back to sda5
> and reupdated grub. Then the entry for sdb5 was still booting up from sda5!
> That is when I went into grub.cfg and discovered the two uuids used for that entry had
> 2 different uuids, for the same entry. The first was sdb5 the next was sda5's uuid.
As I said, by now things could be so mixed up that you either put on a
forensic hat to work out what happened or you do as you did and just
put things straight manually once you have a set of unique UUIDs
(forgive the tautology).
Unless you take the forensic approach, there is no justification for
blaming Grub or Fsck for screwing things up (or apparently doing so).
You just have to accept that some of your assumptions are not based
on evidence, and are probably incorrect.
>>
>> Is sda the same disk as it was when you ran dd? There's no reason why
>> it should be if the kernel races to label them sda and sdb. How did
>> you check?
>>
> Yes, dd run from a stick sdc "dd if=sda5 of=sdb5 bs=1M" then 6,8,9
> 7 was swap created with gparted, why copy it?
Well, you implied you cloned it in your very first paragraph. What
reason do I have to doubt you? Shaky ground, I know.
So let's assume now that you didn't copy it, and partition 7 on the
target was empty/FAT/something/whatever. Each time the system ran,
did you observe the swap file being started? Either of the fstab files
would point to the correct partition because this UUID is one
that didn't get duplicated. So you've either got to examine dmesg
carefully for a line like
Adding 979960k swap on /dev/sda4. Priority:-1 extents:1 across:979960k FS
and check that that is an a ↑ and not a b,
or notice this fact as it flashes by. Why would be looking out for it?
At this point in time, you were confident that the source was still
sda and the copy sdb.
>>
>> Why does a system suddenly run an fsck?
>>
> It run on recovery so I can run update grub before a full boot-up.
> Before the login root prompt it reported the disk problems that never
> had before with the same disks. But I'm pretty sure it checks filesystems
> in every boot because I remember it says hit ctrl-C to skip filechecks.
My question was asked before you said that you had rebooted many
times, so no surprise to me anymore.
↓ that's a "3".
>>
>> > > 3 - Updating grub seemed to have made a bigger mess as now I could
>> > boot up but the partitions were mixed up between sda and sdb parts. It
>> > would boot-up on root / of sda and have a home in sdb9 ....
>>
>> Yes. I would expect grub to be completely confused about what to
>> write in …wherever/grub/grub.cfg and possibly have no idea which
>> MBR it related to, with two disks present.
>>
> With the same 2 disks grub had been updated before, at least 2 times
> recently. I don't understand you reasoning, but again: After all uuid's
> were unique and both fstabs have been edited and double-checked.
In your OP, no such thing had yet been done. Corrected things was your
"4" and my comment here is about "3" (see above).
> update-grub runs and on the next reboot it is asked on menu entry
> Debian on sdb5 to boot and it boots sda5 because on its cfg file it has
> 2 different uuids for the same entry.
> Any explanations?
That's very good of you to make the UUIDs unique, but I'm not sure
what role you think fstab plays in getting grub.cfg sorted out.
Also bear in mind that the copied (and possibly mangled/out-of-date)
grub.cfg can yield input to grub-mkconfig being run on the original
partition.
In the absence of all the information about both disks, and the
then existing grub.cfg files, it wouldn't be worth even speculating
about what grub-mkconfig would come up with.
>>
>> > The first thing I did after dd was done, was to make a txt table of all
>> > the sda* sdb* in order with uuids and rewrote the fstab on sdb, I thought
>> > the sda would be as it was. But wasn't. It wasn't all uuids that transfered
>> > but seem enough for me not to notice they were the same and for sda5
>> > to boot up and discover the discrepancy. I speculate that fsck then instead
>> > of giving sdb a new uuid it altered sda. By the time the check was done
>> > and I run grub-upd some of the uuid in grub belonged to sda partitions and
>> > some on sdb.
>> > By copying from sda to sdb I never thought the original would be affected,
>> > only the target needed fixing. That threw me off. But on the new grub.cfg
>> > both entried would boot from the sda but with remaining partitions being
>> > a mismatch.
>>
>> This makes me wonder whether you've taken on board Pascal's comments
>> re different types of UUIDs.
>>
>> And when did fsck start handing out UUIDs to filesystems, let alone devices.
>>
> I have no clue what fsck did, I assumed at that point that leaving gaps between
> partitions in an extended one was a problem and extended the end of a partition
> to avoid the gap. I am clueless of fsck's capabilities.
>>
>> > > 4 - Eventually to stay safe I booted from live usb and edited all the
>> > partitions, gave them separate labels to avoid any confusion (as "Deb
>> > on Sda6 var..."_) and switched to all new uuids to clear up the mess.
>> > Re-edited fstab on bot sda and sdb (hd0 and hd1 dos5) and rebooted
>> > sda and updated-grub again.
>>
>> And we're left guessing what the contents are.
>> Do remind me, what is msdos3 here?
>>
> I am not on that system now, but it is a different system and it is a logical part.
> I remember 4 is the extended partition and the 5-11 are within it.
I don't think 3 can be a logical partition number.
>>
>> > > 5 - Here is the unexplained part: My sda5 system booted up fine and normal, when I would try sdb5 the second line would say debian clean on sda5!!!
>> > > So I went into grub.cfg to see what is going on. On the menu entry of the second installation the first line of each entry would have the proper sdb5 uuid, the following line - which I believe is the actual command - had the uuid from sda5. So, am I to assume that grub when it updates only replaces the first line of each entry expecting the next uuid will be the same, even if it is not?
>>
>> No idea. Again, we don't know what you've written where, when you say
>> "switched to all new uuids to clear up the mess". And what's the
>> "second installation"? What are you calling an entry, and a menu entry?
>>
> Instead of twiching my eyes to read uuids digit by digit and since once published
> they are unique, at about the 3rd round I usded gparted and asked it to issue new
> uuids, to make sure they are unique.
> Grub at boot comes up and has some entries, ie
> Windows7
> Debian 8
> Debian 8 recovery
> memcheck
> Debian 9
> Debian 9 recovery
> Debian 10
> FreeBSD
> Manjaro
>
> This is what I call menu entries. They are separated in the cfg file bu groups 00
> 10 20 30 ... and contain sub-entries of the each installation, as recovery modes
> and booting other linux images. If they are not called menu entries this is what
> I mean by entry.
The OS that's running is what populates the 10 section. 20 is always
empty for me. 30 contains other OSes, and some of that information is
copied from their grub.cfg files, as Pascal has already pointed out.
This may be why there was odd-looking information in your new grub.cfg.
>>
>> > I know if I boot
>> > up something on sda12 and install or reconfigure grub in there then 12 will
>> > take control of booting all other bootable partitions in the system.
>>
>> Well, that depends on whether you install it or just configure it.
>>
> If it exists in the mbr and grub as a package is installed in sda12 then updating
> takes control of booting from sda5 and on the next boot the grub.cfg that matters
> is on sda12, right? If you I monitor, as I do, to only allow sda5 to ever update
> grub the rest are there installed in case I ever need to use them.
No. Your language is too imprecise to convey any unambiguous meaning.
Grub as a package should be installed in all the instances of linux.
Each will maintain its own /boot/grub/grub.cfg (assuming you don't
put this in a separate partition and share it between them).
In the straightforward case, the last system that "installed grub
in the MBR" is the system whose grub.cfg will be consulted at boot.
"installed grub in the MBR" has nothing to do with installing the
grub package (like apt-get install grub*) and everything to do with
running the program called grub-install.
>>
>> When you generate a new grub.cfg file, grub collects bits and pieces
>> from the grub.cfg files it finds on the various partitions it guesses
>> are linux systems. This new grub.cfg will normally be written in
>> whatever filesystem contains /boot/grub/. However, that's not
>> necessarily the grub.cfg that was read and acted upon when you just
>> booted, nor necessarily the grub.cfg that will be active when you next
>> boot. You can demonstrate this to yourself by adding a word, say, to
>> one of the id strings.
>>
> So let's say I install lubuntu in a single new partition sda100, and reboot the
> system Debian on sda1 for example, and flubuntu is not even there, there is
> a /boot but no /boot/grub and I go and manually create a boot/grub/grub.cfg
> and I write "all work and no play makes Jack a dull boy" will it make a difference
> to Debian's grub? I believe it will find the /boot and the images there and
> write a "menu entry" to properly boot flubuntu and even display the available
> kernels to boot from. Now if I boot flumuntu, install grub, then the "...Jack" will
> be replaced with new entries. This has been my experience.
It may well be, but the language here is too ambiguous to be of any
help if, say, one step didn't work properly and this was your explanation
of what you did.
> Nothing to do with auto-updating and replacing one uuid and leaving an old
> one in there for the same "entry" "pick" single booting option which caused
> unnecessary havoc.
>>
>> A grub-install, the one that writes typically to the MBR, will make
>> future boots read the new configuration file.
>>
>> Thus, if you maintain a backup linux installation on your machine,
>> and the kernel is updated on the backup, the new grub.cfg that is
>> then configured will have no direct effect on booting. However, if
>> your backup system gets an update of its grub package, that will
>> write a new boot image, and now the backup's grub.cfg will be the
>> active one on next booting.
>>
> Perfect agreement, by experience this is exactly how it has worked
> from day one of ever seeing a grub screen. I have not even hinted
> otherwise.
I'm not trying to suggest that you said otherwise, but only that your
language is too ambiguous for somebody to be convinced that you
understand it. You wrote "I know if I boot up something on sda12 and
install or reconfigure grub in there then 12 will take control of
booting all other bootable partitions in the system."
If you "reconfigure grub in there", 12 will *not* take control.
Now you may know what you mean by "install or reconfigure" but you
cannot expect somebody else (me) to read your mind.
In the paragraph you wrote about lubuntu in sda100, you wrote
"will it make a difference to Debian's grub". Again, you know
what you mean. I don't. What's Debian's grub?
Grub is a booting system, like LILO.
Grub is a suite of programs in a Debian .deb file used to configure
and install such a booting system.
Grub is some pieces of code in the MBR and elsewhere (some poorly
defined) that run in th CPU at boot time.
Using the word Grub involves making plain which part of grub
you're talking about, both in space and time.
>>
>> > [various complaints] I think this is grub's fault.
>>
>> I'm not sure blaming grub will help you straighten out your system.
>> I only hope that one of your disks still carries a suitable clone
>> of the other.
>>
> 1st of all my problem was all figured out and both work fine, is not
> a matter of trying to fix anything. I had it all fixed the night/morning
> before I wrote the message.
>
> What I am looking for is an explanation why grub on its own will
> blend two different uuids for the same "entity" and booting action.
> It labeled the entry on its own, Debian 9 on sdb5 and its single
> action contained two different uuids that I did not edit. It would
> then proceed with booting and end up on mounting sda5.
> Because the one id it had was from sdb5 the second
> was from sda5. Sda5 was all correct at that point, but on grub's
> primary and residing entry it does not even use a uuid. Right?
Well, it now appears that you are an experienced practitioner of
blending systems with non-unique UUIDs. All I can do is sit on
the sidelines and challenge your assumptions about what happened.
When you conduct a post-mortem, the first thing you have to do
is throw away all your assumptions and consider only the evidence
and the possibilities.
>From your last paragraph, particularly the "grub on its own"
expression, I don't think you've yet come to terms with that.
>From your last word, it appears you just want someone sitting
on the other side of the world to agree with your assumptions,
someone who has only read reported fragments of evidence
filtered through your assumptions.
Cheers,
David.
Reply to: