Re: How to troubleshoot when stuck at "GRUB loading."?
On 18 September 2013 12:28, olivier sallou <olivier.sallou@gmail.com> wrote:
>
>
>
> 2013/9/18 Anders Ingemann <anders@ingemann.de>
>>
>> On 18 September 2013 08:31, olivier sallou <olivier.sallou@gmail.com>
>> wrote:
>> >
>> >
>> >
>> > 2013/9/17 Anders Ingemann <anders@ingemann.de>
>> >>
>> >> On 17 September 2013 15:46, olivier sallou <olivier.sallou@gmail.com>
>> >> wrote:
>> >> >
>> >> >
>> >> >
>> >> > 2013/9/17 Anders Ingemann <anders@ingemann.de>
>> >> >>
>> >> >> > The virtualbox provider in python branch worked fine at booting
>> >> >> > the
>> >> >> > first bootable disk with grub installed via the loopback.
>> >> >> Honestly, I was never able to get it to work. I think it failed
>> >> >> exactly because it was a loopback device. Can you send a manifest
>> >> >> that
>> >> >> reproduces your setup? Maybe I can work my way up from there.
>> >> >> > did you use grub2 on host machine ? grub1 fails to install over
>> >> >> > loopback
>> >> >> > device and lead to boot error (can't find grub).
>> >> >> Exactly, which is why I am using dmsetup to fake a real hdd, so grub
>> >> >> installs without a hitch :-).
>> >> >
>> >> >
>> >> > I am using grub2 on my computer, it manages correctly the install
>> >> > over
>> >> > loopback.
>> >> > Being able to use grub1 would be perfect, but I do not know how to
>> >> > make
>> >> > it
>> >> > worked. I tried many things to get grub working, but it only worked
>> >> > when
>> >> > I
>> >> > used grub2 :-(
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> > did you look also at your generated device.map and grub.cfg files
>> >> >> > ?
>> >> >> Yes, everything seems fine, I'll have another look though (the
>> >> >> config
>> >> >> is in the pastebin I provided)
>> >> >>
>> >> >> > I know that while trying to setup grub stuff etc... for kvm and
>> >> >> > virtualbox I had issues with auto-generated grub.cfg using
>> >> >> > loopback
>> >> >> > devices
>> >> >> > instead of using disk devices (see loopback keywork in grub.cfg).
>> >> >> Yes, I circumvent that by making the fake hdd, although maybe there
>> >> >> is
>> >> >> some leftover in the grub.cfg... I'll check it out.
>> >> >>
>> >> >> > I also used disk device (/dev/sda for example) instead of disk
>> >> >> > uuids
>> >> >> > in
>> >> >> > boot menu setup of grub (disk uuids may not be the same when
>> >> >> > booting
>> >> >> > on your
>> >> >> > final host).
>> >> >> Actually. They will be exactly the same, I even have problems when
>> >> >> attaching two disks created by the same snapshot because VBox
>> >> >> doesn't
>> >> >> like duplicate UUIDs. At the very least the partition UUIDs will be
>> >> >> the same (which is what I use in fstab).
>> >> >>
>> >> >> Anders
>> >> >>
>> >> >>
>> >> >> On 17 September 2013 08:13, olivier sallou
>> >> >> <olivier.sallou@gmail.com>
>> >> >> wrote:
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > 2013/9/16 Anders Ingemann <anders@ingemann.de>
>> >> >> >>
>> >> >> >> Hey guys
>> >> >> >>
>> >> >> >> I posted to help-grub@gnu.org, asking for help to figure out why
>> >> >> >> my
>> >> >> >> VirtualBox image is stuck at "GRUB loading.". I thought that it
>> >> >> >> might
>> >> >> >> be more relevant over there, but maybe you guys have some ideas
>> >> >> >> as
>> >> >> >> well.
>> >> >> >
>> >> >> > did you look also at your generated device.map and grub.cfg files
>> >> >> > ?
>> >> >> >
>> >> >> > I know that while trying to setup grub stuff etc... for kvm and
>> >> >> > virtualbox I
>> >> >> > had issues with auto-generated grub.cfg using loopback devices
>> >> >> > instead
>> >> >> > of
>> >> >> > using disk devices (see loopback keywork in grub.cfg).
>> >> >> >
>> >> >> > I also used disk device (/dev/sda for example) instead of disk
>> >> >> > uuids
>> >> >> > in
>> >> >> > boot
>> >> >> > menu setup of grub (disk uuids may not be the same when booting on
>> >> >> > your
>> >> >> > final host).
>> >> >> >
>> >> >> > Olivier
>> >> >> >
>> >> >> >>
>> >> >> >> Here's the link to the mailing list:
>> >> >> >> http://lists.gnu.org/archive/html/help-grub/2013-09/threads.html
>> >> >> >> (not
>> >> >> >> a direct link, mailman seems to take its time indexing my post)
>> >> >> >> ...and here's what I wrote
>> >> >> >> ---------------------------
>> >> >> >> Hello everybody
>> >> >> >>
>> >> >> >> I am developing the debian bootstrapper "build-debian-cloud"
>> >> >> >> (https://github.com/andsens/build-debian-cloud) and am having
>> >> >> >> some
>> >> >> >> trouble getting grub to boot my debian installation.
>> >> >> >> Since I bootstrap from a host system (with chroot etc.) I had to
>> >> >> >> use
>> >> >> >> some workarounds to get grub installed onto a loopback device
>> >> >> >>
>> >> >> >> (http://ebroder.net/2009/08/04/installing-grub-onto-a-disk-image/).
>> >> >> >>
>> >> >> >> The setup consists of a vdi image partitioned with GPT into boot,
>> >> >> >> root, swap (in that order).
>> >> >> >> I am unable to get past the "GRUB loading." message when booting
>> >> >> >> the
>> >> >> >> image in VirtualBox.
>> >> >> >>
>> >> >> >> I have a hard time figuring out what is wrong with the setup,
>> >> >> >> especially because the scenario is a bit off the beaten path
>> >> >> >> (mounting
>> >> >> >> vdi as an network block device, bootstrapping, using dmsetup to
>> >> >> >> fake
>> >> >> >> a
>> >> >> >> real hdd etc.). So the usual "just run BootRepair" or "reinstall
>> >> >> >> grub"
>> >> >> >> won't really help to create a stable bootstrapping process.
>> >> >> >> I have run Ubuntus Boot Repair system to check if anything was
>> >> >> >> wrong,
>> >> >> >> but I can't seem to find anything. The output is here:
>> >> >> >> http://paste.ubuntu.com/6115737/
>> >> >> >>
>> >> >> >> The setup can be fully reproduced by cloning my repo and running
>> >> >> >> `./build-debian-cloud manifests/virtualbox.manifest.json` (only
>> >> >> >> tested
>> >> >> >> on debian wheezy).
>> >> >> >> Simply create a new virtual machine in VBox, attach the resulting
>> >> >> >> image and boot it.
>> >> >> >>
>> >> >> >> I would appreciate any help you can offer
>> >> >> >> Anders Ingemann
>> >> >> >> ---------------------------
>> >> >> >>
>> >> >> >>
>> >> >> >> --
>> >> >> >> To UNSUBSCRIBE, email to debian-cloud-request@lists.debian.org
>> >> >> >> with a subject of "unsubscribe". Trouble? Contact
>> >> >> >> listmaster@lists.debian.org
>> >> >> >> Archive:
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> [🔎] CAMcOGXFiLv2+WQ3fXe87wGv6ihs8eTfQBejqb4Vuof99OnrzZg@mail.gmail.com">http://lists.debian.org/[🔎] CAMcOGXFiLv2+WQ3fXe87wGv6ihs8eTfQBejqb4Vuof99OnrzZg@mail.gmail.com
>> >> >> >>
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> >
>> >> >> > gpg key id: 4096R/326D8438 (keyring.debian.org)
>> >> >> >
>> >> >> > Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D
>> >> >> > 8438
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> >
>> >> > gpg key id: 4096R/326D8438 (keyring.debian.org)
>> >> >
>> >> > Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
>> >>
>> >> OK, having done a bit of research, I feel like I need to clear some
>> >> stuff up, so we avoid any further confusion. I *think* I am getting
>> >> this right, please correct me if I am wrong.
>> >>
>> >> > I am using grub2 on my computer, it manages correctly the install
>> >> > over
>> >> > loopback.
>> >> > Being able to use grub1 would be perfect, but I do not know how to
>> >> > make
>> >> > it worked. I tried many things to get grub working, but it only
>> >> > worked when
>> >> > I used grub2 :-(
>> >> This is the main point that got me confused. The bootstrapper doesn't
>> >> install grub1 on virtualbox and never has. It has always been grub2,
>> >> debian switched to grub2 quite a while ago (since squeeze:
>> >> https://wiki.debian.org/Grub). I am a bit puzzled as to how you
>> >> managed to install the bootloader since grub2 gets all confused about
>> >> loopback devices, which is why I haven't been able to install grub2
>> >> with neither your virtualbox version nor mine (yet :-) ).
>> >> Are you sure you didn't accidentally test the wrong image after you
>> >> bootstrapped?
>> >
>> >
>> > Sure, I am currently working on a vagrant provider, which is an extend
>> > of
>> > virtualbox provider and it works fine in virtualbox (at least for vbox
>> > part,
>> > not vagrant...).
>> >
>> > I had a quick look, and the fact is I get it worked because I use grub2
>> > 2.0.x from sid. Previous versions (1.98, 1.99) did not manage correctly
>> > indeed the loopback interface. The version > 2.0 manages it correctly.
>> >
>> > If you are on ubuntu or wheezy,... you get older grub2 release and it
>> > does
>> > not work...
>> > v2.x should be in testing but it seems some bugs prevent migration to
>> > testing.
>> >>
>> >>
>> >> There is one exception to the use of grub1. We need it on EC2 because
>> >> of the paravirtualized (PV) nature of the system. Here the grub
>> >> bootloader is not actually installed to the volume, but rather
>> >> launched from outside the box. That special version of grub then looks
>> >> into the volume and parses the menu.lst file.
>> >> menu.lst is the way grub1 figured out where the inital ram disk
>> >> (initrd) was. The EC2 PV bootloader hasn't been updated yet and can
>> >> therefore not parse the newer grub2 grub.cfg file.
>> >> To that end we create a special grub2 configuration generation script
>> >> which mimicks the old menu.lst layout. This config is then outputted
>> >> by `update-grub` to /boot/grub/grub.cfg, which my bootloader symlinks
>> >> to /boot/grub/menu.lst where PV-Grub (http://wiki.xen.org/wiki/PvGrub)
>> >> can find it.
>> >> Bonus info: This is what Charles Plessy has been working on with the
>> >> "pv-grub-menu" package. Once that is working and packaged we don't the
>> >> special grub2 config files any longer, because the tool generates that
>> >> file.
>> >>
>> >> Now... on virtualbox _we don't need that workaround_. VirtualBox
>> >> generates a full environment for grub to actually load. Meaning vbox
>> >> couldn't care less about what the grub.cfg or menu.lst looks like.
>> >>
>> >> OK, so that was a bit of a tangent. I also want to report back what I
>> >> have figured out so far.
>> >>
>> >> I am fairly certain that my grub.cfg is wrong, it has a `set
>> >> root='/dev/mapper/vdb'`, which surely is not what grub sees at boot,
>> >> mostly because '/dev/mapper' is only used by kpartx and lvm. Then
>> >> again, what would root be set to when you boot from an lvm partitioned
>> >> volume???
>> >> When I change that line to `set root='/dev/sda1'` it still does not
>> >> work. But at this point I figured out that my partitioning may be a
>> >> bit wrong, the boot partition started at sector 0 instead of the usual
>> >> 1MiB mark. Also, I used MB for partition sizes not MiB, meaning my
>> >> 1024MB volume suddenly grew to 1074MB.
>> >>
>> >> There is another issue about grub not being able to figure out where
>> >> the partitions are located on the volume. I am pretty sure however
>> >> that this can be fixed with a proper device.map (btw. the device.map
>> >> file is not read at boot, it is used by the grub tools to figure out
>> >> how to make the grub.cfg file work).
>> >>
>> >> I believe grub should map hd0 as /dev/sda at boot, however even when I
>> >> try to set root to (hd0,gpt1) it still does not work.
>> >>
>> >> Again, if anybody finds an error with what I have written, please
>> >> chime in, I would appreciate it.
>> >>
>> >>
>> >> Anders Ingemann
>> >
>> >
>> >
>> >
>> > --
>> >
>> > gpg key id: 4096R/326D8438 (keyring.debian.org)
>> >
>> > Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
>>
>> > Sure, I am currently working on a vagrant provider
>> Aw maan, I was looking forward to doing that myself :-)
>
>
> Vagrant provider almost work, I have a few updates to do but first tests are
> fine.
>
> However Vagrant needs VirtualBox on host (not image) to *pack* the image in
> a Vagrant box. VirtualBox is in contrib, not free (and contrib backports for
> wheezy).
>
> Should we require virtualbox and launch commands to create the box, even if
> in contrib, or should we generate the vmdk disk and give instructions to the
> user to create the pack box ? (I have a shell script for this for the
> moment)
>
>
>
>>
>> Guess I'll just work on my other plugin idea then: Minimizing the
>> image size by e.g. preventing apt from writing the apt-cache to the
>> bootstrap volume (meaning the vdi won't be expanded).
>
>
> For the moment (what I coded), the virtualbox provider create a raw disk,
> and is converted to vdi only at the end of the process. So vdi will be the
> expanded size, not a compact one.
>>
>>
>> > I had a quick look, and the fact is I get it worked because I use grub2
>> > 2.0.x from sid. Previous versions (1.98, 1.99) did not manage correctly
>> > indeed the loopback interface. The version > 2.0 manages it correctly.
>> Oooh, ok. That explains it!
>>
>> > v2.x should be in testing but it seems some bugs prevent migration to
>> > testing.
>> No matter, we'll just handle it depending on the grub version once
>> it's in debian testing. It'll cut down on the weird hackery I am doing
>> now :-)
>
>
> I do not know however when it gonna move to testing. Issue is related to
> dependencies, so it could last long... :-(
>
> Olivier
>
>
>
> --
>
> gpg key id: 4096R/326D8438 (keyring.debian.org)
>
> Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
> Should we require virtualbox and launch commands to create the box, even if in contrib,
That may be a possibility, though I'd like to keep all utilities free.
Are you sure we can't package with qemu tools only?
> or should we generate the vmdk disk and give instructions to the user to create the pack box ? (I have a shell script for this for the moment)
No, definitely not. The entire philosophy for this bootstrapper is to
avoid exactly those scenarios, no matter how convoluted a packaging
process may be, you can always script it. Once you require manual
steps the iterable nature of the tool goes away. A great advantage of
this tool is that you can modify the script/add plugins, bootstrap,
test and then repeat the whole process without any cumbersome manual
steps. Also, because of the manifest everything is reproducible,
manual steps remove that feature.
> For the moment (what I coded), the virtualbox provider create a raw disk, and is converted to vdi only at the end of the process. So vdi will be the expanded size, not a compact one.
In the WIP branch I use qemu-nbd to bootstrap onto a proper vdi volume :-)
> I had a quick look, and the fact is I get it worked because I use grub2 2.0.x from sid. Previous versions (1.98, 1.99) did not manage correctly indeed the loopback interface. The version > 2.0 manages it correctly.
Oooh, ok. That explains it!
> I do not know however when it gonna move to testing. Issue is related to dependencies, so it could last long... :-(
Doesn't matter, the hack is scripted now, so it's not really an issue
any longer.
p.s.: You forgot to reply to the mailing list btw.
Reply to: