I’m upgrading from CentOS 7 to Alma 9, using Elevate. I went from 7 to Alma 8 with no problems (and much faster than the same process had gone on my laptop!). So I started the upgrade to Alma 9: cleaned up old el7 packages and keys, installed the current Elevate and Leapp, ran the pre-upgrade check, fixed what it asked for, and then sudo leapp upgrade. It got through the first phase and prompted me to re-boot.
But when I did re-boot, the upgrade landed in the grub shell. Now I can’t find the kernel at all.
Here’s what I see.
grub> ls
(lvm/centos_aem-root) (lvm/centos_aem-swap) (lvm/centos_aem-home) (lvm/centos_aem-data) (proc) (hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd2) (hd2,gpt2) (hd2,gpt1)
grub> ls (lvm/centos_aem-root)
(lvm/centos_aem-root) : Filesystem is xfs.
grub> ls (hd0)
(hd0): Filesystem is unknown.
grub> ls (hd0,gpt1)
(hd0,gpt1: Filesystem is xfs.
This does all look sensible: I have LVM set up and those are the appropriate names. I know I need to do “set root”, “linux”, “initrd”, and “boot” from here, but I can’t find the kernel because grub isn’t reading my filesystems.
How do I back out of here to my functioning Alma 8 system? And then what do I need to do to get from there to 9?
Further information. I have two physical drives in the machine. On one of them there is a partition for /boot – it’s /dev/sda1, and it’s an XFS partition. The EFI partition is on the other disk, /dev/nvme0n1p1, and it shows up as an EFI system partition, FAT.
This worked with no problem in CentOS 7, and when I upgraded to Alma 8 this week, it also booted with no difficulty. But after the leapp upgrade to Alma 9, I can’t boot (as described above). It looks like one of those two partitions can’t find the other one.
On my laptop, where the upgrade C7->A8->A9 ran successfully, I’ve only got one physical disk, so the partitions that eventually get mounted as /boot and /boot/efi are in the same place.
So how do I tell Alma 9 where to find the two relevant partitions?
Not sure if this works for you, but I have had EL8 getting stuck at booting into EL9 most of the times.
The way I resolved it, is to go to the hypervisor’s VM console, type poweroff and poweroff the VM. The poweron. Typically, this would be enough to resolve the issue.
If it wasn’t, I had to make sure to give more free space to the /boot partition.
Additionally, you may try:
change /etc/fstab to reference the devices’ UUID rather than /dev/....
remove unnecessary data disks from /etc/fstab if possible.
If we are at GRUB shell, then we are not in /etc/fstab yet.
UEFI loads GRUB from the ESP (EFI System Partition, FAT). (Technically, it loads shimx64.efi that loads grubx64.efi.)
GRUB loads grub.cfg from directory that has grubx64.efi.
GRUB boot entries load kernel and initramfs from /boot/. We did drop to GRUB shell, so we never got that far.
The grub.cfg is in /boot/efi/EFI/almalinux/. The \EFI\almalinux\grub.cfg within ESP.
The /boot/, if it is separate filesystem as it has to be when rest is on LVM volumes, has vmlinuz-* and initramfs-*.img on its “root directory”.
On el7 the boot entries are in the grub.cfg.
On freshly installed el8 the linux boot entries are separate files in /boot/loader/entries/ (i.e. within the /boot filesystem) that are included into grub.cfg with
insmod bslcfg
blscfg
On freshly installed el9 the /boot/efi/EFI/almalinux/grub.cfg is a generated stub that includes the “traditional” grub.cfg: the /boot/grub2/grub.cfg. (On legacy boot GRUB is in /boot/grub2/ and reads the /boot/grub2/grub.cfg directly.)
Example stub:
I have never used ELevate, so I don’t know what it does. Whether it leaves /boot/efi/EFI/almalinux/grub.cfg as it was on el8, or does changes.
It should be possible to boot (USB) AlmaLinux 9 install image into “troubleshoot” mode, which ideally mounts all filesystems and then chroot into them and fix things. (If one can boot from USB/PXE, etc)
Yes, I can boot an install image into “troubleshoot” or “rescue” mode; I have an Alma Live disk and an install disk (the “boot” version), both of which I downloaded and put on DVD in the past few days. And I can get in, mount the disks, chroot to the original root – no difficulty.
But then what? ELevate leaves a minimal /boot/efi/EFI/almalinux/grub.cfg file, which looks exactly like the one you quoted above. (And the one on my functioning Alma 9 machine is the same.) I do have the boot loader entries for the menu, and I do have BLS turned on in the config file /etc/default/grub. The /boot/grub2/grub.cfg file is also where I expect to find it. I did a grub2-mkinstall and grub2-mkconfig (from a rescue boot with chroot) and although that file did get re-written, it did not change. It does contain “insmod blscfg” followed by “blscfg.”
You say UEFI goes to the ESP partition to find shimx64.efi – so that means my system has found that partition. Presumably, then, it hasn’t found the /boot partition, which is on the other physical disk. What do I need to tell it?
(And, no, I have no idea why I set it up this way when I first installed the machine, about 5 or 6 years ago with CentOS 7. It doesn’t make sense to have those two partitions on separate disks, but it did work.)
Don’t know if this will help, but I always label the filesystem partitions with meaningful names. I checked on both Almalinux 8.10 an 9.4, and they both have the same directory tree structure, with the EFI partition mounted on /boot/efi. The grub.cfg that I used indicates that the FS unknown on (hd0) is due to the setup using GPT partition tables. The /boot partition, if I understand your configuration correctly, the kernels should be in the partition (hd0,gpt1). The EFI partition is in (hd2,gpt2), and the filesystem type should be vfat. You should be able to tell your system that the root is on (lvm/centos_aem-root), and the kernel is (kd0,gpt2):vmlinuz-5… for the A lmaLinux 9 kernel, and vmlinuz-4… for the AlmaLinux 8 kernel. The grub file that you will eventually need to correct is in /boot/grub2, but it is generated from files in /etc/grub.d/. Hope this info helps.