AlmaLinux 9.6 server fails to boot normally after motherboard replacement – mdraid / dracut / kernel investigation

Support
1

Hello Community,

I’m looking for some advice from the AlmaLinux community regarding a difficult boot issue on a production server.

I want to make it clear from the beginning that I do not know whether this is an AlmaLinux bug, a kernel issue, a bootloader issue, a BIOS/hardware issue, or simply a configuration problem.

For that reason, I am posting this in the Support section rather than opening a Bug Tracker report. If someone with more experience with AlmaLinux, RHEL, dracut, mdraid or older Intel hardware can point us in the right direction, that would be extremely helpful.

Hardware and environment

The server is a refurbished OVH/Kimsufi bare-metal server.

  • Motherboard: Intel DH67BL

  • CPU/platform: older Intel platform

  • 3 SATA disks

  • Software RAID1 using mdadm

  • AlmaLinux 9.6

  • XFS root filesystem

  • Server is normally managed remotely via SSH/Plesk

The motherboard was recently replaced by OVH after the server became unreachable.

This is actually the second motherboard replacement on this server. The previous motherboard had also been replaced by OVH in November 2025.

The new motherboard has a different network MAC address. We updated the installed AlmaLinux configuration accordingly.

OVH has confirmed that their standard configuration uses PXE/network boot. From their Manager, the customer can select whether the server should boot the OVH Rescue system or boot from the local disk.

What happened

After the recent motherboard replacement, the server did not return normally to its installed operating system.

We therefore booted the machine into the OVH Rescue environment and inspected the installed AlmaLinux system.

The RAID arrays are healthy and fully assembled:

md2 : active raid1 sda2[0] sdb2[1] sdc2[2]
      1921275904 blocks super 1.2 [3/3] [UUU]

md1 : active raid1 sdb1[1] sda1[0] sdc1[2]
      357376 blocks super 1.2 [3/3] [UUU]

The root filesystem is:

/dev/md2: LABEL="root" UUID="fa2b08cf-9214-45d0-afe0-5676f191ceb4" TYPE="xfs"

The RAID UUIDs are:

md1 = 37dfdbfe:68f4c0d3:8538933b:7a45f0d2
md2 = 82233419:0d073144:2db5b4d8:ffbb9aa6

The installed system contains three kernels:

5.14.0-570.60.1.el9_6.x86_64
5.14.0-427.20.1.el9_4.x86_64
5.14.0-427.13.1.el9_4.x86_64

The currently preferred/default kernel is:

5.14.0-570.60.1.el9_6.x86_64

dracut / initramfs investigation

We checked the initramfs for the current kernel.

It contains the mdraid module and mdadm components:

mdraid

etc/mdadm.conf
usr/sbin/mdadm
usr/sbin/mdraid_start
usr/sbin/mdraid-cleanup

The initramfs was generated with:

Arguments: --force --kver '5.14.0-570.60.1.el9_6.x86_64' \
--add 'mdraid' \
--add-drivers 'ahci libata sd_mod raid1'

The installed dracut configuration contains:

add_dracutmodules+=" lvm mdraid "
add_drivers+=" 8139too e1000 e1000e igb r8169 dm-raid raid0 raid1 raid10 raid456 mlx4_en mptspi mptsas ixgbe nvme "
hostonly="yes"

dracut --print-cmdline gives:

rd.md.uuid=82233419:0d073144:2db5b4d8\:ffbb9aa6
rd.md.uuid=37dfdbfe:68f4c0d3:8538933b:7a45f0d2
root=UUID=fa2b08cf-9214-45d0-afe0-5676f191ceb4
rootfstype=xfs
rootflags=rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota

GRUB also contains the correct mdraid and root filesystem references.

For example:

insmod mdraid1x
set root='mduuid/822334190d0731442db5b4d8ffbb9aa6'
search --no-floppy --fs-uuid --set=root fa2b08cf-9214-45d0-afe0-5676f191ceb4

Older kernel

There is also an older kernel:

5.14.0-427.20.1.el9_4.x86_64

Its initramfs also contains mdraid and mdadm.

We initially configured GRUB to boot this older kernel once, because we wanted to determine whether the problem might be related to the newer kernel.

However, there was an important complication in our testing:

the OVH Manager was still configured to boot the Rescue system through PXE.

Therefore, when we rebooted, the machine returned to the OVH Rescue environment instead of actually testing the local AlmaLinux kernel.

So at this point, we have NOT conclusively demonstrated that the 5.14.0-570 kernel itself fails to boot.

This is an important distinction.

Network observation

After one reboot, the server became reachable by ping:

10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev =
22.841/24.743/33.569 ms

However, we subsequently confirmed that the machine was actually running the OVH Rescue environment.

Therefore this ping only proves that the replacement motherboard/network interface and the Rescue environment were functioning correctly. It does not prove that AlmaLinux successfully booted.

Current situation

At this point, the RAID and installed filesystem appear healthy.

We have verified:

  • all three RAID members are present;

  • both RAID arrays are [UUU];

  • the XFS root filesystem is accessible;

  • mdadm is installed;

  • dracut has the mdraid module;

  • the initramfs contains mdraid/mdadm;

  • the RAID UUIDs are consistent;

  • the root filesystem UUID is consistent;

  • GRUB contains the appropriate RAID/root references;

  • the installed older kernel and its initramfs are present.

The server is currently left untouched because it is a production server and has already experienced approximately 72 hours of service interruption during this incident.

I therefore do not want to perform another blind reboot until we have a better understanding of what we should be looking for.

Questions

I would really appreciate advice from anyone familiar with this type of setup.

  1. Does anything in the information above look suspicious or incorrect from an AlmaLinux/dracut/mdraid perspective?

  2. Could hostonly="yes" in dracut cause a problem after a motherboard replacement, even though the initramfs contains mdraid, raid1, ahci, libata and sd_mod?

  3. Are there any known differences between the older 5.14.0-427.x kernels and the newer 5.14.0-570.x kernels that could affect mdraid initialization, SATA detection or booting on an older Intel DH67BL platform?

  4. Could a BIOS update on this older Intel platform affect the way the SATA controller or RAID member disks are exposed to the kernel?

  5. Are there specific logs or diagnostic commands that you would recommend running from the OVH Rescue environment before attempting another boot?

  6. Would you recommend rebuilding the initramfs in a non-hostonly configuration as a diagnostic step?

  7. Are there specific kernel parameters (rd.auto, rd.md.uuid, etc.) that would be useful for diagnosing a possible mdraid discovery problem?

Again, I am not claiming that this is an AlmaLinux bug. At this stage we simply don’t know where the problem lies.

I am posting here rather than immediately opening an AlmaLinux Bug Tracker report because I would first like to establish whether there is actually a reproducible AlmaLinux/kernel issue.

If someone can identify a likely cause or suggest a safe diagnostic procedure, I would be very grateful.

Thank you in advance for any help.

Best regards,

Gérald

2

Hello

I would check the following before suspecting a kernel regression:

  1. Disable PXE/Rescue boot in the OVH Manager and make sure the server is actually booting from the local disk.

  2. Check the BIOS SATA mode and confirm it is set as expected, preferably AHCI.

  3. From the Rescue environment, check which SATA driver is actually in use:

lspci -nnk | grep -A4 -Ei 'SATA|AHCI|IDE'

  1. As a diagnostic step, rebuild the initramfs without host-only mode:

dracut -fv --no-hostonly --add "mdraid" --add-drivers "ahci ata_piix libata sd_mod raid1" --kver 5.14.0-570.60.1.el9_6.x86_64

Then try a local boot again.

At the moment, the RAID arrays, UUIDs, GRUB configuration and mdraid support all look consistent. I would first suspect a BIOS/SATA configuration change after the motherboard replacement, or a host-only initramfs issue.

Only if 5.14.0-570 fails while 5.14.0-427 boots successfully would I start investigating this as a possible kernel regression.