Almalinux 10.2 installer not assembling pre-existing mdadm raid

zencowsaysmu · August 5, 2026, 1:27pm

my home server has been successfully running fedora server for several years, but I’d like to rebase it to almalinux for the slower roll and longer support time.

My main storage is 4 drives, arranged as two mdadm raid1 mirrors, which are connected by LVM into one pool, mounted at /home. The raids are consistent, no errors, etc., and pass their weekly scrubs.

My problem is when booting the almalinux installer, it fails to recognize the existing arrays and errors out with “mdadm: add new device failed: invalid argument”. Checking the dmesg logs gives an error about missing/invalid 1.2 superblocks for each drive.

But again, fedora server has no issues assembling the array, and there are no errors.

How can I get the almalinux installer to read my existing arrays so I can proceed with the install?

Thanks!

redadmin · August 5, 2026, 3:13pm

This does not look like a damaged RAID array.

A likely cause is a compatibility issue between the newer Fedora kernel that last assembled or updated the RAID1 metadata and the older 6.12-based kernel used by the AlmaLinux 10.2 installer.

There have been cases where RAID1 arrays assembled by newer kernels could no longer be imported by older kernels, producing errors such as:

mdadm: add new device failed: Invalid argument
does not have a valid v1.2 superblock
md_import_device returned -22

Since Fedora can still assemble both arrays normally and the arrays pass their consistency checks, I would not recreate the arrays or erase the superblocks.

Please boot Fedora or another recent rescue environment and provide the output of:

uname -r; mdadm --version; cat /proc/mdstat; mdadm --detail /dev/md0; mdadm --detail /dev/md1; for d in /dev/sdXN /dev/sdYN /dev/sdZN /dev/sdWN; do mdadm --examine "$d"; done

Replace the device names with the four actual RAID member partitions.

It would also be useful to check whether the metadata contains an internal bitmap or bad-block log, because older md drivers may reject metadata features written by newer kernels.

For now, do not use --create, --zero-superblock, or --force, as these could make recovery more difficult.

If the arrays can be assembled with a newer rescue kernel but not by the AlmaLinux 10.2 installer, this should probably be reported as an AlmaLinux/RHEL installer or kernel-md compatibility regression.

zencowsaysmu · August 5, 2026, 3:37pm

Thanks. yes, it would not surprise me that kernel differences could cause issues.

uname -r: 7.1.6-201.fc44.x86_64
mdadm - v4.3 - 2024-02-15
cat /proc/mdstat : Personalities : [raid1]
md126 : active raid1 sdc1[0] sdb1[1]
2930133440 blocks super 1.2 [2/2] [UU]
bitmap: 3/22 pages [12KB], 65536KB chunk

md127 : active raid1 sde1[0] sdd1[1]
2930133440 blocks super 1.2 [2/2] [UU]
bitmap: 5/22 pages [20KB], 65536KB chunk

unused devices:

sudo mdadm --detail /dev/md/mirror0
/dev/md/mirror0:
Version : 1.2
Creation Time : Wed Jul 29 12:28:50 2026
Raid Level : raid1
Array Size : 2930133440 (2.73 TiB 3.00 TB)
Used Dev Size : 2930133440 (2.73 TiB 3.00 TB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent

 Intent Bitmap : Internal

   Update Time : Wed Aug  5 11:34:02 2026
         State : clean 
Active Devices : 2

Working Devices : 2
Failed Devices : 0
Spare Devices : 0

Consistency Policy : bitmap

          Name : mythhtpc:mirror0  (local to host mythhtpc)
          UUID : 6a0df689:191073a6:c8daace4:ec1590a6
        Events : 5065

Number   Major   Minor   RaidDevice State
   0       8       33        0      active sync   /dev/sdc1
   1       8       17        1      active sync   /dev/sdb1

sudo mdadm --detail /dev/md/mirror1
/dev/md/mirror1:
Version : 1.2
Creation Time : Wed Jul 29 12:28:40 2026
Raid Level : raid1
Array Size : 2930133440 (2.73 TiB 3.00 TB)
Used Dev Size : 2930133440 (2.73 TiB 3.00 TB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent

 Intent Bitmap : Internal

   Update Time : Wed Aug  5 11:34:52 2026
         State : active 
Active Devices : 2

Working Devices : 2
Failed Devices : 0
Spare Devices : 0

Consistency Policy : bitmap

          Name : mythhtpc:mirror1  (local to host mythhtpc)
          UUID : 770c5e5d:383d0dca:5299a1f7:b655d873
        Events : 8039

Number   Major   Minor   RaidDevice State
   0       8       65        0      active sync   /dev/sde1
   1       8       49        1      active sync   /dev/sdd1

sudo mdadm --examine /dev/sdb1
/dev/sdb1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 6a0df689:191073a6:c8daace4:ec1590a6
Name : mythhtpc:mirror0 (local to host mythhtpc)
Creation Time : Wed Jul 29 12:28:50 2026
Raid Level : raid1
Raid Devices : 2

Avail Dev Size : 5860266895 sectors (2.73 TiB 3.00 TB)
Array Size : 2930133440 KiB (2.73 TiB 3.00 TB)
Used Dev Size : 5860266880 sectors (2.73 TiB 3.00 TB)
Data Offset : 264192 sectors
Super Offset : 8 sectors
Unused Space : before=263912 sectors, after=15 sectors
State : clean
Device UUID : 2dad6308:2b918ffc:9e491ab0:3509d5ad

Internal Bitmap : 8 sectors from superblock
Update Time : Wed Aug 5 11:35:55 2026
Bad Block Log : 512 entries available at offset 264 sectors
Checksum : c5f881fb - correct
Events : 5064

Device Role : Active device 1
Array State : AA (‘A’ == active, ‘.’ == missing, ‘R’ == replacing)

sudo mdadm --examine /dev/sdc1
/dev/sdc1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 6a0df689:191073a6:c8daace4:ec1590a6
Name : mythhtpc:mirror0 (local to host mythhtpc)
Creation Time : Wed Jul 29 12:28:50 2026
Raid Level : raid1
Raid Devices : 2

Avail Dev Size : 5860266895 sectors (2.73 TiB 3.00 TB)
Array Size : 2930133440 KiB (2.73 TiB 3.00 TB)
Used Dev Size : 5860266880 sectors (2.73 TiB 3.00 TB)
Data Offset : 264192 sectors
Super Offset : 8 sectors
Unused Space : before=263912 sectors, after=15 sectors
State : clean
Device UUID : cbf74551:7fe343ea:879fd57c:1c80074b

Internal Bitmap : 8 sectors from superblock
Update Time : Wed Aug 5 11:36:20 2026
Bad Block Log : 512 entries available at offset 264 sectors
Checksum : 667cebd5 - correct
Events : 5064

Device Role : Active device 0
Array State : AA (‘A’ == active, ‘.’ == missing, ‘R’ == replacing)

/dev/sdd1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 770c5e5d:383d0dca:5299a1f7:b655d873
Name : mythhtpc:mirror1 (local to host mythhtpc)
Creation Time : Wed Jul 29 12:28:40 2026
Raid Level : raid1
Raid Devices : 2

Avail Dev Size : 5860266895 sectors (2.73 TiB 3.00 TB)
Array Size : 2930133440 KiB (2.73 TiB 3.00 TB)
Used Dev Size : 5860266880 sectors (2.73 TiB 3.00 TB)
Data Offset : 264192 sectors
Super Offset : 8 sectors
Unused Space : before=264112 sectors, after=15 sectors
State : active
Device UUID : 62c05c4a:cde13f6e:830f6a47:07ca033f

Internal Bitmap : 8 sectors from superblock
Update Time : Wed Aug 5 11:36:41 2026
Checksum : 7a57a1c3 - correct
Events : 8039

Device Role : Active device 1
Array State : AA (‘A’ == active, ‘.’ == missing, ‘R’ == replacing)

/dev/sde1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 770c5e5d:383d0dca:5299a1f7:b655d873
Name : mythhtpc:mirror1 (local to host mythhtpc)
Creation Time : Wed Jul 29 12:28:40 2026
Raid Level : raid1
Raid Devices : 2

Avail Dev Size : 5860266895 sectors (2.73 TiB 3.00 TB)
Array Size : 2930133440 KiB (2.73 TiB 3.00 TB)
Used Dev Size : 5860266880 sectors (2.73 TiB 3.00 TB)
Data Offset : 264192 sectors
Super Offset : 8 sectors
Unused Space : before=264112 sectors, after=15 sectors
State : active
Device UUID : 529cf72a:9adf8d38:4858aaff:261d503f

Internal Bitmap : 8 sectors from superblock
Update Time : Wed Aug 5 11:36:59 2026
Checksum : ddcd1775 - correct
Events : 8039

Device Role : Active device 0
Array State : AA (‘A’ == active, ‘.’ == missing, ‘R’ == replacing)

redadmin · August 6, 2026, 12:23am

Thanks for the detailed output. This confirms that the arrays themselves are healthy.

The important detail is that both arrays were created on July 29, 2026 and are now being used by Fedora kernel 7.1.6. Linux 6.19 introduced a new MD logical-block-size field in version 1.2 metadata. Once this field has been written, kernels older than or equal to 6.18 may reject the array with exactly this error:

mdadm: add new device failed: Invalid argument

The “does not have a valid v1.2 superblock” message is therefore misleading. The checksums are correct, both arrays are complete, and the metadata shown here does not indicate corruption.

This is a known upstream compatibility issue:

Because the AlmaLinux 10.2 installer uses a 6.12-based kernel, it cannot currently assemble MD arrays created or updated by the newer Fedora kernel.

Please do not use --force, --zero-superblock, --create, or --update=force-no-bbl. The internal bitmap and bad-block log are not the primary cause, and modifying them would introduce unnecessary risk.

The safest options are:

Back up the data while Fedora can still assemble both arrays.
Report this as an AlmaLinux/RHEL kernel and installer compatibility issue and request the upstream compatibility fix to be backported.
Use an installer or rescue environment containing a sufficiently recent kernel or the compatibility patch.
If no compatible kernel becomes available, back up the data, recreate the arrays using the AlmaLinux kernel, and restore the data.

It would also help the bug report to include the output of:

for d in md126 md127 sdb sdc sdd sde; do echo “[$d]”; grep -H . /sys/block/$d/queue/{logical_block_size,physical_block_size} 2>/dev/null; grep -H . /sys/block/$d/md/logical_block_size 2>/dev/null; done

At this point, I would treat this as a reproducible kernel compatibility regression rather than a RAID recovery problem.

zencowsaysmu · August 6, 2026, 12:55am

Blarg. The fedora “recreated” mdadm date is because of a test install of debian as server, before I discovered that I hated it compared to rhel/fedoraland. Debian trixie (kernel 6.12) had no issues reading my raid during its install.

Apparently, fedora then upgraded the raid when I restored my clonezilla backup of the boot drive, resulting in fedora re-assembling the arrays and changing their metadata.

Sigh. I guess I’ll stay with fedora server until EL catches up.

redadmin · August 6, 2026, 1:10am

That explains the timeline.

Debian 6.12 could assemble the original metadata, but Fedora later reassembled the arrays and updated the metadata into a form that the AlmaLinux 10.2 installer kernel can no longer import.

In that case, staying on Fedora Server for now is probably the safest and least disruptive option. I would avoid trying to downgrade or manually rewrite the RAID metadata, since both arrays are healthy and the data is currently accessible.

Please keep a verified backup. When EL receives the required md compatibility fix—or moves to a sufficiently new kernel—you should be able to reconsider the migration without recreating the arrays.