AlmaLinux 10 cloud-init not working?

cloud-init works fine with AlmaLinux 9 but fails with AlmaLinux 10. Everything is identical in my setup except for the cloud image used

https://repo.almalinux.org/almalinux/10/cloud/x86_64_v2/images/AlmaLinux-10-GenericCloud-latest.x86_64_v2.qcow2
vs
https://repo.almalinux.org/almalinux/9/cloud/x86_64/images/AlmaLinux-9-GenericCloud-latest.x86_64.qcow2

I should note I experience this same issue with Rocky Linux 9 vs 10, so I’m guessing this is in some common code.

I am using Apache Cloudstack v4.21, if that matters.

I’ve tried these cloud init data variants:

#cloud-config

user:
  name: infra
  lock_passwd: False
  gecos: Infra
  plain_text_passwd: "Test123$"
  sudo: ["ALL=(ALL) NOPASSWD:ALL"]
  shell: /bin/bash
  ssh_authorized_keys:
    - ...
    - ...
#cloud-config

ssh_pwauth: true
password: changeme
chpasswd:
  expire: false
ssh_authorized_keys:
  - ...
  - ...
#cloud-config

users:
  - name: infra
     shell: /bin/bash
     lock_passwd: False
     sudo: ["ALL=(ALL) NOPASSWD:ALL"]
     plain_text_passwd: "Test123$"
     ssh_authorized_keys:
       - ...
       - ...

Hello

I was looking for a source that suggested changing lock_passwd to lock_password, but

I couldn’t find any documentation, but I did find some code that looked like it might work.
Try “lock_password”.
■ Source Code

/root/rpmbuild/SOURCES/cloud-init-24.4/cloudinit/distros/__init__.py

"__init__.py" 1792L, 66714B 1,1 Top
password_key = "passwd"
# Only "plain_text_passwd" and "hashed_passwd"
# are valid for an existing user.
LOG.warning(
"'passwd' in user-data is ignored for existing "
"user %s",
name,
)

# As no password is specified for the existing user in user-data
# Then check if the existing user's hashed password value is
# empty (whether locked or not).
has_existing_password = not (
self._shadow_file_has_empty_user_password(name)
)
else:
if "passwd" in kwargs: 
ud_password_specified = True 
password_key = "passwd" 
if not kwargs["passwd"]: 
ud_blank_password_specified = True 

# Default locking down the account. 'lock_passwd' defaults to True. 
# Lock account unless lock_password is False in which case unlock 
# account as long as a password (blank or otherwise) was specified. 
if kwargs.get("lock_passwd", True): 
self.lock_passwd(name) 
elif has_existing_password or ud_password_specified: 
# 'lock_passwd: False' and either existing account already with 
# non-blank password or else existing/new account with password 
# explicitly set in user-data. 
if ud_blank_password_specified: 
LOG.debug( 
"Allowing unlocking empty password for %s based on empty" 
" '%s' in user-data", 
name, password_key, 
) 

# Unlock the existing/new account 
self.unlock_passwd(name) 
elif pre_existing_user: 
# Pre-existing user with no existing password and none 
# explicitly set in user-data. 
LOG.warning( 
"Not unlocking blank password for existing user %s." 
" 'lock_passwd: false' present in user-data but no existing" 
"password set and no 'plain_text_passwd'/'hashed_passwd'" 
"provided in user-data", 
name, 
) 
else: 
# No password (whether blank or otherwise) explicitly set 
LOG.warning( 
"Not unlocking password for user %s. 'lock_passwd: false'" 
" present in user-data but no 'passwd'/'plain_text_passwd'/" 
"'hashed_passwd' provided in user-data", 
name, 
)

Does not use “lock_password” anywhere in it. (The comment does not count.)


I have not used cloud-init much, so how does the “fail” manifest?

When I did use (for el7 and el9, on OpenStack), I had:

#cloud-config
users:
  - name: slurm
    uid: "1009"
    gecos: "SLURM"
    no_create_home: True
    homedir: /dev/null
    shell: /sbin/nologin
  - name: root
    ssh_authorized_keys:
      - ssh-ed25519 AA..mw IamGroot

Which resembles your third (but you have indentation issue, at least in what you did post).

Sorry, by fail I mean doesn’t run the config at all. The user doesn’t get created as far as I can tell. The SSH keys aren’t usable. Its pretty hard to debug what is going on when you can’t log in at all though.

That said, I’ve narrowed this down to the ‘CloudStack’ provider in cloud-init not working in this version. This provider relies on sending a well-known URL in the dhcp response, which is then used to grab the configuration for cloud-init.
https://cloudinit.readthedocs.io/en/latest/reference/datasources/cloudstack.html

Maybe this is due to some interaction with a dhcp client change in v10 vs v9. I honestly haven’t looked to see how v10 does dhcp vs v9, but my first thought is maybe it moved away from dhcpcd/dhclient and to systemd or NetworkManager’s newer equivalent which maybe doesn’t pass this data through.

I forced my cloudstack instance to use ConfigDrive instead, and the config now works. Since my cloudstack instance is new, this is fine, but its not really going to work for existing instances.

I can tell from the cloud-init logs after switching to ConfigDrive that there is mention of the “CloudStack” provider being attempted, so it is clearly enabled in cloud-init, just can’t get the data from dhcp it looks like.

Does not use “lock_password” anywhere in it. (The comment does not count.)

Thank you for pointing this out. Thanks to you, the cause has become clear.

It seems my initial predictions and investigations were incorrect.

Ultimately, I had been proceeding with the investigation based on the assumption that AlmaLinux 10’s cloud-init intentionally ignores plain_text_passwd by design, and that using hashed_passwd would resolve the issue.