I’m installing AlmaLinux 9.4 on two new dedicated Hetzner-servers that will host a KVM-based virtual server running AlmaLinux 9.4 with Plesk. The virtual server should have its own additional failover IP subdomain. I’ve had this type of setup running successfully for many years on two old Hetzner servers with CentOS 7.
On my old servers the network was setup using the old type of ifcfg files in /etc/sysconfig/network-scripts. With RH/Alma/Rocky version 9 the network should instead be setup using Network Manager and keyfiles in /etc/NetworkManager/system-connections. I have tried to read up on Hetzner’s instructions on how to setup additional IP numbers on https://docs.hetzner.com/robot/dedicate…nal-ip-adresses. However, none of the examples of how to setup a brouted network configuration for virtual KVM based server covers the more recent RH/Alma/Rocky Linux version 9 using Network Manager and nmcli. This is the type of network configuration that is suitable on a Hetzner server:
Has anyone got any hints on how I can use nmcli (Network Manager) to setup this kind of routed-bridged network for additional IP-numbers/IP-subnet on an AlmaLinux 9.4 server?
Physical, which has gateway and eth0 of host as members
Virtual, where each guest and br0 of host are
When virtualization packages are installed, Alma 9 has automatically one virtual subnet defined and enabled by libvirtd. It is called “default” and shows as virbr0 interface on the host. The libvirt does set the host to route between this and the physical subnet. Furthermore, there will be masquerade (sNAT on traffic from default to physical).
If that is not what you want, you can modify (or create new) virtual subnet with libvirt (virsh, virt-manager, or manual edit of config in XML). For example, without masquerade and/or different address range.
What I want is to setup routing from the physical interface (eth0 on the picture, but eno1 in my case) to my virtual bridge interface br0 ,which I will then add additional IP numbers to and use for KVM virtual machines / guests.
Normally you would add the physical interface as a slave to the br0 bridge. However, my hosting provider does not allow different MAC addresses on the different IP addresses, which you would get then, and you need to somehow route the traffic through the physical eno1 interface and gateway.
In other words, the host would bridge, not route, the VM’s to the physical subnet,
and since each VM would then show up in the external subnet with their own MACs …
You have to have one interface with multiple IP addresses and forward traffic (from outside) into the virtual subnet.
It has been possible for a single interface to have more than one address for years now.
On NetworkManager’s connection the ipv4.addresses holds a list of (manually set) IPv4 addresses. See man nm-settings. The first of them seems to be “primary”.
According to man nmcli one would add an address to conn_name with:
nmcli con mod ${conn_name} +ipv4.addresses 192.168.78.4/24
Having addresses on the interface is only part of the puzzle. Then is routing.
When someone on the outside tries to connect to 192.168.78.4, you want to forward (dNAT) that connection to VM (say 192.168.122.4). The NetworkManager does not do that.
The nf_tables ruleset in the kernel does. (You can see it with nft list ruleset )
By default it is firewalld.service that maintains those rules. The libvirt does inject some for the “default” virtual subnet. How they communicate is beyond me. (A) I use only bridged setups – so no rules – and (B) I do use nftables.service for “serious work”.
