Proper configuration of private network inside a local network

Support
1

Hi All, I would ask help in proper configuring a common situation. I founds several tutorials each one covering only a part of the configuration so at the end I’m not sure that everything work coherently, so I would ask your help to setup the system along the whole configuration chain.

final goal: I have the need to create a private network of hosts with alma linux 9 that they should operate as a balanced calculation cluster with one host that act as interface between the private network an the local network (e.g. allowing to connect to all nodes without physically connect to one of the host of private network)

situation:
Actually one host is both connected to private network and local network of the company (with an assigned ip, gateway, dns) and it can communicate on both networks (eg. browsing on website both on internet and on an host on private network). Hosts on private network can communicate (e.g. ping or connect through ssh) each others.

goal: allows for a host on the private network to access to hosts outside, in particular, to have the possibility to download and install additional packages and software.

what is the best (i.e. secure and robust) way to configure hosts?

thanks for your help

2

Lets start with something. You have two subnets: company “LAN” and private “HPC”.

You have one machine, “F” (front), that is member of both subnets.
The F is “just a member” of LAN. The F has IP address on the HPC too.
The F has default route out via a router that is in LAN and does use a DNS server “out there” too.

No other machine in LAN knows about the HPC subnet.

The F should act as router between LAN and HPC.

The easiest way to configure other members of HPC is to let them do what they do by default; call DHCP server for proper config. The DHCP server obviously should run in F and listen only on the HPC IP address of F.

The easiest way to set up DHCP server in F is to use dnsmasq. It can act as DHCP server, DNS server, and TFTP server. If it offers DHCP and DNS to members of HPC, then they have F as their gateway and DNS server.

There are two ways to set up dnsmasq:

  • As standalone service
  • Set NetworkManager to use dnsmasq for DNS resolution and add custom config so that it serves the HPC subnet as you need

The difference is mostly where you put the dnsmasq configuration.

For F to act as router, there are three things:

  • Enable IP forwarding
  • Allow forwarded traffic in firewall
  • Masquerade on the LAN connection so that the LAN will not see HPC

The external zone that firewalld has ought to do most of those, if the LAN connection is put to that zone.

What zone would be ok for the HPC connection? I don’t know. The latest FirewallD has policy objects for inter-zone traffic rules and I have no idea how they work.

The DHCP based configuration is “robust” in the sense that you have the configuration of every subnet member decided at the DHCP server config. You can decide there what IP address each machine gets and that only known machines do get address.

If you want less “dynamic” setup, then you could reconfig each machine to have manual network config. That is relatively trivial with network system role of Ansible. You would need packages ansible-core and rhel-system-roles at F and “inventory variables” for the machines. Again, the config of each machine would be decided (in the Ansible config) in one machine.

Red Hat has documentation about configuration. For example Configuring and managing networking Red Hat Enterprise Linux 9 | Red Hat Customer Portal

PS. What kind of “balanced calculation cluster”? SLURM?

3

thank @jlehtone , I was thinking to use HTCondor since I used it for some year as user. but still open to alternatives suitable for small cluster (i.e 3-5 nodes with 10-14 GPUs in total)

4

Internet access for the cluster nodes is helpful to set up anyway, but something you might also consider is hosting those packages on the host jlehtone is calling “F,” both to reduce traffic out but also to allow you to easily reproduce a certain set of package versions. If one of your cluster nodes gets messed up you have the option of just reinstalling without having to worry about having different versions of software than what you’ve tested works together well in your cluster.

If you have time to learn a giant monster that does that and DHCP and network installs and such (but which potentially results in a coworker-friendly GUI), you could check out Foreman/Katello, but you can also achieve local repositories using, say, reposync and running an httpd yourself.