Fail2ban | stopped working after 8 -> 9 'elevation'

Support
1

Had fail2ban working perfectly on one of my AlmaLinux boxes which had been running AL8. On said box, I recently did the 8 → 9 in-place upgrade via “leapp”. Everything seems to work – except fail2ban. All the fail2ban .conf and .local files were the same, but danged if I can get fail2ban to ban anything.

So, tried the Draconian approach of erasing fail2ban, and starting from scratch. Here is what I did (after erasing fail2ban, nuking all the previous .conf and .local files, and confirming firewalld running):

  sudo dnf install fail2ban fail2ban-firewalld

Then, started and enabled:

 sudo systemctl start fail2ban
 sudo systemctl enable --now fail2ban

Confirmed everything running

  sudo systemctl status fail2ban

No errrors.

Copied jail.conf → jail.local. Tweaked defaults in jail.local to

 [DEFAULT]
 ignoreip = 127.0.0.1/8 ::1
 bantime = 1h
 findtime = 48h
 maxretry = 3

I used a big findtime as a way to make sure that fail2ban found something in /var/log/secure. Manual inspection of secure showed a fair number of bad actors that should be banned. In fact

 fail2ban-regex /var/log/secure /etc/fail2ban/filter.d/sshd.conf

returns

 Failregex = 39056 total

In jail.d, tweaked sshd.local to the following (on this machine, I map sshd to an alternate port, which I’ll call 1234 here)

 [sshd]
 enabled = true
 port = 1234,ssh
 maxretry = 3
 logpath = /var/log/secure
 actions=iptables-multiport

Restarted fail2ban

  sudo systemctl restart fail2ban

But - nada banned.

 sudo fail2ban-client status sshd

shows nothing banned. At all.

So, back into jail.local. Following settings may be ‘involved’ in the problem:

 backend = systemd

In jail.d/sshd./local

 banaction = iptables-multiport

where I also tried

banaction = firewallcmd-rich-rules[actiontype=<multiport>]

but that made no difference. Tried

 [DEFAULT]
  banaction = iptables-multiport
  banaction_allports = iptables-allports

and in jail.d/sshd.local, tried

[sshd]
enabled = true
filter = sshd
port = 1234,ssh,sftp
bantime = 3d
maxretry = 3
logpath = /var/log/secure
action = iptables-allports[name=SSH, protocol=tcp]

But, still nothing banned.

So, am hoping someone can suggest something I should try from my end.

Thanks in advance…

2

Why ‘iptables’? There seems to be also ‘firewalld’ and ‘nftables’ versions.
The actual ruleset in kernel is nf_tables.

In el8 the use of firewalld or iptables, which was mere translator to nft, did autogenerate the iptables main chains that traditional netfilter had. In el9 firewalld no longer does that (and I have not run iptables to know).

So, could the rule additions fail due to lack of chains? Does fail2ban write log somewhere?

3

I use iptables-multiport in alma 9.5
It worked and did not look further for the firewalld action.

Could SeLinux be blocking fail2ban.
It was in my case but more related to the location of the log files.

4

SElinux is in ‘permissive’ mode - same as it was when using AlmaLinux 8, and when everything was working.

5

Why iptables? Simple – worked with AlmaLinux 8. Besides - firewalld is just layered on iptables (like LaTeX is layered on TeX). I’ve tried both optables, and firewalld, and neither works. fail2ban absolutely fails to ad to the firewall rules using AL 9 - whereas is worked perfectly under AL 8.

I think the issue might be related to the following: fail2ban | fail2ban-regex matches directly, but not in filter - Server Fault

6

FirewallD is a frontend that knows more than one backend.
On default el9 install FirewallD uses nf_tables as backend.

On el8 and el9 the iptables is a translator/wrapper that creates nf_tables rules (but cannot translate/support everything).

A new thing in el9_5 is that FirewallD does set “owner” flag on its nf_tables table.
That means that no other process can modify chains in the table of FirewallD.
(This can be changed.)

Consequence:
If fail2ban attempts to use iptables (or nftables) to change rules that are in FirewallD’s tables, it will fail.
If fail2ban asks FirewallD to update rules, then it should be ok.
If fail2ban has a separate table, then it should be ok.

7

Unfortunately, that doesn’t explain why 1\ it was working under el8 – perfectly, 2\ why it stopped working following ‘elevation’ to el9, despite 8-10 different permutations of attempts to get things working (using either iptales of firewalld), and 3\ why other folks seem to be able to get it work. What seems to be the issue is something that happens during the upgrade process.

I think that process tweaks something somewhere (permissions on something?), that breaks fail2ban. One perhaps confirmatory data point – if I installed el9 from scratch in a VM, fail2ban installs and seems to be working as it should. I could try the experiment of install el8 in a VM, get fail2ban working, then ‘elevate it’ from 8 → 9, but thats a lot of work. I’ll opt for the easier solution of simply ditching fail2ban from this particular box, and implemeneting other tools (I’ve alreaded pretty much locked down sshd, including requiring both keys and passwords, so fail2ban probably only provided marginal additional security).

  • List item