Xfsdump - Undersanding logs

Support
1

I moved some backup scripts from CentOS7 to my new Alma9 installation and I’m missing some logs from xfsdump.

The script that does the backups is run by root from

/var/spool/cron/root

The script runs system xfsdumps of /, /root/, and a data directory and all that works just fine. In the past on COS7 I would receive an email that was sent to root and forwarded to my email that showed the output of the three xfsdumps. This no longer happens on Alma9

I’m guessing I need to set some parameter/setting related to xfsdump as to where the log is sent and how handled but I have been unable to locate any information related to that.

If anyone more knowledgeable can point the way please advise.

2

I noticed it also but did not need the email option.
One possibility is to pipe the output of the cron job to a mail command or script that sends an email.
Or read the man of crond to find out more on e.g. MAILTO=root option that i have seen in some scripts.

3

Thanks rvk. I could include mailto lines in the script but I’m trying to understand why the script as-is worked in COS7 but now the report is not being sent in Alma9. Good suggestion on looking into crond some more. However, does anyone understand when xfsdump runs, where does any report get sent or saved?

4

man cron does say in both that default tool for sending mail is sendmail.

Both postfix and sendmail do provide /usr/sbin/sendmail, but I have neither on any of my Alma systems. (I have not installed DNF group mail-server.)

On /var/log/cron I have from every start of `crond.service:

(CRON) INFO (Syslog will be used instead of sendmail.)

In other words, crond is “happy” without ability to send mail.

Do you have postfix installed (or alternative mail sending program)?

5

I do have postfix installed. I’ll look into my /var/log/cron and see what I see there. Is there a cron config file for setting these things. I haven’t dealt much with crond, I better look into more of the documentation. However, how would cron know about xfsdump and any log or report that it generates? I’m still not understanding the whole process I guess.

6

OK, some is coming clear to me. Duh! of course cron knows about the xfsdumps because the script is called and run by root from
/var/spool/cron/root

I can see in /var/log/cron that the script is being run from lines like

Oct 6 05:05:02 dunwellguitar3 CROND[13367]: (root) CMD (/root/backup2usb.sh)
Oct 6 05:06:27 dunwellguitar3 CROND[13335]: (root) CMDEND (/root/backup2usb.sh)

So all I’m lacking is how or why crond knows to send an email. I see in various online suggestions things like

cat /var/log/syslog | grep cron
nano /etc/rsyslog.d/50-default.conf

but those files don’t exist on my Alma9 install. So I’m still not knowing where or how crond decided to put things in /var/log/cron and/or send a mail to root. I still have my COS7 disks so maybe I can rsync a copy to a disk and look around there to see if there are config files for cron somewhere. Any suggestions appreciated.

7

Hi @cosmic11do2

Can you share how you do your backup using xfsdump?
Are you able to do a bare-metal recovery using you backup files?

8

I checked with the guy that originally wrote the script and he says it is OK to share. NOTE!!!: some of the lines got wrapped when I had to convert the text file to a .pdf in order to upload it. Go through carefully in a non-line-wrap editor and correct. Most are comments but some lines are commands so watch it.

Yes, I have used the dump in the past to do a complete restore but mostly I count on it for picking up files or subdir that got lost or corrupted. For complete restore I count on my dd complete disk backups that I do every 6 months. Once I’ve restored one of those I can just do a restore from the xfsdumps to pickup the latest needed, like mail tree etc. My system doesn’t change much, not very dynamic.

backup2usb.pdf (31.2 KB)

9

Back to the question of the xfsdump logs and how they get mailed, I checked my backup of COS7 before I switched to Alma9. Looking in
/var/log/cron (lines got wrapped in transfer)

on alma9

Oct 6 05:05:02 dunwellguitar3 CROND[13367]: (root) CMD (/root/backup2usb.sh)
Oct 6 05:06:27 dunwellguitar3 CROND[13335]: (root) CMDEND (/root/backup2usb.sh)

on cos7 this once. It may have just happened in the middle of the hourly.

Sep 23 05:01:01 dunwellguitar3 run-parts(/etc/cron.hourly)[15548]: starting 0yum-hourly.cron
Sep 23 05:05:01 dunwellguitar3 CROND[15617]: (root) CMD (/root/backup2usb.sh)
Sep 23 05:06:37 dunwellguitar3 run-parts(/etc/cron.hourly)[15700]: finished 0yum-hourly.cron

then all the rest are like

Sep 24 05:05:01 dunwellguitar3 CROND[5972]: (root) CMD (/root/backup2usb.sh)

I looked around on both systems and I still don’t understand the process whereby the root cron process manages to detect the xfsdump output and email a copy to root@localhost. Anyone that is knowledgeable of this process plz. chime in. TIA

10

if there a log file of the mail : /var/log/maillog
are you using SELinux ? Try to run the cron job with this feature disbaled: setenforce 0
maybe schedule a new simple cron job that triggers more often to test.

11

The man cron writes:

When executing commands, any output is mailed to the owner of the crontab (or to the user specified in the MAILTO environment variable in the crontab, if such exists). Any job output can also be sent to syslog

The output. When you run commands, they may write to stdout and stderr. If you run command interactively, then the output shows on the terminal. You can redirect output to a file (e.g. ls -l > filename) or pipe it to another command (e.g. ls -l | grep d). If you have commands in a script file and run that script (interactively), you get output of each command like you hand run them “manually”.

What the cron does is run the /root/backup2usb.sh script. It does then pipe the output that it gets to a program that reads stdin and puts that input into body of an e-mail that it sends to address that it got as command-line parameter (when cron calls it).

The man sendmail(on CentOS 7 did) write:

SYNOPSIS
sendmail [option ...] [recipient ...]

DESCRIPTION
…
By default, Postfix sendmail(1) reads a message from standard input until EOF or until it reads a line with only a . character, and arranges for delivery.

In other words, it is like the cron would run:

/root/backup2usb.sh | sendmail root@localhost

If you do run the /root/backup2usb.sh interactively, then do you get the expected “log text” to the terminal? Do you get the text on the CentOS 7 system?

If you don’t get similar log in both, then the difference is within /root/backup2usb.sh, not in cron.

You can copy-paste text from script to message and use the code tags to keep it neat.
If the text is too long for a single post, then there are paste-bin servers.

12

rvk, my selinux is set to permissive. Good thought on making a short script to un for testing. More on that if I do.

13

jlehton, I think you have it. Many thanks for clarifying the whole process. I must have read that in the man pages but it didn’t stick.

When I run the script interactively I do see the output on the screen.

Yesterday I had already put a MAILTO command into the
/var/spool/cron/root
file pointing to my own personal mail rather than to root. Although the aliases file also has root: myemailhere as well. When the script ran it did try to send a mail message just as you indicated it should. So cron is working as expected. I see in the
/var/log/maillog
file

Oct 11 05:21:49 dunwellguitar3 postfix/local[22873]: 4C9A441B9C20: to=alan@external.com, orig_to=root@external.com.com, relay=local, delay=0, delays=0/0/0/0, dsn=5.1.1, status=bounced (unknown user: “alan”)
Oct 11 05:21:49 dunwellguitar3 postfix/qmgr[10366]: 4C9A441B9C20: removed

Where I have replaced my actual external address with “external”.

So likely it is a mail problem, it looks like mail is trying to send to a local user maybe rather than the email address? I don’t understand the bit about unknown user, where is it thinking about that, locally? There is no alan user on my local dunwellguitar3 machine. I can use mail or sendmail to send emails from the command line to that external address OK.

I also see messages in the log like

Oct 10 13:57:16 dunwellguitar3 postfix/local[7266]: 2AF9341B9C1D: to=root@external.com, relay=local, delay=0, delays=0/0/0/0, dsn=5.2.0, status=bounced (cannot update mailbox /root/~/Mymail for user root. unable to create lock file /root/~/Mymail.lock: No such file or directory)
Oct 10 13:57:16 dunwellguitar3 postfix/qmgr[1806]: 2AF9341B9C1D: removed

Note that it is sending to external root but at my external email address and I have no idea what is generating these messages.
Grasshopper is mighty puzzled again as usual.

14

More Data On Mailing:

So I made a simple script to send out mails to my two external email addresses

cat MailTestScript.sh
#!/bin/bash

echo -e “This is the message from DG to alan@” |mail -s “Subject Here” alan@External-1.com
echo -e “This is the message from DG to mric” |mail -s “Subject Here” dunwell@External-2

Where the External{1/2} are two separate email addresses. When I look in the /var/log/maillog I see that the first External-1 fails and the second goes and I receive it.

Oct 11 16:01:38 dunwellguitar3 postfix/pickup[32613]: 7147B41B9C10: uid=1000 from=
Oct 11 16:01:38 dunwellguitar3 postfix/cleanup[34628]: 7147B41B9C10: message-id=20241011220138.7147B41B9C10@External-1.com
Oct 11 16:01:38 dunwellguitar3 postfix/qmgr[28665]: 7147B41B9C10: from=dunwell@External-1.com, size=385, nrcpt=1 (queue active)
Oct 11 16:01:38 dunwellguitar3 postfix/pickup[32613]: 7386E41B9C17: uid=1000 from=
Oct 11 16:01:38 dunwellguitar3 postfix/cleanup[34628]: 7386E41B9C17: message-id=20241011220138.7386E41B9C17@External-1.com
Oct 11 16:01:38 dunwellguitar3 postfix/qmgr[28665]: 7386E41B9C17: from=dunwell@External-1.com, size=378, nrcpt=1 (queue active)
Oct 11 16:01:38 dunwellguitar3 postfix/local[34634]: 7147B41B9C10: to=alan@External-1r.com, relay=local, delay=0.03, delays=0.02/0/0/0.01,
dsn=5.1.1, status=bounced (unknown user: “alan”)
Oct 11 16:01:38 dunwellguitar3 postfix/cleanup[34628]: 76E5D41B9C20: message-id=20241011220138.76E5D41B9C20@External-1.com
Oct 11 16:01:38 dunwellguitar3 postfix/bounce[34637]: 7147B41B9C10: sender non-delivery notification: 76E5D41B9C20
Oct 11 16:01:38 dunwellguitar3 postfix/qmgr[28665]: 76E5D41B9C20: from=<>, size=2206, nrcpt=1 (queue active)
Oct 11 16:01:38 dunwellguitar3 postfix/qmgr[28665]: 7147B41B9C10: removed
Oct 11 16:01:38 dunwellguitar3 postfix/local[34634]: 76E5D41B9C20: to=dunwell@External-1.com, relay=local, delay=0, delays=0/0/0/0,
dsn=5.2.0, status=bounced (cannot update mailbox /home/dunwell/~/Mymail for user dunwell. unable to create lock file /home/dunwell/~/Mymail.lock: No such file or directory)
Oct 11 16:01:38 dunwellguitar3 postfix/qmgr[28665]: 76E5D41B9C20: removed
Oct 11 16:01:55 dunwellguitar3 postfix/smtp[34635]: 7386E41B9C17: to=dunwell@External-2, relay=mx.External-2[66.96.140.188]:25, delay=17,
delays=0/0.01/17/0.24, dsn=2.0.0, status=sent (250 2.0.0 zNhps4vXLUrv0zNhrsvcfh mail accepted for delivery)
Oct 11 16:01:55 dunwellguitar3 postfix/qmgr[28665]: 7386E41B9C17: removed

Unfortunately, the first email address is the one I want to use and the second is just a spare address that I’m phasing out. Also note in the complaints about External-1 that there are notations like

/home/dunwell/~/Mymail for user dunwell

In my

/etc/postfix/master.cf

I specify the following vars

myhostname = External-1.com
mydomain = External-1.com
myorigin = $mydomain
home_mailbox = ~/Mymail

So that is likely where the ~/Mymail comes from but why does it pre-pend the use info in front of it making it a non-existent subdir?

And this from my logwatch this morning

--------------------- Postfix Begin ------------------------

    1   *Warning: Database is older than source file            1
    1   Miscellaneous warnings                           1

19.366K Bytes accepted 19,831
1.462K Bytes sent via SMTP 1,497
======== ==================================================

   10   Accepted                                   100.00%

   10   Total                                      100.00%

======== ==================================================

   16   Removed from queue                              16
    4   Sent via SMTP                                    4
   12   Bounced (local)                                 12
    6   Notifications sent                               6

    1   Postfix refresh                                  1

    3   Running in backwards compatibile mode            3

---------------------- Postfix End -------------------------

'Umble Leafhopper is powerful confused.

15

Some more mods:
I went in and changed my Home mailbox to be the usual

home_mailbox = Maildir/

The next time I ran the test script it created the dir tree in my home directory with the proper creds. [NOTE: the maildirmake command does not seem to exist in Alma9 postfix/dovecot installations.] However, it is still thinking that mail to alan@External-1.com is actually a local delivery. This I suspect because in main.cf the host name is the same because that is the only domain I own. I don’t know how to make it deliver the mail as an external, not local, mail message. This all worked in COS7 w/o any other configuration so I don’t know where to look or what to change to make this happen.

In retrospect the title of this thread really should say something about mail but I don’t know how to mod the title.