CentOS Linux release 7.2.1511 (Core)
Hi,
Last night our CentOS 7 server rebooted. Seemingly it's a very clean reboot. I can't find a shred of evidence as to why it happened though.
Things I've checked: * sa reports * /var/log/{messages,secure,dmesg,cron} * /var/log/audit/audit.log * lastlog
The host is used for KVM virtualisation and connects via multipathing to our OmniOS SAN via infiniband.
I can provide any logging you may think helpful
Here's an excerpt from /var/log/messages at the time of reboot:
---8<--- Jan 21 23:32:01 daytona systemd: Starting Session 10964 of user root. Jan 21 23:32:12 daytona systemd: Started Amanda Backup System (192.168.0.31:703). Jan 21 23:32:12 daytona systemd: Starting Amanda Backup System (192.168.0.31:703)... Jan 21 23:33:01 daytona systemd: Started Session 10965 of user root. Jan 21 23:33:01 daytona systemd: Starting Session 10965 of user root. Jan 21 23:34:01 daytona systemd: Started Session 10966 of user root. Jan 21 23:34:01 daytona systemd: Starting Session 10966 of user root. Jan 21 23:35:01 daytona systemd: Started Session 10967 of user root. Jan 21 23:35:01 daytona systemd: Starting Session 10967 of user root. Jan 21 23:35:34 daytona systemd: Started Amanda Backup System (192.168.0.31:703). Jan 21 23:35:34 daytona systemd: Starting Amanda Backup System (192.168.0.31:703)... Jan 21 23:36:01 daytona systemd: Started Session 10968 of user root. Jan 21 23:36:01 daytona systemd: Starting Session 10968 of user root. Jan 21 23:36:50 daytona systemd: Started Amanda Backup System (192.168.0.31:703). Jan 21 23:36:50 daytona systemd: Starting Amanda Backup System (192.168.0.31:703)... Jan 21 23:37:02 daytona systemd: Started Session 10969 of user root. ---8<--- Jan 21 23:53:01 daytona systemd: Starting Session 10985 of user root. Jan 21 23:54:01 daytona systemd: Started Session 10986 of user root. Jan 21 23:54:01 daytona systemd: Starting Session 10986 of user root. Jan 21 23:55:01 daytona systemd: Started Session 10987 of user root. Jan 21 23:55:01 daytona systemd: Starting Session 10987 of user root. Jan 21 23:56:01 daytona systemd: Started Session 10988 of user root. Jan 21 23:56:01 daytona systemd: Starting Session 10988 of user root. Jan 21 23:57:01 daytona systemd: Started Session 10989 of user root. Jan 21 23:57:01 daytona systemd: Starting Session 10989 of user root. Jan 21 23:58:01 daytona systemd: Started Session 10990 of user root. Jan 21 23:58:01 daytona systemd: Starting Session 10990 of user root. Jan 22 00:05:10 daytona rsyslogd: [origin software="rsyslogd" swVersion="7.4.7" x-pid="6886" x-info="http://www.rsyslog.com"] start Jan 22 00:05:10 daytona rsyslogd-2307: warning: ~ action is deprecated, consider using the 'stop' statement instead [try http://www.rsyslog.com/e/2307 ] Jan 22 11:02:38 daytona journal: Runtime journal is using 8.0M (max allowed 4.0G, trying to leave 4.0G free of 125.8G available → current limit 4.0G). Jan 22 11:02:38 daytona kernel: Initializing cgroup subsys cpuset Jan 22 11:02:38 daytona kernel: Initializing cgroup subsys cpu Jan 22 11:02:38 daytona kernel: Initializing cgroup subsys cpuacct Jan 22 11:02:38 daytona kernel: Linux version 3.10.0-327.4.4.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) ) #1 SMP Tue Jan 5 16:07:00 UTC 2016 Jan 22 11:02:38 daytona kernel: Command line: BOOT_IMAGE=/vmlinuz-3.10.0-327.4.4.el7.x86_64 root=/dev/mapper/centos-root ro rd.lvm.lv=centos/swap rd.iscsi.firmware crashkernel=auto vconsole.keymap=us vconsole.font=latarcyrheb-sun16 ip=ibft bridge=br0:ibft0 rd.lvm.lv=centos/root LANG=en_AU.UTF-8 Jan 22 11:02:38 daytona kernel: e820: BIOS-provided physical RAM map: Jan 22 11:02:38 daytona kernel: BIOS-e820: [mem 0x0000000000000000-0x0000000000095fff] usable Jan 22 11:02:38 daytona kernel: BIOS-e820: [mem 0x0000000000096000-0x000000000009ffff] reserved Jan 22 11:02:38 daytona kernel: BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved Jan 22 11:02:38 daytona kernel: BIOS-e820: [mem 0x0000000000100000-0x000000007df5cfff] usable Jan 22 11:02:38 daytona kernel: BIOS-e820: [mem 0x000000007df5d000-0x000000007e0c8fff] reserved Jan 22 11:02:38 daytona kernel: BIOS-e820: [mem 0x000000007e0c9000-0x000000007e2d7fff] ACPI NVS Jan 22 11:02:38 daytona kernel: BIOS-e820: [mem 0x000000007e2d8000-0x000000007f352fff] reserved Jan 22 11:02:38 daytona kernel: BIOS-e820: [mem 0x000000007f353000-0x000000007f7fffff] ACPI NVS Jan 22 11:02:38 daytona kernel: BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved ---8<--- Jan 22 11:05:00 daytona systemd: Starting Switch Root. Jan 22 11:05:00 daytona systemd: Starting Switch Root... Jan 22 11:05:00 daytona multipathd: 3600144f00000000000005272fc8f0002: stop event checker thread (139795286001408) Jan 22 11:05:00 daytona systemd: Switching root. Jan 22 11:05:00 daytona journal: Journal stopped Jan 22 00:05:03 daytona journal: Runtime journal is using 8.0M (max allowed 4.0G, trying to leave 4.0G free of 125.8G available → current li mit 4.0G). Jan 22 00:05:03 daytona journal: Runtime journal is using 8.0M (max allowed 4.0G, trying to leave 4.0G free of 125.8G available → current li mit 4.0G). Jan 22 00:05:03 daytona systemd-journald[322]: Received SIGTERM from PID 1 (systemd). Jan 22 00:05:03 daytona kernel: type=1404 audit(1453421101.209:2): enforcing=1 old_enforcing=0 auid=4294967295 ses=4294967295 Jan 22 00:05:03 daytona kernel: type=1403 audit(1453421101.610:3): policy loaded auid=4294967295 ses=4294967295 Jan 22 00:05:03 daytona systemd[1]: Successfully loaded SELinux policy in 425.378ms.
The messages 'Starting Session ...' appear every minute for several hours due to a couple of ssh sessions I left logged in. Is that cause for concern?
Other than that there were a couple of messages from the amanda backup. Then the host reboots. The timestamp at boot is of note: it reverts to UTC and then back to our local timezone at the end of the boot cycle. Is that normal and can it be configured otherwise?
Any help is appreciated.
Kind regards, Tom