[CentOS] CentOS 7 magically rebooted!

Thu Jan 21 23:27:53 UTC 2016
Tom Robinson <tom.robinson at motec.com.au>

CentOS Linux release 7.2.1511 (Core)

Hi,

Last night our CentOS 7 server rebooted. Seemingly it's a very clean reboot. I can't find a shred of
evidence as to why it happened though.

Things I've checked:
* sa reports
* /var/log/{messages,secure,dmesg,cron}
* /var/log/audit/audit.log
* lastlog

The host is used for KVM virtualisation and connects via multipathing to our OmniOS SAN via infiniband.

I can provide any logging you may think helpful

Here's an excerpt from /var/log/messages at the time of reboot:

---8<---
Jan 21 23:32:01 daytona systemd: Starting Session 10964 of user root.
Jan 21 23:32:12 daytona systemd: Started Amanda Backup System (192.168.0.31:703).
Jan 21 23:32:12 daytona systemd: Starting Amanda Backup System (192.168.0.31:703)...
Jan 21 23:33:01 daytona systemd: Started Session 10965 of user root.
Jan 21 23:33:01 daytona systemd: Starting Session 10965 of user root.
Jan 21 23:34:01 daytona systemd: Started Session 10966 of user root.
Jan 21 23:34:01 daytona systemd: Starting Session 10966 of user root.
Jan 21 23:35:01 daytona systemd: Started Session 10967 of user root.
Jan 21 23:35:01 daytona systemd: Starting Session 10967 of user root.
Jan 21 23:35:34 daytona systemd: Started Amanda Backup System (192.168.0.31:703).
Jan 21 23:35:34 daytona systemd: Starting Amanda Backup System (192.168.0.31:703)...
Jan 21 23:36:01 daytona systemd: Started Session 10968 of user root.
Jan 21 23:36:01 daytona systemd: Starting Session 10968 of user root.
Jan 21 23:36:50 daytona systemd: Started Amanda Backup System (192.168.0.31:703).
Jan 21 23:36:50 daytona systemd: Starting Amanda Backup System (192.168.0.31:703)...
Jan 21 23:37:02 daytona systemd: Started Session 10969 of user root.
---8<---
Jan 21 23:53:01 daytona systemd: Starting Session 10985 of user root.
Jan 21 23:54:01 daytona systemd: Started Session 10986 of user root.
Jan 21 23:54:01 daytona systemd: Starting Session 10986 of user root.
Jan 21 23:55:01 daytona systemd: Started Session 10987 of user root.
Jan 21 23:55:01 daytona systemd: Starting Session 10987 of user root.
Jan 21 23:56:01 daytona systemd: Started Session 10988 of user root.
Jan 21 23:56:01 daytona systemd: Starting Session 10988 of user root.
Jan 21 23:57:01 daytona systemd: Started Session 10989 of user root.
Jan 21 23:57:01 daytona systemd: Starting Session 10989 of user root.
Jan 21 23:58:01 daytona systemd: Started Session 10990 of user root.
Jan 21 23:58:01 daytona systemd: Starting Session 10990 of user root.
Jan 22 00:05:10 daytona rsyslogd: [origin software="rsyslogd" swVersion="7.4.7" x-pid="6886"
x-info="http://www.rsyslog.com"] start
Jan 22 00:05:10 daytona rsyslogd-2307: warning: ~ action is deprecated, consider using the 'stop'
statement instead [try http://www.rsyslog.com/e/2307 ]
Jan 22 11:02:38 daytona journal: Runtime journal is using 8.0M (max allowed 4.0G, trying to leave
4.0G free of 125.8G available → current limit 4.0G).
Jan 22 11:02:38 daytona kernel: Initializing cgroup subsys cpuset
Jan 22 11:02:38 daytona kernel: Initializing cgroup subsys cpu
Jan 22 11:02:38 daytona kernel: Initializing cgroup subsys cpuacct
Jan 22 11:02:38 daytona kernel: Linux version 3.10.0-327.4.4.el7.x86_64
(builder at kbuilder.dev.centos.org) (gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) ) #1 SMP Tue
Jan 5 16:07:00 UTC 2016
Jan 22 11:02:38 daytona kernel: Command line: BOOT_IMAGE=/vmlinuz-3.10.0-327.4.4.el7.x86_64
root=/dev/mapper/centos-root ro rd.lvm.lv=centos/swap rd.iscsi.firmware crashkernel=auto
vconsole.keymap=us vconsole.font=latarcyrheb-sun16 ip=ibft bridge=br0:ibft0 rd.lvm.lv=centos/root
LANG=en_AU.UTF-8
Jan 22 11:02:38 daytona kernel: e820: BIOS-provided physical RAM map:
Jan 22 11:02:38 daytona kernel: BIOS-e820: [mem 0x0000000000000000-0x0000000000095fff] usable
Jan 22 11:02:38 daytona kernel: BIOS-e820: [mem 0x0000000000096000-0x000000000009ffff] reserved
Jan 22 11:02:38 daytona kernel: BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
Jan 22 11:02:38 daytona kernel: BIOS-e820: [mem 0x0000000000100000-0x000000007df5cfff] usable
Jan 22 11:02:38 daytona kernel: BIOS-e820: [mem 0x000000007df5d000-0x000000007e0c8fff] reserved
Jan 22 11:02:38 daytona kernel: BIOS-e820: [mem 0x000000007e0c9000-0x000000007e2d7fff] ACPI NVS
Jan 22 11:02:38 daytona kernel: BIOS-e820: [mem 0x000000007e2d8000-0x000000007f352fff] reserved
Jan 22 11:02:38 daytona kernel: BIOS-e820: [mem 0x000000007f353000-0x000000007f7fffff] ACPI NVS
Jan 22 11:02:38 daytona kernel: BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved
---8<---
Jan 22 11:05:00 daytona systemd: Starting Switch Root.
Jan 22 11:05:00 daytona systemd: Starting Switch Root...
Jan 22 11:05:00 daytona multipathd: 3600144f00000000000005272fc8f0002: stop event checker thread
(139795286001408)
Jan 22 11:05:00 daytona systemd: Switching root.
Jan 22 11:05:00 daytona journal: Journal stopped
Jan 22 00:05:03 daytona journal: Runtime journal is using 8.0M (max allowed 4.0G, trying to leave
4.0G free of 125.8G available → current li
mit 4.0G).
Jan 22 00:05:03 daytona journal: Runtime journal is using 8.0M (max allowed 4.0G, trying to leave
4.0G free of 125.8G available → current li
mit 4.0G).
Jan 22 00:05:03 daytona systemd-journald[322]: Received SIGTERM from PID 1 (systemd).
Jan 22 00:05:03 daytona kernel: type=1404 audit(1453421101.209:2): enforcing=1 old_enforcing=0
auid=4294967295 ses=4294967295
Jan 22 00:05:03 daytona kernel: type=1403 audit(1453421101.610:3): policy loaded auid=4294967295
ses=4294967295
Jan 22 00:05:03 daytona systemd[1]: Successfully loaded SELinux policy in 425.378ms.

The messages 'Starting Session ...' appear every minute for several hours due to a couple of ssh
sessions I left logged in. Is that cause for concern?

Other than that there were a couple of messages from the amanda backup. Then the host reboots. The
timestamp at boot is of note: it reverts to UTC and then back to our local timezone at the end of
the boot cycle. Is that normal and can it be configured otherwise?

Any help is appreciated.

Kind regards,
Tom


-- 

Tom Robinson
IT Manager/System Administrator

MoTeC Pty Ltd

121 Merrindale Drive
Croydon South
3136 Victoria
Australia

T: +61 3 9761 5050
F: +61 3 9761 5051
E: tom.robinson at motec.com.au

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: OpenPGP digital signature
URL: <http://lists.centos.org/pipermail/centos/attachments/20160122/1b0fc5cd/attachment-0004.sig>