On Wed, Apr 27, 2016 at 7:05 AM, Chris Adams <linux at cmadams.net> wrote: > Once upon a time, Chris Murphy <lists at colorremedies.com> said: >> On Tue, Apr 26, 2016 at 3:01 PM, Chris Adams <linux at cmadams.net> wrote: >> > Once upon a time, Chris Murphy <lists at colorremedies.com> said: >> >> On Tue, Apr 26, 2016, 2:09 PM Chris Adams <linux at cmadams.net> wrote: >> >> > I have several recently-installed CentOS 7 servers that keep having >> >> > systemd-journald corruption >> >> >> >> Determined with 'journalctl --verify' or another way? > > One system did get into this state overnight, and that said: > > [root at spamscan3 ~]# journalctl --verify > 15bd478: invalid object > File corruption detected at /run/log/journal/f8ade260c5f84b8aa04095c233c041e0/system.journal:15bd478 (of 25165824 bytes, 90%). > FAIL: /run/log/journal/f8ade260c5f84b8aa04095c233c041e0/system.journal (Cannot assign requested address) > (and then a bunch of passes on the rest of the files) > >> There's also this patch as a suggested fix: >> https://bugzilla.redhat.com/show_bug.cgi?id=1292447#c9 > > I'll take a look at that. > >> What version of systemd and rsyslog? systemd-219-19.el7_2.7 and >> rsyslog-7.4.7-12 are current. > > Those are the versions I have. > >> If you're there already you could ry editing >> /etc/systemd/journald.conf and uncommenting Compress=yes and changing >> it to no. > > Thanks, I'm trying that on these servers. Also I wonder if merely restarting the journal daemon solves it: systemctl restart systemd-journald What should happen is it realizes its own logs are corrupt and ignores them, and starts working on new copies. And journalctl should still try to read the old ones but skips the corrupt entries. If that works you could schedule a restart of the journal periodically as a goofy hack work around until it gets fixed. Clearly Red Hat knows about this problem. -- Chris Murphy