The system is an AWS Instance based on a community CentOS 6.4 AMI snapshot.
The vdisk is as follows as shown below [1] The root LVM contains /var/log/
I have attached another block device with ext4 FS.
I copied the files from /var/log to this device (mounted on /mnt) and then changed /etc/fstab to mount this device on /var/log on boot.
However, I do not see anything being logged in /var/log/messages. To test the logging, I used the 'logger' command to log some string; nothing appears in /var/log/messages.
'service rsyslog status' reports the daemon is running.
When I stop rsyslog, umount the /var/log device and then restart rsyslog, I can see that logs are being recorded in /var/log/messages. Using the 'logger' command I can see messages written in /var/log/messages.
man pages of ryslog.conf and rsyslogd show nothing related to logs being on a separate device
Any pointers to fix the problem would be much appreciated.
[1] Disk /dev/xvda: 8589 MB, 8589934592 bytes
Device Boot Start End Blocks Id System /dev/xvda1 * 1 64 512000 83 Linux /dev/xvda2 64 1045 7875584 8e Linux LVM
Disk /dev/mapper/VolGroup-lv_root: 7205 MB, 7205814272 bytes Disk /dev/mapper/VolGroup-lv_swap: 855 MB, 855638016 bytes
-- Arun Khan
In article CAHhM8gD+hFdUyy7uAH3KX2h37cA5fDbBWtJWYckV9Tp3_4nzQw@mail.gmail.com, Arun Khan knura9@gmail.com wrote:
The system is an AWS Instance based on a community CentOS 6.4 AMI snapshot.
The vdisk is as follows as shown below [1] The root LVM contains /var/log/
I have attached another block device with ext4 FS.
I copied the files from /var/log to this device (mounted on /mnt) and then changed /etc/fstab to mount this device on /var/log on boot.
However, I do not see anything being logged in /var/log/messages. To test the logging, I used the 'logger' command to log some string; nothing appears in /var/log/messages.
'service rsyslog status' reports the daemon is running.
When I stop rsyslog, umount the /var/log device and then restart rsyslog, I can see that logs are being recorded in /var/log/messages. Using the 'logger' command I can see messages written in /var/log/messages.
man pages of ryslog.conf and rsyslogd show nothing related to logs being on a separate device
Any pointers to fix the problem would be much appreciated.
Probably rsyslog is being started before /var/log is mounted, and so it is opening files within /var/log on the root device.
When the second device gets mounted on /var/log, the files within the original /var/log are no longer visible, but rsyslog still has open handles to them.
You need to arrange for rsyslog to get restarted or HUPed after the mounting of /var/log.
Cheers Tony
On Wed, Aug 06, 2014 at 04:50:41PM +0000, Tony Mountifield wrote:
Probably rsyslog is being started before /var/log is mounted, and so it is opening files within /var/log on the root device.
rsyslog should start after local mounts are finished.
I suspect it's selinux; /var/log should have a "var_log_t" context and I suspect it doesn't.
John
----- Original Message ----- | On Wed, Aug 06, 2014 at 04:50:41PM +0000, Tony Mountifield wrote: | > | > Probably rsyslog is being started before /var/log is mounted, and | > so it | > is opening files within /var/log on the root device. | | rsyslog should start after local mounts are finished. | | I suspect it's selinux; /var/log should have a "var_log_t" context | and I | suspect it doesn't.
running a restorecon -vv on /var/log should correct that automatically I would think.
SOLVED
On Wed, Aug 6, 2014 at 10:28 PM, James A. Peltier jpeltier@sfu.ca wrote:
----- Original Message ----- | On Wed, Aug 06, 2014 at 04:50:41PM +0000, Tony Mountifield wrote: | > | > Probably rsyslog is being started before /var/log is mounted, and | > so it | > is opening files within /var/log on the root device. | | rsyslog should start after local mounts are finished. | | I suspect it's selinux; /var/log should have a "var_log_t" context | and I | suspect it doesn't.
running a restorecon -vv on /var/log should correct that automatically I would think.
I had suspected SElinux and have it disabled still rsyslogd was not logging on the new device mounted on /var/log/
*** restorecon -vv /var/log does the trick! ***
@ James A. Peltier Thank you!
FWIW - here are the steps
1. service rsyslog stop 2. mount <new var log device> /mnt/ 3. rsync -aP /var/log/ /mnt/ 4. rm -fr /var/log/* 5. umount /mnt 6. mount <new var log device> /var/log/ (also make change to /etc/fstab) 7. restorecon -vv /var/log <<< the solution 8. service rsyslog start. 9. logger "this is a test" 10. tail /var/log/messages to verify that indeed the logger string was logged.
-- Arun Khan
On 08/07/2014 05:48 AM, Arun Khan wrote:
SOLVED
On Wed, Aug 6, 2014 at 10:28 PM, James A. Peltier jpeltier@sfu.ca wrote:
----- Original Message ----- | On Wed, Aug 06, 2014 at 04:50:41PM +0000, Tony Mountifield wrote: | > | > Probably rsyslog is being started before /var/log is mounted, and | > so it | > is opening files within /var/log on the root device. | | rsyslog should start after local mounts are finished. | | I suspect it's selinux; /var/log should have a "var_log_t" context | and I | suspect it doesn't.
running a restorecon -vv on /var/log should correct that automatically I would think.
I had suspected SElinux and have it disabled still rsyslogd was not logging on the new device mounted on /var/log/
*** restorecon -vv /var/log does the trick! ***
@ James A. Peltier Thank you!
FWIW - here are the steps
- service rsyslog stop
- mount <new var log device> /mnt/
- rsync -aP /var/log/ /mnt/
- rm -fr /var/log/*
- umount /mnt
- mount <new var log device> /var/log/ (also make change to /etc/fstab)
- restorecon -vv /var/log <<< the solution
- service rsyslog start.
- logger "this is a test"
- tail /var/log/messages to verify that indeed the logger string was logged.
-- Arun Khan _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
If restorecon fixes the problem, then you never disabled SELinux
If you untar files into a location, you should always run restorecon on the directory to fix the SELinux labels.
In article 20140806165735.GD10943@frodo.gerdesas.com, John R. Dennison jrd@gerdesas.com wrote:
On Wed, Aug 06, 2014 at 04:50:41PM +0000, Tony Mountifield wrote:
Probably rsyslog is being started before /var/log is mounted, and so it is opening files within /var/log on the root device.
rsyslog should start after local mounts are finished.
Ah, ok, thanks. I hadn't actually gone and looked...
I suspect it's selinux; /var/log should have a "var_log_t" context and I suspect it doesn't.
Be interesting to know if that fixes it for the OP.
Cheers Tony
On 2014-08-06, John R. Dennison jrd@gerdesas.com wrote:
On Wed, Aug 06, 2014 at 04:50:41PM +0000, Tony Mountifield wrote:
Probably rsyslog is being started before /var/log is mounted, and so it is opening files within /var/log on the root device.
rsyslog should start after local mounts are finished.
I suspect it's selinux; /var/log should have a "var_log_t" context and I suspect it doesn't.
But would that explain why, when the OP umounts /var/log, the latest logs have been written to /var/log/messages on the / filesystem?
It certainly can't hurt to check both cases: make sure rsyslog is starting after the proper filesystem with /var/log is mounted, and check the selinux contexts to make sure they're correct.
--keith
On Wed, Aug 06, 2014 at 10:20:38AM -0700, Keith Keller wrote:
It certainly can't hurt to check both cases: make sure rsyslog is starting after the proper filesystem with /var/log is mounted, and check the selinux contexts to make sure they're correct.
rsyslog is started with a start priority of 12; long after /etc/rc.d/rc.sysinit finished mounting local filesystems. Syslog's initial ring-buffer is dumped nearly at the end of sysinit so that should be getting dumped to the newly-mounted /var/log as well.
Kind of curious as to why that is not happening.
John
-
Arun Khan -
Daniel J Walsh -
James A. Peltier -
John R. Dennison -
Keith Keller -
tony＠softins.co.uk