Something has killed any writes to /var/log/messages. Syslogd is running. Has been off for some time and I just discovered it. Any hints as to what / where to look since syslogd is running?
Sam
Something has killed any writes to /var/log/messages. Syslogd is running. Has been off for some time and I just discovered it. Any hints as to what / where to look since syslogd is running?
Is the partition full? Are there any messages in the other logs such as audit or security? Does lsof say that anything has the file open?
j
Jason Bradley Nance wrote:
Something has killed any writes to /var/log/messages. Syslogd is running. Has been off for some time and I just discovered it. Any hints as to what / where to look since syslogd is running?
Is the partition full? Are there any messages in the other logs such as audit or security? Does lsof say that anything has the file open?
Nope.. partition plenty of space, and lsof does not show an open. No other messages in any other logs either. I just rebooted thinking that might clear things up, but apparently not.
Nope.. partition plenty of space, and lsof does not show an open. No other messages in any other logs either. I just rebooted thinking that might clear things up, but apparently not.
Is this a default syslog config or has it been modified to maybe redirect to a logging box? Have you messed with your SELinux stuff?
j
Jason Bradley Nance wrote:
Nope.. partition plenty of space, and lsof does not show an open. No other messages in any other logs either. I just rebooted thinking that might clear things up, but apparently not.
Is this a default syslog config or has it been modified to maybe redirect to a logging box? Have you messed with your SELinux stuff?
j
No, logging box -- everything as far as syslog and messages is default. SELINUX not enabled. Perms are correct on the file(s). Can't seem to find anything that would be causing writes to fail, but don't know for sure if anything has written yet either. I'll give a hup to named and see what happens.
No, logging box -- everything as far as syslog and messages is default. SELINUX not enabled. Perms are correct on the file(s). Can't seem to find anything that would be causing writes to fail, but don't know for sure if anything has written yet either. I'll give a hup to named and see what happens.
Have you check the signature on the binaries to make sure someone hasn't replaced your syslog with a cracked one?
BTW, if you are running the chroot'd named most of it's stuff get's written to /var/named/chroot/log/*, not syslog.
j
Jason Bradley Nance wrote:
No, logging box -- everything as far as syslog and messages is default. SELINUX not enabled. Perms are correct on the file(s). Can't seem to find anything that would be causing writes to fail, but don't know for sure if anything has written yet either. I'll give a hup to named and see what happens.
Have you check the signature on the binaries to make sure someone hasn't replaced your syslog with a cracked one?
BTW, if you are running the chroot'd named most of it's stuff get's written to /var/named/chroot/log/*, not syslog.
AFIK, the machine has not been compromised. It's pretty well sealed off with the exception of myself and 2 other very trusted users. Not exposed even on port 80. Named is really only caching, and I do know from past kills, it does write to /var/log/messages. I'm very tempted to boot again and see if something shows up somewhere else, but one of my main jobs just started up and I hate to kill it off due to time constraints.
William L. Maltby wrote:
On Tue, 2006-06-27 at 15:06 -0400, Sam Drinkard wrote:
Jason Bradley Nance wrote:
No, logging box -- everything as far as syslog and messages is default. SELINUX not enabled. <snip>
IIRC, the new(?) dbus used for this? If so, is it running?
HTH
Yep.. dbus is running too
On Tue, 2006-06-27 at 15:06 -0400, Sam Drinkard wrote:
Jason Bradley Nance wrote:
<snip>
AFIK, the machine has not been compromised. It's pretty well sealed off with the exception of myself and 2 other very trusted users. Not exposed even on port 80. Named is really only caching, and I do know from past kills, it does write to /var/log/messages. I'm very tempted to boot again and see if something shows up somewhere else, but one of my main jobs just started up and I hate to kill it off due to time constraints.
Well, if you're not worried about a compromise under these circumstances... ;-)) I'd let your jobs finish and not sweat about it. You said you had plenty of disk space, did you "df -i" to see if you exhausted your i-nodes (unlikely, I know, but no assumptions are warranted now).
Do you have quotas? Any chance they hit someone they weren't supposed to hit? Permissions on the directoy still as they should be?
[wild-bill@wlmlfs08 ~]$ ls -dl /var/log drwxr-xr-x 22 root root 4096 Jun 25 04:02 /var/log
As folks have mentioned in other threads, a chkrootkit run might be appropriate if you can't find the cause.
On Tue, 2006-06-27 at 15:48 -0400, William L. Maltby wrote:
On Tue, 2006-06-27 at 15:06 -0400, Sam Drinkard wrote:
Jason Bradley Nance wrote:
<snip>
<snip>
You might want to to a "man syslogd" and restart logging with debug turned on and some other parameters set. This should not affect your running Honey-Toasted-Number-Crunchers.
William L. Maltby wrote:
<snip>
You might want to to a "man syslogd" and restart logging with debug turned on and some other parameters set. This should not affect your running Honey-Toasted-Number-Crunchers.
Ok.. maybe this will help get me back... few weeks back, talking about shutting down unneeded processes, and servers. I did such. About the only thing I see NOT running that was before, but *shouldn't * affect, or I'd think it would not affect syslogd is portmap.
You are correct Bill.. time for man syslogd and see what I need to do to debug the thing. All the obvious have been checked out.
Thanks....
On Tue, 2006-06-27 at 16:08 -0400, Sam Drinkard wrote:
William L. Maltby wrote:
<snip>
<snip>
Ok.. maybe this will help get me back... few weeks back, talking about shutting down unneeded processes, and servers. I did such. About the only thing I see NOT running that was before, but *shouldn't * affect, or I'd think it would not affect syslogd is portmap.
I can't recall ever seeing logging affected by that. But, the logging does use ports (unix ports, IIRC), so maybe there is some error or some connection I'd never seen? Used to be tcpwrappers and portmap worked together and I always did a deny all and enable local. But that was always only IP related (IP #s, host-domain-names,...).
You are correct Bill.. time for man syslogd and see what I need to do to debug the thing. All the obvious have been checked out.
And I would not be surprised if just restarting it fixed it too! You know how obtuse these damn things can be!
<snip>
Good luck.
William L. Maltby wrote:
On Tue, 2006-06-27 at 16:08 -0400, Sam Drinkard wrote:
William L. Maltby wrote:
<snip>
<snip>
Ok.. maybe this will help get me back... few weeks back, talking about shutting down unneeded processes, and servers. I did such. About the only thing I see NOT running that was before, but *shouldn't * affect, or I'd think it would not affect syslogd is portmap.
I can't recall ever seeing logging affected by that. But, the logging does use ports (unix ports, IIRC), so maybe there is some error or some connection I'd never seen? Used to be tcpwrappers and portmap worked together and I always did a deny all and enable local. But that was always only IP related (IP #s, host-domain-names,...).
You are correct Bill.. time for man syslogd and see what I need to do to debug the thing. All the obvious have been checked out.
And I would not be surprised if just restarting it fixed it too! You know how obtuse these damn things can be!
<snip>
Good luck.
No joy. Q. I see a process called "klogd" is running a different pid from syslogd. I don't ever recall seeing something as klogd before?? I got a sneaking suspicion I stopped something, but if I only knew what besides syslogd was required. Portmap does not apparently need to be running, as nothing still has been written to the log file, and a reboot did not help. Rats.....
On Tue, 2006-06-27 at 16:28 -0400, Sam Drinkard wrote:
William L. Maltby wrote:
On Tue, 2006-06-27 at 16:08 -0400, Sam Drinkard wrote:
William L. Maltby wrote:
<snip>
And I would not be surprised if just restarting it fixed it too! You know how obtuse these damn things can be!
<snip>
Good luck.
No joy. Q. I see a process called "klogd" is running a different pid from syslogd. I don't ever recall seeing something as klogd before?? I got a sneaking suspicion I stopped something, but if I only knew what besides syslogd was required. Portmap does not apparently need to be running, as nothing still has been written to the log file, and a reboot did not help. Rats.....
I have klogd too. I think it's probably related to your prob somehow.
[root@wlmlfs08 InstallUpdate]# grep -irl klogd /etc/rc.d /etc/rc.d/rc1.d/K88syslog /etc/rc.d/rc0.d/K88syslog /etc/rc.d/rc3.d/S12syslog /etc/rc.d/rc4.d/S12syslog /etc/rc.d/rc2.d/S12syslog /etc/rc.d/rc5.d/S12syslog /etc/rc.d/rc6.d/K88syslog /etc/rc.d/init.d/syslog
<snip sig stuff>
Did you get my corection to myself on ls -dl /dev/syslog? Any hope on that?
William L. Maltby wrote:
On Tue, 2006-06-27 at 16:28 -0400, Sam Drinkard wrote:
William L. Maltby wrote:
On Tue, 2006-06-27 at 16:08 -0400, Sam Drinkard wrote:
William L. Maltby wrote:
<snip>
And I would not be surprised if just restarting it fixed it too! You know how obtuse these damn things can be!
<snip>
Good luck.
No joy. Q. I see a process called "klogd" is running a different pid from syslogd. I don't ever recall seeing something as klogd before?? I got a sneaking suspicion I stopped something, but if I only knew what besides syslogd was required. Portmap does not apparently need to be running, as nothing still has been written to the log file, and a reboot did not help. Rats.....
I have klogd too. I think it's probably related to your prob somehow.
[root@wlmlfs08 InstallUpdate]# grep -irl klogd /etc/rc.d /etc/rc.d/rc1.d/K88syslog /etc/rc.d/rc0.d/K88syslog /etc/rc.d/rc3.d/S12syslog /etc/rc.d/rc4.d/S12syslog /etc/rc.d/rc2.d/S12syslog /etc/rc.d/rc5.d/S12syslog /etc/rc.d/rc6.d/K88syslog /etc/rc.d/init.d/syslog
<snip sig stuff>
Did you get my corection to myself on ls -dl /dev/syslog? Any hope on that?
Yes, sure did Bill. I knew what you meant :-). My head hurts! I don't know what else to do at this point. I'm gonna take a break and think about it. I restarted a bunch of services that I think used to be running, but I really believe they have nothing to do with syslogd. As you stated, man syslogd and see how to start in debug mode. Strace starts it and detaches as it should, so nothing to really see there.
Thanks all for the help.. I'm sure I'll be back.....!
Sam
On Tue, 2006-06-27 at 16:49 -0400, Sam Drinkard wrote:
William L. Maltby wrote:
<snip>
I don't know what else to do at this point. I'm gonna take a break and think about it. I restarted a bunch of services that I think used to be running, but I really believe they have nothing to do with syslogd. As you stated, man syslogd and see how to start in debug mode. Strace starts it and detaches as it should, so nothing to really see there.
NO! You need to run the logger in debug where it does no go into BG for strace to capture everything!
Thanks all for the help.. I'm sure I'll be back.....!
Sam
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
William L. Maltby wrote:
On Tue, 2006-06-27 at 16:49 -0400, Sam Drinkard wrote:
William L. Maltby wrote:
<snip>
I don't know what else to do at this point. I'm gonna take a break and think about it. I restarted a bunch of services that I think used to be running, but I really believe they have nothing to do with syslogd. As you stated, man syslogd and see how to start in debug mode. Strace starts it and detaches as it should, so nothing to really see there.
NO! You need to run the logger in debug where it does no go into BG for strace to capture everything!
Bill,
I finally figured that out, but strace generates soooo much output.. I opened syslogd in debug mode, and have been watching it for some bit now. When a write *should* go to messages, it appears that it is going to "UNUSED" for some reason. All the other logging stuff, cron, mail, etc., all appear to be working correctly, so it boils down to theis UNUSED whatever it is. The actual line from the debug is this:
Message from UNIX socket: #3 Message length: 83, File descriptor: 3. logmsg: auth.info<38>, flags 2, from thunder, msg Jun 27 18:00:01 crond(pam_unix) [4006]: session opened for user rob by (uid=0) Called fprintlog, logging to UNUSED Calling select, active file descriptors (max3): 3
Zat help any ?
Sam
On Tue, 2006-06-27 at 19:06 -0400, Sam Drinkard wrote:
William L. Maltby wrote:
On Tue, 2006-06-27 at 16:49 -0400, Sam Drinkard wrote:
<snip>
All the other logging stuff, cron, mail, etc., all appear to be working correctly, so it boils down to theis UNUSED whatever it is. The actual line from the debug is this:
Message from UNIX socket: #3 Message length: 83, File descriptor: 3. logmsg: auth.info<38>, flags 2, from thunder, msg Jun 27 18:00:01 crond(pam_unix) [4006]: session opened for user rob by (uid=0) Called fprintlog, logging to UNUSED Calling select, active file descriptors (max3): 3
Zat help any ?
Sorry I wasn't here for that. Dinner came up and all priorities pale in comparison. :-(^)
Sam
<snip sig stuff>
William L. Maltby wrote:
On Tue, 2006-06-27 at 16:49 -0400, Sam Drinkard wrote:
William L. Maltby wrote:
<snip>
I don't know what else to do at this point. I'm gonna take a break and think about it. I restarted a bunch of services that I think used to be running, but I really believe they have nothing to do with syslogd. As you stated, man syslogd and see how to start in debug mode. Strace starts it and detaches as it should, so nothing to really see there.
NO! You need to run the logger in debug where it does no go into BG for strace to capture everything!
OK.. problem solved. I overlooked an entry I had set up in /etc/syslog.conf that called local8 for the new software I had installed. I removed the logging and stuff is now writing to messages as it should. Gotta figure out now how to enable the local8 logging for the LDM software.
Many thanks to all for your help, and sorry to have caused the problem myself. Should have looked closer at the syslog.conf. Didn't remember making the change.
Sam
--- Sam Drinkard sam@wa4phy.net wrote:
William L. Maltby wrote:
On Tue, 2006-06-27 at 16:49 -0400, Sam Drinkard
wrote:
William L. Maltby wrote:
<snip>
I don't know what else to do at this point. I'm gonna
take a break and think
about it. I restarted a bunch of services that I
think used to be
running, but I really believe they have nothing
to do with syslogd. As
you stated, man syslogd and see how to start in
debug mode. Strace
starts it and detaches as it should, so nothing
to really see there.
NO! You need to run the logger in debug where it
does no go into BG for
strace to capture everything!
OK.. problem solved. I overlooked an entry I had set up in /etc/syslog.conf that called local8 for the new software I had installed. I removed the logging and stuff is now writing to messages as it should. Gotta figure out now how to enable the local8 logging for the LDM software.
Many thanks to all for your help, and sorry to have caused the problem myself. Should have looked closer at the syslog.conf. Didn't remember making the change.
Sam
-- Sam W.Drinkard -- sam@wa4phy.net NOAA Cooperative Observer KAGS (snow) http://wa4phy.net Augusta Area Mesonet
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Now this is a good example of having that one software that Alex suggested (dconf) so you do not run into this problem again.... :-)
Steven
"On the side of the software box, in the 'System Requirements' section, it said 'Requires Windows or better'. So I installed Linux."
OK.. problem solved. I overlooked an entry I had set up in /etc/syslog.conf that called local8 for the new software I had installed.
So back at the very beginning when I asked you to verify your package...
rpm -V sysklogd
That would have told you that /etc/syslog.conf was changed.
In the future you should be a little more thorough when following the advice of people taking time out of their day trying to debug problems for you remotely. If you don't understand what they are asking you to do, ask.
j
Jason Bradley Nance wrote:
OK.. problem solved. I overlooked an entry I had set up in /etc/syslog.conf that called local8 for the new software I had installed.
So back at the very beginning when I asked you to verify your package...
rpm -V sysklogd
That would have told you that /etc/syslog.conf was changed.
In the future you should be a little more thorough when following the advice of people taking time out of their day trying to debug problems for you remotely. If you don't understand what they are asking you to do, ask.
j
Jason,
It was not the package itself. It's OK. it was the syslog.conf that I had made an error in an entry I didn't catch at first. There is no such thing as a "local8" and that is what I had for logging. That was what was killing the whole shebang. I knew the syslog had changed, I just didn't catch the error by being in a hurry.
Sorry to trouble you.....
-s
It was not the package itself. It's OK. it was the syslog.conf that I had made an error in an entry I didn't catch at first. There is no such thing as a "local8" and that is what I had for logging. That was what was killing the whole shebang. I knew the syslog had changed, I just didn't catch the error by being in a hurry.
You said you had DEFAULT install. Here's the exact quote:
Me: "Is this a default syslog config or has it been modified to maybe redirect to a logging box?"
You: "No, logging box -- everything as far as syslog and messages is default."
Redirecting all logging to a different level is not default.
The point is if you would have ran the command and seen that the config file had changed you could have looked into it and saw that there was something messed up.
j
-- Jason Bradley Nance What would you cry for? aitrus@tresgeek.net Swallow your pride for? Tres Geek What would you go wild for? http://tresgeek.net/ -Poe, "Wild" --
Jason Bradley Nance wrote:
It was not the package itself. It's OK. it was the syslog.conf that I had made an error in an entry I didn't catch at first. There is no such thing as a "local8" and that is what I had for logging. That was what was killing the whole shebang. I knew the syslog had changed, I just didn't catch the error by being in a hurry.
You said you had DEFAULT install. Here's the exact quote:
Me: "Is this a default syslog config or has it been modified to maybe redirect to a logging box?"
You: "No, logging box -- everything as far as syslog and messages is default."
Redirecting all logging to a different level is not default.
The point is if you would have ran the command and seen that the config file had changed you could have looked into it and saw that there was something messed up.
j
-- Jason Bradley Nance What would you cry for? aitrus@tresgeek.net Swallow your pride for? Tres Geek What would you go wild for? http://tresgeek.net/ -Poe, "Wild"
I *DID* look at the config file, and I knew when I looked, it had changed. When I stated it was a default install, I meant there was no logging box involved. If you insist on continuing on with this dialog because I didn't do as you stated, then take it offline. Nobody wants to hear two grown men bitch at each other. As far as I'm concerned, this discussion is over. I made the mistake, and I admitted it. What more do you want? A confession?
On Tue, 2006-06-27 at 19:12 -0400, Sam Drinkard wrote:
William L. Maltby wrote:
On Tue, 2006-06-27 at 16:49 -0400, Sam Drinkard wrote:
William L. Maltby wrote:
<snip>
I don't know what else to do at this point. I'm gonna take a break and think
<snip>
Many thanks to all for your help, and sorry to have caused the problem myself. Should have looked closer at the syslog.conf. Didn't remember making the change.
Welcome to the hoard! :-)) 90%+ of all probs we have are caused by ourselves.
"They say your memory is the second thing to go..." "Really? That's interesting. What's the first?" "Um... can't remember".
Sam
Glad it's solved!
On Tue, 2006-06-27 at 16:28 -0400, Sam Drinkard wrote:
William L. Maltby wrote:
On Tue, 2006-06-27 at 16:08 -0400, Sam Drinkard wrote:
<snip>
No joy. Q. I see a process called "klogd" is running a different pid from syslogd. I don't ever recall seeing something as klogd before?? I got a sneaking suspicion I stopped something, but if I only knew what besides syslogd was required. Portmap does not apparently need to be running, as nothing still has been written to the log file, and a reboot did not help. Rats.....
Do the thing Rodrigo mentioned...
Then I see 2 options: check the filedes it has (on /proc) and try executing syslogd under strace, to see exactly what is happening.
That will nail it down quick.
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Tue, 2006-06-27 at 16:28 -0400, Sam Drinkard wrote:
William L. Maltby wrote:
On Tue, 2006-06-27 at 16:08 -0400, Sam Drinkard wrote:
<snip>
No joy. Q. I see a process called "klogd" is running a different pid from syslogd. I don't ever recall seeing something as klogd before?? I got a sneaking suspicion I stopped something, but if I only knew what besides syslogd was required. Portmap does not apparently need to be running, as nothing still has been written to the log file, and a reboot did not help. Rats.....
OK. You have entered the zone where you will *quickly* look at things and jump to conclusions, like seeing a "reasonable" date on the /etc/syslog.conf file and saying "It hasn't changed". But something could change it and "touch" the date. Or an untar or cpio extract carries the date of the original file... you see where I'm going.
So this is the time to start at square one. I think it is Rodrigo's suggestion. And one that is easy is to actually look at the logfile contents and date. Here's mine for comparison, but yours could have minor variations I guess.
[root@wlmlfs08 InstallUpdate]# ls -l /etc/sysl* -rw-r--r-- 1 root root 938 Oct 4 2005 /etc/syslog.conf
I've attached the contents so uninterested parties don't have to be bored with the details.
<snip sig stuff>
Any boot-time params changed (grub.conf) that might have unexpected effects? Any system configuration changes (chkconfig...)
chkconfig --list
any help?
I'm out of ideas now.
William L. Maltby wrote:
AFIK, the machine has not been compromised. It's pretty well sealed off with the exception of myself and 2 other very trusted users. Not exposed even on port 80. Named is really only caching, and I do know from past kills, it does write to /var/log/messages. I'm very tempted to boot again and see if something shows up somewhere else, but one of my main jobs just started up and I hate to kill it off due to time constraints.
Well, if you're not worried about a compromise under these circumstances... ;-)) I'd let your jobs finish and not sweat about it. You said you had plenty of disk space, did you "df -i" to see if you exhausted your i-nodes (unlikely, I know, but no assumptions are warranted now).
Do you have quotas? Any chance they hit someone they weren't supposed to hit? Permissions on the directoy still as they should be?
[wild-bill@wlmlfs08 ~]$ ls -dl /var/log drwxr-xr-x 22 root root 4096 Jun 25 04:02 /var/log
As folks have mentioned in other threads, a chkrootkit run might be appropriate if you can't find the cause.
There is no way this machine could be compromised from outside. It just can't happen. Plenty of i-nodes, plenty of disk space, no quotas, all the lock files are correct, directory perms are OK, file perms are OK, etc. It may be time to reboot anyhow and see if it comes back, or if something pops up during the reboot -- hang the run -- I need the log files to make sure some other software is working, and it appears that the logging for it is bombed too, even tho it's got it's own logging facility, it does use syslog to write. Have tried with and without it active, and no joy.
There's gotta be something strange.. now that I think about it, my daily log got really short sometime back, but don't remember exactly when. I assumed it was due to stopping a lot of processes. Hmmm.... someone tell me what processes besides syslog and dbus are required for it.. I may have stepped on my thingy myself!
On Tue, 2006-06-27 at 16:00 -0400, Sam Drinkard wrote:
William L. Maltby wrote:
<snip>
There is no way this machine could be compromised from outside. It just can't happen. Plenty of i-nodes, plenty of disk space, no quotas, all the lock files are correct, directory perms are OK, file perms are OK, etc. It may be time to reboot anyhow and see if it comes back, or if something pops up during the reboot -- hang the run -- I need the log files to make sure some other software is working, and it appears that the logging for it is bombed too, even tho it's got it's own logging facility, it does use syslog to write. Have tried with and without it active, and no joy.
There's gotta be something strange.. now that I think about it, my daily log got really short sometime back, but don't remember exactly when. I assumed it was due to stopping a lot of processes. Hmmm.... someone tell me what processes besides syslog and dbus are required for it.. I may have stepped on my thingy myself!
The last thing I can think of, barring kernel problems or compromise...
[wild-bill@wlmlfs08 ~]$ ls -dl /var/log drwxr-xr-x 22 root root 4096 Jun 25 04:02 /var/log
<snip sig stuff>
On Tue, 2006-06-27 at 16:04 -0400, William L. Maltby wrote:
On Tue, 2006-06-27 at 16:00 -0400, Sam Drinkard wrote:
William L. Maltby wrote:
<snip>
<snip>
The last thing I can think of, barring kernel problems or compromise...
[wild-bill@wlmlfs08 ~]$ ls -dl /var/log drwxr-xr-x 22 root root 4096 Jun 25 04:02 /var/log
That was supposed to be
[wild-bill@wlmlfs08 ~]$ ls -l /dev/log srw-rw-rw- 1 root root 0 Jun 23 21:21 /dev/log
<snip sig stuff>
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Tue, 2006-06-27 at 16:00 -0400, Sam Drinkard wrote:
William L. Maltby wrote:
<snip>
Hmmm.... someone tell me what processes besides syslog and dbus are required for it.. I may have stepped on my thingy myself!
I have nothing to offer a man who *can* do that! =>:-O
<snip>
Jason Bradley Nance wrote:
Nope.. partition plenty of space, and lsof does not show an open. No other messages in any other logs either. I just rebooted thinking that might clear things up, but apparently not.
Is this a default syslog config or has it been modified to maybe redirect to a logging box? Have you messed with your SELinux stuff?
j
Nothing being written. killed -1 named, and still have 0 in messages.
-s
On Tue, 2006-06-27 at 15:02 -0400, Sam Drinkard wrote:
Jason Bradley Nance wrote:
Nope.. partition plenty of space, and lsof does not show an open. No other messages in any other logs either. I just rebooted thinking that might clear things up, but apparently not.
Is this a default syslog config or has it been modified to maybe redirect to a logging box? Have you messed with your SELinux stuff?
j
Nothing being written. killed -1 named, and still have 0 in messages.
Your syslogd config file still intact?
-s
William L. Maltby wrote:
On Tue, 2006-06-27 at 15:02 -0400, Sam Drinkard wrote:
Jason Bradley Nance wrote:
Nope.. partition plenty of space, and lsof does not show an open. No other messages in any other logs either. I just rebooted thinking that might clear things up, but apparently not.
Is this a default syslog config or has it been modified to maybe redirect to a logging box? Have you messed with your SELinux stuff?
j
Nothing being written. killed -1 named, and still have 0 in messages.
Your syslogd config file still intact?
Yes, /etc/syslogd.conf is still intact, and nothing has been changed at all. The problem apparently started at least on May 28, which is the date of /var/log/messages.4, and it and all the rest are size of 0. Just wondering if the last update I did had something to do with it. According to the yum log, a new kernel was installed on the 30th, and between there and now, there have been a few updates to spamassassin, mysql, mailman, mysql-server, kdebase and sendmail. Nothing else updated.
On Tue, 2006-06-27 at 15:30 -0400, Sam Drinkard wrote:
William L. Maltby wrote:
On Tue, 2006-06-27 at 15:02 -0400, Sam Drinkard wrote:
Jason Bradley Nance wrote:
<snip>
Your syslogd config file still intact?
Yes, /etc/syslogd.conf is still intact, and nothing has been changed at all. The problem apparently started at least on May 28, which is the date of /var/log/messages.4, and it and all the rest are size of 0. Just wondering if the last update I did had something to do with it. According to the yum log, a new kernel was installed on the 30th, and between there and now, there have been a few updates to spamassassin, mysql, mailman, mysql-server, kdebase and sendmail. Nothing else updated.
Maybe a "find -type f -newer SomeFileName" in the /etc and /var directories will get a pointer for you? If it's related to that install.
And maybe a "find / -name '*messag*'" just in case it's off in some other (chrooted) directory? Any rpmsave or rpmnew files laying around that might have been needed? Done a "dmesg"?
<snip sig stuff>
William L. Maltby wrote:
On Tue, 2006-06-27 at 15:30 -0400, Sam Drinkard wrote:
William L. Maltby wrote:
On Tue, 2006-06-27 at 15:02 -0400, Sam Drinkard wrote:
Jason Bradley Nance wrote:
<snip>
Your syslogd config file still intact?
Yes, /etc/syslogd.conf is still intact, and nothing has been changed at all. The problem apparently started at least on May 28, which is the date of /var/log/messages.4, and it and all the rest are size of 0. Just wondering if the last update I did had something to do with it. According to the yum log, a new kernel was installed on the 30th, and between there and now, there have been a few updates to spamassassin, mysql, mailman, mysql-server, kdebase and sendmail. Nothing else updated.
Maybe a "find -type f -newer SomeFileName" in the /etc and /var directories will get a pointer for you? If it's related to that install.
And maybe a "find / -name '*messag*'" just in case it's off in some other (chrooted) directory? Any rpmsave or rpmnew files laying around that might have been needed? Done a "dmesg"
Yep.. just did a find, and there are all kind of message* files, many of which I know what are, others I have no clue, but I suspect they are OK. dmesg does not give any clues either.
How about a lock file somewhere? Lemme check that....
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Tue, Jun 27, 2006 at 02:38:32PM -0400, Sam Drinkard wrote:
Something has killed any writes to /var/log/messages. Syslogd is running. Has been off for some time and I just discovered it. Any hints as to what / where to look since syslogd is running?
Are you sure syslogd is not writing to /var/log/messages.1 ?
Try: fuser /var/log/messages*
- -- Rodrigo Barbosa "Quid quid Latine dictum sit, altum viditur" "Be excellent to each other ..." - Bill & Ted (Wyld Stallyns)
Rodrigo Barbosa wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Tue, Jun 27, 2006 at 02:38:32PM -0400, Sam Drinkard wrote:
Something has killed any writes to /var/log/messages. Syslogd is running. Has been off for some time and I just discovered it. Any hints as to what / where to look since syslogd is running?
Are you sure syslogd is not writing to /var/log/messages.1 ?
Try: fuser /var/log/messages*
That returns nothing
On Tuesday 27 June 2006 15:34, Jason Bradley Nance wrote:
Did you ever check the signatures on the binaries like I suggested?
j
Honestly this looks like a compromised machine from what i see. syslog doesn't just stop working like this
you can do
lsattr /var/log/*
and see if there are some odd attributes set on these files.
Jason Bradley Nance wrote:
Did you ever check the signatures on the binaries like I suggested?
j
Yep.. everything OK in that department. Still looking and working.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Tue, Jun 27, 2006 at 03:43:50PM -0400, Sam Drinkard wrote:
Jason Bradley Nance wrote:
Did you ever check the signatures on the binaries like I suggested?
Yep.. everything OK in that department. Still looking and working.
Then I see 2 options: check the filedes it has (on /proc) and try executing syslogd under strace, to see exactly what is happening.
- -- Rodrigo Barbosa "Quid quid Latine dictum sit, altum viditur" "Be excellent to each other ..." - Bill & Ted (Wyld Stallyns)
-
Charles Lacroix -
Jason Bradley Nance -
Rodrigo Barbosa -
Sam Drinkard -
Steven Vishoot -
William L. Maltby