On Sun, 2009-07-05 at 17:32 +0300, fabian wrote:

...

2009/7/5 fabian fabian@baladia.gov.kw

...

i notced recently that the cpu utilization peaks to abt 100 % at 4 am every day and becomes normall at 10 am

cpu 4.5%us 2.3% sy 0.0%ni 2% id 95% wa

notice the 95% wa

It looks like you might be misreading the idle % for CPU activity. A 95% idle shows the system is not doing much.

so sorry abouyt my typing

its cpu==>4.5%us sy==> 2.3% ni==> 0.0% id==> 2 % wa==> 95%

so the wa=95 %

this starts about 4 am and stops about 10 am then the CPU idle time is always between 90% to 99%

----

Your in Time Wait there! Look at your disk i/o and processor ques for clam av and havp along with squid.

Clam AV in sync with havp and squid with lots of connections can bring a server to its knees real quick. You can start by looking at your config for clamd that controls the squid scanning and logging. If no problems there then look elsewhere. look at your running processes.

John

Dear Guys

Thanks and apprecite your quick replies By the way i had stated solving the problem of almost 100% CPU utulization starting everyday at 4 am and ending at 10 am in a very crude way top was reporting 95% wa

i started to check the crontab file and noticed that the cron.daily entry in the file sarts at 4 am

and in the directory i found

-rwxr-xr-x 1 root root 133 Jul 6 16:59 00webalizer -rwxr-xr-x 1 root root 379 Jul 6 17:02 0anacron lrwxrwxrwx 1 root root 39 Nov 9 2008 0logwatch -> /usr/share/logwatch/scripts/logwatch.pl drwxr-xr-x 2 root root 4096 Jul 5 21:25 backup -rw-r--r-- 1 root root 62 Jul 6 17:37 clamav -rwxr-xr-x 1 root root 118 Jul 6 17:00 cups -rwxr-xr-x 1 root root 456 Nov 3 2008 freshclam -rwxr-xr-x 1 root root 26 Jul 6 17:00 havp -rwxr-xr-x 1 root root 180 Dec 2 2007 logrotate -rwxr-xr-x 1 root root 418 Jan 6 2007 makewhatis.cron -rwxr-xr-x 1 root root 137 Jul 6 17:23 mlocate.cron -rwxr-xr-x 1 root root 2181 Jun 21 2006 prelink -rwxr-xr-x 1 root root 114 Jul 6 17:01 rpm -rwxr-xr-x 1 root root 290 Jul 5 21:24 tmpwatch

i started movies some of thes files to difereent directory and finally found that it was the below file that was the culprit

-rwxr-xr-x 1 root root 290 Jul 5 21:24 tmpwatch ------- the contents of the file was

/usr/sbin/tmpwatch -x /tmp/.X11-unix -x /tmp/.XIM-unix -x /tmp/.font-unix \ -x /tmp/.ICE-unix -x /tmp/.Test-unix 240 /tmp /usr/sbin/tmpwatch 720 /var/tmp for d in /var/{cache/man,catman}/{cat?,X11R6/cat?,local/cat?}; do if [ -d "$d" ]; then /usr/sbin/tmpwatch -f 720 "$d" fi done ----------------

now everything works fine without this file in the /etc/cron.daily directory so the tmpwatch was the culprit cause the CPU wait state to almost 99 % for almost 6 hrs but jus would like to know if this particular script has any signifance or any perfomance issue and my system will run perfect without the file

apprecite you help

regards

Fabian

fabian wrote:

...

...

2009/7/5 fabian fabian@baladia.gov.kw

...

i notced recently that the cpu utilization peaks to abt 100 % at 4 am every day and becomes normall at 10 am

cpu 4.5%us 2.3% sy 0.0%ni 2% id 95% wa

notice the 95% wa

It looks like you might be misreading the idle % for CPU activity. A 95% idle shows the system is not doing much.

so sorry abouyt my typing

its cpu==>4.5%us sy==> 2.3% ni==> 0.0% id==> 2 % wa==> 95%

so the wa=95 %

this starts about 4 am and stops about 10 am then the CPU idle time is always between 90% to 99%

Wait means the kernel is waiting for some device i/o command(s) to complete. The CPU is idle but can't do anything useful without the results from the device, probably disk or network. Starting at 4 am makes the 'updatedb'run that builds the database for the locate command a likely culprit. Do you have a lot of files, slow disks, or perhaps some network mounts that are included?

-- Les Mikesell lesmikesell@gmail.com

---

CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos

-- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.

Dear All,

I had solved the problem earlier of CPU wait state being almost 100% and had solved the problem of disbling the cron job of tmpwatch

-rwxr-xr-x 1 root root 290 Jul 5 21:24 tmpwatch

which is a cron.daily file

now more investigations reveleved to me that

since im running havp 0.89 with squid i have a directory created by havp in

/var/tmp

drwxrwx--- 2 havp havp 71536640 Jul 14 18:15 havp

now if i do a cd /var/tmp i go to tmp directory which is fine

then if i say

cd havp

my machine hangs and i see that the cpu wait state shoots upto 100% now i tried goin in asingle user mode and the same problem

i ty to delete this directory i cannot

actually since the tmpwatch cron job was clearing files in /var/tmp/havp it was hangging

apprecite some help n advice how to get reid of this directory

Regards

Fabian

fabian wrote:

...

Dear Guys

Thanks and apprecite your quick replies By the way i had stated solving the problem of almost 100% CPU utulization starting everyday at 4 am and ending at 10 am in a very crude way top was reporting 95% wa

i started to check the crontab file and noticed that the cron.daily entry in the file sarts at 4 am

and in the directory i found

-rwxr-xr-x 1 root root 133 Jul 6 16:59 00webalizer -rwxr-xr-x 1 root root 379 Jul 6 17:02 0anacron lrwxrwxrwx 1 root root 39 Nov 9 2008 0logwatch -> /usr/share/logwatch/scripts/logwatch.pl drwxr-xr-x 2 root root 4096 Jul 5 21:25 backup -rw-r--r-- 1 root root 62 Jul 6 17:37 clamav -rwxr-xr-x 1 root root 118 Jul 6 17:00 cups -rwxr-xr-x 1 root root 456 Nov 3 2008 freshclam -rwxr-xr-x 1 root root 26 Jul 6 17:00 havp -rwxr-xr-x 1 root root 180 Dec 2 2007 logrotate -rwxr-xr-x 1 root root 418 Jan 6 2007 makewhatis.cron -rwxr-xr-x 1 root root 137 Jul 6 17:23 mlocate.cron -rwxr-xr-x 1 root root 2181 Jun 21 2006 prelink -rwxr-xr-x 1 root root 114 Jul 6 17:01 rpm -rwxr-xr-x 1 root root 290 Jul 5 21:24 tmpwatch

i started movies some of thes files to difereent directory and finally found that it was the below file that was the culprit

-rwxr-xr-x 1 root root 290 Jul 5 21:24 tmpwatch

the contents of the file was

/usr/sbin/tmpwatch -x /tmp/.X11-unix -x /tmp/.XIM-unix -x /tmp/.font-unix \ -x /tmp/.ICE-unix -x /tmp/.Test-unix 240 /tmp /usr/sbin/tmpwatch 720 /var/tmp for d in /var/{cache/man,catman}/{cat?,X11R6/cat?,local/cat?}; do if [ -d "$d" ]; then /usr/sbin/tmpwatch -f 720 "$d" fi done

---

now everything works fine without this file in the /etc/cron.daily directory so the tmpwatch was the culprit cause the CPU wait state to almost 99 % for almost 6 hrs but jus would like to know if this particular script has any signifance or any perfomance issue and my system will run perfect without the file

apprecite you help

regards

Fabian

man tmpwatch shows that this is designed to cleanup tmp directories - you may want to see why your tmp files are either very many or have some other issues HTH Rob

-- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.

---

CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos

fabian wrote:

...

apprecite some help n advice how to get reid of this directory

Is /var/ on a separate file system? It sounds like there is just a massive amount of files there, if /var is split out then you can move all other data off of /var and reformat it, and copy the data back.

Otherwise perhaps just run rm -rf havp and wait till it finishes, maybe take a few minutes, hours, or days.

Do you know what havp is? I'd suggest finding it's configuration and fixing it so it doesn't create so many files.

nate

Thanks nate for ur immediate reply

really apprecite

by the way i had tried to rm -rf havp but it was jus takin a real long time actually the var partition is on different file system but the partition also servers as a data for our ftp server

i will try to move the data

but was jus wondering if somethin easier could be there to solve it

as of now after disabling the cron script the systems being workin fine for almost a week with no problems

but have to find the solulation for the problem

regards

fabian

---

CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos

-- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.

fabian wrote:

...

...

fabian wrote:

...

apprecite some help n advice how to get reid of this directory

Is /var/ on a separate file system? It sounds like there is just a massive amount of files there, if /var is split out then you can move all other data off of /var and reformat it, and copy the data back.

Otherwise perhaps just run rm -rf havp and wait till it finishes, maybe take a few minutes, hours, or days.

Do you know what havp is? I'd suggest finding it's configuration and fixing it so it doesn't create so many files.

nate

Thanks nate for ur immediate reply

really apprecite

by the way i had tried to rm -rf havp but it was jus takin a real long time actually the var partition is on different file system but the partition also servers as a data for our ftp server

i will try to move the data

but was jus wondering if somethin easier could be there to solve it

as of now after disabling the cron script the systems being workin fine for almost a week with no problems

but have to find the solulation for the problem

Is this really a (normal) side effect of enabling mandatory locks on the file system as havp claims to require? (And the reason that unix-like systems rarely use mandatory locks)?

-- Les Mikesell lesmikesell@gmail.com

Once gain thanks so much guys i just managed to get rid of the /var/tmp/havp directory actually i was little in hurry for my earlier reply

what i did was i went to single user mode and did a rm -rf /var/tmp/havp and it took almost more thn 4 hrs to delete the directory and contents more thn 40 gb stuff housed in it

anyway way i put missing scripts back in my crontab files and will check tomorrow when all the users start usng the server.

I really do apprecite all ur help thanks once again

regards Fabian

also i try to see to restrict the size if i can in havp

---

CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos

-- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.

On Monday 06 July 2009, "fabian" fabian@baladia.gov.kw wrote:

now everything works fine without this file in the /etc/cron.daily directory so the tmpwatch was the culprit cause the CPU wait state to almost 99 % for almost 6 hrs but jus would like to know if this particular script has any signifance or any perfomance issue and my system will run perfect without the file

If that script is running for more than a few seconds you must have a lot of stuff in your tmp directory. You should probably fix that problem instead of disabling the script. All it does is cleanup cruft in temporary directories.