Hi,
I have been steadily migrating machines, about 50-70 of them to Centos 4.3, mostly using Opteron Dual Core/ Tyan boxes of Supermicro origin.
They perform apache, mysql, tomcat and qmail work, depending where they exist
The main reason from the programmer side (i'm a sysadmin/ engineer) is the larger "heap size" to satisfy some of the needs. Since Java/Tomcat applications can also be I/O intensive, i'm looking at things from the systems side to improve on, kernel settings, filesystem choice, RAID configuration, etc.
Is there anyone out there in the Centos 4.3 community that's looking at similar optimation interests, and if so, what have you found thus far? Mostly i'm curious about stability, features, bugs and other aspects native to Centos 64-bit/4.3 and running Tomcat/ Java on it. Right now we are at Java 1.4, but will likely make the jump to 1.5 within the next 180 days.
So far the developers are happy with the first box I made for them, but I did not customize the kernel, or make any in-depth adjustments, just handed them a larger heap size and they seemed quite happy with that in itself.
Clustering, keeping alot of single machines separate, getting one large machine with alot of mem/cpu/i/o capacity, these are all being examined. Just curious what you think, or if you know of any great resources on the subject.
I'm in a similar boat wih regard to Mysql, but that's another story, and I have alot of actual research on that to share.
-karlski
I'm in a similar boat wih regard to Mysql, but that's another story, and I have alot of actual research on that to share.
Most of the tuning I've done so far has been filesystem related, and application specific with respect to recommendations for tunables in /etc/sysctl.conf, but I'd be very interested to hear about what you've got for mysql tuning. Anything you'd be willing to share with the community I'm sure would be warmly welcomed.
Java eh? I guess that means memory hog?
Mostly Opterons you say...so...what are the none Opterons running? Have you noticed swapping on non-Opteron boxes?
Opterons have automatic memory handling advantages without tuning under the Linux kernel.
As for I/O, it depends on what kind of patterns you have and what kind of behaviour do you require. Do you have lots of files in a single directory? Do you require sync? What is your RAID for?
I have had plenty experience in squeezing performance out of 2.6 kernels but not that much with Centos 4.x boxes except for discovering that Centos 4.x kernels don't cut it. As to why I found out later but that only applies to Intel-based processors apparently...that is the bad vm patch Redhat applied to their kernels of which the removal will come in Update 4.
RHEL 4 limiting themselves to ext3 leads to there being less to try...
Anyway, I am very into optimization since that was how I did not have ask for extra machines for the mail server clusters that I use to watch over in my previous job.
Yup, many of the boxes are at 95% of both CPU's. (Dual-core 2Ghz Opteron 64-bit Centos 4.3 with 3 GB of RAM)
2.6.9-34.ELsmp #1 SMP Thu Mar 9 06:23:23 GMT 2006 x86_64 x86_64 x86_64
Mem: 3963184k total, 3943972k used, 19212k free, 15676k buffers Swap: 4184892k total, 11692k used, 4173200k free, 165400k cached
Filesystem Size Used Avail Use% Mounted on /dev/i2o/hda2 2.0G 343M 1.6G 18% / /dev/i2o/hda1 99M 20M 75M 21% /boot none 1.9G 0 1.9G 0% /dev/shm /dev/i2o/hda7 1.8G 42M 1.6G 3% /tmp /dev/i2o/hda3 4.1G 2.0G 2.0G 50% /usr /dev/i2o/hda5 122G 29G 88G 25% /var
What's strange is that for the same applications, running on exactly the same machines, and load balanced, the performance is different, one box is much busier per se.
Can you enlighten me on why perhaps the 4.x wasn't cutting it? Mostly the intel-based you mean right? -we are solidly opteron looking toward the future.
Interested to find out what tools and commands you usually use to prove that java is *just really busy* vs. some kind of tuning issue?
Do you think installing tools like sar and iostat are worthwhile, or is it really just a matter of vmstat and lsof, and looking at settings in /proc?
Any tools you'd recommend? Any Whitepapers?
When I saw the artcile in Linux Journal about what you can do with CentOS, I laughed because it was way too simplified of an explanation, -we are doing so much with it.
We *do* run some pretty intensive stuff, just want to get all my facts/figures straight before making the case to add more memory.
And if so, which choice to make there, (e.g. Mysql recommends ECC RAM). Probably going with ECC.
With technologies like SAS, it was always about balancing I/O across multiple spindles, but this current environ only uses about 8 disks per server, RAID 1 i'm fairly certain.
-karl
Java eh? I guess that means memory hog?
Mostly Opterons you say...so...what are the none Opterons running? Have you noticed swapping on non-Opteron boxes?
Opterons have automatic memory handling advantages without tuning under the Linux kernel.
As for I/O, it depends on what kind of patterns you have and what kind of behaviour do you require. Do you have lots of files in a single directory? Do you require sync? What is your RAID for?
I have had plenty experience in squeezing performance out of 2.6 kernels but not that much with Centos 4.x boxes except for discovering that Centos 4.x kernels don't cut it. As to why I found out later but that only applies to Intel-based processors apparently...that is the bad vm patch Redhat applied to their kernels of which the removal will come in Update 4.
RHEL 4 limiting themselves to ext3 leads to there being less to try...
Anyway, I am very into optimization since that was how I did not have ask for extra machines for the mail server clusters that I use to watch over in my previous job. _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
-
Feizhou -
Jim Perrin -
Karl R. Balsmeier