[CentOS] pagecache corruption on Tyan S3870

Tue Mar 6 22:08:08 UTC 2007
Dan Halbert <halbert at bbn.com>

To follow up on issues we are having with the Tyan S3870 (K8SSA) Opteron motherboards:

We actually saw another problem with these boxes, but only with i386 Linux (CentOS, FC6, etc.). A certain compute-intensive application that also read about 10MB of data files would get wrong answers when several instances were run in parallel. (Interestingly, a yum update I ran on the box also got occasional strange errors.)

This was an easier error to check for, since I could reproduce the error in a few minutes.

After systematically trying many different memory swaps and BIOS settings (including memory timings), I discovered that booting with "noapic" fixed the problem above. We haven't yet completed an x86_64 pagecache corruption test with "noapic", but I am pretty suspicious that these problems are related. Running with maxcpus=1 also fixes the problem, which confirms it's an smp-related problem.

I'll report back one more time if noapic fixes our pagecache problems.

Tyan updated the BIOS for this board a few versions back to fix a booting problem with x86_64 Redhat. Some people worked around that problem with noapic. I wonder if the BIOS still has some problems...

Thanks for all your suggestions,