[CentOS] Re: Fighting some install issues with a new box -- ah, ACPI trump, maybe sheer number of busses

[Bryan J. Smith]

On Tue, 2005-07-26 at 16:05 -0700, Sean O'Connell wrote:
> I may not have mentioned that it was POSTing, but I guess it didn't
> strike me as significant (I wouldn't have expected to see it if it
> hadn't POSTed :).

You mentioned it later, which made me switch my focus away from the BIOS
to the kernel again.  So your initial instinct was correct, and I should
have realized that.

> I believe the kernels use ACPI to enumerate PCI buses. Maybe a kernel
> guru could chime in? :) As one of the potential kernel parameters is
> pci=noacpi
> noacpi                  [IA-32] Do not use ACPI for IRQ routing
>                         or for PCI scanning.

Yeah, now it makes sense.  I bet ACPI trumps the PCI setting, and is
probably set to "pci=nobios" by default in kernel 2.6.  I'm sure that
works for Intel and lower-end AMD board with only a few PCI channels.

And not this monster.  ;->

BTW, I'm running RHEL3 on the S2895s I asssembled, which are kernel 2.4
without ACPI, hence why I didn't run into it.  I loaded the nvnet for
the NIC (didn't trust forcedeth at the time), and didn't worry about any
other peripherals since I was using the 3Ware card.

> For S&G, I did try booting the machine with acpi=off and it panicd :)

Yep.

> > [ BTW, where did you find this suggestion? ]
> I was re-reading the various boot flag options in kernel-paramaters.txt
> (yum install kernel-doc), and I found a few that looked promising (see
> below). I tried pci=biosirq first (you can see why :), but then tried
> pci=bios

I'd:  

1)  Add a RHEL Bugzilla entry
2)  Drop a note to Tyan (so they are aware)
3)  Drop a note to 3Ware (so people are aware)

> bios                    [IA-32] force use of PCI BIOS, don't access
>                         the hardware directly. Use this if your machine
>                         has a non-standard PCI host bridge.

What do they mean by "non-standard"?

Everything's been pretty much "non-standard" every since AMD left GTL
and adopted EV6.  But even then, there wasn't much difference between
bridged (Intel GTL) and switched (AMD EV6), and the latter emulated GTL
from the APIC's standpoint.

Now with HyperTransport, we have a _real_ system interconnect -- no more
fudging a peripheral interconnect like PCI as a system interconnect.
But even then, HyperTransport is supposed handle this.

The only thing I can think of is the fact that unlike connecting an AMD
8131 either directly to the CPU or an AMD 8151, which are the same
vendor, connecting an AMD 8131 to a nForce Pro 2200 or nForce Pro 2050
may result in the kernel getting confused on vendor IDs at the APIC/ACPI
level.  So maybe it just ignores the fact that the AMD 8131 is there.

Then again, people previously connected the AMD 8131 to the nForce3 too,
and didn't have an issue.  So I don't know what's up here -- other than
the sheer number of PCI channels!

I mean, the sucker's got:  
2x20 PCIe channels (the nForce Pro 2200 and the nForce Pro 2050)
   2 PCI-X channels (the AMD8131)
   1 legacy PCI channel (the nForce Pro 2200)

> I was also pondering the use of (but wasn't really sure how to go about
> the value of N).
> lastbus=N               [IA-32] Scan all buses till bus #N. Can be useful
>                         if the kernel is unable to find your secondary buses
>                         and you want to tell it explicitly which ones they are.

Bus assignment is arbitrary, that's the problem.
I guess it's the sheer number of PCI busses.
More than anything I've seen, other than on a HP DL585 or Sun Sunfire
v40z.

> I'm fairly sure, but the newer BIOS seemed to have a few more options
> and reorganizes things a bit more nicely.

I no longer have the S2895 systems I assembled, so I can't try it.

> If someone has an RHEL account/entitlement (I don't), they could file a
> bug against the upstream kernel.

???

You don't have to have a RHEL account/entitlement to file a bug in
Bugzilla.  Just create a new account and have fun!
  https://bugzilla.redhat.com/bugzilla/createaccount.cgi  

-- 
Bryan J. Smith                                     b.j.smith at ieee.org 
--------------------------------------------------------------------- 
It is mathematically impossible for someone who makes more than you
to be anything but richer than you.  Any tax rate that penalizes them
will also penalize you similarly (to those below you, and then below
them).  Linear algebra, let alone differential calculus or even ele-
mentary concepts of limits, is mutually exclusive with US journalism.
So forget even attempting to explain how tax cuts work.  ;->