[CentOS] [OT] Memory Models and Multi/Virtual-Cores -- Software IOTLB

Bryan J. Smith b.j.smith at ieee.org
Wed Jun 29 06:15:01 UTC 2005


On Wed, 2005-06-29 at 00:26 -0400, Peter Arremann wrote:
> RedHat and others don't want to have to support two separate kernels - so
> they limit IO to the lowest 4GB no matter if you're running an Opteron
> or EM64T. 

On Wed, 2005-06-29 at 00:01 -0500, Bryan J. Smith wrote:
> ?  I was unaware this is how they handled Opteron.  I thought Red Hat
> _dynamically_ handled EM64T separately in their x86-64 kernels, and that
> was a major performance hit.

Looking again at the release notes ...

http://www.centos.org/docs/3/release-notes/as-amd64/RELEASE-NOTES-U2-
x86_64-en.html#id3938207

>From the looks of it, it's not just whether memory mapped I/O is above
4GiB, but _any_ direct memory access (DMA) by a device where either the
source or destination is above 4GiB.  I.e., the memory mapped I/O might
be below 4GiB, but the device might be executing a DMA transfer to user
memory above 4GiB.

That's where the "Software IOTLB" comes in, _only_enabled_ on EM64T.

If I remember back to the March 2004 onward threads on the LKML, that's
how they dealt with it -- using pre-allocated kernel bounced buffers
below 4GiB.  A Linux/x86-64 kernel _always_ uses an I/O MMU -- it is
just software for EM64T if either the source or destination address of a
DMA transfer is above 4GiB.

I don't think it really matters where the memory mapped I/O is itself.
Although it obviously is advantageous if it is setup under 4GiB on EM64T
-- because it would only need the "bounce buffers" when a DMA transfer
is to user memory above 4GiB, instead of _always_ if the memory mapped
I/O was above 4GiB.


-- 
Bryan J. Smith                                     b.j.smith at ieee.org 
--------------------------------------------------------------------- 
It is mathematically impossible for someone who makes more than you
to be anything but richer than you.  Any tax rate that penalizes them
will also penalize you similarly (to those below you, and then below
them).  Linear algebra, let alone differential calculus or even ele-
mentary concepts of limits, is mutually exclusive with US journalism.
So forget even attempting to explain how tax cuts work.  ;->





More information about the CentOS mailing list