Paul Heinlein heinlein@madboa.com wrote:
I'm spec-ing a development server for some Haskell hackers. In particular, they need to be able to build and run the Glasgow Haskell Compiler (ghc), the x86_64 port of which isn't quite there yet. My current thought is to get an Opteron-based system, but load it (initially, at least) with the 32-bit i386 version of CentOS. That buys me near-term ghc compatibility. In the future, should ghc become 64-bit clean, I'll be able to upgrade to a full 64-bit OS.
AMD designed x86-64 to fully support running 32-bit applications _regardless_ what actual addressing model is being used by the kernel. In other words, the kernel can run the processor addressing in either legacy 36-bit mode (64GiB) or 52-bit mode (16EiB) mode, and still run 32-bit applications. The key in how AMD does this is by continuing to use 48-bit virtual addresses for both, which is how the Translation Lookahead Buffer (TLB) of the i486 onward works. The 52-bit addressing mode also uses the same Processor Address Extensions (PAE) approach as 36-bit mode.
[ NOTE: AMD has reserved a true "flat" 64-bit mode for future x86-64 implementations. Until then, the 48-bit virtual/52-bit physical register is how current Athlon64/Opterons are implemented. They are actually a 40-bit (1TiB) physical platform limitation, which is tied to its current interconnect logic (long story). ]
When you run a Linux/x86-64 kernel, you get the 52-bit (flat) mode. When you run a Linux/x86-64 kernel, you get the 36-bit mode (LOWMEM/HIGHMEM).
If you run 32-bit applications either, you will have 4GiB user-space limitations. You must also call only 32-bit libraries from 32-bit applications. That's about the only issue with x86-64 releases, the fact that you may require an i386 library, instead of a x86_64 library.
The alternative is to build the server around high-end ia32 chips and leave the Opteron for another day.
It is never recommended because of performance issues with 32-bit/4GiB models -- especially as you pass 1GiB of memory, and definitely when you exceed 4GiB.
Especially if you have a lot of I/O.
Does anyone have experience running a 32-bit-only OS on Opterons? Are there any gotchas?
None on Linux/x86-64 I'm aware of. GNU has been largely 64-bit clean since the mid-to-late '90s (thanx to Alpha, MIPS and other developments). This is unlike Windows (which was always released as 32-bit on Alpha, MIPS, etc..., long story).
Windows XP 64-bit Edition and Windows 2003 Server have many because the Win32 codebase is _not_ 64-bit clean like GNU has been for a long time. Microsoft's Windows 64 OSes use largely Win32 on Win64 (WoW) subsystem which is not very fast, instead of providing true Win64 services and programs.
The Opteron is currently a true 40-bit (1TiB) interconnect, 48-bit (256TiB) virtual, 52-bit (4TiB) register platform, including offering an I/O MMU for "safe" I/O transfers above 4GiB of memory (including any I/O card memory mapped to ranges above 4GiB, or using DMA to programs in memory above 4GiB).
In fact, for servers with 4+GiB of memory, Intel EM64T processors have the _same_ I/O limitations ("bounce buffers") as IA32 processors, whereas AMD Opterons do not.