On 04/05/2011 09:00 AM, rainer@ultra-secure.de wrote:
That is really a no-brainer. In the time it takes to re-build such a "RAID", another disk might just fail and the "R in "RAID" goes down the toilet. Your 19-disk RAID5 just got turned into 25kg of scrap-metal.
As for ZFS - we're using it with FreeBSD with mixed results. The truth is, you've got to follow the development very closely and work with the developers (via mailinglists), potentially testing patches/backports from current - or tracking current from the start. It works much better with Solaris. Frankly, I don't know why people want to do this ZFS on Linux thing. It works perfectly well with Solaris, which runs most stuff that runs on Linux just as well. I wouldn't try to run Linux-binaries on Solaris with lxrun, either.
During my current work building a RAID-6 VM Host system (currently testing with SL-6 but later CentOS-6) I had a question rolling around in the back of my mind whether or not I should consider building the Host with OpenSolaris (or the OpenIndiana fork) and ZFS RAID-Z2, which I had heard performs somewhat better on Solaris. I'd then run CentOS Guest OS instances with VirtualBox.
But ... I've been reading about some of the issues with ZFS performance and have discovered that it needs a *lot* of RAM to support decent caching ... the recommendation is for a GByte of RAM per TByte of storage just for the metadata, which can add up. Maybe cache memory starvation is one reason why so many disappointing test results are showing up.
Chuck
But ... I've been reading about some of the issues with ZFS performance and have discovered that it needs a *lot* of RAM to support decent caching ... the recommendation is for a GByte of RAM per TByte of storage just for the metadata, which can add up. Maybe cache memory starvation is one reason why so many disappointing test results are showing up.
Yes, it uses most of any available RAM as cache. Newer implementations can use SSDs as a kind of 2nd-level cache ("L2-ARC"). Also, certain on-disk logs can be written out to NVRAMs directly, speeding up things even more. Compared with Cache-RAM in RAID-Controllers, RAM for servers is dirt-cheap.
The philosophy is: why put tiny, expensive amounts of RAM into the RAID-controller and have it try to make guesses on what should be cached and what not - if we can add RAM to the server directly at a fraction of the cost and let the OS handle _everything_ short of moving the disk-heads over the platters.
IMO, it's a brilliant concept.
Do you know if there is a lot of performance-penalty with KVM/VBox, compared to Solaris Zones?
On Apr 5, 2011, at 4:19 PM, rainer@ultra-secure.de wrote:
But ... I've been reading about some of the issues with ZFS performance and have discovered that it needs a *lot* of RAM to support decent caching ... the recommendation is for a GByte of RAM per TByte of storage just for the metadata, which can add up. Maybe cache memory starvation is one reason why so many disappointing test results are showing up.
Yes, it uses most of any available RAM as cache. Newer implementations can use SSDs as a kind of 2nd-level cache ("L2-ARC"). Also, certain on-disk logs can be written out to NVRAMs directly, speeding up things even more. Compared with Cache-RAM in RAID-Controllers, RAM for servers is dirt-cheap.
The philosophy is: why put tiny, expensive amounts of RAM into the RAID-controller and have it try to make guesses on what should be cached and what not - if we can add RAM to the server directly at a fraction of the cost and let the OS handle _everything_ short of moving the disk-heads over the platters.
The problem is the volatility of system RAM.
Those very expensive RAID write-back caches are usually battery backed (if they aren't don't buy them). It's the battery and the technology to recover the write cache after failure that makes it so expensive.
Look at the cost of capacitor backed SSDs vs non-capacitor backed SSDs.
When I ran a ZFS NFS server I used a RAID controller with battery backed write-back cache and the ZIL spread across the pool. Even with each disk as RAID0 setup, it was able to perform just as well as a similar pool using a standard SAS controller and a good SSD drive but the SSD setup cost twice as much and unlike the SSD the NVRAM doesn't burnout or need trimming.
IMO, it's a brilliant concept.
Do you know if there is a lot of performance-penalty with KVM/VBox, compared to Solaris Zones?
Solaris zones is not hardware virtualization it is a container technology like 'jail' or openvz but with resource management.
-Ross
-
Chuck Munro -
rainer＠ultra-secure.de -
Ross Walker