[CentOS] 8-15 TB storage: any recommendations?

[Christopher Chan]

Warren Young wrote:
> On 1/7/2010 6:01 PM, Christopher Chan wrote:
>>> I'm not recommending OpenSolaris on purpose.
>> Serious system administrators are not Linux fans I don't think.
> 
> I think I must have been sent back in time, say to 1997 or so, because I 
> can't possibly be reading this in 2010.  I base this on the fact that 
> your statement logically means there are no serious Linux sysadmins, 

Huh? How did YOU get from what I wrote to 'no serious Linux sysadmins'. 
I used to be Linux for everything too. Grow up.

> which is of course is so much hooey that no one believes this any more 
> in the time I come from.  Therefore, I must have been sent far enough 
> back in time that such statements were still uttered with complete 
> seriousness.

Given that there are others besides me on this list that have pointed 
elsewhere other than Linux for stuff like firewalling and I suspect 
probably throughout the history of the list, I think you have a serious 
case of tinted corneas.

> 
> I guess the other possibility is that someone's gatewayed a Usenet 
> advocacy group to this list.
> 
>> I find pkg on OpenSolaris to be more akin to yum or apt than ports
> 
> In some ways, sure.  Ports is definitely a different way of doing 
> things, though, I think, not a bad one.

Right.

> 
> There are several areas where OpenSolaris' package system falls down:
> 
> 1. No free updates.  Even if you just want security fixes, you have to 
> buy a support contract.  (If you think this is reasonable, why are you 
> here on the CentOS list, a place for discussing a solution to a 
> different but similar problem?)

I do not know what you are talking about. No free updates? OpenSolaris 
happens to be open source FYI. Maybe you should first learn about 
something before you start dissing it. Fanboys like you are what give 
Linux a bad name.

Even Solaris has free updates and I have been able to go and download 
patches. Last I did that anyway. But recently I brought this up about 
Solaris on the OpenSolaris irc channel and there are tools that automate 
all this and again, you do not need a support contract. But good luck 
trying file bug reports or get support.


> 
> 2. There is no upgrade path from release to release other than 
> "reinstall", and releases are spaced just 6 months apart.  Between this 
> and the previous problem, it means I have to reinstall my server every 6 
> months to keep up to date if I don't want to buy a support contract. 
> Those serious sysadmins where you come from like this sort of thing?  In 
> my world, we prefer OSes with long term support so we can stay current 
> on a release for years at a time.

??? What on earth are you on about? Are you talking about Ubuntu? 
Fedora? ???

I have never "reinstalled" my two OpenSolaris servers and I have 
upgraded from one release to another smoothly.

> 
> 3. The main package repo is pretty sparse.  If you want anything even a 
> little bit nonstandard you end up downloading sources from the Internet 
> and compiling by hand, which may not even succeed since Solaris is down 
> in the third tier or so of popularity these days.  At least with 
> FreeBSD's ports, you're pretty much guaranteed that it will build and 
> install with "sudo make install clean", even chasing down dependencies 
> for you automatically.

Hang on. I thought we were talking about OpenSolaris? I don't use 
Solaris because I don't like its 'dual' package management system.


> 
> 4. At least back in the 2008.05 and 2008.11 days when I last tried to 
> really use OpenSolaris, I found IPS to be quite immature.  I managed, 
> twice, to render a machine unbootable simply by removing packages I 
> thought I didn't need, using the GUI package manager.  No warnings, just 
> boot...bang.  Now maybe I'm being unrealistic, but I would think one of 
> the basic requirements for a package manager is that it know enough 
> about dependencies to refuse to let me uninstall core system components.

I like the 'I thought I didn't need' part. Especially since OpenSolaris 
gives you the ability to clone boot environments to experiment on so 
that if you do mess up, you can just boot back into a working boot 
environment. Sorry, no sympathy from me on this score if that is what 
you are going to base your reason not to use OpenSolaris at all.


>> and then there is always nexenta if I
>> really want a complete GNU userland and apt/dpkg.
> 
> How many different machines have you tried it on?  Perhaps you have been 
> lucky, and have found that it installs on everything you want it to run 
> on.

Nexenta? One. A Dell. alpha5 too. Had to drop it since the Nexenta guys 
were short handed and did not have an update to a release that had an 
iscsi fix that was necessary for exporting to Windows. I make a point to 
build hardware that works with the matrix of possible operating systems 
I will use.


> 
> In my experience, both NCP and NexentaStor made me jump through quite a 
> few hoops to find a hardware configuration they were happy with.  Even 
> after I got them working, neither seemed valuable enough to bother 
> sticking with them, compared to OSes I already know and trust to just run.

No comment. I have not had a reason to want to commit time and energy 
try Nexenta again after alpha5.

> 
>> Does it support direct sharing/exporting as nfs/cifs/iscsi
> 
> NFS, yes, that's how I'm using it.
> 
> CIFS, no, as there is no CIFS support in FreeBSD's kernel.  Of course, 
> you can always just use Samba.
> 
> iSCSI, no, because there isn't yet any iSCSI serving support in FreeBSD 
> of any kind.  Since I didn't want my ZFS pools to be directly attached 
> to another machine, but rather shared among multiple machines in 
> traditional file-server manner, this didn't cause a problem for me.

Just pointing out that FreeBSD cannot completely take on all roles that 
a person may use OpenSolaris for.

> 
> Let me bounce this ball back in your court: how about AFS, for the Macs 
> in your organization?  ZFS has no direct support for it on either 
> platform, but at least on FreeBSD and most Linuxes, it's a supported 
> package, available on demand, already preconfigured for that system. 

AFS? Hmm, all our Macs are standalone. Too bad. Anyway, I probably would 
have gone NFS if we were to roll out Macs everywhere instead of the 
Music room. Or given GlusterFS a shot. I have not had time to look at AFS.

> All you have to do is do local customizations to the configuration, set 
> it to start automatically, and you're done.  With OpenSolaris, it's a 
> fully manual process.
> 
>> Does it support using ZFS for booting
> 
> Not as part of the OS installer, but it can be done:
> 
> http://lulf.geeknest.org/blog/freebsd/Setting_up_a_zfs-only_system/
> 
> This doesn't interest me because it shares the same limitation as on 
> Solaris, which is that it will only work with a mirror.  I don't want to 
> dedicate two disks just to the OS if I want a RAID-Z pool for actual data.
> 
> My solution for high root FS reliability was to put it on a CF card.  In 
> addition to being solid state, it has a few side benefits:
> 
> - It lets me use an otherwise unused ATA connection.
> 
> - It's small enough that I can mount it in otherwise dead space in the 
> chassis, instead of taking up a precious disk bay.
> 
> Once I got the system installed, I moved some top-level trees into 
> dedicated ZFS pools, so my root filesystem is now quite small and rarely 
> touched.

But still no mirror. Not that it matters I guess in your case.

> 
>> lot more on vinum than there is on zfs in the FreeBSD manual.
> 
> I did most of my FreeBSD ZFS setup using the Solaris ZFS Admin Guide 
> PDF.  Everything it asked me to do worked fine on FreeBSD.
> 
> Yes, I'm sure you can point to places where a thing will work on Solaris 
> and not on FreeBSD, but I haven't found anything that actually *matters* 
> to me yet.


Just because it does not matter to you does not mean you can say that 
FreeBSD can replace or be used in lieu of OpenSolaris when you do not 
know what is wanted.