On 06/02/2012 02:16 PM, Boris Epstein wrote: > On Sat, Jun 2, 2012 at 6:16 AM, Johnny Hughes <johnny at centos.org> wrote: > >> On 06/01/2012 10:26 PM, Boris Epstein wrote: >>> On Fri, Jun 1, 2012 at 6:36 PM, John R Pierce <pierce at hogranch.com> >> wrote: >>> >>>> On 06/01/12 2:27 PM, Boris Epstein wrote: >>>>> I believe that unfsd (http://unfs3.sourceforge.net/ ) now does have >>>>> multi-threaded capability and as such should be fairly well scalable. I >>>> am >>>>> using it on CentOS 6.2 and it seems to become all but unusable when >> more >>>>> then 3-4 users connect to it. Is that normal? What sort of experience >>>> have >>>>> other people had? >>>> yeesh, wtf ? >>>> >>>> latest version: 0.9.22 2009-01-05 >>>> >>>> WHY?!??! what problem is this supposed to solve over the built in >>>> native Linux NFS, which supports a lot more than just NFSv3? >>>> >>>> >>>> maybe in 2003, when Linux NFS was sketchy, this made sense. >>>> >>>> >>>> -- >>>> john r pierce N 37, W 122 >>>> santa cruz ca mid-left coast >>>> >>>> >>>> >>>> John, >>> The native NFS only supports the local file system (on the local disk). >>> What we have here is an NFS gateway to a distributed file system, in our >>> case MooseFS ( http://www.moosefs.org/ ). >>> >> >> You might take a look at GlusterFS for your distributed file system if >> most of your nodes are on the same 100mbit or 1Gbit network. GlusterFS >> is the new "big thing" that Red Hat is going to support and we use it on >> the CentOS infrastructure and like it quite well. It is also very easy >> to maintain and you can mount it via the glusterfs client or via NFS. >> It does not work real well across a slower internet like in multiple >> datacenters, but if your machines are all on a fast network with each >> other, I highly recommend it. >> >> >> John, > > I agree with you that GlusterFS is not bad - though neither is MooseFs, > based on all accounts, and MooseFS is very simple and lightweight, which > was why we chose it. At any rate, at this point this is what we are using. > All we need is an NFS gateway that would scale to 10-20 sessions without > losing too much performance. > > And yes, it could be that it is my MooseFS that is underperforming - I am > studying that possibility too. MooseFS is really only designed to host large files and to be useful if you care about throughput but not latency. GlusterFS is going to perform much better as a regular filesystem due to its consistent hashing approach and is just as simple and lightweigt as MooseFS. But why can't you mount MooseFS locally and then export it using the regular nfs implementation? Regards, Dennis PS: You might also take a look at Ceph at ceph.com and Sheepdog at www.osrg.net/sheepdog. Both two very interesting contenders. You can find some interesting benchmarks for a 1000 node Sheepdog cluster here: http://sheepdog.taobao.org/people/zituan/sheepdog1k.html Regards, Dennis