On Sat, Jun 2, 2012 at 8:50 AM, Dennis Jacobfeuerborn <dennisml at conversis.de > wrote: > On 06/02/2012 02:16 PM, Boris Epstein wrote: > > On Sat, Jun 2, 2012 at 6:16 AM, Johnny Hughes <johnny at centos.org> wrote: > > > >> On 06/01/2012 10:26 PM, Boris Epstein wrote: > >>> On Fri, Jun 1, 2012 at 6:36 PM, John R Pierce <pierce at hogranch.com> > >> wrote: > >>> > >>>> On 06/01/12 2:27 PM, Boris Epstein wrote: > >>>>> I believe that unfsd (http://unfs3.sourceforge.net/ ) now does have > >>>>> multi-threaded capability and as such should be fairly well > scalable. I > >>>> am > >>>>> using it on CentOS 6.2 and it seems to become all but unusable when > >> more > >>>>> then 3-4 users connect to it. Is that normal? What sort of experience > >>>> have > >>>>> other people had? > >>>> yeesh, wtf ? > >>>> > >>>> latest version: 0.9.22 2009-01-05 > >>>> > >>>> WHY?!??! what problem is this supposed to solve over the built in > >>>> native Linux NFS, which supports a lot more than just NFSv3? > >>>> > >>>> > >>>> maybe in 2003, when Linux NFS was sketchy, this made sense. > >>>> > >>>> > >>>> -- > >>>> john r pierce N 37, W 122 > >>>> santa cruz ca mid-left coast > >>>> > >>>> > >>>> > >>>> John, > >>> The native NFS only supports the local file system (on the local disk). > >>> What we have here is an NFS gateway to a distributed file system, in > our > >>> case MooseFS ( http://www.moosefs.org/ ). > >>> > >> > >> You might take a look at GlusterFS for your distributed file system if > >> most of your nodes are on the same 100mbit or 1Gbit network. GlusterFS > >> is the new "big thing" that Red Hat is going to support and we use it on > >> the CentOS infrastructure and like it quite well. It is also very easy > >> to maintain and you can mount it via the glusterfs client or via NFS. > >> It does not work real well across a slower internet like in multiple > >> datacenters, but if your machines are all on a fast network with each > >> other, I highly recommend it. > >> > >> > >> John, > > > > I agree with you that GlusterFS is not bad - though neither is MooseFs, > > based on all accounts, and MooseFS is very simple and lightweight, which > > was why we chose it. At any rate, at this point this is what we are > using. > > All we need is an NFS gateway that would scale to 10-20 sessions without > > losing too much performance. > > > > And yes, it could be that it is my MooseFS that is underperforming - I am > > studying that possibility too. > > MooseFS is really only designed to host large files and to be useful if you > care about throughput but not latency. GlusterFS is going to perform much > better as a regular filesystem due to its consistent hashing approach and > is just as simple and lightweigt as MooseFS. > > But why can't you mount MooseFS locally and then export it using the > regular nfs implementation? > > Regards, > Dennis > > PS: You might also take a look at Ceph at ceph.com and Sheepdog at > www.osrg.net/sheepdog. Both two very interesting contenders. You can find > some interesting benchmarks for a 1000 node Sheepdog cluster here: > http://sheepdog.taobao.org/people/zituan/sheepdog1k.html > > Regards, > Dennis > _______________________________________________ > Dennis, Thanks for a thoughtful reply. I believe the regular NFS does not allow you to export non-local directories. That was so a few years ago; I didn't even check for myself this time around as people are saying this is still the case. Perhaps I should check. When you are saying that MooseFS is high latency - what sort of latency should I expect when accessing a file, though? There's a whole community of happy MooseFS users out there; I am not sure they'd be so happy if you had to wait for 30 seconds to just start reading a file. We could tolerate some latency here, by the way. Boris.