[CentOS] Slow NFS writes

Mon Oct 20 18:42:57 UTC 2008
Craig White <craigwhite at azapple.com>

On Mon, 2008-10-20 at 13:57 -0400, Joshua Baker-LePain wrote:
> On Mon, 20 Oct 2008 at 10:49am, Craig White wrote
> 
> > I was using a 648 Gb 'PSD' file which surely is beyond caching.
> 
> Your initial email stated "648 Megabyte Photoshop file (PSD)".  Also, 
> your transfer times are on the order of 20 seconds.  Unless you have a 
> network running at 32 GB/s, I don't think you mean 648 GB.
----
you're correct, brain not fully engaged yet. Transferring 549 GB did
take me around 18 hours or so.
----
> 
> As an aside, it'd be fun to watch Photoshop try to open a file over half a 
> terabyte in size.
----
I was playing around with that but what I felt was that I was testing
that particular Macintosh/Photoshop ability to VM rather than server
speed.
----
> > You definitely were correct in your assertion and stupid me should have
> > tested it from another Linux box.
> >
> > $ time cp BackgroundGraphic.psd /home/filesystems/srv-adv/
> >
> > real    0m18.547s
> > user    0m0.015s
> > sys     0m3.306s
> >
> > so the problem isn't NFS slow writes...it's slow NFS writes from
> > Macintosh client  ;-(
> 
> Get rid of the Macs.  Problem solved.  ;)
----
Yeah but that isn't gonna happen...client is an Advertising Agency.

after adjusting settings some more, I got this when copying the file
from cli (using time command)...

0.005u 8.161s 0:19.49 41.8%     0+0k 0+20737io 0pf+0w

but you may be correct in terms of caching speeding things up. That
happened to be the largest PSD file I could find but I suppose I could
just create a dummy file and copy that to test. The 19.49 seconds speed
is much the same as I found using the AFP or SMB connections via the GUI
(Finder) allowing for the inaccuracy of my ability to start/stop the
time.

So I created a 1.5 gigabyte file using dd and copied that. What I did
figure out was that using the command line, copying the file took about
46 seconds via NFS mount but using the Finder to copy the same exact
file took almost 2 minutes longer to copy. There's obviously a lot of
latency in Macintosh Finder operations when 'writing' a file to an NFS
mount.

That same latency doesn't exist when using SMB or AFP mounts (copying
this same file using AFP took 37 seconds). I didn't find any caching
impact when copying any of these files and have found NFS usable, but
clearly a second class performer and thus, not worth considering.

For purposes of leaving a searchable footprint on the topic, this is
what I found to be the best NFS settings on the Macintosh client
(network is 1000BaseT)...

-P (secure/privileged port or otherwise you have to specify insecure on
NFS export)
-3 (to ensure NFS v3)
-T (to ensure TCP)
-intr (for softer landing if NFS server is unavailable)
-r=32768
-w=32768

Craig