I guess I'm sort of surprised and I expected better performance
I have a new server set up with RAID 10 drives (6)
Repeated a number of times and though I am clumsy with stop watch timing, these numbers appear to be close enough for government work...
Server, CentOS 5.2 and updated earlier today, just installed a week ago.
Client, Macintosh G4, OS X 10.4.11
NFS Mount is done with the following options... -P (privileged ports) intr -r=32768 -w=32768 I tried doubling the size of the read/write windows to 65536 but it seemed to make little difference.
Task, Read / Write 648 Megabyte Photoshop file (PSD) Win2K = Win2K server (slow), RAID 5, Symantec EndPoint (ugh), retiring this server AFP = Netatalk from new CentOS Server SMB = Samba from new CentOS Server NFS = see above options, same CentOS Server
Copy To Win2K AFP SMB NFS 1m40.053s 0m22.566s 0m23.817s 2m11.849s Copy From Win2K AFP SMB NFS 1m34.478s 0m20.709s 0m20.823s 0m23.487s
NFS read performance was slightly slower than AFP/SMB but the write performance was poor.
I suppose that the answer is not so important because if the performance was equal to AFP and/or SMB, they'd probably just use AFP anyway but I did want to register my shock (or my ignorance).
Craig
On Sat, 18 Oct 2008 at 11:21pm, Craig White wrote
Server, CentOS 5.2 and updated earlier today, just installed a week ago.
Client, Macintosh G4, OS X 10.4.11
*snip*
Copy To Win2K AFP SMB NFS 1m40.053s 0m22.566s 0m23.817s 2m11.849s
Copy From Win2K AFP SMB NFS 1m34.478s 0m20.709s 0m20.823s 0m23.487s
Do you have any Linux clients to test with? That way you could determine whether the problem is on the server or the client side. ISTR hearing bad things about Apple's NFS implementation (shocking, I know). You also want to test with larger files (at least 2x RAM of the server or client, whichever is larger) to make sure you're not just seeing cache effects.
On Mon, 2008-10-20 at 13:21 -0400, Joshua Baker-LePain wrote:
On Sat, 18 Oct 2008 at 11:21pm, Craig White wrote
Server, CentOS 5.2 and updated earlier today, just installed a week ago.
Client, Macintosh G4, OS X 10.4.11
*snip*
Copy To Win2K AFP SMB NFS 1m40.053s 0m22.566s 0m23.817s 2m11.849s
Copy From Win2K AFP SMB NFS 1m34.478s 0m20.709s 0m20.823s 0m23.487s
Do you have any Linux clients to test with? That way you could determine whether the problem is on the server or the client side. ISTR hearing bad things about Apple's NFS implementation (shocking, I know). You also want to test with larger files (at least 2x RAM of the server or client, whichever is larger) to make sure you're not just seeing cache effects.
---- I was using a 648 Gb 'PSD' file which surely is beyond caching.
You definitely were correct in your assertion and stupid me should have tested it from another Linux box.
$ time cp BackgroundGraphic.psd /home/filesystems/srv-adv/
real 0m18.547s user 0m0.015s sys 0m3.306s
so the problem isn't NFS slow writes...it's slow NFS writes from Macintosh client ;-(
I'll play around a bit more but it's clear that no matter what I do, NFS writes will never be faster than AFP or SMB from a Macintosh so given the extra pain of setup, it's never going to happen.
Thanks
Craig
On Mon, 20 Oct 2008 at 10:49am, Craig White wrote
I was using a 648 Gb 'PSD' file which surely is beyond caching.
Your initial email stated "648 Megabyte Photoshop file (PSD)". Also, your transfer times are on the order of 20 seconds. Unless you have a network running at 32 GB/s, I don't think you mean 648 GB.
As an aside, it'd be fun to watch Photoshop try to open a file over half a terabyte in size.
You definitely were correct in your assertion and stupid me should have tested it from another Linux box.
$ time cp BackgroundGraphic.psd /home/filesystems/srv-adv/
real 0m18.547s user 0m0.015s sys 0m3.306s
so the problem isn't NFS slow writes...it's slow NFS writes from Macintosh client ;-(
Get rid of the Macs. Problem solved. ;)
On Mon, 2008-10-20 at 13:57 -0400, Joshua Baker-LePain wrote:
On Mon, 20 Oct 2008 at 10:49am, Craig White wrote
I was using a 648 Gb 'PSD' file which surely is beyond caching.
Your initial email stated "648 Megabyte Photoshop file (PSD)". Also, your transfer times are on the order of 20 seconds. Unless you have a network running at 32 GB/s, I don't think you mean 648 GB.
---- you're correct, brain not fully engaged yet. Transferring 549 GB did take me around 18 hours or so. ----
As an aside, it'd be fun to watch Photoshop try to open a file over half a terabyte in size.
---- I was playing around with that but what I felt was that I was testing that particular Macintosh/Photoshop ability to VM rather than server speed. ----
You definitely were correct in your assertion and stupid me should have tested it from another Linux box.
$ time cp BackgroundGraphic.psd /home/filesystems/srv-adv/
real 0m18.547s user 0m0.015s sys 0m3.306s
so the problem isn't NFS slow writes...it's slow NFS writes from Macintosh client ;-(
Get rid of the Macs. Problem solved. ;)
---- Yeah but that isn't gonna happen...client is an Advertising Agency.
after adjusting settings some more, I got this when copying the file from cli (using time command)...
0.005u 8.161s 0:19.49 41.8% 0+0k 0+20737io 0pf+0w
but you may be correct in terms of caching speeding things up. That happened to be the largest PSD file I could find but I suppose I could just create a dummy file and copy that to test. The 19.49 seconds speed is much the same as I found using the AFP or SMB connections via the GUI (Finder) allowing for the inaccuracy of my ability to start/stop the time.
So I created a 1.5 gigabyte file using dd and copied that. What I did figure out was that using the command line, copying the file took about 46 seconds via NFS mount but using the Finder to copy the same exact file took almost 2 minutes longer to copy. There's obviously a lot of latency in Macintosh Finder operations when 'writing' a file to an NFS mount.
That same latency doesn't exist when using SMB or AFP mounts (copying this same file using AFP took 37 seconds). I didn't find any caching impact when copying any of these files and have found NFS usable, but clearly a second class performer and thus, not worth considering.
For purposes of leaving a searchable footprint on the topic, this is what I found to be the best NFS settings on the Macintosh client (network is 1000BaseT)...
-P (secure/privileged port or otherwise you have to specify insecure on NFS export) -3 (to ensure NFS v3) -T (to ensure TCP) -intr (for softer landing if NFS server is unavailable) -r=32768 -w=32768
Craig
Client, Macintosh G4, OS X 10.4.11
NFS Mount is done with the following options... -P (privileged ports) intr -r=32768 -w=32768 I tried doubling the size of the read/write windows to 65536 but it seemed to make little difference.
Task, Read / Write 648 Megabyte Photoshop file (PSD) Win2K = Win2K server (slow), RAID 5, Symantec EndPoint (ugh), retiring this server AFP = Netatalk from new CentOS Server SMB = Samba from new CentOS Server NFS = see above options, same CentOS Server
Copy To Win2K AFP SMB NFS 1m40.053s 0m22.566s 0m23.817s 2m11.849s
Copy From Win2K AFP SMB NFS 1m34.478s 0m20.709s 0m20.823s 0m23.487s
NFS read performance was slightly slower than AFP/SMB but the write performance was poor.
I had a similar problem with a freebsd box awhile ago and the solution was to mount an nfs share with much lower r/w buffer size (2048?). There also was something in the logs related to nfs server timeouts or server not responding.
HTH
-
A. Kirillov -
Craig White -
Joshua Baker-LePain