Hi,
I'm currently using nis/nfs3/autofs in a small network (20 boxes), and planning on using a more secure/elegant method. The thing is, which solution to adopt ? The network is mainly composed of Centos boxes, and a couple MS/Win ones. ldap/kerberos/nfs4 ? Directory Server ? Anything else ? Another point is, we have several servers with a local /data. Is there any solution to make each /data accessible to each server without having to maintain an awful fstab list per server ? (no way to deploy gfs).
Regards, Laurent.
On Wed, 2008-10-15 at 09:52 +0200, Laurent Wandrebeck wrote:
Hi,
I'm currently using nis/nfs3/autofs in a small network (20 boxes), and planning on using a more secure/elegant method. The thing is, which solution to adopt ? The network is mainly composed of Centos boxes, and a couple MS/Win ones. ldap/kerberos/nfs4 ? Directory Server ? Anything else ? Another point is, we have several servers with a local /data. Is there any solution to make each /data accessible to each server without having to maintain an awful fstab list per server ? (no way to deploy gfs).
Without knowing more specifics, you could always try using the /net automount... as in: /net/servername/data
It's ugly, and rarely used, but it works for small networks...
-I
2008/10/15 Ian Forde ian@duckland.org:
Without knowing more specifics, you could always try using the /net automount... as in: /net/servername/data
It's ugly, and rarely used, but it works for small networks...
OK, here are some more details: each /data is between 1 and 8 TB, network is gbps. Generally, we process data locally for efficiency/latency (processing often touches several tens or hundreds of GB) , but sometimes a box can be a bit overloaded, and we want to process data on another server, using nfs mounts. Our datacenter keeps growing, and mounting every /data on every box is becoming ugly. I'm willing to know if there's a cleaner solution. automount could do the trick, but it's ugly, as you said :) Thanks, Laurent
Laurent Wandrebeck wrote:
2008/10/15 Ian Forde ian@duckland.org:
Without knowing more specifics, you could always try using the /net automount... as in: /net/servername/data
It's ugly, and rarely used, but it works for small networks...
OK, here are some more details: each /data is between 1 and 8 TB, network is gbps. Generally, we process data locally for efficiency/latency (processing often touches several tens or hundreds of GB) , but sometimes a box can be a bit overloaded, and we want to process data on another server, using nfs mounts. Our datacenter keeps growing, and mounting every /data on every box is becoming ugly. I'm willing to know if there's a cleaner solution.
Is there any way you would/could consider a centralized storage solution like netapp or similar? Yes, it could be costly but you *are* currently tossing back and forth up to 160 TB of data on discreet storage. Do you do backups? Do you have 20 server boxes/towers? Or are you using rackmount blade servers? HW raid cards? What size disks?
Sorry for all the prying questions - just that your computing situation intrigues me. I'd understand your reluctance to give out this info.
2008/10/15 Toby Bluhm tkb@alltechmedusa.com:
Is there any way you would/could consider a centralized storage solution like netapp or similar? Yes, it could be costly but you *are* currently tossing back and forth up to 160 TB of data on discreet storage. Do you do backups? Do you have 20 server boxes/towers? Or are you using rackmount blade servers? HW raid cards? What size disks?
Sorry for all the prying questions - just that your computing situation intrigues me. I'd understand your reluctance to give out this info.
A centralized storage solution is impossible due to our (awfully) low IT budget. Only important data is backuped (/home and a couple other things), as we can't afford to save several TB. 3 servers are rack ones, others are towers. A bit of history: when I get employed there, we had 400GB, 1 box per user, 100mbps network, local user accounts...we are now at 30+TB, twice more boxes than users... Everytime we had to work on a new satellite, generally a new box came in and was dedicated to store and process data of this new sat. Everytime, it was a noname box, with classical hardware and a 3ware card (sometimes, I even had to use software raid *sigh*). We're always close to full capacity, and work in emergency is my daily companion (as I'm the only IT guy, having to do lots of things others than admin) Disks are, depending on the box, from 200GB to 1TB, 4 up to 24 ones. raid is mostly 5, 10 on a couple others (home server, db server) I know the way it was deployed isn't the best, unfortunately, struggling with low time and budget, it was difficult to do it a different way. I hope I answered to your questions, do not hesitate to ask if you need more details. Regards, Laurent
Laurent Wandrebeck wrote: . .
A centralized storage solution is impossible due to our (awfully) low IT budget.
I'm used to that. "We need this, this, this and that. Here's a dollar."
Only important data is backuped (/home and a couple other things), as we can't afford to save several TB. 3 servers are rack ones, others are towers. A bit of history: when I get employed there, we had 400GB, 1 box per user, 100mbps network, local user accounts...we are now at 30+TB, twice more boxes than users... Everytime we had to work on a new satellite, generally a new box came in and was dedicated to store and process data of this new sat. Everytime, it was a noname box, with classical hardware and a 3ware card (sometimes, I even had to use software raid *sigh*). We're always
I rather enjoy using SW raid.
close to full capacity, and work in emergency is my daily companion (as I'm the only IT guy, having to do lots of things others than admin) Disks are, depending on the box, from 200GB to 1TB, 4 up to 24 ones. raid is mostly 5, 10 on a couple others (home server, db server) I know the way it was deployed isn't the best, unfortunately, struggling with low time and budget, it was difficult to do it a different way.
Kinda what I figured - a conglomeration of stuff. Sounds like a situation I'd find myself in. Actually, I kinda like it.
Anyway, how about collapsing your storage down to a few roll-your-own NFS servers? Perhaps the smaller boxes could easily be moved to one server, the heavy hitters left as is & the medium boxes folded into 2-3 servers.
That said, NFS server performance on generic hardware & Linux always seems to be somewhat of an issue. While I'm not a huge fan of Sun, a few OpenSolaris boxes with ZFS could be quite nifty.
With the only resources being myself & (relatively) inexpensive generic HW, that would be my approach.
Hi,
On Wed, Oct 15, 2008 at 06:05, Laurent Wandrebeck l.wandrebeck@gmail.com wrote:
2008/10/15 Ian Forde ian@duckland.org:
Without knowing more specifics, you could always try using the /net automount... as in: /net/servername/data
It's ugly, and rarely used, but it works for small networks...
automount could do the trick, but it's ugly, as you said :)
automount is not ugly, what is ugly is to use paths that include the name of the server, in that if you change the server name the path of the files will change. This is also ugly because you end up having cross-mounts, in which machine A mounts a volume from machine B and machine B mounts a volume from machine A, so when you want to shut them down they may hang one waiting for the other one to come up (and with fstab instead of automount, you have the same problem when you boot up).
automount is actually quite a good tool if you really need to do this kind of stuff, which in your case you will probably have to anyway. The setup with automount is actually good in that volumes will be kept mounted only while they're used (if you use a short enough timeout), and in your case it seems that they will be seldomly used, so you would not have NFS mounted filesystems most of the time.
I sure recommend you to move from NIS to LDAP, for your network size OpenLDAP should be good enough, but you may want to look into a Directory Server if you want something more robust (although it will be harder to set up). When you implement LDAP, make sure you implement it over SSL if you don't want your passwords going unencrypted over the network, or use LDAP for user information only and Kerberos for authentication.
NFSv3 -> NFSv4 also looks good, but I would say this tends to be a more risky upgrade, since NFS3 is quite stable and NFS4 is still somewhat new and you may end up having some surprises with it. Personally I will still stick with NFSv3 for a while.
HTH, Filipe
On Oct 15, 2008, at 9:51 AM, "Filipe Brandenburger" <filbranden@gmail.com
wrote:
Hi,
On Wed, Oct 15, 2008 at 06:05, Laurent Wandrebeck l.wandrebeck@gmail.com wrote:
2008/10/15 Ian Forde ian@duckland.org:
Without knowing more specifics, you could always try using the /net automount... as in: /net/servername/data
It's ugly, and rarely used, but it works for small networks...
automount could do the trick, but it's ugly, as you said :)
automount is not ugly, what is ugly is to use paths that include the name of the server, in that if you change the server name the path of the files will change. This is also ugly because you end up having cross-mounts, in which machine A mounts a volume from machine B and machine B mounts a volume from machine A, so when you want to shut them down they may hang one waiting for the other one to come up (and with fstab instead of automount, you have the same problem when you boot up).
Try to write your own auto mount maps that mount to descriptive mount points rather than server names:
/archive/00, /archive/01...
automount is actually quite a good tool if you really need to do this kind of stuff, which in your case you will probably have to anyway. The setup with automount is actually good in that volumes will be kept mounted only while they're used (if you use a short enough timeout), and in your case it seems that they will be seldomly used, so you would not have NFS mounted filesystems most of the time.
I sure recommend you to move from NIS to LDAP, for your network size OpenLDAP should be good enough, but you may want to look into a Directory Server if you want something more robust (although it will be harder to set up). When you implement LDAP, make sure you implement it over SSL if you don't want your passwords going unencrypted over the network, or use LDAP for user information only and Kerberos for authentication.
If all your doing is serving up mount maps or netgroups then ldap is over kill, definitely don't put passwords in nis (or ldap) use kerberos for those.
A small user base can be handled more easily via nis then ldap you don't need to put passwords in passwd use kerberos for those.
NFSv3 -> NFSv4 also looks good, but I would say this tends to be a more risky upgrade, since NFS3 is quite stable and NFS4 is still somewhat new and you may end up having some surprises with it. Personally I will still stick with NFSv3 for a while.
For best interoperability use v3.
-Ross
On Wed, Oct 15, 2008 at 7:14 AM, Ross Walker rswwalker@gmail.com wrote:
If all your doing is serving up mount maps or netgroups then ldap is over kill, definitely don't put passwords in nis (or ldap) use kerberos for those.
A small user base can be handled more easily via nis then ldap you don't need to put passwords in passwd use kerberos for those.
Maybe it's just me, but I find sentences like the above two to be terribly confusing since they lack any punctuation to separate the separables. Since I am just learning about this area, I need all the clarity I can get. Perhaps others are in a similar situation, so, please use punctuation of some kind - separate lines, commas, semicolons, etc.
Thanks.
mhr
On Oct 15, 2008, at 2:47 PM, MHR mhullrich@gmail.com wrote:
On Wed, Oct 15, 2008 at 7:14 AM, Ross Walker rswwalker@gmail.com wrote:
If all your doing is serving up mount maps or netgroups then ldap is over kill, definitely don't put passwords in nis (or ldap) use kerberos for those.
A small user base can be handled more easily via nis then ldap you don't need to put passwords in passwd use kerberos for those.
Maybe it's just me, but I find sentences like the above two to be terribly confusing since they lack any punctuation to separate the separables. Since I am just learning about this area, I need all the clarity I can get. Perhaps others are in a similar situation, so, please use punctuation of some kind - separate lines, commas, semicolons, etc.
Sigh...
I resist top posting and trim and thread my replies, stay on topic, am polite, all while tapping out on my iPhone display.
But that ain't enough no, now I have to watch my run on sentences!
Sheesh, from now on MHR, your name will be Grumpy.
So Grumps, if my answers bring up more questions then why not just ask for clarification rather then get all over my poor punctuation?
-Ross
On Wed, Oct 15, 2008 at 7:13 PM, Ross Walker rswwalker@gmail.com wrote:
Sigh...
I resist top posting and trim and thread my replies, stay on topic, am polite, all while tapping out on my iPhone display.
But that ain't enough no, now I have to watch my run on sentences!
Sheesh, from now on MHR, your name will be Grumpy.
So Grumps, if my answers bring up more questions then why not just ask for clarification rather then get all over my poor punctuation?
Oh, such ammunition! >:^)
That's what you get for using an iPhone!
No, wait, that's cruel.
Ross, you're better than that!
Hmm, that doesn't really say it, either.
Y'know, I can't really think up a good comeback. Grump, grump, grump....
Wait! I know:
So, what did that sentence really mean?
<RBFG>
mhr (grumpy)
On Wed, Oct 15, 2008 at 11:08 PM, MHR mhullrich@gmail.com wrote:
On Wed, Oct 15, 2008 at 7:13 PM, Ross Walker rswwalker@gmail.com wrote:
Sigh...
I resist top posting and trim and thread my replies, stay on topic, am polite, all while tapping out on my iPhone display.
But that ain't enough no, now I have to watch my run on sentences!
Sheesh, from now on MHR, your name will be Grumpy.
So Grumps, if my answers bring up more questions then why not just ask for clarification rather then get all over my poor punctuation?
Oh, such ammunition! >:^)
That's what you get for using an iPhone!
No, wait, that's cruel.
Ross, you're better than that!
Hmm, that doesn't really say it, either.
Y'know, I can't really think up a good comeback. Grump, grump, grump....
Wait! I know:
So, what did that sentence really mean?
Basically, in a nutshell what I was trying to get across is:
1) Keep passwords in local passwd files or Kerberos, using NIS or LDAP for passwords is generally not a good idea as there are too many ways these can be compromised. I realize one can hack Heimdal Kerberos and OpenLDAP to work together keeping Kerberos information in LDAP like Active Directory does, but it is a complex unsupported hack that is sure to break at some point if either side is upgraded. If that's what you want, go out and buy an Active Directory server and integrate it into your Linux environment.
2) Use of LDAP for most small environments is overkill. NIS for auto-mount maps and account information (passwords stripped), is more then adequate here, but as the organization grows you may find NIS harder to manage then LDAP, so at that time I would migrate from NIS to LDAP. Of course there may be other reasons to use LDAP over NIS, such as third party application support where third party application configuration information is distributed through LDAP. Of course your choice will be based on your requirements independant of what anybody like myself says.
I hope that helps clarify things.
-Ross
On Thu, Oct 16, 2008 at 7:22 AM, Ross Walker rswwalker@gmail.com wrote:
Basically, in a nutshell what I was trying to get across is:
- Keep passwords in local passwd files or Kerberos, using NIS or LDAP for
passwords is generally not a good idea as there are too many ways these can be compromised. I realize one can hack Heimdal Kerberos and OpenLDAP to work together keeping Kerberos information in LDAP like Active Directory does, but it is a complex unsupported hack that is sure to break at some point if either side is upgraded. If that's what you want, go out and buy an Active Directory server and integrate it into your Linux environment.
- Use of LDAP for most small environments is overkill. NIS for auto-mount maps
and account information (passwords stripped), is more then adequate here, but as the organization grows you may find NIS harder to manage then LDAP, so at that time I would migrate from NIS to LDAP. Of course there may be other reasons to use LDAP over NIS, such as third party application support where third party application configuration information is distributed through LDAP. Of course your choice will be based on your requirements independant of what anybody like myself says.
I hope that helps clarify things.
Indeed, and awesomely so.
Many thanks.
mhr (no grump here :-)
2008/10/15 Filipe Brandenburger filbranden@gmail.com:
Hi,
<snip>
automount is actually quite a good tool if you really need to do this kind of stuff, which in your case you will probably have to anyway. The setup with automount is actually good in that volumes will be kept mounted only while they're used (if you use a short enough timeout), and in your case it seems that they will be seldomly used, so you would not have NFS mounted filesystems most of the time.
agreed, I guess there's no other way.
<snip>
NFSv3 -> NFSv4 also looks good, but I would say this tends to be a more risky upgrade, since NFS3 is quite stable and NFS4 is still somewhat new and you may end up having some surprises with it. Personally I will still stick with NFSv3 for a while.
I'm currently reading http://www-theorie.physik.unizh.ch/~dpotter/howto/kerberos (ldap/kerberos/nfs4 howto), and I think I'll follow that, minus nfs4. I've taken a quick look at Directory Server, it seems nice but a bit overkill for the size of the network. Thx a lot for your advice. Laurent.
-
Filipe Brandenburger -
Ian Forde -
Laurent Wandrebeck -
MHR -
Ross Walker -
Toby Bluhm