I'm looking to find a Linux NAS solution for a Windows network. Looking into FreeNAS and OpenFiler. Anyone using these or any other solution? Any comments/suggestions?
Thanks,
Ed
On Tue, 2006-01-31 at 07:42, Ed Morrison wrote:
I'm looking to find a Linux NAS solution for a Windows network. Looking into FreeNAS and OpenFiler. Anyone using these or any other solution? Any comments/suggestions?
Any linux box running samba will work. If you want something that is appliance-like but can supply other services (email, web, ftp, etc.) if you want them, look at SME server at www.contribs.org. Version 7 that is in pre-release now is based on Centos but the administration is all through simple web forms.
Les Mikesell wrote:
On Tue, 2006-01-31 at 07:42, Ed Morrison wrote:
I'm looking to find a Linux NAS solution for a Windows network. Looking into FreeNAS and OpenFiler. Anyone using these or any other solution? Any comments/suggestions?
Any linux box running samba will work. If you want something that is appliance-like but can supply other services (email, web, ftp, etc.) if you want them, look at SME server at www.contribs.org. Version 7 that is in pre-release now is based on Centos but the administration is all through simple web forms.
Interesting project. I just searched through the web pages and didn't see a list of the various tools/packages used. What does it use for a mail server?
Cheers,
On Tue, 2006-01-31 at 10:12, Chris Mauritz wrote:
I'm looking to find a Linux NAS solution for a Windows network. Looking into FreeNAS and OpenFiler. Anyone using these or any other solution? Any comments/suggestions?
Any linux box running samba will work. If you want something that is appliance-like but can supply other services (email, web, ftp, etc.) if you want them, look at SME server at www.contribs.org. Version 7 that is in pre-release now is based on Centos but the administration is all through simple web forms.
Interesting project. I just searched through the web pages and didn't see a list of the various tools/packages used. What does it use for a mail server?
It's an odd mix of qmail with a replacement front-end in perl (qpsmtpd) that can do spam/virus scans, maildir format delivery and dovecot for imap/pop. Performance is pretty good. It also has hoarde/imp for webmail and one of the management simplifications is that if you add a 'group', you get a unix permissions group that can be used for file sharing and an email group address at the same time.
Thank you to everyone that replied. I will take your advice and subsequuent discussion and use it in my decision.
Thanks again, Ed
Ed Morrison wrote:
I'm looking to find a Linux NAS solution for a Windows network. Looking into FreeNAS and OpenFiler. Anyone using these or any other solution? Any comments/suggestions?
From the longtime experience direction, be sure to figure out the
snapshotting upfront; a good snapshot rotation can save you so much pain it's not funny; many commercial NAS (NetApp, SnapServer, EMC, etc) have this integrated. When looking at a DIY linux NAS, you may have to roll it up.
We snapshot roughly like so: [8a,12p,4p M-F] [nightly 2 nights retention] -- this setup is enough to provide instantanious file restore without having to deal with the backups. (classic "hey IT, I deleted a file last night...")
Personally I've used rdiff as a snapshot-esque tool and it works pretty well, but not very good when the underlying link dies; it's not the exact right tool. rsnapshot is also in the same vein, but again it's not quite the exact snapshot type of thing that a NetApp provides. I bet there are better choices out there...
hth, -te
On Tue, 2006-01-31 at 09:10 -0800, Troy Engel wrote:
Ed Morrison wrote:
I'm looking to find a Linux NAS solution for a Windows network. Looking into FreeNAS and OpenFiler. Anyone using these or any other solution? Any comments/suggestions?
From the longtime experience direction, be sure to figure out the
snapshotting upfront; a good snapshot rotation can save you so much pain it's not funny; many commercial NAS (NetApp, SnapServer, EMC, etc) have this integrated. When looking at a DIY linux NAS, you may have to roll it up.
We snapshot roughly like so: [8a,12p,4p M-F] [nightly 2 nights retention] -- this setup is enough to provide instantanious file restore without having to deal with the backups. (classic "hey IT, I deleted a file last night...")
Personally I've used rdiff as a snapshot-esque tool and it works pretty well, but not very good when the underlying link dies; it's not the exact right tool. rsnapshot is also in the same vein, but again it's not quite the exact snapshot type of thing that a NetApp provides. I bet there are better choices out there...
---- OK - I'm intrigued...any references about snapshots?
This may prove useful
Craig
http://en.wikipedia.org/wiki/Snapshot_%28computer_storage%29 Linux: LVM Win2003: VSS
bye, Ago
OK - I'm intrigued...any references about snapshots?
This may prove useful
Craig
On Tue, 2006-01-31 at 11:10, Troy Engel wrote:
Ed Morrison wrote:
I'm looking to find a Linux NAS solution for a Windows network. Looking into FreeNAS and OpenFiler. Anyone using these or any other solution? Any comments/suggestions?
From the longtime experience direction, be sure to figure out the
snapshotting upfront; a good snapshot rotation can save you so much pain it's not funny; many commercial NAS (NetApp, SnapServer, EMC, etc) have this integrated. When looking at a DIY linux NAS, you may have to roll it up.
I think there are still problems with LVM2 snapshots. And, you have to realize that there are certain disasters where the snapshot will melt too.
We snapshot roughly like so: [8a,12p,4p M-F] [nightly 2 nights retention] -- this setup is enough to provide instantanious file restore without having to deal with the backups. (classic "hey IT, I deleted a file last night...")
Personally I've used rdiff as a snapshot-esque tool and it works pretty well, but not very good when the underlying link dies; it's not the exact right tool. rsnapshot is also in the same vein, but again it's not quite the exact snapshot type of thing that a NetApp provides. I bet there are better choices out there...
A really nice tool for nightly backups with a history kept on-line is backuppc http://backuppc.sourceforge.net/. It probably isn't practical to do runs more often than daily, but the compression/linking scheme allows keeping a much longer history than you would expect. You can also designate 'owners' for a machine and allow them to restore their own files or grab older copies back through the web interface. Since the backups generally run at night you may be able to stick some big drives in a desktop machine that would otherwise be idle to run it.
Les Mikesell wrote:
A really nice tool for nightly backups with a history kept on-line is backuppc http://backuppc.sourceforge.net/. It probably
I've played with this a little, and my comment in the negative is that it's a do-or-die on a whole system. You don't get fine grained control of exactly what you want (for instance, I may only want /etc/ and /var/named/ of the DNS/DHCP server, D:\My Documents\ of a shared Windows PC, etc) and pretty much have to suck the whole thing up.
This really isn't a *snapshot* feature, per se - it's a backup. A snapshot (should be|is) a lot more space conservative and lean running that won't affect operation of the NAS in realtime. The hardcore snapshot feature of NetApp/SnapServer really is great, it's hard to beat. I may be a linux nut, but sometimes a commercial solution is the right answer for a specific situation.
-te
Troy Engel wrote:
Les Mikesell wrote:
A really nice tool for nightly backups with a history kept on-line is backuppc http://backuppc.sourceforge.net/. It probably
I've played with this a little, and my comment in the negative is that it's a do-or-die on a whole system. You don't get fine grained control of exactly what you want (for instance, I may only want /etc/ and /var/named/ of the DNS/DHCP server, D:\My Documents\ of a shared Windows PC, etc) and pretty much have to suck the whole thing up.
This really isn't a *snapshot* feature, per se - it's a backup. A snapshot (should be|is) a lot more space conservative and lean running that won't affect operation of the NAS in realtime. The hardcore snapshot feature of NetApp/SnapServer really is great, it's hard to beat. I may be a linux nut, but sometimes a commercial solution is the right answer for a specific situation.
openfiler 1.x supports snapshots along with a multitude of options on handling them in various ways.
On Tue, 2006-01-31 at 11:30, Troy Engel wrote:
A really nice tool for nightly backups with a history kept on-line is backuppc http://backuppc.sourceforge.net/. It probably
I've played with this a little, and my comment in the negative is that it's a do-or-die on a whole system. You don't get fine grained control of exactly what you want (for instance, I may only want /etc/ and /var/named/ of the DNS/DHCP server, D:\My Documents\ of a shared Windows PC, etc) and pretty much have to suck the whole thing up.
That's not really true. You have all the facilities that the underlying transports (rsync/tar/or smb shares) provide to specify what to include or exclude. On the Windows side you can just share the My Documents folder if you want. If your machines all have different layouts, you'll have to make a separate config file for each target, but that's not difficult and it's a one-time job.
This really isn't a *snapshot* feature, per se - it's a backup.
Agreed, but you get some of the features of snapshots, like having many versions on-line at once and browsable by their owner as well as the admin(s), plus the extra safety of having the copy on a different machine.
A snapshot (should be|is) a lot more space conservative and lean running that won't affect operation of the NAS in realtime.
Backuppc is even more space-conservative than filesystem snapshots but it does do a lot of work to accomplish that.
The hardcore snapshot feature of NetApp/SnapServer really is great, it's hard to beat. I may be a linux nut, but sometimes a commercial solution is the right answer for a specific situation.
Yes, I've used a NetApp (actually the Dell rebranded one), but those snapshots don't solve all your problems. It's rare, but if you lose 2 drives at once the raid will fail - and the snapshots are annoying when you try to make copies elsewhere since you always have to remember to exclude them.
Les Mikesell wrote:
From the longtime experience direction, be sure to figure out the
snapshotting upfront; a good snapshot rotation can save you so much pain it's not funny; many commercial NAS (NetApp, SnapServer, EMC, etc) have this integrated. When looking at a DIY linux NAS, you may have to roll it up.
I think there are still problems with LVM2 snapshots.
we're hoping to see a major improvement in that situation ( lvm2 / kernel 2.6 / snapshots ) with the CentOS 4.3 release. not too far now :)
On Tue, 2006-01-31 at 11:35, Karanbir Singh wrote:
From the longtime experience direction, be sure to figure out the
snapshotting upfront; a good snapshot rotation can save you so much pain it's not funny; many commercial NAS (NetApp, SnapServer, EMC, etc) have this integrated. When looking at a DIY linux NAS, you may have to roll it up.
I think there are still problems with LVM2 snapshots.
we're hoping to see a major improvement in that situation ( lvm2 / kernel 2.6 / snapshots ) with the CentOS 4.3 release. not too far now :)
Do you know if it is possible to image-copy a snapshot off to another system? A backuppc archive contains so many hardlinks that normal copy techniques are not practical. I currently raid-mirror to external firewire drives that are rotated offsite periodically, but it would probably work better to freeze an LVM snapshot and copy that instead.
Les Mikesell wrote:
From the longtime experience direction, be sure to figure out the
snapshotting upfront; a good snapshot rotation can save you so much pain it's not funny; many commercial NAS (NetApp, SnapServer, EMC, etc) have this integrated. When looking at a DIY linux NAS, you may have to roll it up.
I think there are still problems with LVM2 snapshots.
we're hoping to see a major improvement in that situation ( lvm2 / kernel 2.6 / snapshots ) with the CentOS 4.3 release. not too far now :)
Do you know if it is possible to image-copy a snapshot off to another system? A backuppc archive contains so many hardlinks that normal copy techniques are not practical. I currently raid-mirror to external firewire drives that are rotated offsite periodically, but it would probably work better to freeze an LVM snapshot and copy that instead.
hi Les,
you can reverse mount and add-as-share the snapshot, over whatever service you like ( smb, ftp, http-web-dab, nfs ) and backup using whatever means you like.
- K
On Wed, 2006-02-01 at 04:49, Karanbir Singh wrote:
I think there are still problems with LVM2 snapshots.
we're hoping to see a major improvement in that situation ( lvm2 / kernel 2.6 / snapshots ) with the CentOS 4.3 release. not too far now :)
Do you know if it is possible to image-copy a snapshot off to another system? A backuppc archive contains so many hardlinks that normal copy techniques are not practical. I currently raid-mirror to external firewire drives that are rotated offsite periodically, but it would probably work better to freeze an LVM snapshot and copy that instead.
you can reverse mount and add-as-share the snapshot, over whatever service you like ( smb, ftp, http-web-dab, nfs ) and backup using whatever means you like.
That's not what I meant. It takes an impractical amount of time to copy the files from a mounted backuppc archive with millions of hardlinks. I'd like instead to do an image copy of the partition block device either to a physical disk partition or a file that could be loopback-mounted later if needed. This should be the same as copying real partitions with dd, except I'd want the frozen state of the snapshot so the filesystem could remain active while the copy runs. Someone on the backuppc list has mentioned doing this so I think the snapshots present a normal-looking block device interface - but I think they've had trouble with LVM2.
It would be even nicer if you could use rsync to update a remote image using only the bandwidth needed to transfer the differences (again a horrible thing to do through a filesystem with millions of symlinks). Does the size of a snapshot and the locations of the used portion stay the same between snapshots?
On Tuesday 31 January 2006 09:10, Troy Engel wrote:
From the longtime experience direction, be sure to figure out the
snapshotting upfront; a good snapshot rotation can save you so much pain it's not funny; many commercial NAS (NetApp, SnapServer, EMC, etc) have this integrated. When looking at a DIY linux NAS, you may have to roll it up.
We snapshot roughly like so: [8a,12p,4p M-F] [nightly 2 nights retention] -- this setup is enough to provide instantanious file restore without having to deal with the backups. (classic "hey IT, I deleted a file last night...")
We've combined our contractual requirement for nightly off-site backups and snapshotting using Linux and rsync. I've put it together into a project written in PHP that you're free to use:
http://www.effortlessis.com/backupbuddy
It uses the hard link option of rsync to minimize disk usage for unchanged files, resulting in great incremental stores on fairly modest disk usage.
It saves backups on a power of two (or three, if you want), EG: Today, Yesterday, Two days ago, Four days ago, Eight days ago, Sixteen days ago... [repeat until disk space expires]
I've created a web-based tool that allows for recovery when the inevitable "Hey, I deleted a file last week..." request is made, I just give them a URL and they handle it themselves. (However, due to its nature, it's not part of the bbuddy release)
I'm about to release an update to this script that handles expiring old branches more sanely, so you may want to wait until the next release comes out. (I'm waiting to see that it's stable before I release, and given the nature of backups, that might take another week or two)
Combine this with samba (Read-only) and perhaps LDAP authentication for another variation on this theme.
-Ben
Benjamin Smith wrote:
We've combined our contractual requirement for nightly off-site backups and snapshotting using Linux and rsync. I've put it together into a project written in PHP that you're free to use:
So here's a question from another direction regarding your project and others that have this in common: is there a technical reason you can't use a push mode to an rsync daemon, and require passwordless SSH instead?
At home I have a basic FC4 laptop hidden under the stereo that runs MPD to stream music, and it doubles as a basic backup server (external USB disk). This laptop has a static IP, but all others are on DHCP.
What makes sense to me is that the server runs rsyncd in a target mode, something like:
[laptop1] path = /mtn/usbdrive/backups/laptop1 (other options...)
Then from the client, it simply runs the backup client indicating who the server is and it's target, supplying the password (rsync user@server1::laptop1 etc.). To me this seems a lot more architecturally simple (and if you want SSH, you set it up after-the-fact as the transport) and so forth.
Yet, no project (I can find, rdiff-backup would be nice) does this - is there a reason? Why does everyone insist on dealing with SSH and keys? (ignore the encryption benefit)
-te
On Tue, 31 Jan 2006, Troy Engel wrote:
Benjamin Smith wrote:
We've combined our contractual requirement for nightly off-site backups and snapshotting using Linux and rsync. I've put it together into a project written in PHP that you're free to use:
So here's a question from another direction regarding your project and others that have this in common: is there a technical reason you can't use a push mode to an rsync daemon, and require passwordless SSH instead?
At home I have a basic FC4 laptop hidden under the stereo that runs MPD to stream music, and it doubles as a basic backup server (external USB disk). This laptop has a static IP, but all others are on DHCP.
What makes sense to me is that the server runs rsyncd in a target mode, something like:
[laptop1] path = /mtn/usbdrive/backups/laptop1 (other options...)
Then from the client, it simply runs the backup client indicating who the server is and it's target, supplying the password (rsync user@server1::laptop1 etc.). To me this seems a lot more architecturally simple (and if you want SSH, you set it up after-the-fact as the transport) and so forth.
Yet, no project (I can find, rdiff-backup would be nice) does this - is there a reason? Why does everyone insist on dealing with SSH and keys? (ignore the encryption benefit)
Interested in helping me develop it ? It seems pretty easy to do and enormously useful for myself. (especially if it can also include rsnapshot kind of functionality as well)
I'm taking up some new challenges lately to build some very simple tools that simplify (or automate) all the work I've done until recently.
I'll be announcing another one the coming days regarding kickstart management and PXE booting (as part of Yam).
Kind regards, -- dag wieers, dag@wieers.com, http://dag.wieers.com/ -- [all I want is a warm bed and a kind word and unlimited power]
Dag Wieers wrote:
Interested in helping me develop it ? It seems pretty easy to do and enormously useful for myself. (especially if it can also include rsnapshot kind of functionality as well)
In what extra time I have I've been working on an asset management system (php/mysql) based on an ancient copy of phpMyInventory, but quite different by now. The intent is to get it to a stable state and get it released - so, afraid my hands are full for right now...
-te
On Tue, 2006-01-31 at 17:08, Troy Engel wrote:
Dag Wieers wrote:
Interested in helping me develop it ? It seems pretty easy to do and enormously useful for myself. (especially if it can also include rsnapshot kind of functionality as well)
In what extra time I have I've been working on an asset management system (php/mysql) based on an ancient copy of phpMyInventory, but quite different by now. The intent is to get it to a stable state and get it released - so, afraid my hands are full for right now...
Are you aware of the RT (http://www.bestpractical.com/rt/) addon: for asset tracking: http://atwiki.chaka.net/?
Les Mikesell wrote:
Are you aware of the RT (http://www.bestpractical.com/rt/) addon: for asset tracking: http://atwiki.chaka.net/?
Yup - apples and oranges, two different target audiences. :)
-te
On Tue, 2006-01-31 at 16:07, Troy Engel wrote:
Then from the client, it simply runs the backup client indicating who the server is and it's target, supplying the password (rsync user@server1::laptop1 etc.). To me this seems a lot more architecturally simple (and if you want SSH, you set it up after-the-fact as the transport) and so forth.
Yet, no project (I can find, rdiff-backup would be nice) does this - is there a reason? Why does everyone insist on dealing with SSH and keys? (ignore the encryption benefit)
If you have more than a couple of backup clients, it is necessary for the server to handle the scheduling to control the load and ssh keys are the easy way to allow automated remote commands. By the way, backuppc can use smb shares or rsyncd targets instead of ssh if you like and can handle dhcp addresses when using the smb share method.