I am planning to use ZFS on my Centos 5.2 systems. The data I am storing is very large text files where each file can range from 10M to 20G. I am very interested on the compression feature of ZFS, and it seems no other native Linux FS supports it.
My question are: Is ZFS stable? How does it scale for very large filesytems, ie, 2TB to 9TB? How is the performance of fuse? I plan to use it on my archive server first, so data reliability is very important
Any thoughts or ideas?
TIA
Mag Gam wrote:
I am planning to use ZFS on my Centos 5.2 systems. The data I am storing is very large text files where each file can range from 10M to 20G. I am very interested on the compression feature of ZFS, and it seems no other native Linux FS supports it.
My question are: Is ZFS stable? How does it scale for very large filesytems, ie, 2TB to 9TB? How is the performance of fuse? I plan to use it on my archive server first, so data reliability is very important
Any thoughts or ideas?
I'd be surprised if anyone is using zfs/fuse/linux combinations seriously. Why not just run your archive server on opensolaris or freebsd where zfs runs natively? Or if you want to pretend it is a linux distro, perhaps nexenta http://www.nexenta.org/os or their commercial nexentastor version would work. They have a mostly-ubuntu userland running on an opensolaris kernel.
On Sun, Dec 28, 2008 at 8:02 PM, Les Mikesell lesmikesell@gmail.com wrote:
Mag Gam wrote:
I am planning to use ZFS on my Centos 5.2 systems. The data I am storing is very large text files where each file can range from 10M to 20G. I am very interested on the compression feature of ZFS, and it seems no other native Linux FS supports it.
My question are: Is ZFS stable? How does it scale for very large filesytems, ie, 2TB to 9TB? How is the performance of fuse? I plan to use it on my archive server first, so data reliability is very important
Any thoughts or ideas?
I'd be surprised if anyone is using zfs/fuse/linux combinations seriously. Why not just run your archive server on opensolaris or freebsd where zfs runs natively? Or if you want to pretend it is a linux distro, perhaps nexenta http://www.nexenta.org/os or their commercial nexentastor version would work. They have a mostly-ubuntu userland running on an opensolaris kernel.
Yes, agreed with Les Mikesell. Running zfs on Linux is not a good idea at all, at least for this moment :-)
On Sun, 28 Dec 2008 20:27:10 +0100 Vnpenguin vnpenguin@vnoss.org wrote:
On Sun, Dec 28, 2008 at 8:02 PM, Les Mikesell lesmikesell@gmail.com wrote:
Mag Gam wrote:
Any thoughts or ideas?
I'd be surprised if anyone is using zfs/fuse/linux combinations seriously. Why not just run your archive server on opensolaris or freebsd where zfs runs natively? Or if you want to pretend it is a linux distro, perhaps nexenta http://www.nexenta.org/os or their commercial nexentastor version would work. They have a mostly-ubuntu userland running on an opensolaris kernel.
Yes, agreed with Les Mikesell. Running zfs on Linux is not a good idea at all, at least for this moment :-)
It works (at least I didn't have any data loss in the two months I was using it), but is painfully slow (fuse has its price). Nexenta is the next best thing if you want "kinda linux like" environment, altough linux zones on solaris might also be worth checking.
On Dec 28, 2008, at 9:17 PM, Jure Pečar wrote:
Nexenta is the next best thing if you want "kinda linux like" environment, altough linux zones on solaris might also be worth checking.
Definitely, even if support for kernel 2.6 is not completely working (at least not 2 months ago...). Check this http://opensolaris.org/os/community/brandz/
d
Davide Cittaro davide.cittaro@ifom-ieo-campus.it
Am 28.12.2008 um 20:02 schrieb Les Mikesell:
Mag Gam wrote:
I am planning to use ZFS on my Centos 5.2 systems. The data I am storing is very large text files where each file can range from 10M to 20G. I am very interested on the compression feature of ZFS, and it seems no other native Linux FS supports it.
My question are: Is ZFS stable? How does it scale for very large filesytems, ie, 2TB to 9TB? How is the performance of fuse? I plan to use it on my archive server first, so data reliability is very important
Any thoughts or ideas?
I'd be surprised if anyone is using zfs/fuse/linux combinations seriously. Why not just run your archive server on opensolaris or freebsd where zfs runs natively? Or if you want to pretend it is a linux distro, perhaps nexenta http://www.nexenta.org/os or their commercial nexentastor version would work. They have a mostly-ubuntu userland running on an opensolaris kernel.
Even in FreeBSD-land, you'd need to run the very latest CURRENT. No advised for production.
Go for opensolaris or Solaris 10U6.
Regards, Rainer
On Dec 28, 2008, at 7:16 PM, Mag Gam wrote:
I am planning to use ZFS on my Centos 5.2 systems. The data I am storing is very large text files where each file can range from 10M to 20G. I am very interested on the compression feature of ZFS, and it seems no other native Linux FS supports it.
Even if fuse implementation of ZFS looks rather stable, I won't suggest it in a production environment... We strongly wanted ZFS and we chose for Solaris 10 for our file server.
My question are: Is ZFS stable? How does it scale for very large filesytems, ie, 2TB to 9TB? How is the performance of fuse? I plan to use it on my archive server first, so data reliability is very important
ZFS really is great. We are now managing three 18Tb archives. It is not only reliable, it comes with zpool and zfs commands that really make it easy to manage! If you don't want Solaris, you can use FreeBSD 7 which supports native ZFS.
d
Davide Cittaro davide.cittaro@ifom-ieo-campus.it
thanks everyone for your fair and balanced opinions and experiences!
On Sun, Dec 28, 2008 at 2:38 PM, Davide Cittaro davide.cittaro@ifom-ieo-campus.it wrote:
On Dec 28, 2008, at 7:16 PM, Mag Gam wrote:
I am planning to use ZFS on my Centos 5.2 systems. The data I am storing is very large text files where each file can range from 10M to 20G. I am very interested on the compression feature of ZFS, and it seems no other native Linux FS supports it.
Even if fuse implementation of ZFS looks rather stable, I won't suggest it in a production environment... We strongly wanted ZFS and we chose for Solaris 10 for our file server.
My question are: Is ZFS stable? How does it scale for very large filesytems, ie, 2TB to 9TB? How is the performance of fuse? I plan to use it on my archive server first, so data reliability is very important
ZFS really is great. We are now managing three 18Tb archives. It is not only reliable, it comes with zpool and zfs commands that really make it easy to manage! If you don't want Solaris, you can use FreeBSD 7 which supports native ZFS.
d
Davide Cittaro davide.cittaro@ifom-ieo-campus.it
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Sun, Dec 28, 2008, Davide Cittaro wrote:
On Dec 28, 2008, at 7:16 PM, Mag Gam wrote:
I am planning to use ZFS on my Centos 5.2 systems. The data I am storing is very large text files where each file can range from 10M to 20G. I am very interested on the compression feature of ZFS, and it seems no other native Linux FS supports it.
Even if fuse implementation of ZFS looks rather stable, I won't suggest it in a production environment... We strongly wanted ZFS and we chose for Solaris 10 for our file server.
My question are: Is ZFS stable? How does it scale for very large filesytems, ie, 2TB to 9TB? How is the performance of fuse? I plan to use it on my archive server first, so data reliability is very important
ZFS really is great. We are now managing three 18Tb archives. It is not only reliable, it comes with zpool and zfs commands that really make it easy to manage! If you don't want Solaris, you can use FreeBSD 7 which supports native ZFS.
I would go with Opensolaris. There has been quite a bit of messaging on the freebsd lists on zfs that give me the impression that zfs on freebsd is not really ready for prime time.
Bill
On Dec 29, 2008, at 7:09 AM, John R Pierce wrote:
Bill Campbell wrote:
I would go with Opensolaris.
for a dedicated production storage server, I would go with Solaris 10. unless there's some specific feature/capability you need thats only in OpenSolaris.
Totally agree. Solaris 10 is known for its stability. OpenSolaris includes some advanced capabilities that will be included into Solaris (especially on zfs and kernel side).
Solaris : OpenSolaris = CentOS : Fedora
(more or less...)
d
On Mon, Dec 29, 2008 at 2:54 AM, in message
8E388B67-1D39-4095-95C5-132B02E4F63C@ifom-ieo-campus.it, Davide Cittaro davide.cittaro@ifom-ieo-campus.it wrote:
On Dec 29, 2008, at 7:09 AM, John R Pierce wrote:
Bill Campbell wrote:
I would go with Opensolaris.
for a dedicated production storage server, I would go with Solaris 10. unless there's some specific feature/capability you need thats only in OpenSolaris.
Totally agree. Solaris 10 is known for its stability. OpenSolaris includes some advanced capabilities that will be included into Solaris (especially on zfs and kernel side).
Solaris : OpenSolaris = CentOS : Fedora
(more or less...)
d
I agree in general with most every opinion. Especially Davide's comment above. Very good analogy Open Solaris may be your best choice. I would suggest you do pay attention to Solaris itself. It's free (as in beer) from Sun & it works.
Here at the JHU libraries we manage about 1/2 PB of online data varying from images, audio, scanned documents, etc. in a ZFS instance on some massive storage. We evaluated all the iterations of ZFS on various OS's. ZFS/fuse was eliminated fairly quickly along with BSD. For the critical stuff we use Solaris on Sun H/W. For general storage it's Solaris_x86 on generic x86 H/W.
Tony Placilla aplacilla@jhu.edu Sr. UNIX Systems Administrator The Sheridan Libraries Johns Hopkins University
I agree in general with most every opinion. Especially Davide's comment above. Very good analogy Open Solaris may be your best choice. I would suggest you do pay attention to Solaris itself. It's free (as in beer) from Sun & it works.
Except for patches unless you want to browse Sun's website regularly to download them.
You also get more hardware support on OpenSolaris and support from Sun for OpenSolaris but I suppose the latter option is probably better done with Solaris 10 + support which includes access to patch management. Unless you like the way things are done over here in Linux land which is one tool to manage them all and not one tool to install packages and another tool to install patches to packages.
On Mon, Dec 29, 2008 at 7:24 PM, in message
49596A48.4000104@bradbury.edu.hk, Christopher Chan christopher.chan@bradbury.edu.hk wrote:
I agree in general with most every opinion. Especially Davide's comment
above. Very good analogy
Open Solaris may be your best choice. I would suggest you do pay attention to Solaris itself. It's free (as in
beer) from Sun & it works.
Except for patches unless you want to browse Sun's website regularly to download them.
You also get more hardware support on OpenSolaris and support from Sun for OpenSolaris but I suppose the latter option is probably better done with Solaris 10 + support which includes access to patch management. Unless you like the way things are done over here in Linux land which is one tool to manage them all and not one tool to install packages and another tool to install patches to packages.
Agreed.
Only the OP knows the criticality of his data & whether or not he needs support & at what level.
The root answer is that if he wants to use ZFS (which is a *good* choice) he should use some flavor of Solaris
Tony Placilla aplacilla@jhu.edu Sr. UNIX Systems Administrator The Sheridan Libraries Johns Hopkins University
On 2008-12-30, 15:32 GMT, Tony Placilla wrote:
The root answer is that if he wants to use ZFS (which is a *good* choice) he should use some flavor of Solaris
I would just add that RHEL 5.3 (and thus CentOS 5.3) when it happens, will have ext4fs as a technology preview, which may fulfill OT needs as well.
Matěj
ext4 isn't going to help too much. Our biggest concerns are: compression, and unlimited inodes
On Mon, Jan 5, 2009 at 8:36 AM, Matej Cepl mcepl@redhat.com wrote:
On 2008-12-30, 15:32 GMT, Tony Placilla wrote:
The root answer is that if he wants to use ZFS (which is a *good* choice) he should use some flavor of Solaris
I would just add that RHEL 5.3 (and thus CentOS 5.3) when it happens, will have ext4fs as a technology preview, which may fulfill OT needs as well.
Matěj
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Mag Gam wrote:
I am planning to use ZFS on my Centos 5.2 systems. The data I am storing is very large text files where each file can range from 10M to 20G. I am very interested on the compression feature of ZFS, and it seems no other native Linux FS supports it.
My question are: Is ZFS stable? How does it scale for very large filesytems, ie, 2TB to 9TB? How is the performance of fuse? I plan to use it on my archive server first, so data reliability is very important
Any thoughts or ideas?
Try OpenSolaris. It comes with a package manager that is getting there and upgrades from one version to newer versions have been relatively painless for me. These upgrades are not the 'wipe system out and install latest version of distro' type.
You will have to learn stuff unique to Solaris land though if you are not already familiar with some of the ways things are done in Solaris.
Mag Gam wrote:
I am planning to use ZFS on my Centos 5.2 systems. The data I am storing is very large text files where each file can range from 10M to 20G. I am very interested on the compression feature of ZFS, and it seems no other native Linux FS supports it.
My question are: Is ZFS stable? How does it scale for very large filesytems, ie, 2TB to 9TB? How is the performance of fuse? I plan to use it on my archive server first, so data reliability is very important
Any thoughts or ideas?
Did you look at Ext4 ? http://en.wikipedia.org/wiki/Ext4
"The ext4 filesystem can support volumes with sizes up to 1 exbibyte and files with sizes up to 16 TiB"
At least for linux it looks better that ZFS via fuse.
-
Bill Campbell -
Christopher Chan -
Davide Cittaro -
John R Pierce -
Jure Pečar -
Les Mikesell -
Mag Gam -
Matej Cepl -
Rainer Duffner -
Sergej kandyla -
Tony Placilla -
Vnpenguin