Hi,
we plan to set up a big file storage for media files like uncompressed movies from student film projects, dvd images etc.
It should be some sort of archive and will not bee accessed by more than may be 5 people at the same time.
The iSCSI RAID we have is about 26TB netto and I'm again faced with the question: How many partitions, which filesystem, which mount options etc.
For the User it would be the most simpel thing, to have one big filesystem she/he could fill with all the data and dont has to search e.g. on multiple volumes.
On the other hand, if one big filesystem crashes or has do be checked it will destroy a lot of data or the check will take hours ...
Any suggestions pro or cons are welcome! :-)
My favourite for now is 3 to 4 filesystems with the default ext4 settings. (Redhat EL 5.7, may be soon 6.1)
Thanks and best regards. Götz
Greetings,
2011/10/28 Götz Reinicke goetz.reinicke@filmakademie.de:
Hi,
The iSCSI RAID we have is about 26TB netto and I'm again faced with the question: How many partitions, which filesystem, which mount options etc.
My favourite for now is 3 to 4 filesystems with the default ext4 settings. (Redhat EL 5.7, may be soon 6.1)
You should seriously consider XFS (supported natively in RHEL and hence Centos 6) for any single filesystem >16TB. (EXT4 supports only upto 16TB)
If you are going to have an RHCS for HA, then GFS2. But I cant comment on its performance for streaming applications etc.
Am 28.10.11 11:11, schrieb Rajagopal Swaminathan:
Greetings,
2011/10/28 Götz Reinicke goetz.reinicke@filmakademie.de:
Hi,
The iSCSI RAID we have is about 26TB netto and I'm again faced with the question: How many partitions, which filesystem, which mount options etc.
My favourite for now is 3 to 4 filesystems with the default ext4 settings. (Redhat EL 5.7, may be soon 6.1)
You should seriously consider XFS (supported natively in RHEL and hence Centos 6) for any single filesystem >16TB. (EXT4 supports only upto 16TB)
If you are going to have an RHCS for HA, then GFS2. But I cant comment on its performance for streaming applications etc.
Thanks for your suggestions! I'll have a look at XFS.
BTW: We will not use that storage for streaming! It is 'just' a big space for files with ftp and smb access.
Regards . Götz
Am 28.10.11 11:11, schrieb Rajagopal Swaminathan:
Greetings,
2011/10/28 Götz Reinicke goetz.reinicke@filmakademie.de:
Hi,
The iSCSI RAID we have is about 26TB netto and I'm again faced with the question: How many partitions, which filesystem, which mount options etc.
My favourite for now is 3 to 4 filesystems with the default ext4 settings. (Redhat EL 5.7, may be soon 6.1)
You should seriously consider XFS (supported natively in RHEL and hence Centos 6) for any single filesystem >16TB. (EXT4 supports only upto 16TB)
BTW 2 . I thought, the max. filesystemsize is 1EB and not 16TB.
The max FILEsize should be 16TB ...
and
http://en.wikipedia.org/wiki/Ext4
/Götz
Greetings,
2011/10/28 Götz Reinicke goetz.reinicke@filmakademie.de:
BTW 2 . I thought, the max. filesystemsize is 1EB and not 16TB.
The max FILEsize should be 16TB ...
Apologies for missing to give you the link
http://www.redhat.com/rhel/compare/
Check under "Filesystems and Storage Limits" section.
An one more thing: forget 32 bit here. Use 64 bit only.
HTH
Am 28.10.11 11:58, schrieb Rajagopal Swaminathan:
Greetings,
2011/10/28 Götz Reinicke goetz.reinicke@filmakademie.de:
BTW 2 . I thought, the max. filesystemsize is 1EB and not 16TB.
The max FILEsize should be 16TB ...
Apologies for missing to give you the link
http://www.redhat.com/rhel/compare/
Check under "Filesystems and Storage Limits" section.
An one more thing: forget 32 bit here. Use 64 bit only.
Thx. Yes I'l usually go with 64 bit only.
BTW 3. So far I found the information, that the 1EB ist the theory, but the usertools for managing the ext4 are limited to 16TB for safety reasons...
regards . Götz
Greetings,
2011/10/28 Götz Reinicke goetz.reinicke@filmakademie.de:
Thx. Yes I'l usually go with 64 bit only.
BTW 3. So far I found the information, that the 1EB ist the theory, but the usertools for managing the ext4 are limited to 16TB for safety reasons...
Another important consideration would be, if and when the organisation needs support and are ready to pay for it, say, only for production systems, Redhat has always been there.
From: Götz Reinicke goetz.reinicke@filmakademie.de
The iSCSI RAID we have is about 26TB netto and I'm again faced with the question: How many partitions, which filesystem, which mount options etc. For the User it would be the most simpel thing, to have one big filesystem she/he could fill with all the data and dont has to search e.g. on multiple volumes. On the other hand, if one big filesystem crashes or has do be checked it will destroy a lot of data or the check will take hours ...
Splitting the space, if you have the option, has advantages... You already mentioned fsck. You could assign different partitions to different groups of people. You could, depending on your RAID level, create several arrays on different drives to limit disk contention (but you will lose either some space and/or some speed)...
JD
2011/10/28 Götz Reinicke goetz.reinicke@filmakademie.de
The iSCSI RAID we have is about 26TB netto and I'm again faced with the question: How many partitions, which filesystem, which mount options etc.
My vote goes to XFS, if only one server needs acces to the LUN's; and GPFS (not GFS) if you need a cluster filesystem.
BR Bent (130 TB in one XFS installation, and 600 TB in one gpfs cluster)
For the User it would be the most simpel thing, to have one big filesystem she/he could fill with all the data and dont has to search e.g. on multiple volumes.
On the other hand, if one big filesystem crashes or has do be checked it will destroy a lot of data or the check will take hours ...
Any suggestions pro or cons are welcome! :-)
My favourite for now is 3 to 4 filesystems with the default ext4 settings. (Redhat EL 5.7, may be soon 6.1)
Thanks and best regards. Götz
Götz Reinicke IT-Koordinator
Tel. +49 7141 969 420 Fax +49 7141 969 55 420 E-Mail goetz.reinicke@filmakademie.de
Filmakademie Baden-Württemberg GmbH Akademiehof 10 71638 Ludwigsburg www.filmakademie.de
Eintragung Amtsgericht Stuttgart HRB 205016
Vorsitzender des Aufsichtsrats: Jürgen Walter MdL Staatssekretär im Ministerium für Wissenschaft, Forschung und Kunst Baden-Württemberg
Geschäftsführer: Prof. Thomas Schadt
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Fri, 2011-10-28 at 10:34 +0200, Götz Reinicke wrote:
we plan to set up a big file storage for media files like uncompressed movies from student film projects, dvd images etc. It should be some sort of archive and will not bee accessed by more than may be 5 people at the same time. The iSCSI RAID we have is about 26TB netto and I'm again faced with the question: How many partitions, which filesystem, which mount options etc.= For the User it would be the most simpel thing, to have one big filesystem she/he could fill with all the data and dont has to search e.g. on multiple volumes.
Use LVM.
Divide it into numerous partitions. Initialize the partitions as Physical Volumes. Create a Volume Group containing the Physical Volumes. Then create the volumes you allocate storage as using Logical Volumes. Your LV can span as many, or as few, PVs as you like.
To the SAN each PV is a partition, you can move it, delete it, etc.. In the OS you can move data between PVs, migrate LVs between PVs, etc.. Even connect to another SAN, attach PVs there to you volume group, and migrate the LVs to those PVs - this migrating data from one SAN to another. Then drop the PVs on the old SAN from the VG.
-
Adam Tauno Williams -
Bent Terp -
Götz Reinicke -
John Doe -
Rajagopal Swaminathan