hi , our centos os has an ext3 file system. and i cant create any more directories, it gives me a too many links error, even when doing a manual mkdir.
is there any workaround for this? without changing it to a different file system like reiserFS? we dont have a reiserfs module in the os so it will have to be installed. and can it be possible to install a reiserFS without having to reformat the os? just mount it after installing? please advise
chitgoks wrote:
hi , our centos os has an ext3 file system. and i cant create any more directories, it gives me a too many links error, even when doing a manual mkdir.
is there any workaround for this? without changing it to a different file system like reiserFS? we dont have a reiserfs module in the os so it will have to be installed. and can it be possible to install a reiserFS without having to reformat the os? just mount it after installing? please advise
first,
# tune2fs -l /dev/hda2
(or whatever) and note how many inodes on this file system. I'm assuming you're out of free inodes. If this isn't the case, then stop right now, I'm wrong.
assuming you do need to increase the inode count, backup all the files on this file system to temporary media
then
# mkfs.ext3 -N XXXXX /dev/hda2
where XXXXX is some larger number of inodes suitable for your application. I wouldn't increase by any more than the free block count shown in that tune2fs report.
then remount the file system and restore your backup.
hi john
ext3 does have its limit right? because i am planning on upgrading the file system to reiser instead. does retuning the file system always have you require to back up your data?
chitgoks wrote:
hi john
ext3 does have its limit right? because i am planning on upgrading the file system to reiser instead. does retuning the file system always have you require to back up your data?
ext3's inode limits are set when you create the file system. the defaults are generally adequate, unless you're going to create very many very tiny files (say, <= 4K).
AFAIK, ReiserFS is completely unsupported by RHEL and CentOS. You'll be on your own.
On Thu, Sep 27, 2007, John R Pierce wrote:
chitgoks wrote:
hi john
ext3 does have its limit right? because i am planning on upgrading the file system to reiser instead. does retuning the file system always have you require to back up your data?
ext3's inode limits are set when you create the file system. the defaults are generally adequate, unless you're going to create very many very tiny files (say, <= 4K).
AFAIK, ReiserFS is completely unsupported by RHEL and CentOS. You'll be on your own.
I would avoid reiserfs as I've seen far too many cases where there has been massive data loss with it. I've used xfs extensively on SuSE systems without problems, but haven't tried it on centos as it's not supported by the base systems.
Bill -- INTERNET: bill@celestial.com Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way FAX: (206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676
The very powerful and the very stupid have one thing in common. Instead of altering their views to fit the facts, they alter the facts to fit their views ... which can be very uncomfortable if you happen to be one of the facts that needs altering. -- Doctor Who, "Face of Evil"
Am Freitag, 28. September 2007 schrieb Bill Campbell:
I would avoid reiserfs as I've seen far too many cases where there has been massive data loss with it. I've used xfs extensively on SuSE systems without problems, but haven't tried it on centos as it's not supported by the base systems.
Works great using the XFS Module from CentOSPlus. We use it for the data partition of our fileserver.
Andreas Micklei schrieb:
Am Freitag, 28. September 2007 schrieb Bill Campbell:
I would avoid reiserfs as I've seen far too many cases where there has been massive data loss with it. I've used xfs extensively on SuSE systems without problems, but haven't tried it on centos as it's not supported by the base systems.
Works great using the XFS Module from CentOSPlus. We use it for the data partition of our fileserver.
I just ran the script:
#!/bin/tcsh -f @ number = 0 loop: @ number += 1 echo touch $number touch $number if ( $number < 100000 ) goto loop
on a XFS partition of a CentOS 5 x86_64 system with kmod-xfs-0.4-1.2.6.18_8.1.8.el5 and xfsprogs-2.8.20-1.el5.centos installed. The script ran smoothly and took
46.238u 224.379s 4:37.93 97.3% 0+0k 0+0io 0pf+0w
to complete; all the files are there. So 100000 files in a single directory is not a problem for XFS.
HTH, Kay
Salam,
CentOs or RedHat flavours donot come up with "Reiserfs support by default". u can install it using like
yum install enablerepo=centosplus reiserfs-utils
please confirm the command as i used this long time ago....
Suppose, u want to /var to be reiserfs supported, u have to format /var with resierfs, there is no need to format whole OS.
Long time, i used this scenario, but unfortunately donot remember the procedure i know when i did, it was tricky.... soon as i get any help i will update.
Regards
Umair Shakil ETD
On 9/28/07, chitgoks chitgoks@gmail.com wrote:
hi , our centos os has an ext3 file system. and i cant create any more directories, it gives me a too many links error, even when doing a manual mkdir.
is there any workaround for this? without changing it to a different file system like reiserFS? we dont have a reiserfs module in the os so it will have to be installed. and can it be possible to install a reiserFS without having to reformat the os? just mount it after installing? please advise
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Fri, Sep 28, 2007, chitgoks wrote:
hi , our centos os has an ext3 file system. and i cant create any more directories, it gives me a too many links error, even when doing a manual mkdir.
I don't know whether it's applicable here, but I ran into a problem years ago where there was a limit on the number of directories at the root of a file system, but only at the root of the system. The solution was to create symbolic links in the top level directory to subdirectories.
I haven't seen this problem on Linux systems, and we have some /home directories with about 10,000 subdirectories.
Bill -- INTERNET: bill@celestial.com Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way FAX: (206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676
Anyone who thinks Microsoft never does anything truly innovative isn't paying attention to the part of the company that pushes the state of its art: Microsoft's legal department. --Ed Foster, InfoWorld Gripe Line columnist
my issue involves close to a 100,000 directories and more :D
well then, even formatting just a directory to support reiserfs, id have to backup my data just to be on the safe side. thanks for the tip guys. ill let you know
john yeah. a few images per directory (for now, maybe in the future more images). the directory name would point to the username.
robert, thanks. now im sure that the limit is 32k since you confirmed it as well. ive read in other posts regarding this.
On 9/28/07, John R Pierce pierce@hogranch.com wrote:
chitgoks wrote:
my issue involves close to a 100,000 directories and more :D
are there files in these 100,000 directories?
if you're storing a large amount of small data items, perhaps you should consider using a database, rather than a file system. _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On 9/27/07, chitgoks chitgoks@gmail.com wrote:
john yeah. a few images per directory (for now, maybe in the future more images). the directory name would point to the username.
The traditional way to deal with this would be to split up the usernames alphabetically, e.g., make subdirectories of the top directory for each of the 26 letters of the alphabet and then put the directories matching the usernames in the subdirectory corresponding to their first letter.
a/alice b/bob b/bill c/charlie e/etc
If you have more than 32000 users starting with any particular letter, such as "s", then you do the same but with the first two letters:
s/m/smith s/t/stevens
Of course all of this is just a trivial case of a hashing algorithm where the extra layers of directories correspond to hash buckets, so you can switch to a more complex hash if you prefer.
On Fri, Sep 28, 2007 at 01:38:42PM +0800, chitgoks wrote:
john yeah. a few images per directory (for now, maybe in the future more images). the directory name would point to the username.
robert, thanks. now im sure that the limit is 32k since you confirmed it as well. ive read in other posts regarding this.
It is 32,000 entries. As others have said, it's to do with the number of hard links an object can have, and making a directory results in an extra link to the current directory.
(Note: you're not limitted in number of files in this way, merely subdirectories)
Personally, I feel that having 32,000 entries is a bad idea. Especially if all those dierctories will be owned by a different person! Just do an "ls -l" and it will result in 32,000 lookups for each different username.
If you want to maintain a consistent structure like this then you can do what someone else suggested (split by first letter, for example) and if you _want_ to just say /home/user then you can create a symlink tree /home/user -> /realdisk/u/user
chitgoks wrote:
hi , our centos os has an ext3 file system. and i cant create any more directories, it gives me a too many links error, even when doing a manual mkdir.
is there any workaround for this? without changing it to a different file system like reiserFS? we dont have a reiserfs module in the os so it will have to be installed. and can it be possible to install a reiserFS without having to reformat the os? just mount it after installing? please advise
The issue is the link count in the inode for the parent directory. Each subdirectory has a ".." link that points back to the parent, and the maximum link count for an inode in an EXT2 or EXT3 filesystem is 32000. A directory begins life with a link count of 2, so there is a hard limit of 31998 subdirectories.
The limit for reiserfs is larger (max unsigned integer - 1000), so that would avoid the problem. Yes, you can certainly create a reiserfs filesystem and mount it without having to reformat other filesystems on your machine. You will lose whatever data was on the partition that you reformat.
-
Andreas Micklei -
Bart Schaefer -
Bill Campbell -
chitgoks -
John R Pierce -
Kay Diederichs -
Robert Nichols -
Stephen Harris -
umair shakil