On Tue, 2005-09-06 at 16:20 +0200, Manuel BERTRAND wrote: > Hi list, > this problem is already known and I'm sorry to bother if an acceptable > workaround was already debated on the list. > I was getting trouble with a 'grep something /var/log*' which caused the > "Memory exhausted" message. With some deeper search I found the lastlog > file in /var/log/ to be 1.2T sized. This seems to come from the > nfsnobody's uid to be 4294967294 on x86_64 system (and -1 on i386) and > the pre-allocation space for every uid (so from 0 to > ...4294967294...hum..) from lastlog. > Since I do not use NFS at all, can I just erase the nfsnobody from > /etc/passwd and start with a blank lastlog file (this looks like a great > way to nuke my system)? Or do I need to explicitly exclude this file > from all my search? How do you -x86_64 users- deal with this? It seems > the mainstream people don't see this as an issue. According to https://bugzilla.redhat.com/bugzilla/show_bug.cgi? id=149407#c22 there is a fix for this in nfs-utils-1.0.6-60.EL4, but I don't think it's the correct fix (meaning that, while it may solve the symptom, it doens't solve the root problem). Although the file is listed as being 1.2T, it's a sparse file, meaning that it doesn't really take up too much space on disk. The problem we're having is that grep and many other utilities don't handle sparse files well. There are a few ways to deal with this. I'm presenting them in order, from "most correct" to "easiest to implement". "Correct" in this case means "most directly solving the root problem," and not necessarily "most practical." 1) Redesign the kernel's vfs layer to handle sparse files automatically even when accessed by programs that don't know how to handle sparse files. I'm not sure how this might be done. At the moment, several programs (including grep and tar) attempt to read the entire 1.2TB file, instead of just the non-zero bits. I'm not sure if this is a problem with grep & tar, with open(), or with the vfs. 2) Change the format of lastlog so it doesn't use a sparse file. This isn't trivial, but I think it's probably the best solution in the long run (unless #1 happens). 3) Make grep, tar, rpm, rsync, scp, cp, and so on handle sparse files better. This probably needs to be done anyway. 4) Remove the nfs-utils package, remove /var/log/lastlog, and recreate it. This should make lastlog much smaller, unless you have a lot of users on your system. 5) Exclude lastlog from any greps you do in /var/log. For an end user, #5 is the easiest, followed by #4. References: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=64891 https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=138676 https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=144538 https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=145305 https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=146214 https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=149407 https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=156809 https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=163273 https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=164614 https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=165058