Hello everyone -
I have a question regarding UID and GID numbers. First, a bit of background:
Yesterday I suffered a complete power failure. My UPS batteries ran everything for an hour, but that was not long enough. My CentOS6 server shut itself down, just like it should. When the power came back up, the perfectly running server was no longer working. The boot hard drive was complete toast. ARGH!
OK, I have backups and spare hard drives. CentOS6 is getting a bit long in the tooth. I have to reformat and reinstall CentOS before restoring backups anyway, so I decided now is the time to move to CentOS7. I chose to rip the bandage!
The CentOS6 installation was quite old. Back in those days the default UID for new users was 500. That's how I set up my user account on the machine. My main workstation is Fedora 29, but it started life MANY cycles ago and also has my UID=500. That all works nicely with the nfs share coming off the server.
On newer builds I have wrestled with having my UID=1000, as is the new standard. File permissions on the nfs share were never quite right. It was not irritating enough to make me desperate, but it was annoying. I got around it with a lot of 777 and 666 permissions settings. I looked at rpc-idmapd but was never able to make it work.
Now with CentOS7 my server user account is UID=1000. The nfs share is on a separate drive (not the one that failed), and all of the files on it are still owned by UID=500. It works from my main workstation, but is really unclean.
So now the questions:
What will it take to change all the files to the new UID? What will it take to change my main workstation to the new UID **without** having to create a new user account?
I think I can do this in two steps.
0) backup, backup, backup! 1) On the server - use "find" to find all files owned by UID=500. Chown them to UID=1000. Repeat for gid=500. 2) Tricky - On the workstation, boot to non-gui. Login as root. Repeat the same two "find" commands as on the server. Edit the /etc/passwd and /etc/group files to show the new UID and GID numbers.
What does this do to the shadow files? Are there other places I need to look for the UID and GID numbers?
Thanks!
On Thu, Feb 14, 2019 at 11:04:11AM -0600, Bill Gee wrote:
I think I can do this in two steps. 0) backup, backup, backup!
This is already running and you've tested the restore process, right?
- On the server - use "find" to find all files owned by UID=500. Chown
them to UID=1000. Repeat for gid=500.
Yes.
- Tricky - On the workstation, boot to non-gui. Login as root. Repeat the
same two "find" commands as on the server. Edit the /etc/passwd and /etc/group files to show the new UID and GID numbers.
Yes. Although order does not matter -- personally I'd make the account change first.
Also, you can use `usermod -u` and `usermod -g` (possibly both at once) and this will correctly change ownership of all files in the home directory (but not outside of that).
What does this do to the shadow files? Are there other places I need to look for the UID and GID numbers?
shadow (and gshadow) are name based, so shouldn't be a problem. You may need to change some spool files in /var in addition to in /home.
Nothing else *should* be using the numeric values. (Possibly some tar files?)
Hello everyone -
Update: Many thanks to Matt Miller for the tip on usermod options. That worked very well! I did not know those options existed and would never have thought to look for them.
After making and testing backups, I started with my main workstation. Rebooted in runmode=3, then ran the usermod -u and -g options. I did this in two runs. I first had to uninstall docker since it had taken over GID=1000. No big deal, I am not using it. After the usermod programs ran, I then did a "find -uid=500" with an exec option to change ownership. Repeat for changing GID. It found a few dozen files that were not in my home directory.
Rebooted main workstation. Everything came back up, no errors. So far after about a day of use it is working just fine.
On the server I ran the two "find" commands against the entire file system. It took about half an hour to run. No surprise there as it was finding and changing several hundred thousand files. I ran the uid change in one terminal and the gid change in another. Between the two of them they consumed about 90% on both processor cores.
I did not reboot the server since I made no changes to the user account on it.
Testing from several workstation - everything gets the permissions I expect. A few odd things that used to get blocked are now working. WooHoo!
With all that done I made a fresh complete backup of the server. That backup should have all the new uid and gid associations in it.
Next step is to revert to more sensible permissions. No more 777 and 666. That will take a while. It's not critical, so I will do it in spare (!) time.
Thanks!
On 2/16/19 12:14 PM, Bill Gee wrote:
...After the usermod programs ran, I then did a "find -uid=500" with an exec option to change ownership. Repeat for changing GID. It found a few dozen files that were not in my home directory.
On the server I ran the two "find" commands against the entire file system. It took about half an hour to run. No surprise there as it was finding and changing several hundred thousand files. I ran the uid change in one terminal and the gid change in another. Between the two of them they consumed about 90% on both processor cores.
Hi Bill,
Just sharing some wisdom/experience. Too bad it's after the bulk of your work. :-)
It should not have been consuming much cpu. This is an I/O-based activity.
My guess is you used something like
find -uid=500 -exec chown 1000 {} ;
This will start a chown process for each file, changing only one file at a time. That's a lot of work the system has to do for each file! But you probably know chown (and similar utilities) can take multiple file arguments, and 'find' can help you take advantage grouping many arguments with the '+' operator to -exec:
find -uid=500 -exec chown 1000 {} +
And you could combine the bulk of your I/O work of chgrp-ing for all of your files at the same time by using 'chown 1000:1000'.
You might want to check your symlinks ('find -type l -ls'). You may have changed the ownership of what they point to rather than the link itself, precisely what the chown -h (--no-dereference) option is good for. Don't forget to revisit what the permissions should be on the link targets. Bad things would happen if the user had a symlink to /etc/passwd and/or /etc/shadow in their home directory!
I also like to use -xdev to avoid traversing into any unexpected filesystems. So I end up with:
find /somedir -xdev -uid=500 -exec chown -h newID:newGroup {} +
Hope that helps for the next time or someone else!
[This is off the top of my head, so beware of any mistakes...these commands have significant consequences.]
Best regards,
Chris Schanzle
In article 2f86eabc-697f-4f57-3a0a-f2e5da13d9d7@nist.gov, Chris Schanzle via CentOS centos@centos.org wrote:
My guess is you used something like
find -uid=500 -exec chown 1000 {} ;
This will start a chown process for each file, changing only one file at a time. That's a lot of work the system has to do for each file! But you probably know chown (and similar utilities) can take multiple file arguments, and 'find' can help you take advantage grouping many arguments with the '+' operator to -exec:
find -uid=500 -exec chown 1000 {} +
Well I never knew that! Thanks. For many years I have been doing: find ... -print0 | xargs -0 ...
Ah, I see the newer syntax was introduced in CentOS 5. :-)
Cheers Tony
On Wednesday, February 20, 2019 9:17:34 AM CST Chris Schanzle wrote:
On 2/16/19 12:14 PM, Bill Gee wrote:
...After the usermod programs ran, I then did a "find -uid=500" with an exec option to change ownership. Repeat for changing GID. It found a few dozen files that were not in my home directory.
On the server I ran the two "find" commands against the entire file system. It took about half an hour to run. No surprise there as it was finding and changing several hundred thousand files. I ran the uid change in one terminal and the gid change in another. Between the two of them they consumed about 90% on both processor cores.
Hi Bill,
Just sharing some wisdom/experience. Too bad it's after the bulk of your work. :-)
It should not have been consuming much cpu. This is an I/O-based activity.
My guess is you used something like
find -uid=500 -exec chown 1000 {} ;
This will start a chown process for each file, changing only one file at a time. That's a lot of work the system has to do for each file! But you probably know chown (and similar utilities) can take multiple file arguments, and 'find' can help you take advantage grouping many arguments with the '+' operator to -exec:
find -uid=500 -exec chown 1000 {} +
And you could combine the bulk of your I/O work of chgrp-ing for all of your files at the same time by using 'chown 1000:1000'.
You might want to check your symlinks ('find -type l -ls'). You may have changed the ownership of what they point to rather than the link itself, precisely what the chown -h (--no-dereference) option is good for. Don't forget to revisit what the permissions should be on the link targets. Bad things would happen if the user had a symlink to /etc/passwd and/or /etc/shadow in their home directory!
I also like to use -xdev to avoid traversing into any unexpected filesystems. So I end up with:
find /somedir -xdev -uid=500 -exec chown -h newID:newGroup {} +
Hope that helps for the next time or someone else!
[This is off the top of my head, so beware of any mistakes...these commands have significant consequences.]
Best regards,
Chris Schanzle
Thanks Chris and Simon -
I was not concerned about CPU usage while running thousands of instances of chown and chgrp. It was expected, and this server does not handle any user except me. I will keep that '+' operator in mind for the next time I need it. It will not be needed often, but when needed it will be very useful.
I ran chown and chgrp separately because there are some files that are owned by me but have other groups - and a few which I do not own but are in my group. Using "chown user:group" to change both would have overridden those few odd files.
I checked for symlinks and did not find any that were changed by mistake. That's one bullet I seem to have dodged by luck.
Bill Gee
-
Bill Gee -
Chris Schanzle -
Matthew Miller -
tony＠softins.co.uk