I have seen vi do this action when it didn't understand a keycode on teh terminal you are using properly... change the case of a few letters next to the cursor. But IIRC that was busybox vi.
Is it crazy to propose someone opened /etc/passwd in vi, and saved it out without noticing this had happened?
If you suspect your box has been rooted, then perhaps it is time to do some checking.
rpm -Va
Also, have you ever updated the box?