[CentOS] ssh issues with centos 6

I have a new centos 6 install.  Randomly i get out of memory messages and
my ssh file transfer dies.  I have tried this in filezilla as well as
winscp.  I have disabled motd in ssh and i also had entropy problems so i
enacted   rngd -r /dev/urandom -o /dev/random but did not stop the out of
memory errors.  I am seeing the below in the logs.  It only occurs during
ssh file transfers.

Feb 28 17:18:19 web1 t of memory [13604]