[CentOS] Sendmail novrfy filter by ip address?

I have a virus and spam filter device that can do VRFY commands to
reject invalid email before it gets to the next mail hop.  How can I
configure the SMTP server to only allow VRFY commands from one
particular IP address, and nowhere else?  I don't want spammers to be
able to hammer on the gateway looking for valid addresses to send to.