I did wonder that myself. I have now amended to Dovecot definition in jail.conf to:
[dovecot]
port = pop3,pop3s,imap,imaps,submission,sieve,25,1025,465,587 logpath = %(dovecot_log)s backend = %(dovecot_backend)s
I then unbanned and banned each IP address manually with
Did you reload the configuration? ("fail2ban-client reload")
What action are you using - you mention ipset, are you using iptables- ipset-proto4? I don't know anything about ipset, but can you see what ports are being blocked in the fail2ban-dovecot set (just to make sure it is doing the correct thing).
If you manually add an IP address to the *exim* jail, does it get blocked?
P.