If I understood it correctly, only your side needs IMAP.

From there outwards the protocol (degrades to/can use) regular mail/smtp, so you can indeed connect with any address capable of receiving email.