A mail server that only talks TLS is not following the SMTP protocol and is not a part of the global system commonly understood with the term e-mail. Maybe it would be a great idea to migrate the whole world to such a configuration, but in practice it wouldn't give me much confidence. If my server A hands something off to B for it to be delivered to C, then I have no control over whether the link between B and C is secured, so e2e is the only way to be sure.