Because anything that's vulnerable to collision attacks is theoretically vulnerable to preimage attacks. Where I went wrong was assuming that preimage attacks were practical, but as you've rightly said, there's been no known exploits because of their extreme difficulty.
So it's one of those situations where everyone was right: it's so impractical to exploit that it's as good as not vulnerable even though it's mathematically possible.