You're still covered, but you're paying a cost (missing work, being sick) and so is the insurance provider/govt, that you wouldn't otherwise have paid, and it's the non-vaccinator's fault, and they aren't paying for that damage under your system.

Not really a flaw in your system as much as just something interesting to think about.