I looked into it further and I think I'm mostly wrong about <svg>. You can embed javascript into it, but if you don't, then as far as I can tell it works fine for me. Sorry!
That makes sense…and I think that's where the potential security vulnerability comes into play - if you allow untrusted users to upload SVGs, then they could potentially be uploading javascript to your site, offering a vector for XSS. As far as I can tell, adding your own SVGs to your site doesn't open up any novel security issues.
