What if my origin server is behind Google Cloud CDN anyways? I mean it is possible for me to run my own datacenter with a direct connection to the users IP but that is obviously unreasonable. As soon as I outsource anything it can now see the users IP.
I guess this particular case seems somewhat reasonable, but where is the line.
Also note that Google Fonts is a lot more that just hosting a download. It has different font files for different browsers for max compatibility as well as font splitting so that you aren't downloading too many glyphs and weights that you don't need for this page. Reimplementing Google Fonts isn't trivial.
The same does not apply to sideside CDNs.
However, I believe NPM CDNs are the same scenario, so I'd start serving packages from the server as well.