That is correct. If you actually use a browser remotely, you would need to use something like RDP with the WebAuthn Virtual Channel enabled, which unfortunately I think is currently only available by Microsoft. Some remote control software like Teamviewer has USB passthrough, but I've no idea if that works with Yubikeys (I doubt it).
So yes, working with what I'd call a "thin client setup" is something where Yubikeys are probably not a good fit, unless the protocol for that setup would support some kind of direct USB forward that actually works with Yubikeys...
So yes, working with what I'd call a "thin client setup" is something where Yubikeys are probably not a good fit, unless the protocol for that setup would support some kind of direct USB forward that actually works with Yubikeys...