You certainly don't need a full on Keycloak installation here, if you don't want to go that far. There's various OIDC providers, some more complex than others!
If you already have LDAP or some other backing auth, setting up Dex for OIDC is pretty easy. Took me less than an hour or so.
If you want something fancier Authelia isn't too bad, I got that running in an evening and hooking it up to Tailscale took another hour or two. Most of that spent figuring out how I want to do webfinger.
2. I set up Webfinger first, so assuming you're setting it up from scratch you can either run a Webfinger server yourself, or just configure the paths in whatever web server you have for your base domain. I didn't feel like running Yet Another Server and since the Tailnet's only for me I just plugged the following section into Caddy:
Where webfinger.json is file containing the response tailscale is looking for from their doc. You can verify it works right at https://webfinger.net/lookup/ .
I guess I'm doing overkill then. I actually use Keycloak for Tailscale. It also serves as authentication for my Nextcloud and Mastodon instances, so maybe slightly less overkill.