I use kanidm with oauth2-proxy. No issues so far, it was pretty easy to set up.
Note that the connection to kanidm needs to be TLS even if you have a reverse proxy!
EDIT: currently using 80MB RAM for two users and three Service Providers.
I also moved away my domains and the ones of the hackerspace I manage, mainly to:
I also use Migadu, they have been great so far!
desec.io for DNS, also great and supported by Traefik for DNS-01 ACME challenge.
I think you can create a group for friends and a group for family. If you want more separation I think Authentik handles multi-tenancy as well
Maybe you could try tryton? It’s modular and you can add a lot of useful functionality for businesses, like stocks/orders etc
There’s SwiftFin, but it’s been a while from the last update (iOS app was updated recently though) and there’s a number of issues. It’s usable though (I’m using it).