Just use wireguard as VPN and bind ssh only to that interface. You loose public access but I couldn’t think of a reason why I want other devices than my own to connect anyway. You have to make sure that ssh starts after wireguard though or it can’t bind the port.