Checking the inventory service log file during upgrade you can see the service just throwing many java stack traces.
Restarting the sso service may not be required. First attempt may be to just remove and readd the identity source. If this works and the domain shows up properly then upgrade will work fine.