1. Make sure that the hostname in PAM matches the hostname in the RSA server. The RSA server I used for my tests was not able to resolve the IP Address to the Name, so I set the hostname to the IP address. After doing this, RSA and LDAP+RSA started working.
2. Do not remove the sdopts.rec config file. While this might not have been used prior to 3.x it seems to be needed now. Please note that there appears to be a difference in behavior regarding this file. In some cases it can be an empty file, which on some versions can be created(if needed) by a support engineer doing a "touch /var/ace/sdopts.rec" in an ssh session. At some point an upload of an empty sdopts.rec file was allowed, but most recently this became disallowed. If that is the case, create your sdopts.rec to look something like the following, with multiple entries for a cluster:
CLIENT_IP=<PAM IP here>
In another case a customer was using an RSA cluster. They needed to put all the IP addresses into the sdopts.rec, along with a priority, in order to prevent connecting to an RSA server that would respond too slowly, because of its location.
If sdopts.rec is imported there will be a prompt to clear the node secret in pam. This used to require that the node secret be cleared on the RSA server too, but there was a change made at some point. The RSA Security Web Console does not clear autogenerated node secrets, such as those from PAM.
3. Import the sdconf.rec file again. Doing so will prompt you delete the node secret in PAM. The same comment about clearing the node secret on the RSA server holds true here.
There is a folder on PAM, in /var/ace, which corresponds to the hostname configured on PAM and the RSA server. This folder is supposed to be cleared when the node secret is deleted, but in some instances it is not. This can be checked by support if RSA authentication is not working. Note that the node secret is no longer stored in a dedicated file, and a missing check for it in the PAM UI doesn't mean that there is a problem. But clearing the node secret still has the value of clearing the client folder and restarting the RSA client in PAM.
4. Note that PAM 3.X uses a newer RSA client library that communicates with the RSA server port 5500 using the TCP protocol rather than the UDP protocol used by the older client in PAM 2.X. If you had a firewall rule that opened port 5500 from PAM to the RSA server selectively for UDP, you will have to update that rule to accommodate the change to TCP.
5. In one case it turned out that the customer had configured the wrong Root certificate on the RSA server. This was corrected and RSA authentication worked immediately.
If none of the suggestions above work for you, please open a ticket. Make sure to let us know the current state of the RSA configuration on the system you upgraded? Which of the .rec files are populated?