SAML assertion was getting truncated at the relying party when the attributes are being sent in an assertion and if user is a part of groups which has size more than 1024.
Federation versions running from R12Sp3Cr10, R12.5, R12.51 and R12.52
In R12 SP3:
The following issues occur:
The directory attributes appear truncated at the relying party.
The following message appears in the smtracedefault.log file:
[WARNING: Response attribute will be trimmed. [attr = SMUSERGRP:memberOf] [actual attr len = number] [ response attr len = number]]
Note: In the Warning message, SMUSERGRP represents the variable name and memberOf represents the attribute value. The error message is specific to your configuration.
This issue comes when the size of the header/attribute which is larger than 1024 will be sent in an assertion as the relying party/consumer/service provider will receive a truncated attribute in an assertion and will not be able to authorize the users based on that.
As in previous versions there was no limit and it was based on purely the web server size limit for header, but after 12 SP3, the things got changed. So, if the header size of the user attribute size is more than 1024 then we need to increase the value of the following fields to resolve the issue.
We need to change the value of the below fields in the file name "EntitlementGenerator.properties" and this file name would be present in the path "<<PS_Installation_path>>\config\properties"
So, based on the type of federation, we can increase the value of the field in the above file.