The JWT Authentication Scheme fails to authenticate token with many claims and this is observed for the JWT token size over 4k.
The following output can be observed in this situation (smtracedefault.log):
[11/14/2022][08:49:21][5892][][SmAuthUser.cpp:784][ServerTrace][][][][][][][][ Parsing JWT Request of type SIGNED][][][][SmJWTAuthScheme:: Parsing JWT Request of type SIGNED][][][]
[11/14/2022][08:49:21][5892][][SmAuthUser.cpp:784][ServerTrace][][][][][][][][ JIJwtTokenParser.JWTState.SIGNED RSA][][][][SmJWTAuthScheme:: JIJwtTokenParser.JWTState.SIGNED RSA][][][]
[11/14/2022][08:49:21][5892][][SmAuthUser.cpp:784][ServerTrace][][][][][][][][ Cert alias will be used for validation of token][][][][SmJWTAuthScheme:: Cert alias will be used for validation of token][][][]
[11/14/2022][08:49:21][5892][][SmAuthUser.cpp:784][ServerTrace][][][][][][][][Cert alias will be used for validation: oidc-sign-token][][][][SmJWTAuthScheme:: Cert alias will be used for validation: oidc-sign-token][][][]
[11/14/2022][08:49:21][5892][][SmAuthUser.cpp:784][ServerTrace][][][][][][][][ IJwtTokenParser.JWTState. Validating with alias.][][][][SmJWTAuthScheme:: IJwtTokenParser.JWTState. Validating with alias.][][][]
[11/14/2022][08:49:21][5892][][SmAuthUser.cpp:784][ServerTrace][][][][][][][][ Token verification failed with alias:oidc-sign-token,Algorithm:RS256][][][][SmJWTAuthScheme:: Token verification failed with alias:oidc-sign-token,Algorithm:RS256][][][]
[11/14/2022][08:49:21][5892][][SmAuthUser.cpp:784][ServerTrace][][][][][][][][ Exptected state of JWT Token for validation:SIGNONLY,but receieved state is:NONE][][][][SmJWTAuthScheme:: Exptected state of JWT Token for validation:SIGNONLY,but receieved state is:NONE][][][]
Policy Server 12.8SP6
Upgrade the PolicyServer to 12.8SP8, when this one will be available to benefit from fix DE551905.
Defect fixes in the service packs, in this case 12.8 SP8, can be verified from the below link once it is GA.
Defects Fixed in Service Packs
https://techdocs.broadcom.com/us/en/symantec-security-software/identity-security/siteminder/12-8/release-notes/service-packs.html