In most cases the data is referenced against the email address. The root cause for most performance issues is that the LDAP Server has no index based on the reference (i.e., usually the email address).
Since every LDAP lookup initially resolved the email address, and all follow up queries are based on the email reference, every lookup can take a very long time. Symantec has seen instances of over 2 minutes and, as a result, even the unloading of the plug-in.
Adding an index for the used reference, for example the emails, within the LDAP server can bring the query resolution to < 1 seconds (!!)