On the EEM -> Configure -> User Store -> LDAP Attribute Mapping -> Mapping name -> <your custom map>
The top section "User Lookup" has a "User Search Filter"
That is where you would want to include filters to restrict what users are seen.
The lower part of the window "Group Attribute Mapping" has a
"Group Search Filter" that is where you would filter the groups, not users, seen by EEM.
Example of a filter for the user's restricting them to be ONLY users that are members of my specific Administrators's group.
And here is an example of a Group filter restricting the groups seen by EEM to only ones that start with oak, Plat or Adm:
NOTE - You should get an ldif of a user so you can confirm EXACTLY how their attributes are defined so you know how to set up the filters.