Although the installation of the "CA IT Client Manager r12 Smart Card Reader" cannot be avoided at install time, it can be removed after the installation by running the following command on a DOS prompt window run as Administrator:
"%SDROOT%\..\bin\AMD64\rcmirrorinstall.exe" -scremove
After doing so, the Card Reader driver should be uninstalled and the rest of the Remote Control plugin functionality remain intact.