This is a normal behavior for a VMware HA/DRS cluster that is using strict admission control.
Disabling strict admission control (allowing virtual machines to power on even if they violate constraints) should allow a host to enter maintenance mode in this situation. A bug was discovered that would not allow a host to enter maintenance mode even if strict admission control was disabled. However, this was resolved in VirtualCenter 2.5 Update 3 and disabling strict admission control should now allow hosts to enter maintenance mode correctly.
To work around the issue, temporarily disable VMware HA in the cluster settings. After disabling the HA in the cluster settings, you can put the ESX Server host into Maintenance Mode, do the work required, and then re-enable HA on your cluster.
Note: DRS needs to be enabled on your cluster in Fully Automated mode if you want VirtualCenter to migrate your running virtual machines automatically to other hosts when placing your host in Maintenance Mode.