Replacing a storage controller in MD3860 or NetApp E5660 storage array

Products

• Security Analytics

Issue/Introduction

If one of the storage array controllers for either the Dell MD3860 or the Netapp E5660 is having a problem or is Offline, you may have to replace the controller.  

You may also see this error in the recovery guru file in the storage bundle:  Failure Entry 2: OFFLINE_CTL-Recovery Failure Type Code: 28

The controller has failed or has a red X.

The controller is offline.

Resolution

To replace the controller, follow these steps.  The controllers are hot-swappable.

Dell MD3860

NetApp E5600 series

1. Confirm that the controller is showing Offline in SANtricity (Right click > Advanced > Place > Offline).  If it is still Online, change it to Offline.
2. Do not remove fiber cables
3. Unseat the failed Controller about 1 inch.  The power will remain on while the controller shuts down
4. Once all lights are off on the controller, label and remove fiber cables (including the SFP modules if the new controller does not have any).
5. Completely remove the Controller
6. Insert the new Controller, but do not fully seat
7. Confirm that all fiber cables and SFPs are firmly secure and seated all the way.
8. Slide the new controller all the way in making sure it is flush with the other controller.  The black locking lever should be fully closed.
9. Wait for about 5-10 minutes for the storage array to recognize the new controller and synchronize the configuration.
10. Look for the 7-segment LED two-digit indicator.  If there are two 'zeros' (Dell MD3860) or if there are two "nines" (Netapp), the controller is fully booted and synchronized.
11. If the Preferred Path error is still showing in SANtricity, redistribute the volumes again.  See this article for details on this procedure: NON_PREFERRED_PATH on Security Analytics storage
12. Generate a new storage log bundle and send it to technical support to confirm all looks healthy with the new controller.