Disk drive failed in Security Analytics storage array

Products

• Security Analytics

Issue/Introduction

Security Analytics storage arrays contain many hard drives.  These drives can fail at any given moment.  All Security Analytics storage arrays are configured with redundant RAID 5 volumes.  Managed SAN storage arrays such as the Dell MD3860f, the NetApp E5660 or the Dell VA084 storage arrays may have hot spares configured to assist with hard drive failures.   

Other errors that may indicate a failed drive

"DEGRADED_VOLUME-Recovery Failure" 

"VOLUME_HOT_SPARE_IN_USE-Recovery Failure"

"Volume not on preferred path due to AVT/RDAC failover"

"IMPENDING_DRIVE_FAILURE_RISK_MED-Recovery Failure"

"UNRESPONSIVE DISK"  

"NO_REDUNDANCY_DRIVE-Recovery Failure"

Drive - Loss of Path Redundancy

Cause

A hard drive has failed and needs to be replaced.

 

Resolution

If you encounter any failed hard drives in the system, make sure the system is under a current support contract and open a case with technical support to get the drive replaced.  Support will need any supporting log files such as the Customer Service Report (CSR) and any support files available from the storage arrays themselves.  

If the failed drive is in the main head unit or your external storage consists of the Promise J5300 devices, generate a CSR from the Settings > System page in the Security Analytics GUI.

The following knowledge base articles will help in determining the hardware and in generating logs.

• What is my storage array type or brand and how do I get parts replaced?
• How do I collect hardware logs for Dell head units?
• Collecting logs for Dell VA084/U840 or VA012/U144 storage arrays
• How to generate a support bundle for Dell MD3860 or NetApp attached storage
• Using RACADM or iDRAC to generate a Dell TSR
• Failed hard drive replacement on Netapp E5660 storage array
• How to replace a Disk Drive in Carrier (DDIC) in an ME4 Series 5U enclosure

A valid shipping address will also be required with a contact name and phone number. Some support contracts include the option for a field engineer to replace the drive.  The support team can help to confirm this.  

Attach the supporting log files to the case using the support portal.

Additional Information

To review the specific information about a drive located in the head unit or in one of the J5300 JBODs, you can run megacli -pdinfo -physdrv [E:S] -aX.  Using lsi-show can help you find the location of the drive.

Find the adapter number in the Controller Information, in the first column under Id.  It is either 0, 1, or 2.

Find the Enclosure Id under Enclosure Information and find the number in the first column under Id. 

Find the Slot id in Physical Drive Information in the second column under the label "E:S".  In the example below it is the second parameter of 32:0 or 32:1.  The Slot would be 0 or 1.

For example, enclosure 32, Slot 2, Adapter 0.  The command would be megacli -pdinfo -physdrv [32:2] -a0

megacli -pdinfo -physdrv [32:2] -a0
                                     
Enclosure Device ID: 32
Slot Number: 2
Device Id: 2
Sequence Number: 2
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SAS
Raw Size: 931.512 GB [0x74706db0 Sectors]
Non Coerced Size: 931.012 GB [0x74606db0 Sectors]
Coerced Size: 931.0 GB [0x74600000 Sectors]
Firmware state: Online, Spun Up
SAS Address(0): 0x5000c50083c50a85
SAS Address(1): 0x0
Connected Port Number: 6(path0) 
Inquiry Data: SEAGATE ST91000642SS    SERIAL_NUMBER            
FDE Capable: Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None 
Device Speed: 6.0Gb/s 
Link Speed: 6.0Gb/s 
Media Type: Hard Disk Device

Exit Code: 0x00

 

Run the command "lsi-show" from the command line.  The sample lsi-show output may look something like this.

Controller Information
Id   Serial No        Firmware             Encl LD   PD   Product Name
a0   SERIAL_NUMBER    4.250.00-4402        1    1    8    PERC H730P Mini

Enclosure Information
Id   Drives
32   0   1   2   3   4   5   6   7  

Logical Drive Information
Id   Size         Status       Stripe   Raid Level   Cache
0    6.364 TB     Optimal      64 KB    Primary-5    WriteBack, ReadAdaptive, Direct, Write Cache OK if Bad BBU
  by id: 0,1,2,3,4,5,6,7
  by es: 32:0,32:1,32:2,32:3,32:4,32:5,32:6,32:7

Physical Drive Information
Id   E:S      Status                         FC       Size           Inquiry
0    32:0     Online, Spun Up                None     931.512 GB     SEAGATE ST91000642SS    SERIAL_NUMBER
1    32:1     Online, Spun Up                None     931.512 GB     SEAGATE ST91000642SS    SERIAL_NUMBER
2    32:2     Online, Spun Up                None     931.512 GB     SEAGATE ST91000642SS    SERIAL_NUMBER
3    32:3     Online, Spun Up                None     931.512 GB     SEAGATE ST91000642SS    SERIAL_NUMBER
4    32:4     Online, Spun Up                None     931.512 GB     SEAGATE ST91000642SS    SERIAL_NUMBER
5    32:5     Online, Spun Up                None     931.512 GB     SEAGATE ST91000642SS    SERIAL_NUMBER
6    32:6     Online, Spun Up                None     931.512 GB     SEAGATE ST91000642SS    SERIAL_NUMBER
7    32:7     Online, Spun Up                None     931.512 GB     SEAGATE ST91000642SS    SERIAL_NUMBER