Re: Failing disk advice
On Mon, Mar 06, 2017 at 12:17:03PM +0100, Mirko Parthey wrote:
> On Sun, Mar 05, 2017 at 08:38:27PM -0800, David Christensen wrote:
> > On 03/05/2017 01:02 PM, Gregory Seidman wrote:
> > >I have a disk that is reporting SMART errors. It is an active disk in
> > >a (kernel, not hardware) RAID1 configuration. I also have a hot spare
> > >in the RAID1, and md hasn't decided it should fail the disk and switch
> > >to the hot spare. Should I proactively tell md to fail the disk (and
> > >let the hot spare take over), or should I just wait until md notices a
> > >problem?
> >
> > I'm confused by "I also have a hot spare in the RAID1". Do you have a
> > two-member RAID1 with a hot spare, or a three-member RAID1? I would
> > prefer the latter:
> >
> > https://manpages.debian.org/jessie/mdadm/md.4.en.html
>
> Refining this advice a bit, I would convert the spare to a full RAID
> member now, without explicitly failing the disk that reports SMART
> errors first.
> Assuming you have a two-member RAID1 with a hot spare, the command
> should be similar to this (untested):
> mdadm -G /dev/mdX -n 3
> This ensures you keep redundancy during further maintenance actions.
I was unaware that this was possible. I've run it and mdadm -D reports that
it is now in the "clean, degraded, rebuilding" state. Thank you! I wish I
had room in my system to add the fourth (which I've ordered) without
removing the failing disk, but I do not.
> Which SMART errors do you get, and who reports them?
I get emails sent to root:
This message was generated by the smartd daemon running on:
host name: XXXXXX
DNS domain: YYYYYY
The following warning/error was logged by the smartd daemon:
Device: /dev/sdc [SAT], 8 Currently unreadable (pending) sectors
Device info:
ST31500341AS, S/N:9VS43CV9, WWN:5-000c50-0208aa9a3, FW:CC1H, 1.50 TB
For details see host's SYSLOG.
You can also use the smartctl utility for further investigation.
The original message about this issue was sent at Wed Dec 14 00:51:36 2016 EST
Another message will be sent in 24 hours if the problem persists.
...and...
This message was generated by the smartd daemon running on:
host name: XXXXXX
DNS domain: YYYYYY
The following warning/error was logged by the smartd daemon:
Device: /dev/sdc [SAT], 8 Offline uncorrectable sectors
Device info:
ST31500341AS, S/N:9VS43CV9, WWN:5-000c50-0208aa9a3, FW:CC1H, 1.50 TB
For details see host's SYSLOG.
You can also use the smartctl utility for further investigation.
The original message about this issue was sent at Wed Dec 14 00:51:37 2016 EST
Another message will be sent in 24 hours if the problem persists.
(Yes, I know, I've been letting it do this since mid-December, which is not
great.)
> What is the output of the following command for the failing drive?
> smartctl -A /dev/sdY
# smartctl -A /dev/sdc
smartctl 6.4 2014-10-07 r4002 [i686-linux-3.16.0-4-686-pae] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 119 099 006 Pre-fail Always - 205161943
3 Spin_Up_Time 0x0003 100 091 000 Pre-fail Always - 0
4 Start_Stop_Count 0x0032 099 099 020 Old_age Always - 1055
5 Reallocated_Sector_Ct 0x0033 099 099 036 Pre-fail Always - 41
7 Seek_Error_Rate 0x000f 092 060 030 Pre-fail Always - 1743842168
9 Power_On_Hours 0x0032 039 039 000 Old_age Always - 53898
10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0
12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 85
184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0
187 Reported_Uncorrect 0x0032 097 097 000 Old_age Always - 3
188 Command_Timeout 0x0032 100 098 000 Old_age Always - 133146017827
189 High_Fly_Writes 0x003a 007 007 000 Old_age Always - 93
190 Airflow_Temperature_Cel 0x0022 060 040 045 Old_age Always In_the_past 40 (Min/Max 26/45 #502)
194 Temperature_Celsius 0x0022 040 060 000 Old_age Always - 40 (0 18 0 0 0)
195 Hardware_ECC_Recovered 0x001a 038 023 000 Old_age Always - 205161943
197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 8
198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 8
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 1
240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 53897 (15 186 0)
241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 917595486
242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 1262569510
> Regards,
> Mirko
Thanks for the help so far,
--Greg
Reply to: