Re: Software RAID 5 SATA array crashed
Alvin Oga wrote:
hi ya
the only major piece of info that is missing:
a) cat /proc/mdstat
adar@adar:~$ cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid5] [multipath] [raid6]
[raid10] [faulty]
md0 : active raid5 sdb[0] sde[4] sdf[3] sdd[2] sdc[1]
1172229120 blocks level 5, 64k chunk, algorithm 2 [5/5] [UUUUU]
unused devices: <none>
b) what exactly was the commands you type vs the english
explainations
Here is the pertinent command history. I didn't know how to use mdadm,
so much of it is probably meaningless. On another terminal, at some
point, I issued "mdrun", which finally brought up the array and began
the resync process. Unfortunately, I'm not sure when within this command
history I used mdrun.
89 sudo mdadm
90 sudo mdadm -q
91 sudo mdadm --help
92 sudo mdadm --misc -Q
93 sudo mdadm --misc -Q /dev/md0
94 sudo mdadm --misc -Q /dev/sdb
95 sudo mdadm --misc -Q /dev/sdb1
96 sudo mdadm --misc -Q /dev/md0
97 sudo mdadm --assemble -Q /dev/md0
98 sudo mdadm --assemble /dev/md0
99 sudo mdadm --assemble --scan /dev/md0
100 cat /proc/mdstat
101 sudo mdadm --misc -Q /dev/md0
102 sudo mdadm --detail --misc -Q /dev/md0
103 sudo mdadm --detail -Q /dev/md0
104 sudo mdadm -Q --detail /dev/md0
105 sudo mdadm -Q --detail /dev/md0 | less
106 mount
107 sudo mount -a
108 cat /proc/mdstat
109 sudo mdadm -Q --detail /dev/md0 | less
110 fdisk /dev/sdc
111 /sbin/fdisk /dev/sdc
112 ls /dev/sd*
113 sudo /sbin/fdisk /dev/sdc
114 mdadm --help
115 sudo mdadm -Q --assemble --scan -E /dev/md0 | less
116 sudo mdadm --assemble --scan -E /dev/md0 | less
117 sudo mdadm --misc --scan -E /dev/md0 | less
118 sudo mdadm --misc --scan -E /dev/md0
119 sudo mdadm --misc -S /dev/md0
120 sudo mdadm --misc --scan -E /dev/md0
121 sudo mdadm --misc --scan -E -D /dev/md0
122 sudo mdadm --misc -E /dev/md0
123 sudo mdadm -E /dev/md0
124 sudo mdadm /dev/md0 -E
125 sudo mdadm --misc /dev/md0 -E
126 sudo mdadm -v --misc /dev/md0 -E
127 sudo mdadm -v --misc /dev/md0
128 sudo mdadm -v --misc --scan /dev/md0
129 sudo mdadm /dev/md0
130 sudo mdadm --detail /dev/md0
131 sudo mdadm --detail /dev/md0 | less
132 sudo mdadm --manage /dev/md0\
133 sudo mdadm --manage /dev/md0
134 sudo mdadm --manage /dev/md0 -r /dev/sdc1
135 sudo mdadm --manage /dev/md0 -r /dev/sdc
136 sudo mdadm --manage /dev/md0 -r /dev/sdc1
137 sudo mdadm --manage /dev/md0 -f /dev/sdc1
138 cat /proc/mdstat
139 sudo mdadm --misc /dev/md0 -R
140 cat /proc/mdstat
141 dmesg | less
142 sudo less /var/log/syslog
143 sudo mdadm --misc /dev/md0 -q /dev/sdc
144 sudo mdadm --misc /dev/md0 -Q /dev/sdc
145 sudo mdadm --misc /dev/md0 -Q /dev/sdc --examine
146 sudo mdadm --misc /dev/md0 -Q /dev/sdc --examine | less
147 sudo mdadm --misc /dev/md0 -Q /dev/sda --examine | less
148 sudo mdadm --misc /dev/md0 -Q /dev/sdb --examine | less
149 sudo mdadm --misc /dev/md0 -Q /dev/sdc --examine | less
150 sudo mdadm --misc /dev/md0 -Q /dev/sdd --examine | less
151 sudo mdadm --misc /dev/md0 -Q /dev/sde --examine | less
152 sudo mdadm --misc /dev/md0 -Q /dev/md0 --examine | less
153 sudo mdadm --misc /dev/md0 -Q /dev/md0 --examine
154 sudo mdadm --misc /dev/md0 --detail
155 sudo mdadm --misc /dev/md0 --detail --scan
156 less /etc/mdadm/mdadm.conf
157 man mdadm.conf
158 sudo mdadm --examine --scan
159 sudo mdadm --examine\
160 sudo mdadm --examine
161 sudo mdadm --scan
162 sudo mdadm --misc --scan
163 sudo mdadm --misc /dev/md0 --scan
164 sudo mdadm --misc /dev/md0 --scan --detail
165 sudo mdadm --query /dev/md0
166 sudo mdadm --query /dev/md0 --detail
167 less /proc/partitions
168 man badblocks
169 shutdown -h now
170 sudo shutdown -h now
171 /bin/dmesg
172 /bin/dmesg |less
173 psaux
174 dme
175 top
176 dmesg | less
177 less /proc/mdstat
178 less /dev/md0
179 mount
180 mdadm
181 sudo mdadm
182 sudo mdadm --help
183 man mdadm
184 sudo mdadm
185 man mdadm
186 man mdadm
187 man mdadm
188 ps aux
189 dmesg | less
190 cat /proc/mdstat
191 sudo mdadm --misc /dev/md0
192 sudo mdadm --misc /dev/md0 --query
193 sudo mdadm --misc /dev/md0 --examine
194 sudo mdadm --misc /dev/md0 --run
195 cat /proc/mdstat
196 sudo mdadm --assemble /dev/md0 --update=summaries
197 sudo mdadm --assemble /dev/md0 -v --update=summaries | less
198 sudo mdadm --misc /dev/md0 -s
199 sudo mdadm --assemble /dev/md0 -v --update=summaries | less
200 sudo mdadm --misc /dev/md0 -S
201 sudo mdadm --assemble /dev/md0 -v --update=summaries | less
202 sudo mdadm --stop
203 sudo mdadm --stop /dev/md0
204 sudo mdadm --assemble /dev/md0 -v --update=summaries | less
205 sudo mdadm --assemble /dev/md0 -v --update=summaries 2>&1 | less
206 sudo mdadm --stop /dev/md0
207 sudo mdadm --assemble /dev/md0 -v --update=summaries 2>&1 | less
208 man mdadm
209 man mdadm
210 man mdadm
211 man mdadm
212 cat /proc/mdstat
213 xfsinfo
214 man xfs_repair
215 xfs_repair -n
216 sudo xfs_repair -n
217 sudo xfs_repair -n /dev/md0
218 man xfs_repair
219 man xfs_checkl
220 man xfs_check
221 man xfs
222 sudo xfs_check
223 sudo xfs_check /dev/md0
224 less /dev/md0
225 less -f /dev/md0
226 sudo less -f /dev/md0
227 cat /proc/mdstat
228 fdisk /dev/md0
229 sudo fdisk /dev/md0
230 sudo mdadm --start
231 sudo mdadm --assembl /dev/md0 --run
232 sudo mdadm --assemble /dev/md0 --run
233 sudo mdadm --stop /dev/md0
234 ls /dev/md0
235 sudo less -f /dev/md0
236 sudo mdadm --assemble /dev/md0 --run
237 sudo mdadm --stop /dev/md0
238 sudo mdadm --assemble /dev/md0 --verbose --run
239 sudo mdadm --assemble /dev/md0 --verbose --verbose --run
240 sudo mdadm --stop /dev/md0
241 sudo mdadm --assemble /dev/md0 --verbose --verbose --run
242 sudo mdadm --stop /dev/md0
243 dmesg
244 sudo shutdown -h now
245 ps aux
246 dmesg
247 cat /proc/dmesg
248 cat /proc/mdstat
249 cat /proc/mdstat
250 sudo fdisk /dev/md0
251 y
252 exit
253 ps aux
254 man xfs
255 man xfs_admin
256 man xfs_info
257 man xfs_repair
258 ps aux
259 ls /usr/sbin/xfs_*
260 man xfs_io
261 man xfs_ncheck
262 man xfs_rtcp
263 man xfs_quota
264 man xfs_mkfile
265 man xfs_freeze
266 man xfs_d
267 man xfs_db
268 man xfs_db
269 xfs_db
270 sudo xfs_db
271 sudo xfs_db /dev/md0
272 man xfs_check
273 sudo xfs_check /dev/md0
274 sudo xfs_check -v /dev/md0
275 sudo xfs_check -vvv /dev/md0
276 man xfs_check
277 sudo xfs_check -b 1 /dev/md0
278 sudo xfs_check -b 10 /dev/md0
279 sudo xfs_check -i 10 /dev/md0
280 apt-cache search xfs | less
281 apt-cache show mondo
282 apt-cache search xfs | less
283 apt-cache show xfsdump
284 apt-cache show xfsprogs
285 man uuid
286 apt-cache show libuuid1
287 apt-cache search fs tools
288 apt-cache search fs recovery
289 apt-cache show testdisk
290 ps aux
291 dmesg
292 exit
293 dd
294 man dd
295 man dd
296 dd if=/dev/md0 bs=512 count=1
297 sudo dd if=/dev/md0 bs=512 count=1
298 sudo dd if=/dev/md0 bs=512 count=1 | less
299 man less
300 sudo dd if=/dev/md0 bs=2048 count=1 | less
301 sudo dd if=/dev/md0 bs=65536 count=1 | less
302 sudo dd if=/dev/md0 | less
303 sudo dd if=/dev/md0 | strings | less
304 man od
305 sudo xfs_repair -n -o /dev/md0
306 sudo xfs_repair -n -o assume_xfs /dev/md0
307 cat /proc/mdstat
308 which xfs_repaid
309 /sbin/xfs_repair -v
310 /sbin/xfs_repair --version
311 man xfs_repaid
312 man xfs_repair
313 dd if=/dev/md0 of=/dev/stdout |less
314 sudo dd if=/dev/md0 of=/dev/stdout |less
315 exit
316 ps aux
317 mail
318 rm /home/adar/mbox
319 ps aux
320 cat /proc/mdstat
321 ls
322 sudo mount -a
323 sudo xfs_check /dev/md0
324 sudo xfs_repair -n /dev/md0
On Sat, 17 Sep 2005, Adar Dembo wrote:
Sep 11 23:46:57 localhost kernel: md: kicking non-fresh sdc1 from array!
Sep 11 23:46:57 localhost kernel: md: unbind<sdc1>
obviously bad
Note the message about sdc being non-fresh. Also note that the array is
both DIRTY and DEGRADED.
degraded means it works as if it was one giant disk, and if any of the
remaining disks dies, you are most likely to lose all data on all disks
unless you happen to know how to recover tdata when 2 of the 5 disks
are considered bad ... ( raid allows for 1 of 5 to go bad in your config )
I played around with mdadm but I could never get the array to start. All
the exact commands you used and in what order makes all the difference
the output of /proc/mdstat is what you should be looking at
it'd be 100x simpler to reformat and restore from backups
unless you know why you did which commands in which order
say based on some software-raid-repair-howto for which each
failure situation is probably different
c ya
alvin
Reply to: