Cleaning up after disk crashing season hasn’t been fun, but I am pleased with what I’ve managed to recover from the worst crash.
I wanted to get the latest data from the dead webserver. It was in MySQL, and stored in /var/lib/mysql. Unfortunately, the /var/lib directory no longer existed.
I didn’t want to try to recover it in place — with so many bad blocks, things can only get worse — so I copied the entire partition to a file on my laptop (the one with the shiny new disk):
ssh deadserver dd if=/dev/hda1 conv=noerror > hda1.img(You need the conv=noerror or else dd will stop when it hits the first bad block.)
So, then I had most of a corrupt filesystem image. To make it useful I used the loop driver:
losetup /dev/loop0 hda1.imgNow I could try reiserfsck to see what I could recover. I started with
reiserfsck --rebuild-sb /dev/loop0to rebuild the superblock: even it if hadn’t been affected by the physical disk corruption, it would certainly be confused by it new home in a looped image that probably wasn’t the same size as the original partition. Next step was
reiserfsck --rebuild-tree /dev/loop0to try to find the contents of the missing directories. I finished it off with
reiserfsck --check /dev/loop0to make sure it was happy.
Now I can just
mount /dev/loop0 /mntand have a look in /mnt/lost+found. The data is there!
Thanks for the great pointers! I used them today with great success. Too bad all of the filenames can’t be restored too, but files inside recovered directories seem to be intact. Huzzah!
I wanted to point out that your “reiserfsck –rebuild-tree /dev/loop/0” command is missing the “-S” switch which is required to make reiserfsck scan the whole disk (deleted blocks included) while rebuilding the tree. Feel free to remove this comment if you’re able to change the text to add the -S.
How to understand the lost+found filenames. They are numbers of the form XXXXX_YYYYY and I found out that the first part (i.e.) XXXXX is for the directory inode.
That is, let’s suppose we find a file called 3003_4004 in /home/lost+found. And we have a user called “foo”, whose files we want to recover. If we do
ls -i /home
it will show […] 3003 foo […]; gladly telling us that the directory /home/foo has the inode 3003 and that file 3003_4004 once belonged to that directory.
Now for an advanced situation: we have a 500542 file in /home/lost+found and there is also a directory called 30035005. Well, from the last paragraph we learned that 30035005 belongs to /home/foo. But WAIT! 500542 belongs to the directory 5005! So u can copy 500542 into 30035005 and then 3003_5005 to /home/foo then, after crawling into the files/dirs, rename them properly.
Absolutely genius bit of advice!
Unbelievably this is the second hard drive I well and truely trashed in less than a month, but following the advice on FC5 I successfully recovered all of the crucial data from the user accounts!
Just wish I found this page the first time around!
By the way, not sure if its a newer version or something, but had to type: reiserfsck –check –rebuild-tree /dev/loop0 on my FC5 system.
Cheers, Adam
just my 2cent. Add also the “sync” option to dd to obtain “conv=noerror,sync”. Without this option when dd finds an error it just skips bad block and go on. In this way all reference to sequent data blocks become wrong. With sync option dd pads bad block with zeros and the image geometry is correct.
Hm, tried out this howto bec. deleted some files for oops.
But on setting new sb block it asks for blocksize. How do i get original blocksize? Already tried with fdisk and mkfs in gentoo….
With 4096 it doesnt rebuild tree because “metadata missing”
Thought i’d lost 250gb of files due to a couple of dodgy sectors. Worked a treat!
Great HowTo. Recovered lot of important data. Thanks a lot. Could you tell me one more thing, please? I’d like to recover some data from hdd formated with XFS. Which commands I should use instead of reiserfsck -–rebuild-tree -S /dev/loop0 and reiserfsck –-check /dev/loop0?? I’ve tried xfscheck and xfsrepair but something goes wrong. Once again thanks for help.
Best regards Winnetou
‘Winnetou’ according to http://oss.sgi.com/projects/xfs/faq.html#undelete there is no undelete option :(
muzyk10 for ReiserFs also doesn’t exist undelete tool but using this HowTo I have recovered some very important data ;) So I suppose that there is some kind of similar “magic trick” for XFS ;))
Dear Marty May I trnaslate, your article, to Polish for those who have lost their data and don’t understand English? Ofc I’ll mention You as an author.
Great! I recovered my customer’s data from a corrupted hdd. The dd copying takes a long time expecially if you have a lot of bad blocks but DON’T use bs option (eg bs=1M) to speed up copying or you’ll risk to loose data. Thanks a lot.