Cleaning up after disk crashing season hasn’t been fun, but I am pleased with what I’ve managed to recover from the worst crash.
I wanted to get the latest data from the dead webserver. It was in MySQL, and stored in /var/lib/mysql. Unfortunately, the /var/lib directory no longer existed.
I didn’t want to try to recover it in place — with so many bad blocks, things can only get worse — so I copied the entire partition to a file on my laptop (the one with the shiny new disk):
ssh deadserver dd if=/dev/hda1 conv=noerror > hda1.img
(You need the conv=noerror or else dd will stop when it hits the first bad block.)
So, then I had most of a corrupt filesystem image. To make it useful I used the loop driver:
losetup /dev/loop0 hda1.img
Now I could try reiserfsck to see what I could recover. I started with
reiserfsck –rebuild-sb /dev/loop0
to rebuild the superblock: even it if hadn’t been affected by the physical disk corruption, it would certainly be confused by it new home in a looped image that probably wasn’t the same size as the original partition. Next step was
reiserfsck –rebuild-tree /dev/loop0
to try to find the contents of the missing directories. I finished it off with
reiserfsck –check /dev/loop0
to make sure it was happy.
Now I can just
mount /dev/loop0 /mnt
and have a look in /mnt/lost+found. The data is there!
15 comments ↓
Thanks for the great pointers! I used them today with great success. Too bad all of the filenames can’t be restored too, but files inside recovered directories seem to be intact. Huzzah!
I wanted to point out that your “reiserfsck –rebuild-tree /dev/loop/0″ command is missing the “-S” switch which is required to make reiserfsck scan the whole disk (deleted blocks included) while rebuilding the tree. Feel free to remove this comment if you’re able to change the text to add the -S.
How to understand the lost+found filenames. They are numbers of the form XXXXX_YYYYY and I found out that the first part (i.e.) XXXXX is for the directory inode.
That is, let’s suppose we find a file called 3003_4004 in /home/lost+found. And we have a user called “foo”, whose files we want to recover. If we do
ls -i /home
it will show [...] 3003 foo [...]; gladly telling us that the directory /home/foo has the inode 3003 and that file 3003_4004 once belonged to that directory.
Now for an advanced situation: we have a 5005_42 file in /home/lost+found and there is also a directory called 3003_5005. Well, from the last paragraph we learned that 3003_5005 belongs to /home/foo. But WAIT! 5005_42 belongs to the directory 5005! So u can copy 5005_42 into 3003_5005 and then 3003_5005 to /home/foo then, after crawling into the files/dirs, rename them properly.
Absolutely genius bit of advice!
Unbelievably this is the second hard drive I well and truely trashed in less than a month, but following the advice on FC5 I successfully recovered all of the crucial data from the user accounts!
Just wish I found this page the first time around!
By the way, not sure if its a newer version or something, but had to type:
reiserfsck –check –rebuild-tree /dev/loop0
on my FC5 system.
Cheers,
Adam
just my 2cent. Add also the “sync” option to dd to obtain “conv=noerror,sync”. Without this option when dd finds an error it just skips bad block and go on. In this way all reference to sequent data blocks become wrong. With sync option dd pads bad block with zeros and the image geometry is correct.
Hm, tried out this howto bec. deleted some files for oops.
But on setting new sb block it asks for blocksize. How do i get original blocksize? Already tried with fdisk and mkfs in gentoo….
With 4096 it doesnt rebuild tree because “metadata missing”
[...] Reiserfs filesystem recovery Datenwiederherstellung nach “Worst Crash” http://martian.org/marty/2003/09/05/reiserfs-filesystem-recovery [...]
[...] External Links: * ReiserFS Data Recovery (2) [...]
[...] Now you can remount your drive. Any data you may recover using this method will be found in the lost+found directory of that drive. Yes, you’ll have to scour through a bunch of cryptic filenames to find what you’re looking for but it may be worth it in the long run, especially if you’ve lost some very valuble documents. For a more in-depth howto, refer to this page. [...]
[...] I found a very good tutorial on how to undelete stuff on a ReiserFS - and also another one about how to recover from a crash. [...]
[...] The process is lengthy but works nicely. Please follow the instructions here. [...]
Thought i’d lost 250gb of files due to a couple of dodgy sectors. Worked a treat!
Great HowTo. Recovered lot of important data. Thanks a lot. Could you tell me one more thing, please? I’d like to recover some data from hdd formated with XFS. Which commands I should use instead of reiserfsck -–rebuild-tree -S /dev/loop0 and reiserfsck –-check /dev/loop0?? I’ve tried xfs_check and xfs_repair but something goes wrong. Once again thanks for help.
Best regards
Winnetou
‘Winnetou’ according to http://oss.sgi.com/projects/xfs/faq.html#undelete
there is no undelete option
muzyk10 for ReiserFs also doesn’t exist undelete tool but using this HowTo I have recovered some very important data
So I suppose that there is some kind of similar “magic trick” for XFS ;))
Dear Marty
May I trnaslate, your article, to Polish for those who have lost their data and don’t understand English? Ofc I’ll mention You as an author.
Leave a Comment