Cory Foy

Monday, January 01, 2007

Recovering deleted emails from Thunderbird

So tonight I was switching around some windows, and accidently managed to delete (shift-delete nonetheless) all the emails from my inbox in one of my accounts. This is a lot of email, and while I had some older backups, I hoped that maybe I could restore them (even from a shift-delete).

The first thing that I hoped I had going for me is that generally when you delete an email, it really just flags it, not removes it. When you do a compact folder, it goes through and cleans up those deleted emails. So I immediately closed the application and dropped to a command prompt. This looked promising:

foyc@dilbert ~/.thunderbird/[account] $ ls -l Inbox
-rw-rw-rw- 1 foyc users 843590900 Jan 1 17:56 Inbox

Ok, it's large enough to hopefully still have the emails in it. So I copy it out to a working folder. Now let's see what is inside of it. I do a tail -n 1000 Inbox and see these interesting headers:

X-Mozilla-Status: 0008
X-Mozilla-Status2: 00010000

So off to the web I go. Christian Eyrich has a great page explaining the status codes. The relevant bit is that 0x0008 is "Already Gone". Changing that to 0x0001 "Message has been read" should bring them back. Since it has been a long time since I've compacted, I'm probably going to undelete a bunch of email that I thought was gone, but that's worth it here.

So, let's make the change. Just to check myself, let's see if the messages look to be deleted:

tail -n 50000 Inbox | grep 'X-Mozilla-Status'
X-Mozilla-Status: 0008
X-Mozilla-Status2: 00010000
X-Mozilla-Status: 0008
X-Mozilla-Status2: 00010000
X-Mozilla-Status: 0008
X-Mozilla-Status2: 00000000
...(etc)

Ok! sed should be able to help us out nicely here, so let's try that out:

foyc@dilbert ~ $ sed 's/X-Mozilla-Status: 0008/X-Mozilla-Status: 0001/g' Inbox > FixedInbox

And let's sanity check the change:

foyc@dilbert ~ $ ls -l FixedInbox
-rw-r--r-- 1 foyc users 843590900 Jan 1 18:41 FixedInbox
foyc@dilbert ~ $ ls -l Inbox
-rw-r--r-- 1 foyc users 843590900 Jan 1 18:17 Inbox

tail -n 50000 FixedInbox | grep 'X-Mozilla-Status'
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00010000
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00010000
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
...(etc)

Ok, so let's see if this fixes it. I'm going to move the existing Inbox, copy the FixedInbox to the same location (renaming it Inbox) and fire up Thunderbird.

foyc@dilbert ~/.thunderbird/[account] $
mv Inbox Inbox-deleted
foyc@dilbert ~/.thunderbird/[account] $ mv ~/FixedInbox Inbox

and...viola! All of my emails (including *lots* I thought were gone) are back!

It's times like this I am glad for open standards and the ability to look at things like the data file. If Thunderbird had stored the emails as binary, or some proprietary format, I probably would have either had to go buy special software, or just chalked it up to not being careful and backing up that file more often. But thanks to their willingness to be open, I was able to fix a stupid mistake myself.

posted by Cory Foy at 6:49 PM

4 Comments:

  • Thanks! You helped save my bacon - the only thing I may add is that the X-Mozilla-Status can be things like 0018 - meaning, that the message has been replied to and deleted, so a global replace of may miss some emails that have had other things done to them like replied or forwarded and then deleted.

    By Anonymous Anonymous, at 4:23 PM  

  • This post has been removed by the author.

    By Blogger Laura, at 11:03 AM  

  • Excellent post! I too deleted my entire inbox with a shift-delete. OOPS! Your procedure worked to recover what looks to be a LOT of my deleted mails. Including about 14,000 spam messages! I did this on a windows box using the GnuWin32 version of sed.

    The only difference was that I had to use double quotes in the command.

    C:\Program Files\GnuWin32\bin>sed "s/X-Mozilla-Status: 0008/X-Mozilla-Status: 00
    01/g" Inbox > FixedInbox

    Thanks for the info, it saved my emails!

    By Anonymous Stonewall, at 10:51 AM  

  • Thanks for the post! To add to what "anonymous" posted, I used the following to get the following unique statuses:
    grep -E '^X-Mozilla-Status:' Inbox | sort -u | cut -d' ' -f2 > statuses

    I then used the following ruby to mask 0x0008:
    while line = gets
      oval = line.to_i(16)
      nval = oval & 0b1111111110111
      puts "#{line.strip} (#{oval.to_s(2).rjust(13, '0')}) => #{nval.to_s(16).rjust(4, '0')} (#{nval.to_s(2).rjust(13, '0')})"
    end

    Save it in translate.rb or something. That'll show you the conversion in both binary & hex. (ruby translate.rb statuses)

    Finally, I took that and built a sed script similar to the following:
    sed -e '/^X-Mozilla-Status:/ { s/0008/0000/; s/0009/0001/; s/000a/0002/; s/000b/0003/; s/000d/0005/; s/000f/0007/; s/0018/0010/; s/0019/0011/; s/001a/0012/; s/001b/0013/; s/001f/0017/; s/1009/1001/; s/100b/1003/; s/1019/1011/; s/101b/1013/ }' Inbox > Inbox.fixed

    By Blogger Brian, at 2:01 PM  

Post a Comment

Links to this post:

Create a Link

<< Home