Backing Up An IMAP Account

One of my IMAP accounts is taking up over 400 mb on the server and is causing problems, such as bounced messages, so I set out to download all the old messages to my computer and then wipe the data from the server. Unfortunately, Mail.app (my primary mail client) doesn’t have any sort of Export or ‘Copy to local machine’ function, so I had to develop a work around.

My first stop was Rui Camo’s excellent imapbackup python script. This script downloads each IMAP mailbox (folder) into a mbox file, which can then be imported into Mail.app or any other decent mail program. One nice feature is that if called from within a directory with existing mbox files, it will check to see if the messages have been already downloaded and will then simply add any new messages to the mbox files. Unfortunately, there seems to be some problems with Python on OS X that can cause the script to fail, the first being an occasional error I get in Python’s IMAPLib related to authentication and the second being a memory leak that Camo’s work-around unfortunately doesn’t seem to fix for my version of Python (2.5.2). In the end I was able to download about a third of my folders.

However, I think my IMAP server may be partially at fault for imapbackup not working completely. Hoping to find something else that worked, I downloaded Pierre Duquesne’s derivative program imapsave (which unlike imapbackup also supports IMAPS). imapsave had more success: even when it said there were errors with directories, it seems to have been able to download almost all the directories.

I then was confronted with a problem: for five directories both scripts had been able to download messages but ended up with different sized mbox files. mbox files are just text files so I could just use the diff and patch programs but, hoping for something that natively understood mbox files, I started poking around on Google and found Mark Sapiro’s mbmerge script. Running his script in a bash loop (specifically, I ran: for i in 'Lilypad' 'Moving to Amsterdam' 'Sent Messages' 'UofC Alumni Club'; do python mbmerge.py "INBOX.$i.mbox" "INBOX.$i" > "INBOX.$i.merged.mbox"; done) did the trick.

Unfortunately neither method was able to download three of my folders. I then tried to copy the three folders in Mail.app from the IMAP server to the local mailboxes I was able to import (I imported them using Files/Import Messages…/Other and they’re stored at ~/Library/Mail/Mailboxes). Unfortunately it didn’t seem to copy all the message: for instance, the local version had 450 messages while the directory on the IMAP server has 660.

Solution? Download Thunderbird! Thunderbird also doesn’t have any export features but it does save its local mailboxes in the mbox format. This means you can drag IMAP folders to ‘Local Folders’. All the messages will be downloaded and placed in the Thunderbird profile folder as a mbox file. For example, my Family mailbox is at ~/Libary/Thunderbird/Profiles/zf7qxltr.default/Mail/Local Folders/Family. Note that the file doesn’t have the ‘.mbox’ extension, though it is an mbox file. There’s also a Family.msf file there, but I have no clue what it’s for – it’s only a few hundred kb while Family is 45 mb. In the end I would have been better off skipping all the Python scripts and just using Thunderboard to create all my mbox files. The reason I’m not just switching to Thunderbird completely is that it doesn’t integrate with the OS X address book and I prefer Mail.app’s UI.

N.B. If you want to sync between IMAP servers and are comfortable with installing a few Perl modules, check out imapsync. It is supposed to even be able to transfer whole mailboxes to Gmail. Also, I understand that Mail.app’s poor handling of IMAP accounts is fixed in OS X 10.5 (I have 10.4.11).

About Peter

The author.
This entry was posted in Life, Mac OS X, Technology and tagged , , , , , , , , , , , , . Bookmark the permalink.

4 Responses to Backing Up An IMAP Account

  1. Peter says:

    I don’t want all my mail at Gmail.

  2. Rui Carmo says:

    You know, leopard’s mail.app does allow you to download and archive mailboxes…

  3. Peter says:

    Another reason to upgrade then…

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>