Knowledgebase

This post explains how to use imapsync on Linux to successfully migrate all our user’s emails from one server to another.

Take note, this requires root access. 

Before we start

This method assumes that 

1. have a number of email accounts with an existing email provider and that you can access these accounts using the IMAP protocol.
2. You have all the email dresses of the email accounts
3. It also assumes that you have already created the very same email addresses in the new server 

Finally, it assumes you know your way around bash and a Linux terminal shell and that you have a Linux installation with a recent version of imapsync installed.

The users text file

The users.txt file contains one user on each line with their source username, source password, destination username and destination password. All these fields are separated by semi-colons. Like this example:

[email protected];user1sourcepassword;[email protected];user1destinationpassword [email protected];user2sourcepassword;[email protected];user2destinationpassword [email protected];user3sourcepassword;[email protected];user3destinationpassword [...] [email protected];user99sourcepassword;[email protected];user99destinationpassword

The script

The migration.sh script is very simple and you can customise it as you wish. At very least you will need to configure the SERVER1 (source) and SERVER2 (destination) server addresses. For testing you should pay attention to the JUSTCONNECT and DRY options that are commented out. For tweaking how it handles the mapping of your current folders to the new folders pay particular attention to the regextrans2 lines. Do some tests on a single or couple of dummy user accounts!

#!/bin/bash

#Configure servers
SERVER1=imap.source.com
SERVER2=imap.destination.com

#Uncomment to hide folder sizes
#FAST="--nofoldersizes"

#Uncomment to do a dry run (no actual changes)
#DRY="--dry"

#Uncomment to just sync folders (no messages)
#JUSTFOLDERS="--justfolders"

#Uncomment to just connect (no syncs at all)
#JUSTCONNECT="--justconnect"

#Set the path to your imapsync binary
imapsync=imapsync

#Users file
if [ -z "$1" ]
then
echo "No users text file given."
exit
fi

if [ ! -f "$1" ]
then
echo "Given users text file \"$1\" does not exist"
exit
fi

{ while IFS=';' read u1 p1 u2 p2; do

$imapsync --usecache --tmpdir /var/tmp \
--host1 ${SERVER1} --user1 "$u1" \
--password1 "$p1" --ssl1 \
--host2 ${SERVER2} \
--port2 993 --user2 "$u2" \
--password2 "$p2" --ssl2 \
${FAST} ${DRY} ${JUSTFOLDERS} ${JUSTCONNECT} \
--regextrans2 's{Sent$}{[Gmail]/Sent Mail}' \
--regextrans2 's{Sent Items$}{[Gmail]/Sent Mail}' \
--regextrans2 's{Sent Messages$}{[Gmail]/Sent Mail}' \
--regextrans2 's{Drafts$}{[Gmail]/Drafts}' \
--exclude 'INBOX.Trash|INBOX.spam|INBOX.Apple Mail To Do'

done ; } < $1

Running the script

In the simplest case, just run the script (called migration.sh) with the users filename as the first argument:

./migration.sh users.txt

If the script fails to run (maybe with error "/bin/bash^M: bad interpreter: No such file or directory)  chmod the file migration.sh to 777 and then run 

$ sed -i -e 's/\r$//' setup.sh


The script can take a long time to run, so I suggest using nohup and redirecting the output to a log file:

nohup ./migration.sh users.txt > migrationlog.txt 2>&1 &

Doing the migration

The script is designed to be run multiple times, and on subsequent runs it will ignore any messages that have already been transferred. This means that subsequent runs will be much faster than the initial run. 

Examining the log files
The log files are a bit verbose so here is a nice awk for helping to examine them:

awk '($1 == "Host1:") || ($1 == "Host2:") || ($1 == "Transfer") || ($1 == "Messages"