Backup your server to Memstore using Duplicity

We are frequently asked about using rsync with Memstore, our cloud storage solution. Although we have been working on that, at the time of writing we can't fully support rsync because Memstore is an object storage solution and rsync expects to work on a real file system.

Instead of rsync, I recommend you use Duplicity for backups. It's a very efficient backup tool that uses the rsync algorithm and provides some interesting extras such as encryption, signing packages and Swift API support (which is compatible with the Memstore API).

Let's say we want to backup our sever running Debian. We don't need to create a special user because Duplicity won't work with a container user (it needs to create the container), so we're going to use the administrator account for our backup.

Now we need to install Duplicity using the apt-get package manager:

apt-get install duplicity

Note: Duplicity version 0.5.18 or later is required in order to work with Memstore.

We can install Swift support with:

apt-get install python-swiftclient

Once we have the required packages installed, we can create a small script to backup our files regularly with cron. We can put that file in /root/backup2memstore.sh:

#!/bin/bash

# duplicity will create that container for you
UPLOAD_TO_CONTAINER=backup

# what do we want to back up
BAKPATH=/home
# activity log
LOG=/root/backup.log

export SWIFT_AUTHURL=https://auth.storage.memset.com/v1.0
export SWIFT_USERNAME=mymemstore.admin
export SWIFT_APIKEY=your-memstore-password
export PASSPHRASE=passphrase-for-encryption

duplicity --log-file=$LOG $BAKPATH swift://$UPLOAD_TO_CONTAINER > /dev/null
exit $?s

Set the right permissions for the file with:

chmod +x /root/backup2memstore.sh

Now you can use this script to backup your server daily setting a line in your crontab such as:

MAILTO=mymail@mydomain.dom
0 5 * * * /root/backup2memstore.sh

If there is any problem with the backup, you'll get an email at mymail@mydomain.dom and you can check the logs in the configured log file (/root/backup.log in the example).

Using this post as an example, a quick introduction to Duplicity commands would be:

Verify a backup

duplicity verify swift://backup /home

Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Mon May 28 12:34:15 2012
Verify complete: 1425 files compared, 0 differences found.

Delete backups older than 15 days

duplicity remove-older-than 15D swift://backup 

Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Mon May 28 12:34:15 2012
No old backup sets found, nothing deleted.

Restore a backup

duplicity restore swift://backup /restored-home

Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Mon May 28 12:34:15 2012

I encourage you to read Duplicity's documentation on line for further details.