Published on 2020-05-31 by Kenneth FlakBack to Tech Research
Ever since I abandoned iLife I have been missing the now discontinued TimeCapsule: a solid, local, incremental backup solution that doesn't require me to manually attach a USB hard drive to my computer and run a script. I use the excellent rsync.net for cloud backup, but I consider this more of a last resort: the thought of downloading 300 GB of data from a remote server to restore my system does not inspire enthusiasm.
My first thought was to connect a backup hard drive to the internet router's USB port. Many friendly yet fruitless phone calls later I was told by my internet provider that I would need a real IT person to set this up. And that they used to have one at some point in time.
A plausible solution arose when I got my hands on a Raspberry Pi 4 and discovered the excellent rsnapshot script.
rsync to create periodic snapshots of a system, using hard links wherever possible.
Perfect for my needs, in other words.
Setting it up for automated backup on the Raspberry Pi does require a bit of tinkering with configuration files and
systemd services, but luckily all the necessary information is readily available on the Arch Linux wiki.
The final result of my efforts is that
rsnapshot runs as a
systemd service on various intervals, copying over new and modified files from my computer to an external hard drive attached to the Rpi.
On the computer run:
sudo pacman -S rsnapshot
This will pull in
rsnapshot and a couple of dependencies, most importantly
rsync on the Rpi:
sudo pacman -S rsync
In order to be able to push the backup from the computer to the Rpi we need to set up an NFS share. This is a very useful protocol for sharing drives and directories on the local network.
nfs-utils on both computer and Rpi:
sudo pacman -S nfs-utils
On the Rpi create the share directory. This is where we will mount the backup harddrive:
sudo mkdir /srv/nfs/backup
Change permissions recursively to enable everybody on the local network full access. Only recommended if you fully trust the network!
sudo chmod -R 777 /srv/nfs/backup
/etc/exports on the Rpi:
Add this to the end of the file:
Again: this will allow anyone on the network to access the hard drive! You could also specify explicitly the computer you want to grant access to:
Start the server and enable it on reboot:
sudo systemctl enable --now nfs-server.service
Go back to your computer, create the relevant directory and mount the shared drive:
sudo mkdir /run/media/backup sudo mount <ip_of_rpi>:/srv/nfs/backup /run/media/backup
To make the mount persistent across reboots, add this line to the computer's
<ip_of_rpi>:/srv/nfs/backup /run/media/backup auto,nofail 0 0
Now everything is set up and ready. The last thing we need to do is to mount the actual hard drive. On the Rpi run:
The external hard drive will be called something like
Make a note of the hard drive's UUID.
This is a unique identifier that you will need when making the mount persistent.
Mount the disk (replace
sda1 with your disk identifier):
sudo mount /dev/sda1 /srv/nfs/backup
To make this persistent across reboots put this line in the
/etc/fstab file on your Rpi:
UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /run/media/backup ext4 noauto,nofail,noexec,nouser,rw,async 0 2
Replace all the x-es with the UUID you discovered earlier.
This is where things get interesting. On the computer back up the default
rsnapshot configuration file:
sudo cp /etc/rsnapshot.conf /etc/rsnapshot.conf.default
Next, open up the file for editing:
The configuration file is packed with useful information. Here I have extracted the lines that need to be uncommented in order for the setup to work:
snapshot_root /run/media/backup/ # This is a removable disk, so we don't want to accidentally create a new snapshot directory no_create_root 1 # Arch Linux-specific binary paths cmd_cp /usr/bin/cp cmd_rm /usr/bin/rm cmd_rsync /usr/bin/rsync cmd_ssh /usr/bin/ssh cmd_logger /usr/bin/logger cmd_du /usr/bin/du cmd_rsnapshot_diff /usr/bin/rsnapshot-diff # Amount of backups to keep retain hourly 24 retain daily 7 retain weekly 4 # how much chatter do we want verbose 2 loglevel 3 # location of lockfile lockfile /var/run/rsnapshot.pid # Directories to ignore: exclude "/home/<user>/.cache" exclude "/home/<user>/.stack" exclude "/home/<user>/.tmp" exclude "/home/<user>/.thumbnails" exclude "/home/<user>/.cargo" exclude "/home/<user>/build" exclude "/home/<user>/node_modules" exclude "/home/<user>/Downloads" # paths to back up -> path in the snapshot directory # note that the next lines are separated with TABS, not SPACES backup /home/<user>/ localhost/ backup /etc/ localhost/ backup /usr/local/ localhost/
Test the configuration by running:
rsnapshot -t hourly
The next thing we need to do is to set up the
systemd service that will run the backup.
The content of this file is:
[Unit] Description=rsnapshot [%I] backup Requires=run-media-backup.mount After=run-media-backup.mount [Service] Type=oneshot Nice=19 IOSchedulingClass=idle ExecStart=/usr/bin/rsnapshot %I
One thing to note is the section:
This guarantees that the service will only run if the removable hard drive is attached and mounted.
We want to run the backup on hourly, daily and weekly intervals. To achieve this we need to create these files:
sudoedit /etc/systemd/system/rsnapshot-hourly.timer sudoedit /etc/systemd/system/rsnapshot-daily.timer sudoedit /etc/systemd/system/rsnapshot-weekly.timer
The files look like this:
[Unit] Description=rsnapshot hourly backup [Timer] # run hourly OnCalendar=*-*-* *:00:00 Persistent=true Unitfirstname.lastname@example.org [Install] WantedBy=timers.target
[Unit] Description=rsnapshot daily backup [Timer] # run daily OnCalendar=09:30 Persistent=true Unitemail@example.com [Install] WantedBy=timers.target
[Unit] Description=rsnapshot weekly backup [Timer] # run hourly OnCalendar=Monday *-*-* 10:30:00 Persistent=true Unitfirstname.lastname@example.org [Install] WantedBy=timers.target
The very last thing we want to do before enabling the timers is to make sure the service works as advertised:
sudo systemctl start email@example.com
This will probably take a long time on the first run, depending on how much data you have and the speed of the network and hard drive. After you are satisfied everything works, enable the timers:
sudo systemctl enable --now rsnapshot-hourly.timer sudo systemctl enable --now rsnapshot-daily.timer sudo systemctl enable --now rsnapshot-weekly.timer
If all the stars are aligned and every link in the chain works you should now have a working backup solution that requires minimal input in your daily life. All your files will now be available as a regular file system on the external hard drive.
The first time I ran this setup I started the backup in the evening and woke up to find a valiantly working laptop and Rpi, all lights ablink and fans ablowing. The combined effort of 10 hours of work was a glorious 14 GB of data out of the ~300 GB I needed to back up. I can live with a relatively slow performance when the backup happens regularly and in small batches, but for a first run this becomes unbearable. So I unceremoniously unmounted the network drive from laptop and Rpi, stopped the
rsnapshot-*.timers, killed the NFS server on the Rpi and plugged the USB drive into the laptop instead. The beauty of the
nfs setup is that
rsnapshot doesn't care if the drive is mounted via NFS or physically. So for the initial backup run I did this on my computer:
sudo mount /dev/sdb1 /run/media/backup sudo rsnapshot -D hourly
-D flag ensures a running commentary on which files get copied over. And it is so, so, so much faster than the network version.
If you have any feedback or comments, please get in touch!