Restic upgrade on Debian Buster

A while ago I switched backups from Duplicity to Restic. About time: I was using Duplicity for many years (I think I started using it around 2010, long before Restic became available) and it served me well. But recently I ran into more and more issues, especially with archives getting larger and larger. There is an 11 years old open bug in the Duplicity bugtracker, which describes a showstopper for backing up larger archives. And it doesn’t look like this will be solved anytime soon. Therefore it was time for something new.

Since I’m rolling out my backups with Ansible, it was relatively easy to create a set of scripts for Restic which use almost the same infrastructure as the old Duplicity backups. That works as expected on all our laptops. But the Raspberry Pi, which does the fileserver backups, seem to had a problem. Backups took way longer than before, jumped from 30-60 minutes (depending on the amount of changes) to constantly around 10 hours.

After some investigation (means: --verbose --verbose --verbose debugging), it turns out that Restic identifies most of the files as new, even though they did not change at all. Some background information: the Raspberry mounts the QNAP fileserver using the SMB3 protocol. The mount -t cifs uses the serverino option, but apparently that is not enough to provide a stable inode number. And if the inode for a file changes, Restic assumes it is a new file.

On the bright side, because the content of the files do not change, the deduplication still works, and no additional content is added to the backup. The size of the backup does not increase. Still, Restic fetches all the data from the server, and that takes a long time.

Restic* 0.9.6 has an option to work around that specific problem: –ignore-inode*. This option will ignore any changes in the file inode, and use other means to identify if the file changed or not.

Unfortunately the Raspbian on the Raspberry is based on Debian Buster (Stable, as of March 2020), and only ships Restic 0.9.4 - and that version does not have the --ignore-inode option.

I had to choose between updating everything to Testing (Bullseye), or figure out how to upgrade just one package.

The first step was checking the dependencies for the Restic package. The Restic in Bullseye depends only on libc6 (>= 2.4), and Buster already has libc6 in version 2.28. So that will work. The Bullseye version also suggests the packages libjs-jquery and libjs-underscore, but does not add any versions - any installed version should do. Both packages are available in Buster. On the dependency side nothing is blocking a single package upgrade.

Debian is using a mechanism called pinning to specify which version can be installed from which source. By default, everything comes from the packages repository, and everything has the same priority. But this can be changed, by adding files to /etc/apt/preferences.d/. Using my Ansible Playbook (I install the Raspberry using Ansible), I did the following:

1
2
3
4
5
6
7
8

- name: Install apt-pinning
  copy:
    src: '{{ playbook_dir }}/files/apt-pinning.txt'
    dest: /etc/apt/preferences.d/50pinning.pref
    owner: root
    group: root
    mode: 0755
  register: apt_pinning

The apt-pinning.txt file has the following content:

# 500 <= P < 990: causes a version to be installed unless there is a
# version available belonging to the target release or the installed
# version is more recent

Package: *
Pin: release a=stable
Pin-Priority: 900

# 100 <= P < 500: causes a version to be installed unless there is a
# version available belonging to some other distribution or the installed
# version is more recent

Package: *
Pin: release a=testing
Pin-Priority: 400

Package: restic
Pin: release a=testing
Pin-Priority: 900

stable packages become a 900 priority, and testing packages a 400 priority. Only the restic package in testing has a 900 priority.

In order to use testing packages, Debian also needs to know where to get these packages from:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

- name: Install raspi.list
  copy:
    src: '{{ playbook_dir }}/files/raspi.list'
    dest: /etc/apt/sources.list.d/raspi.list
    owner: root
    group: root
    mode: 0755
  register: raspi_list

- name: Refresh cache
  apt:
    update_cache: yes
  when: apt_pinning.changed or raspi_list.changed

An updated source list is installed, and when either the pinning file or the source list file changes, Ansible will update the apt cache. The source list file:

deb http://archive.raspberrypi.org/debian/ buster main
# Uncomment line below then 'apt-get update' to enable 'apt-get source'
#deb-src http://archive.raspberrypi.org/debian/ buster main

deb http://raspbian.raspberrypi.org/raspbian/ bullseye main
# Uncomment line below then 'apt-get update' to enable 'apt-get source'
#deb-src http://raspbian.raspberrypi.org/raspbian/ bullseye main

So far, so good. Last step: install the package from testing. Ansible allows to specify which release should be used for a package:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

- name: basic packages
  apt:
    name:
      - cifs-utils
      - vim
      - screen
      - jq
      - fuse
    state: present

- name: basic packages from testing
  apt:
    name:
      - restic
    state: latest
    default_release: testing

After the Playbook ran, a quick version check shows:

1
2

root@raspberrypi:/root# restic version
restic 0.9.6 compiled with go1.13.4 on linux/arm

The runtime for the backup is now 30-45 minutes. That’s expected.