Restic upgrade on Debian Buster
A while ago I switched backups from "Duplicity" to "Restic". About time: I was using Duplicity for many years (I think I started using it around 2010, long before "Restic" became available) and it served me well. But recently I ran into more and more issues, especially with archives getting larger and larger. There is an 11 years old open bug in the Duplicity bugtracker, which describes a showstopper for backing up larger archives. And it doesn't look like this will be solved anytime soon. Therefore it was time for something new.
Since I'm rolling out my backups with Ansible, it was relatively easy to create a set of scripts for Restic which use almost the same infrastructure as the old Duplicity backups. That works as expected on all our laptops. But the Raspberry Pi, which does the fileserver backups, seem to had a problem. Backups took way longer than before, jumped from 30-60 minutes (depending on the amount of changes) to constantly around 10 hours.
After some investigation (means: --verbose --verbose --verbose debugging), it turns out that Restic identifies most of the files as new, even though they did not change at all. Some background information: the Raspberry mounts the QNAP fileserver using the SMB3 protocol. The "mount -t cifs" uses the "serverino" option, but apparently that is not enough to provide a stable inode number. And if the inode for a file changes, Restic assumes it is a new file.
On the bright side, because the content of the files do not change, the deduplication still works, and no additional content is added to the backup. The size of the backup does not increase. Still, Restic fetches all the data from the server, and that takes a long time.
Restic 0.9.6 has an option to work around that specific problem: "--ignore-inode". This option will ignore any changes in the file inode, and use other means to identify if the file changed or not.
Unfortunately the Raspbian on the Raspberry is based on Debian Buster (Stable, as of March 2020), and only ships Restic 0.9.4 - and that version does not have the "--ignore-inode" option.
I had to choose between updating everything to Testing (Bullseye), or figure out how to upgrade just one package.
The first step was checking the dependencies for the Restic package. The Restic in Bullseye depends only on "libc6 (>= 2.4)", and Buster already has libc6 in version 2.28. So that will work. The Bullseye version also suggests the packages "libjs-jquery" and "libjs-underscore", but does not add any versions - any installed version should do. Both packages are available in Buster. On the dependency side nothing is blocking a single package upgrade.
Debian is using a mechanism called "pinning" to specify which version can be installed from which source. By default, everything comes from the packages repository, and everything has the same priority. But this can be changed, by adding files to "/etc/apt/preferences.d/". Using my Ansible Playbook (I install the Raspberry using Ansible), I did the following:
- name: Install apt-pinning
copy:
src: '{{ playbook_dir }}/files/apt-pinning.txt'
dest: /etc/apt/preferences.d/50pinning.pref
owner: root
group: root
mode: 0755
register: apt_pinning
The "apt-pinning.txt" file has the following content:
# 500 <= P < 990: causes a version to be installed unless there is a
# version available belonging to the target release or the installed
# version is more recent
Package: *
Pin: release a=stable
Pin-Priority: 900
# 100 <= P < 500: causes a version to be installed unless there is a
# version available belonging to some other distribution or the installed
# version is more recent
Package: *
Pin: release a=testing
Pin-Priority: 400
Package: restic
Pin: release a=testing
Pin-Priority: 900
"stable" packages become a "900" priority, and "testing" packages a "400" priority. Only the "restic" package in "testing" has a "900" priority.
In order to use "testing" packages, Debian also needs to know where to get these packages from:
- name: Install raspi.list
copy:
src: '{{ playbook_dir }}/files/raspi.list'
dest: /etc/apt/sources.list.d/raspi.list
owner: root
group: root
mode: 0755
register: raspi_list
- name: Refresh cache
apt:
update_cache: yes
when: apt_pinning.changed or raspi_list.changed
An updated source list is installed, and when either the pinning file or the source list file changes, Ansible will update the apt cache. The source list file:
deb http://archive.raspberrypi.org/debian/ buster main
# Uncomment line below then 'apt-get update' to enable 'apt-get source'
#deb-src http://archive.raspberrypi.org/debian/ buster main
deb http://raspbian.raspberrypi.org/raspbian/ bullseye main
# Uncomment line below then 'apt-get update' to enable 'apt-get source'
#deb-src http://raspbian.raspberrypi.org/raspbian/ bullseye main
So far, so good. Last step: install the package from testing. Ansible allows to specify which release should be used for a package:
- name: basic packages
apt:
name:
- cifs-utils
- vim
- screen
- jq
- fuse
state: present
- name: basic packages from testing
apt:
name:
- restic
state: latest
default_release: testing
After the Playbook ran, a quick version check shows:
root@raspberrypi:/root# restic version
restic 0.9.6 compiled with go1.13.4 on linux/arm
The runtime for the backup is now 30-45 minutes. That's expected.
Trackbacks
No Trackbacks
Comments
Display comments as Linear | Threaded
BUFU on :
BUFU on :
Andreas 'ads' Scherbaum on :