webhook service with TLS and Let's Encrypt certificate

For a number of services, I need a system/service which can receive web hooks, and act when such a trigger is received.

Just a few examples:

• GitHub can send web hooks when something changes in a repo (in any repository you administer, go to Settings -> Webhooks, and add your own hook)
• Tasker for Android can send HTTP(s) requests
• JIRA can send web hooks when certain events occur
• openHAB can send messages to other services

Now it would be useful to have your own receiver for web hooks, and run any task you want. There are a number of tools out there, which can solve this problem. I settled with webhook. In addition, I deploy everything using Ansible, therefore I had to write a bit of code in order to automate this process.

Let’s start with installing webhook:

1
2
3
4
5
6

- name: webhook packages
  apt: name={{ item }} state=present
  with_items:
    - webhook
    - python3-yaml
  register: webhook_install

Let’s Encrypt requires that certificate renewal happens on port 80 (if you are using the HTTP-01 challenge mechanism). Doing the certificate renewal is beyond this blog post, there are several possible methods. Let’s just assume there is another service running which renews the certificate every once in a while, and webhook has access to the certificate and can use it.

By default, webhook will not use encryption. But of course we want that. Encryption can be enabled using the -secure option, and providing the certificate (-cert option) and the key (-key option). Let’s update everything in the systemd service file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# update systemd service file for webhooks
# TLS certificate is Let's Encrypt
- name: /lib/systemd/system/webhook.service
  lineinfile:
    dest: /lib/systemd/system/webhook.service
    regexp: "{{ item.regexp }}"
    line: "{{ item.line }}"
    state: "{{ item.state }}"
  with_items:
    - { regexp: '^ExecStart', line: 'ExecStart=/usr/bin/webhook -nopanic -port 6500 -hotreload -verbose -hooks /etc/webhook.conf -secure -cert /etc/ssl/signed.crt -key /etc/ssl/domain.key', state: present }
  register: webhook_service

It also needs a webhook.conf, with the actual configuration of what you want to listen to:

1
2
3
4
5
6
7
8

- name: Upload webhook.conf
  template:
    src: webhook.conf
    dest: /etc/webhook.conf
    owner: root
    group: root
    mode: 0640
  register: etc_webhook_conf

Make sure (mode: 0640) that not everyone can read this file - most likely you have secrets configured in it. Documentation how to write a webhook config is here, examples are here.

The next part is a bit more tricky: although the above systemd service file specifies the -hotreload option, this only applies to changes in the webhooks.conf. webhook will not recognize when the certificate is renewed, and needs to be restarted for that. I’m using another systemd service for that - and because of the way systemd handles that, you need two files: the service file, and a file for the timer. Wasn’t it easy when you just created a single line in a cron job?

webhook-restart.service:

[Unit]
Description=restart webhook

[Service]
Type=oneshot
ExecStart=/bin/systemctl restart webhook
TimeoutStopSec=900
KillMode=process

webhook-restart.timer:

[Timer]
OnCalendar=*-*-* 3:00:00
Persistent=true

[Install]
WantedBy=timers.target

Upload everything to the server:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

- name: Upload webhook restart service
  template:
    src: containers/files/webhooks/webhook-restart.service
    dest: /etc/systemd/system/webhook-restart.service
    owner: root
    group: root
    mode: 0640
  register: webhook_restart_service

- name: Upload webhook restart timer
  template:
    src: containers/files/webhooks/webhook-restart.timer
    dest: /etc/systemd/system/webhook-restart.timer
    owner: root
    group: root
    mode: 0640
  register: webhook_restart_time

Enable and start the service and the timer:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

# the service is not automatically registered and started
- name: enable webhook service
  service:
    name: webhook
    state: started
    enabled: yes
  when: webhook_install.changed

- name: enable webhook-restart service
  service:
    name: webhook-restart
    state: started
    enabled: yes

- name: enable webhook-restart timer
  service:
    name: webhook-restart.timer
    state: started
    enabled: yes

And if something has changed, systemd must be reloaded, and webhook must be restarted:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

- name: force systemd to reread configs
  systemd:
    daemon_reload: yes
  when: webhook_service.changed or webhook_restart_service.changed or webhook_restart_time.changed

- name: restart webhook
  service:
    name: webhook
    state: restarted
  when: webhook_service.changed

- name: reload webhook service
  shell: killall -USR1 webhook
  when: etc_webhook_conf.changed

That’s it. The webhook service is now up and running on port 6500. webhook itself does not do any host verification. It will just listen to any hostname which points to this server.