Skip to content

Read a JSON text into a variable in Ansible without parsing it

For one project I need to insert the content of a local file into another file on the remote system, and the first file happens to be JSON. The JSON file is in compact format (jq --compact-output) and is supposed to stay this way. When Ansible reads the content of the file, it determines that the content is JSON, and parses the content into the variable - and along the way is uncompressing the format. Not what I want.

 

Continue reading "Read a JSON text into a variable in Ansible without parsing it"

Does an openHAB Item exist?

In a more complex openHAB2 Rule I'm writing, I need to find out if a certain Item exist. If I try to access it when it does not exist, the Rule will fail. Not good.

I could pass in a parameter to the framework which builds the Rule, but that is cumbersome and requires changes on several places.

Let's see if I can have the Rule figure this out.

 

Continue reading "Does an openHAB Item exist?"

Ansible and string comparisation for IDs

Was running in a stupid problem where Ansible (version 2.9.x) throws an error when a variable is defined. The Play:

- name: Check if id is defined
  fail:
    msg: "No id defined for {{ inventory_hostname }}!"
  when: id is not defined or id|length == 0

And the error:

TASK [Check if id is defined] *****************************************
fatal: [xxx.xxx.xxx.xxx]: FAILED! => {"msg": "The conditional check 'id is not defined or id|length == 0' failed. The error was: Unexpected templating type error occurred on ({% if id is not defined or id|length == 0 %} True {% else %} False {% endif %}): object of type 'int' has no len()\n\nThe error appears to be in '/path/to/playbook/configuration.yml': line 57, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n    - name: Check if id is defined\n      ^ here\n"}

Ok, it tries to compare an integer, fine. There are two ways to fix this problem:

1: Change the type to a string by updating the inventory:

From:

id=1

to:

id="1"

But this might have other consequences, as the other parts of the Playbook might depend on this being an integer.

2: Compare as string

when: id is not defined or id|string|length == 0

This casts the integer to a string, and then the length() function works.

Ansible, "copy" module and "become"

For testing I did spin up a couple of new virtual (Ubuntu 20.04 LTS) instances, installed PostgreSQL, and wanted to copy over a .sql file to install in the database:

- name: copy files to PostgreSQL data directory
  copy:
    src: "files/{{ item }}"
    dest: "{{ item }}"
    mode: 0700
  become: yes
  become_user: postgres
  loop:
    - file1.sql
    - file2.sql

And ran into a meaningless error message:

TASK [copy files to PostgreSQL data directory] ***********
fatal: [xxx.xxx.xxx.xxx]: FAILED! => {"msg": "Failed to set permissions on the temporary files Ansible needs to create when becoming an unprivileged user (rc: 1, err: chown: changing ownership of '/var/tmp/ansible-tmp-1618521951.848439-176484068031965/': Operation not permitted\nchown: changing ownership of '/var/tmp/ansible-tmp-1618521951.848439-176484068031965/source': Operation not permitted\n}). For information on working around this, see https://docs.ansible.com/ansible/become.html#becoming-an-unprivileged-user"}

 

 

Continue reading "Ansible, "copy" module and "become""

Make Ansible "postgresql_ping" fail if the database does not exist

Ansible has a very useful module "postgresql_ping" which checks connectivity to the database server. I'm using it in quite a few Playbooks as first step just to ensure that the database server is present - this fails early if there is a problem which otherwise just prevents the rest of the Playbook to work properly.

TASK [Check if database is available]
[WARNING]: PostgreSQL server is unavailable: could not connect to server: No such file or directory         Is the server running locally and accepting         connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
fatal: [127.0.0.1]: FAILED! => {"changed": false, "failed_when_result": true, "is_available": false, "server_version": {}}

 

However this module does not check if the database exists, just if the server is reachable. Example Playbook:

- name: Check if database is available
  postgresql_ping:
    db: "testdb"
  become: yes
  become_user: postgres

When I run the Playbook:

TASK [Check if database is available]
[WARNING]: PostgreSQL server is unavailable: FATAL:  database "testdb" does not exist
ok: [127.0.0.1]

 

As you can see, the database "testdb" does not exist. Which for the module is a reason to raise a warning, but not  a reason to fail.

One possible solution is to let this module do it's work, and add a "postgresql_db" call next, which ensures that the database is created. But not every Playbook is supposed to create and populate a database, and has all the required parameters (owner, encoding, template ect) available. Therefore it would be nice if "postgresql_ping" fails early if the database in question doesn't exist. That's possible, with two more lines of code:

- name: Check if database is available
  postgresql_ping:
    db: "testdb"
  become: yes
  become_user: postgres
  register: ping_database
  failed_when: ping_database.warnings is defined

And the Playbook run:

TASK [Check if database is available]
[WARNING]: PostgreSQL server is unavailable: FATAL:  database "testdb" does not exist
fatal: [127.0.0.1]: FAILED! => {"changed": false, "failed_when_result": true, "is_available": false, "server_version": {}}

Together with "any_errors_fatal: True" this ends the entire Playbook early enough before I have to debug the problem later on.

openHAB: stop a ChromeCast

There are a number of ChromeCasts in our setup. I'm adding (physical) switches to turn displays on and off, and primarily control the volume. When I turn the display off, I also want the ChromeCast to stop doing whatever it is doing right now (stop streaming), and go back to the default application (for video CC).

The openHAB ChromeCast binding has a separate channel for that:

Switch    chromecast_chromecast_dfb5af5187ce1135b276239130aef282_stop     { channel="chromecast:chromecast:dfb5af5187ce1135b276239130aef282:stop" }

However since the channel is named "stop", it has to be "enabled" with "ON". Doesn't make sense? It is as it is:

chromecast_chromecast_dfb5af5187ce1135b276239130aef282_stop.sendCommand(ON)

And the result:

2021-01-18 17:13:20.305 [ome.event.ItemCommandEvent] - Item 'chromecast_chromecast_dfb5af5187ce1135b276239130aef282_stop' received command ON
2021-01-18 17:13:20.321 [nt.ItemStatePredictedEvent] - chromecast_chromecast_dfb5af5187ce1135b276239130aef282_stop predicted to become ON
2021-01-18 17:13:20.443 [vent.ItemStateChangedEvent] - chromecast_chromecast_dfb5af5187ce1135b276239130aef282_appName changed from <...> to UNDEF
2021-01-18 17:13:20.451 [vent.ItemStateChangedEvent] - chromecast_chromecast_dfb5af5187ce1135b276239130aef282_appId changed from <...> to UNDEF
2021-01-18 17:13:20.455 [vent.ItemStateChangedEvent] - chromecast_chromecast_dfb5af5187ce1135b276239130aef282_statustext changed from Casting: <...> to UNDEF
2021-01-18 17:13:20.457 [vent.ItemStateChangedEvent] - chromecast_chromecast_dfb5af5187ce1135b276239130aef282_idling changed from OFF to ON
2021-01-18 17:13:23.888 [vent.ItemStateChangedEvent] - chromecast_chromecast_dfb5af5187ce1135b276239130aef282_appName changed from UNDEF to Backdrop
2021-01-18 17:13:23.893 [vent.ItemStateChangedEvent] - chromecast_chromecast_dfb5af5187ce1135b276239130aef282_appId changed from UNDEF to E8C28D3C

 

Online indicator for remote controllable power plugs in openHAB

Recently I installed a number of new power sockets (like this one). The Hue bridge can not only add each switch to a light group, but also reports each plug as a Thing in openHAB. There I thought it will be a nice touch if openHAB actually reports when it sees a device plugged in. The power socket has a LED which turns on when the plug is on. The Hue bridge reports OFFLINE and ONLINE. That's useful.

The plan is to turn the LED light - and the power plug - on and off for a few second.

 

Continue reading "Online indicator for remote controllable power plugs in openHAB"

Restic backup

Was asked quite a few times how I do my backups with Restic.

For more than 10 years I was using "Duplicity" for backups, but in 2019 I changed to Restic. The main reason for the change was that Duplicity still can't handle "Big Data", as in: larger directories. In 2009 someone opened an issue on the Duplicity bugtracker, and this problem still exists as of today. For about two years I was shifting around the problem, excluding files, trying to make the sigfile smaller. But at some point I decided that it is enough and I need to change the tool.

Duplicity knows two backup modes: "full backup" and "incremental backup". Once in a while you take a full backup, and then you add incremental backups to that full backup. In order to restore a certain backup you need the full backup and the incremental backups. Therefore my go-to mode was to always have two full backups and a couple incremental backups in-between. Even if something goes wrong with the latest full backup, I can still go back to the previous full backup (of course with some changes lost, but that's still better than nothing). When taking a new full backup, the oldest one is only deleted when the new one is completed. Accordingly when a new incremental backup is created, it's a new set of files. Removing the backup removes all the files from this incremental backup. That worked well, but needed scheduling. Over time I wrote a wrapper script around Duplicity, which did schedule new full and incremental backups.

Restic works in a different way. There is no concept of "full backup" and "incremental backup". Basically every backup is a full backup, and Restic figures out which files changed, got deleted, or added. Also it does deduplication: if files are moved around, or appear multiple times, they are not added multiple times into the backup. Deduplication is something which Duplicity can't do. But because Restic can do deduplication, there is no common set of files which belong to a single snapshot. Data blobs from one backup can stay in the repository forever, removing snapshots might not remove any files at all.

Restic on the other hand needs "prune" to remove old data. A snapshot can be removed according to the policy specified, but this does not remove the data from the backup directory. A "prune" run will go over the data and remove any block which is no longer needed.

My first question - after figuring out which other backup tool to use: shall I replicate the wrapper script, or try something else? Given that the backup doesn't need complex scheduling, I decided against writing a complex wrapper. And since I am now deploying all devices with Ansible, I decided to integrate this into my Playbooks, and deploy a set of shell scripts. The goal was to have a small number of dedicated scripts doing the daily backup work, and another set of "helper" scripts which I can use to inspect the backup, modify it, or restore something.

My main goals for this: "small number of programs/scripts" (Unix style: each tool does one job), "rapid development" (don't spend weeks writing another scheduler), "rapid deployment" (re-run Playbooks and let Ansible deploy this to all devices).

 

Continue reading "Restic backup"

openHAB: Configuration model '....rules' has errors, therefore ignoring it: no viable alternative at input '...'

The openHAB rule system is not very helpful by pointing out when it's missing something, or when there is an error to actually point to the problem.

Was debugging an error message for a while and couldn't figure out what is wrong:

Configuration model '....rules' has errors, therefore ignoring it: [152,5]: no viable alternative at input '...'

The line number specified was somewhere else in the file, and had nothing to do with the rule in question. After removing almost everything from the rule, except a logInfo() message, the error still happened - and then it occured to me: I forgot to specify "Item".

My faulty code was:

rule "Rule Name"
when
    Sensor_name received update
then

where it should have been:

rule "Rule Name"
when
    Item Sensor_name received update
then

Error message totally not helpful ...