Skip to content

Find fire stations in OpenStreetMap using Overpass Turbo

My kid got a new project: find fire stations. Google Maps is not really helpful here, obviously. And searching in the OpenStreetMap web interface is also a lot of work. But Overpass Turbo is able to find all the places.

 

Continue reading "Find fire stations in OpenStreetMap using Overpass Turbo"
  • Twitter
  • Bookmark Find fire stations in OpenStreetMap using Overpass Turbo at del.icio.us
  • Facebook
  • Google Bookmarks
  • FriendFeed
  • Digg Find fire stations in OpenStreetMap using Overpass Turbo
  • Bloglines Find fire stations in OpenStreetMap using Overpass Turbo
  • Technorati Find fire stations in OpenStreetMap using Overpass Turbo
  • Fark this: Find fire stations in OpenStreetMap using Overpass Turbo
  • Bookmark Find fire stations in OpenStreetMap using Overpass Turbo at YahooMyWeb
  • Bookmark Find fire stations in OpenStreetMap using Overpass Turbo at Furl.net
  • Bookmark Find fire stations in OpenStreetMap using Overpass Turbo at reddit.com
  • Bookmark Find fire stations in OpenStreetMap using Overpass Turbo at blinklist.com
  • Bookmark Find fire stations in OpenStreetMap using Overpass Turbo at Spurl.net
  • Bookmark Find fire stations in OpenStreetMap using Overpass Turbo at Simpy.com
  • Bookmark Find fire stations in OpenStreetMap using Overpass Turbo at blogmarks
  • Bookmark Find fire stations in OpenStreetMap using Overpass Turbo with wists
  • wong it!
  • Bookmark using any bookmark manager!
  • Stumble It!
  • Identi.ca

GSoC 2021 completed

The Google Summer of Code 2021 for the PostgreSQL Project is wrapped up. The timeline this year was shortened to half, compared to previous years. That’s good, because smaller projects can be worked on, and students have a chance to cope with a changing environment at home and university. On the other hand, the shorter time doesn’t allow diving into more complex projects. Nevertheless, with the help of all mentors, six students successfully concluded their projects.

 

Continue reading "GSoC 2021 completed"
  • Twitter
  • Bookmark GSoC 2021 completed at del.icio.us
  • Facebook
  • Google Bookmarks
  • FriendFeed
  • Digg GSoC 2021 completed
  • Bloglines GSoC 2021 completed
  • Technorati GSoC 2021 completed
  • Fark this: GSoC 2021 completed
  • Bookmark GSoC 2021 completed at YahooMyWeb
  • Bookmark GSoC 2021 completed at Furl.net
  • Bookmark GSoC 2021 completed at reddit.com
  • Bookmark GSoC 2021 completed at blinklist.com
  • Bookmark GSoC 2021 completed at Spurl.net
  • Bookmark GSoC 2021 completed at Simpy.com
  • Bookmark GSoC 2021 completed at blogmarks
  • Bookmark GSoC 2021 completed with wists
  • wong it!
  • Bookmark using any bookmark manager!
  • Stumble It!
  • Identi.ca

Avoid linebreaks in Hugo shortcodes

Shortcodes in Hugo are a neat and poweful system to avoid repating the same piece of text over and over again. Let's say I have the following text:

Nunc in odio id magna molestie congue. Donec erat nulla, pulvinar eget volutpat non, molestie at nisi. Curabitur nec tristique felis. Cras imperdiet, ante et vestibulum iaculis, tellus ipsum pulvinar felis, at viverra est tellus et eros. In nec dignissim lectus, bibendum hendrerit ex. Praesent lobortis eget justo non vehicula.

Nulla et neque cursus libero tristique laoreet nec a ligula. Fusce sit “amet” scelerisque erat. Quisque lorem lectus, lobortis vitae mattis non, tincidunt sed felis. Donec sit “amet” erat nibh.

Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. In et imperdiet dui. In ut enim pharetra, blandit purus vel, malesuada est. Morbi sollicitudin eget leo nec dignissim. Praesent sed leo quis purus pretium aliquet sed quis arcu. Pellentesque facilisis tellus nulla, dignissim laoreet quam maximus et.

And I want to link the "amet" word, then I can create a shortcode:

layouts/shortcodes/amet.html

And in this shortcode I place the text:

<a href="https://www.lipsum.com/feed/html">amet</a>

In the Markdown source this is:

Nulla et neque cursus libero tristique laoreet nec a ligula. Fusce sit "{{< amet >}}" scelerisque erat. Quisque lorem lectus, lobortis vitae mattis non, tincidunt sed felis. Donec sit "{{< amet >}}" erat nibh.

Pretty easy. However Hugo by default does a little bit too much: while generating the static content, it adds a line break. Which in the browser results in:

Nulla et neque cursus libero tristique laoreet nec a ligula. Fusce sit “amet ” scelerisque erat. Quisque lorem lectus, lobortis vitae mattis non, tincidunt sed felis. Donec sit “amet ” erat nibh.
                                                                            ^                                                                                                         ^

There is a nasty little space between the word and the quotation mark. The generated HTML source shows the problem:

  <p>Nunc in odio id magna molestie congue. Donec erat nulla, pulvinar eget volutpat non, molestie at nisi. Curabitur nec tristique felis. Cras imperdiet, ante et vestibulum iaculis, tellus ipsum pulvinar felis, at viverra est tellus et eros. In nec dignissim lectus, bibendum hendrerit ex. Praesent lobortis eget justo non vehicula.</p>
<p>Nulla et neque cursus libero tristique laoreet nec a ligula. Fusce sit &ldquo;<a href="https://www.lipsum.com/feed/html">amet</a>
&rdquo; scelerisque erat. Quisque lorem lectus, lobortis vitae mattis non, tincidunt sed felis. Donec sit &ldquo;<a href="https://www.lipsum.com/feed/html">amet</a>
&rdquo; erat nibh.</p>

Usually a line break in HTML is not a big deal, the browsers deal with that. But here an unnecessary space is added while rendering the content. Hugo makes it complicated to avoid this problem. There is no default setting to turn this off, however the shortcode can have an extra Hugo tag to avoid the training linebreak:

<a href="https://www.lipsum.com/feed/html">amet</a>{{- / Strip trailing newline. / -}}

By adding the {{- -}}, the training line break is not included in the final HTML output, and the quotation mark comes right after the shortcode content.

  • Twitter
  • Bookmark Avoid linebreaks in Hugo shortcodes at del.icio.us
  • Facebook
  • Google Bookmarks
  • FriendFeed
  • Digg Avoid linebreaks in Hugo shortcodes
  • Bloglines Avoid linebreaks in Hugo shortcodes
  • Technorati Avoid linebreaks in Hugo shortcodes
  • Fark this: Avoid linebreaks in Hugo shortcodes
  • Bookmark Avoid linebreaks in Hugo shortcodes at YahooMyWeb
  • Bookmark Avoid linebreaks in Hugo shortcodes at Furl.net
  • Bookmark Avoid linebreaks in Hugo shortcodes at reddit.com
  • Bookmark Avoid linebreaks in Hugo shortcodes at blinklist.com
  • Bookmark Avoid linebreaks in Hugo shortcodes at Spurl.net
  • Bookmark Avoid linebreaks in Hugo shortcodes at Simpy.com
  • Bookmark Avoid linebreaks in Hugo shortcodes at blogmarks
  • Bookmark Avoid linebreaks in Hugo shortcodes with wists
  • wong it!
  • Bookmark using any bookmark manager!
  • Stumble It!
  • Identi.ca

Delete directories recursive on Hetzner Storage Box

Among other external solutions, I store some data on Storage Boxes from Hetzner. The Storage Box allows you to have sub-accounts, so for every server and system storing data there, I use a separate account. For each sub-account, one can select a subdirectory where the data is stored, and the sub-account then can only see this data. The Admin account can see all data, and see all directories.

The usual way I access the Storage Box from other systems is by using sftp protocol from ssh (don't confuse this with the "other" sftp). That's all good, until I remove a sub-account and want to delete the subdirectory with the data. The server doesn't know "rm -r" for recursive deletion, which means I have to traverse into every directory, delete all files, then delete the empty directories. And the encrypted backup I'm using is creating plenty of subdirectories.

Or I find a better tool.

 

Continue reading "Delete directories recursive on Hetzner Storage Box"
  • Twitter
  • Bookmark Delete directories recursive on Hetzner Storage Box at del.icio.us
  • Facebook
  • Google Bookmarks
  • FriendFeed
  • Digg Delete directories recursive on Hetzner Storage Box
  • Bloglines Delete directories recursive on Hetzner Storage Box
  • Technorati Delete directories recursive on Hetzner Storage Box
  • Fark this: Delete directories recursive on Hetzner Storage Box
  • Bookmark Delete directories recursive on Hetzner Storage Box at YahooMyWeb
  • Bookmark Delete directories recursive on Hetzner Storage Box at Furl.net
  • Bookmark Delete directories recursive on Hetzner Storage Box at reddit.com
  • Bookmark Delete directories recursive on Hetzner Storage Box at blinklist.com
  • Bookmark Delete directories recursive on Hetzner Storage Box at Spurl.net
  • Bookmark Delete directories recursive on Hetzner Storage Box at Simpy.com
  • Bookmark Delete directories recursive on Hetzner Storage Box at blogmarks
  • Bookmark Delete directories recursive on Hetzner Storage Box with wists
  • wong it!
  • Bookmark using any bookmark manager!
  • Stumble It!
  • Identi.ca

PostgreSQL Project @ GSoC 2021

Wow! The PostgreSQL Project got all 7 proposals accepted into Google Summer of Code 2021!

This year Google changed the participation terms a bit, and cut the time for the students in half. This is supposed to help students who can’t work full-time from home, especially in light of the global pandemic situation. It also means smaller projects, which are easier to handle even for students new to the project.

The PostgreSQL project got a great number of initial applications (29), and we talked with many of the students about refining their proposals. 27 out of the 29 applications were finally submitted by the students. Some are duplicates, some are clearly just copied from somewhere, but many propose good ideas.

After talking with available mentors, and “recruiting” a few more, we settled on 7 final applications, and submitted them to Google.

As usual many of the proposals are not directly developing code for core PostgreSQL, but work on tools and applications from the PostgreSQL ecosystem. Expect some great output over the following months.
 

  • Twitter
  • Bookmark PostgreSQL Project @ GSoC 2021 at del.icio.us
  • Facebook
  • Google Bookmarks
  • FriendFeed
  • Digg PostgreSQL Project @ GSoC 2021
  • Bloglines PostgreSQL Project @ GSoC 2021
  • Technorati PostgreSQL Project @ GSoC 2021
  • Fark this: PostgreSQL Project @ GSoC 2021
  • Bookmark PostgreSQL Project @ GSoC 2021 at YahooMyWeb
  • Bookmark PostgreSQL Project @ GSoC 2021 at Furl.net
  • Bookmark PostgreSQL Project @ GSoC 2021 at reddit.com
  • Bookmark PostgreSQL Project @ GSoC 2021 at blinklist.com
  • Bookmark PostgreSQL Project @ GSoC 2021 at Spurl.net
  • Bookmark PostgreSQL Project @ GSoC 2021 at Simpy.com
  • Bookmark PostgreSQL Project @ GSoC 2021 at blogmarks
  • Bookmark PostgreSQL Project @ GSoC 2021 with wists
  • wong it!
  • Bookmark using any bookmark manager!
  • Stumble It!
  • Identi.ca

Ansible and string comparisation for IDs

Was running in a stupid problem where Ansible (version 2.9.x) throws an error when a variable is defined. The Play:

- name: Check if id is defined
  fail:
    msg: "No id defined for {{ inventory_hostname }}!"
  when: id is not defined or id|length == 0

And the error:

TASK [Check if id is defined] *****************************************
fatal: [xxx.xxx.xxx.xxx]: FAILED! => {"msg": "The conditional check 'id is not defined or id|length == 0' failed. The error was: Unexpected templating type error occurred on ({% if id is not defined or id|length == 0 %} True {% else %} False {% endif %}): object of type 'int' has no len()\n\nThe error appears to be in '/path/to/playbook/configuration.yml': line 57, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n    - name: Check if id is defined\n      ^ here\n"}

Ok, it tries to compare an integer, fine. There are two ways to fix this problem:

1: Change the type to a string by updating the inventory:

From:

id=1

to:

id="1"

But this might have other consequences, as the other parts of the Playbook might depend on this being an integer.

2: Compare as string

when: id is not defined or id|string|length == 0

This casts the integer to a string, and then the length() function works.

  • Twitter
  • Bookmark Ansible and string comparisation for IDs at del.icio.us
  • Facebook
  • Google Bookmarks
  • FriendFeed
  • Digg Ansible and string comparisation for IDs
  • Bloglines Ansible and string comparisation for IDs
  • Technorati Ansible and string comparisation for IDs
  • Fark this: Ansible and string comparisation for IDs
  • Bookmark Ansible and string comparisation for IDs at YahooMyWeb
  • Bookmark Ansible and string comparisation for IDs at Furl.net
  • Bookmark Ansible and string comparisation for IDs at reddit.com
  • Bookmark Ansible and string comparisation for IDs at blinklist.com
  • Bookmark Ansible and string comparisation for IDs at Spurl.net
  • Bookmark Ansible and string comparisation for IDs at Simpy.com
  • Bookmark Ansible and string comparisation for IDs at blogmarks
  • Bookmark Ansible and string comparisation for IDs with wists
  • wong it!
  • Bookmark using any bookmark manager!
  • Stumble It!
  • Identi.ca

Ansible, "copy" module and "become"

For testing I did spin up a couple of new virtual (Ubuntu 20.04 LTS) instances, installed PostgreSQL, and wanted to copy over a .sql file to install in the database:

- name: copy files to PostgreSQL data directory
  copy:
    src: "files/{{ item }}"
    dest: "{{ item }}"
    mode: 0700
  become: yes
  become_user: postgres
  loop:
    - file1.sql
    - file2.sql

And ran into a meaningless error message:

TASK [copy files to PostgreSQL data directory] ***********
fatal: [xxx.xxx.xxx.xxx]: FAILED! => {"msg": "Failed to set permissions on the temporary files Ansible needs to create when becoming an unprivileged user (rc: 1, err: chown: changing ownership of '/var/tmp/ansible-tmp-1618521951.848439-176484068031965/': Operation not permitted\nchown: changing ownership of '/var/tmp/ansible-tmp-1618521951.848439-176484068031965/source': Operation not permitted\n}). For information on working around this, see https://docs.ansible.com/ansible/become.html#becoming-an-unprivileged-user"}

 

 

Continue reading "Ansible, "copy" module and "become""
  • Twitter
  • Bookmark Ansible, &quot;copy&quot; module and &quot;become&quot; at del.icio.us
  • Facebook
  • Google Bookmarks
  • FriendFeed
  • Digg Ansible, &quot;copy&quot; module and &quot;become&quot;
  • Bloglines Ansible, &quot;copy&quot; module and &quot;become&quot;
  • Technorati Ansible, &quot;copy&quot; module and &quot;become&quot;
  • Fark this: Ansible, &quot;copy&quot; module and &quot;become&quot;
  • Bookmark Ansible, &quot;copy&quot; module and &quot;become&quot; at YahooMyWeb
  • Bookmark Ansible, &quot;copy&quot; module and &quot;become&quot; at Furl.net
  • Bookmark Ansible, &quot;copy&quot; module and &quot;become&quot; at reddit.com
  • Bookmark Ansible, &quot;copy&quot; module and &quot;become&quot; at blinklist.com
  • Bookmark Ansible, &quot;copy&quot; module and &quot;become&quot; at Spurl.net
  • Bookmark Ansible, &quot;copy&quot; module and &quot;become&quot; at Simpy.com
  • Bookmark Ansible, &quot;copy&quot; module and &quot;become&quot; at blogmarks
  • Bookmark Ansible, &quot;copy&quot; module and &quot;become&quot; with wists
  • wong it!
  • Bookmark using any bookmark manager!
  • Stumble It!
  • Identi.ca

Make Ansible "postgresql_ping" fail if the database does not exist

Ansible has a very useful module "postgresql_ping" which checks connectivity to the database server. I'm using it in quite a few Playbooks as first step just to ensure that the database server is present - this fails early if there is a problem which otherwise just prevents the rest of the Playbook to work properly.

TASK [Check if database is available]
[WARNING]: PostgreSQL server is unavailable: could not connect to server: No such file or directory         Is the server running locally and accepting         connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
fatal: [127.0.0.1]: FAILED! => {"changed": false, "failed_when_result": true, "is_available": false, "server_version": {}}

 

However this module does not check if the database exists, just if the server is reachable. Example Playbook:

- name: Check if database is available
  postgresql_ping:
    db: "testdb"
  become: yes
  become_user: postgres

When I run the Playbook:

TASK [Check if database is available]
[WARNING]: PostgreSQL server is unavailable: FATAL:  database "testdb" does not exist
ok: [127.0.0.1]

 

As you can see, the database "testdb" does not exist. Which for the module is a reason to raise a warning, but not  a reason to fail.

One possible solution is to let this module do it's work, and add a "postgresql_db" call next, which ensures that the database is created. But not every Playbook is supposed to create and populate a database, and has all the required parameters (owner, encoding, template ect) available. Therefore it would be nice if "postgresql_ping" fails early if the database in question doesn't exist. That's possible, with two more lines of code:

- name: Check if database is available
  postgresql_ping:
    db: "testdb"
  become: yes
  become_user: postgres
  register: ping_database
  failed_when: ping_database.warnings is defined

And the Playbook run:

TASK [Check if database is available]
[WARNING]: PostgreSQL server is unavailable: FATAL:  database "testdb" does not exist
fatal: [127.0.0.1]: FAILED! => {"changed": false, "failed_when_result": true, "is_available": false, "server_version": {}}

Together with "any_errors_fatal: True" this ends the entire Playbook early enough before I have to debug the problem later on.

  • Twitter
  • Bookmark Make Ansible &quot;postgresql_ping&quot; fail if the database does not exist at del.icio.us
  • Facebook
  • Google Bookmarks
  • FriendFeed
  • Digg Make Ansible &quot;postgresql_ping&quot; fail if the database does not exist
  • Bloglines Make Ansible &quot;postgresql_ping&quot; fail if the database does not exist
  • Technorati Make Ansible &quot;postgresql_ping&quot; fail if the database does not exist
  • Fark this: Make Ansible &quot;postgresql_ping&quot; fail if the database does not exist
  • Bookmark Make Ansible &quot;postgresql_ping&quot; fail if the database does not exist at YahooMyWeb
  • Bookmark Make Ansible &quot;postgresql_ping&quot; fail if the database does not exist at Furl.net
  • Bookmark Make Ansible &quot;postgresql_ping&quot; fail if the database does not exist at reddit.com
  • Bookmark Make Ansible &quot;postgresql_ping&quot; fail if the database does not exist at blinklist.com
  • Bookmark Make Ansible &quot;postgresql_ping&quot; fail if the database does not exist at Spurl.net
  • Bookmark Make Ansible &quot;postgresql_ping&quot; fail if the database does not exist at Simpy.com
  • Bookmark Make Ansible &quot;postgresql_ping&quot; fail if the database does not exist at blogmarks
  • Bookmark Make Ansible &quot;postgresql_ping&quot; fail if the database does not exist with wists
  • wong it!
  • Bookmark using any bookmark manager!
  • Stumble It!
  • Identi.ca

Blog website crawlers and bots in Apache2

Found a couple more bots crawling my website, and from the look at online resources it seems I catched a few of the bad guys. Crawlers which ignore the robots.txt standard, and just crawl a website for content.

Decided to do something against it, and added a filter in Apache2.

The way I have my webserver setup is that I have templates for every website (they all have different configs), and deploy them using Ansible. Parts of the website configuration which are the same, or at least similar, are handled by includes.

 

Continue reading "Blog website crawlers and bots in Apache2"
  • Twitter
  • Bookmark Blog website crawlers and bots in Apache2 at del.icio.us
  • Facebook
  • Google Bookmarks
  • FriendFeed
  • Digg Blog website crawlers and bots in Apache2
  • Bloglines Blog website crawlers and bots in Apache2
  • Technorati Blog website crawlers and bots in Apache2
  • Fark this: Blog website crawlers and bots in Apache2
  • Bookmark Blog website crawlers and bots in Apache2 at YahooMyWeb
  • Bookmark Blog website crawlers and bots in Apache2 at Furl.net
  • Bookmark Blog website crawlers and bots in Apache2 at reddit.com
  • Bookmark Blog website crawlers and bots in Apache2 at blinklist.com
  • Bookmark Blog website crawlers and bots in Apache2 at Spurl.net
  • Bookmark Blog website crawlers and bots in Apache2 at Simpy.com
  • Bookmark Blog website crawlers and bots in Apache2 at blogmarks
  • Bookmark Blog website crawlers and bots in Apache2 with wists
  • wong it!
  • Bookmark using any bookmark manager!
  • Stumble It!
  • Identi.ca

Create child pages in Redmine

Once or twice a year I have to setup a new Redmine project, which we use to organize conferences for PostgreSQL Europe.

Every single time I'm trying to figure out how the child pages for the meeting notes work.

Steps required:

  • Create a page "MeetingMinutes", doesn't need to have any content, but it must exist
  • Create the first child page with meeting minutes, as parent page select "MeetingMinutes"
  • On the main wiki page insert a link with: "{{child_pages(MeetingMinutes)}}"

Every time I forget step 1), because this page is only required as parent page, and serves no further purpose. And then I'm wondering why I can't create child pages.

  • Twitter
  • Bookmark Create child pages in Redmine at del.icio.us
  • Facebook
  • Google Bookmarks
  • FriendFeed
  • Digg Create child pages in Redmine
  • Bloglines Create child pages in Redmine
  • Technorati Create child pages in Redmine
  • Fark this: Create child pages in Redmine
  • Bookmark Create child pages in Redmine at YahooMyWeb
  • Bookmark Create child pages in Redmine at Furl.net
  • Bookmark Create child pages in Redmine at reddit.com
  • Bookmark Create child pages in Redmine at blinklist.com
  • Bookmark Create child pages in Redmine at Spurl.net
  • Bookmark Create child pages in Redmine at Simpy.com
  • Bookmark Create child pages in Redmine at blogmarks
  • Bookmark Create child pages in Redmine with wists
  • wong it!
  • Bookmark using any bookmark manager!
  • Stumble It!
  • Identi.ca