Skip to content

2 + 2 * 4 = 16?

Jens Kubieziel recently posted a Twitter poll: 2 + 2 * 4

The available options:

  • 16
  • 15
  • 14
  • 13

Notably absent is the correct result (at least for our ordinary mathematics). This poll is a No-Win Situation: you can't win, because the correct result is not available as an option.

Nevertheless I had an idea, but for this I had to wait until the poll closes.

2 + 2 <strong /> 4 = 16

 

Continue reading "2 + 2 * 4 = 16?"

ctid and other PostgreSQL table internals

While attending a conference (yay) the discussion turned to system columns, and especially ctid. This was in context of backups, and the physical position of data on disk. The question was if that can be used to determine if an incremental backup is required, and if it is enough to just copy that field where the ctid points to. Not quite, but I’d like to use this question to describe the system columns more in detail.

In a PostgreSQL table one may find - depending on the server version - the following additional system columns:

  • tableoid
  • xmin
  • xmax
  • cmin
  • cmax
  • ctid
  • oid

Most of the columns exist in every table, but are excluded when doing a SELECT *. However one can include the column(s) in a query, and then PostgreSQL will return the values for the columns as well.

 

 

Continue reading "ctid and other PostgreSQL table internals"

PostgreSQL Upgrades are hard!

Together with Lætitia Avrot and Nikolay Samokhvalov I was invited to participate in a Community Panel (YouTube video) about PostgreSQL Upgradability at Postgres Vision 2022. The panel was moderated by Bruce Momjian and initiated and organized by Jimmy Angelakos. Bruce did talk with each of us before, which helped a lot to guide the discussion in the right direction. The recording of the panel discussion is available on the Postgres Vision website.

During this panel each of us provided examples for how easy or complicated PostgreSQL upgrades still are.
 

 

Continue reading "PostgreSQL Upgrades are hard!"

PostgreSQL Project @ GSoC 2022

As is a good tradition, the PostgreSQL Project participates in Google Summer of Code (GSoC). Last year we submitted 7 projects for 7 students - and got all 7 projects accepted. This year we got quite a few more good proposals from students, and more mentors are helping. Guess what? Google accepted all 12 proposals!

Google modified the program again. For 2021 they cut the time for every project in half, to accommodate for the at-home work during the Covid-19 pandemic. This turned out to be suboptimal, and many larger projects need more time. This year students can choose between “medium” (175 hours) and “large” (350 hours) projects. This gives everyone a chance to scope the project accordingly.

 

 

Continue reading "PostgreSQL Project @ GSoC 2022"

My PostgreSQL database is empty!

The one thing I like to do is challenge PostgreSQL - and readers - to the extreme. A while ago I posted about the Beer o'clock. Building Zoé was another interesting example. And at some point I blog about Dancing with the Stars as well. Today it's all about nothing. Empty. Where is all the data gone?

A while ago I posted this screenshot on Twitter, and as far as I can tell no one found the correct answer.

The screenshot looks like there's nothing. Empty names, schema name, role name, database name, even no data. And yet this all comes together and somehow works. Let's dig in.

 

Continue reading "My PostgreSQL database is empty!"

Not so equal texts in PostgreSQL - and how to compare texts in a more elaborate way

Christophe Pettus posted an interesting challenge here. Two strings which on the surface look equal, but if you ask PostgreSQL to compare them, they are not equal.

Now let me start with a note: Twitter totally screws this challenge up.

How so? Although the two strings are different in the original, when posting this to Twitter the strings are made equal. Where is the fun in that?

I asked Christophe for the original query:

INSERT INTO t VALUES (E'Zo\u0065\u0301', E'Zo\u00e9');

And you end up with the following texts in the table:

SELECT * FROM t;
  a  |  b  
-----+-----
 Zoé | Zoé
(1 row)

If you translate the UTF-8 strings into hex, you get "0x5a 0x6f 0x65 0xcc 0x81" and "0x5a 0x6f 0xc3 0xa9". Clearly they are different.

However if you convert the two strings from the Tweet, you get "0x5a 0x6f 0xc3 0xa9" and "0x5a 0x6f 0xc3 0xa9". Same string. Poor Twitter.

Checking the hex values was actually one of my first ideas when I spotted this challenge. But nevertheless based on my experience from my "Data Types in PostgreSQL" and "Advanced Data Types in PostgreSQL" talks, I figured it should be possible to "solve" this puzzle even if the strings are in fact equal.

Buckle up! We are about to dive deep into how extendible PostgreSQL really is!

 

Continue reading "Not so equal texts in PostgreSQL - and how to compare texts in a more elaborate way"

GSoC 2021 completed

The Google Summer of Code 2021 for the PostgreSQL Project is wrapped up. The timeline this year was shortened to half, compared to previous years. That’s good, because smaller projects can be worked on, and students have a chance to cope with a changing environment at home and university. On the other hand, the shorter time doesn’t allow diving into more complex projects. Nevertheless, with the help of all mentors, six students successfully concluded their projects.

 

Continue reading "GSoC 2021 completed"

PostgreSQL Project @ GSoC 2021

Wow! The PostgreSQL Project got all 7 proposals accepted into Google Summer of Code 2021!

This year Google changed the participation terms a bit, and cut the time for the students in half. This is supposed to help students who can’t work full-time from home, especially in light of the global pandemic situation. It also means smaller projects, which are easier to handle even for students new to the project.

The PostgreSQL project got a great number of initial applications (29), and we talked with many of the students about refining their proposals. 27 out of the 29 applications were finally submitted by the students. Some are duplicates, some are clearly just copied from somewhere, but many propose good ideas.

After talking with available mentors, and “recruiting” a few more, we settled on 7 final applications, and submitted them to Google.

As usual many of the proposals are not directly developing code for core PostgreSQL, but work on tools and applications from the PostgreSQL ecosystem. Expect some great output over the following months.
 

Make Ansible "postgresql_ping" fail if the database does not exist

Ansible has a very useful module "postgresql_ping" which checks connectivity to the database server. I'm using it in quite a few Playbooks as first step just to ensure that the database server is present - this fails early if there is a problem which otherwise just prevents the rest of the Playbook to work properly.

TASK [Check if database is available]
[WARNING]: PostgreSQL server is unavailable: could not connect to server: No such file or directory         Is the server running locally and accepting         connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
fatal: [127.0.0.1]: FAILED! => {"changed": false, "failed_when_result": true, "is_available": false, "server_version": {}}

 

However this module does not check if the database exists, just if the server is reachable. Example Playbook:

- name: Check if database is available
  postgresql_ping:
    db: "testdb"
  become: yes
  become_user: postgres

When I run the Playbook:

TASK [Check if database is available]
[WARNING]: PostgreSQL server is unavailable: FATAL:  database "testdb" does not exist
ok: [127.0.0.1]

 

As you can see, the database "testdb" does not exist. Which for the module is a reason to raise a warning, but not  a reason to fail.

One possible solution is to let this module do it's work, and add a "postgresql_db" call next, which ensures that the database is created. But not every Playbook is supposed to create and populate a database, and has all the required parameters (owner, encoding, template ect) available. Therefore it would be nice if "postgresql_ping" fails early if the database in question doesn't exist. That's possible, with two more lines of code:

- name: Check if database is available
  postgresql_ping:
    db: "testdb"
  become: yes
  become_user: postgres
  register: ping_database
  failed_when: ping_database.warnings is defined

And the Playbook run:

TASK [Check if database is available]
[WARNING]: PostgreSQL server is unavailable: FATAL:  database "testdb" does not exist
fatal: [127.0.0.1]: FAILED! => {"changed": false, "failed_when_result": true, "is_available": false, "server_version": {}}

Together with "any_errors_fatal: True" this ends the entire Playbook early enough before I have to debug the problem later on.