Skip to content

Google Summer of Code 2019 - PostgreSQL participates with 5 projects

For the 13th year, the PostgreSQL Project is participating in Google Summer of Code (GSoC). This project is a great opportunity to let students learn about Open Source projects, and help them deliver new features. It is also a chance to engage the students beyond just one summer, and grow them into active contributors.

In GSoC, students first learn about the Open Source organization, and either pick a summer project from the list provided by the org, or submit their own idea for review. After a “community bonding” period, the students have time to implement their idea, under supervision of mentors from the Open Source organization. There is also an incentive: first, Google pays the students for their work on improving Open Source projects. And second, having a completed GSoC project in a CV is well recognized.

Continue reading "Google Summer of Code 2019 - PostgreSQL participates with 5 projects"

How long will a 64 bit Transaction-ID last in PostgreSQL?

At FOSDEM someone asked how long 64 bit Transaction-IDs will last.

To refresh: PostgreSQL is currently using 32 bits for the TXID, and is good for around 4 billion transactions:

fosdem=# SELECT 2^32;
  ?column?  
------------
 4294967296
(1 row)

That will not last very long if you have a busy database, doing many writes over the day. MVCC keeps the new and old versions of a row in the table, and the TXID will increase with every transaction. At some point the 4 billion transactions are reached, the TXID will overrun, and start again at the beginning. The way transactions are working in PostgreSQL, suddenly all data in your database will become invisible. No one wants that!

To limit this problem, PostgreSQL has a number mechanism in place:

  • PostgreSQL splits transaction ids into half: 2 billion in the past are visible, 2 billion in the future are not visible - all visible rows must live in the 2 billion in the past, at all times.
  • Old, deleted row versions are enevtually removed by VACUUM (or Autovacuum), the XID is no longer used.
  • Old row versions, which are still live, are marked as "freezed" in a table, and assigned a special XID - the previously used XID is no longer needed. The problem here is that every single table in every database must be Vacuumed before the 2 billion threshold is reached.
  • PostgreSQL uses lazy XIDs, where a "real" transaction id is only assigned if the transaction changes something on disk - if a transaction is read only, and does not change anything, no transaction id is consumed.

 

 

Continue reading "How long will a 64 bit Transaction-ID last in PostgreSQL?"

Google Code-In 2018 - Halftime

The PostgreSQL Project participates in Google Code-In (GCI) 2018. This is a program which allows pre-university students to pick up tasks defined by the partnering open source projects, learn about these projects, and also win a prize (certificates, t-shirts, hoodies, but also a trip to Google HQ).

 

Every project creates a number different tasks, some technical, some design based, some about updating documentation, or validating bugs. Whatever is useful in order to get to know the project better. Students can select tasks and submit their work. Mentors from the project then evaluate the work, and either approve it or send it back to the student because more work is needed.

 

Now we are halfway into this year's competition, it's time to run the numbers.

 

Continue reading "Google Code-In 2018 - Halftime"

Using Makefiles to build PostgreSQL

For a long time I was using a Makefile to quickly build, start, stop and then wipe a predefined PostgreSQL version. That comes handy if you just want to test something on an older version, without actually installing the software. Everything happens in a single directory, even a different port is assigned.

When I needed that setup recently, I ran into unrelated build errors:

relpath.c:21:10: fatal error: catalog/pg_tablespace_d.h: No such file or directory
 #include "catalog/pg_tablespace_d.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

Can't be - pg_tablespace_d.h is included in the tarball I'm using.

 

 

Continue reading "Using Makefiles to build PostgreSQL"

How we manage the PostgreSQL FOSDEM devroom

The PostgreSQL Project was assigned a devroom at FOSDEM again this year. We got a room with approx. 200 seats, and as usual every last seat was taken for many of the talks. This requires that we manage the room very effectively and carefully, both to avoid that people walk in and out at all times and disturb the speaker, but also in order to keep the exits free at all times.

 

For a speaker it is quite disturbing if the door in the back opens, someone walks in, down all the way to the front, tries to find a seat, walk around to the other side, maybe leaves again, or just stays somewhere and blocks the exit. The worst case is when there is a free seat somewhere in the middle of a row, and everybody has to get up to let the new attendee in.

 

The FOSDEM team requires that we keep the exits free, in case the room needs to be evacuated. If people are staying on the steps along the wall, it is not possible to clear the room effectively and quickly.

 

 

Continue reading "How we manage the PostgreSQL FOSDEM devroom"

PostgreSQL @ FOSDEM 2018

For more than 10 years, the PostgreSQL Project is present at FOSDEM. We also organize a separate PostgreSQL Day on Friday.

This blog post presents useful information about both the PGDay and the booth and devroom at FOSDEM.

 

Continue reading "PostgreSQL @ FOSDEM 2018"

PostgreSQL @ FOSDEM 2017 - Review

FOSDEM 2017 is over, and hopefully everyone is back home. Time for a short review.

Once again, PostgreSQL ran a pre-conference to FOSDEM, called PGDay. One day just filled with awesome PostgreSQL talks, and meeting community members.

Many speakers uploaded or linked their talks to the PostgreSQL Wiki. If the slides for a talk are missing, please contact the speaker and ask them to upload or link the slides. (For the record, my slides are here)

The FOSDEM video team already uploaded the recordings, and the PostgreSQL Volunteers reviewed every single video. Find them here. Devrim also did short recordings of most of the talks, and posted them on Twitter.

The Hacker Public Radio did interviews with many projects, including PostgreSQL. Find the interview here.

The PostgreSQL Project had a Devroom on Sunday. The schedule is part of the regular PGDay/FOSDEM schedule. And of course we managed to fill the room to the last seat, standing room only. If only people would obey the "Room is full" sign on the door ;-)

We also had a booth (pic 2, pic 3) in the main building, where you can talk with community members about anything around PostgreSQL. Also we had some nice merchandise. Matching the bad weather, the umbrellas went very well. We promise to have more next year.

Last but not least, Slonik visited FOSDEM. Here is just a short selection of pictures.

Hope to see you all next year!

 

P.S.: The next PostgreSQL Europe Conference will be in Warsaw, in October. Details can be found here.

Slides from my talk "Tour de Data Types: VARCHAR2 or CHAR(255)?" at FOSDEM 2017

I had the opportunity to talk about the many datatypes in PostgreSQL, in the PostgreSQL Devroom at FOSDEM 2017. It's quite interesting how many people only use the basic data types and don't know the broad range of types which PostgreSQL is offering.

My slides are available here: *click*

PostgreSQL @ FOSDEM 2017

Like the past 10 or so years, the PostgreSQL Project is present at FOSDEM 2017. We also organize a separate PostgreSQL Day on Friday.

This blog post presents vital information about both the PGDay and the booth and devroom at FOSDEM.

 

Continue reading "PostgreSQL @ FOSDEM 2017"

First PostgreSQL Meetup in Hamburg

InnoGames is hosting a PostgreSQL Meetup Tuesday next week in Hamburg, in their office in Hammerbrook. You have a wonderful view over the skyline of Hamburg from the rooftoop meeting room.

Please sign up here: First Meetup at InnoGames.

Also if you are intersted in speaking at another Meetup, or you want to host a Meetup, please let the organizers know.