The PostgreSQL Project participates in Google Summer of Code (GSoC) 2020, with 3 projects. After the “Community Bonding” period finished last week, we are now in the active development phase - “Coding” as Google calls it. All three projects make good progress!
The project defined a number of milestones, and evaluated the current database structure. Modifications are required on this front, and will be applied over the following days. Also the structure for sending data from the client to the project server is re-evaluated and modified. The student started with the database design modifications, and also with documenting the changes and terminology used.
PL/Java build system
Thanks to the ongoing work setting up the continuous integration, PL/Java's master branch - which will become the 1.6 release - is now getting regular CI builds against several PostgreSQL and Java versions on amd64 Ubuntu and Mac OS X, and the student has moved on to setting up the same for Windows. We had a goal to enable test options in the CI builds that were otherwise impractically strict, and identify ways to filter the output down to a manageable volume which is exposing real issues. Through a combination of fixes to some real PL/Java warnings, and a small state machine now keeping known non-PL/Java ones out of the log, the project now builds -Xcheck:jni clean. The first actual bug found through the students work got fixed before the bonding period ended - the bug had been there for fifteen years.
The later part of the work will involve more straight-up coding, to replace a Maven plugin now used in PL/Java's build that isn't quite suited to the need. The proposal outlined a few reasons for that and the preliminary work has already uncovered more reasons. It was already a goal of that work to improve the signal-to-noise ratio of diagnostics from that plugin, so already solving a similar problem for -Xcheck:jni was a good warmup.
First 2 weeks are going almost as we planned, the student started working on the first task during the phase of proposals. In the first week he updated his PR and we merged it. The first feature already works. This week he updated it with several improvements (mostly refactorings) and started working on test coverage of the first feature and also making some drafts on the second feature. Now we have discussion about design details of the second feature. I hope it will help to implement it according to the current plan.
Thank you to all mentors for the status update!
The PostgreSQL main website has a new page: "Related Projects".
This page lists the projects which help running and maintaining the PostgreSQL project, the infrastructure, and other things like the translations for press releases. For each project it lists links to the source, as well as information where to send updates, patches, or input.
If you want to get involved in one of the projects, that's your starting point. If a project is missing, please send a note or a patch to pgsql-www.
Many thanks to Jonathan S. Katz for polishing the patch, and making it look nice!
SCALE 18x is taking place during the next week, in Pasadena, CA. I will present a 3 hours workshop about PostgreSQL data types on Friday morning.
In order to attend, you need a SCALE registration. No special knowledge is required for the workshop. If you want to follow the exercises, you need a PostgreSQL database installed on your computer.
See you there!
Pavlo recently pointed out that the pg_sleep() function in PostgreSQL has two alternatives: pg_sleep_for() and pg_sleep_until().
What do they do?
Continue reading "SELECT pg_sleep_until('#800Monies');"
The PostgreSQL Project is present with a booth at FOSDEM ever since 2007. Since 2008 we organize a Devroom, since 2013 we have our own PGDay on the Friday before FOSDEM. This year marks the 8th FOSDEM PGDay.
This blog post presents useful information about the PGDay, the booth and Devroom at FOSDEM.
Continue reading "PostgreSQL @ FOSDEM 2020"
For the 13th year, the PostgreSQL Project is participating in Google Summer of Code (GSoC). This project is a great opportunity to let students learn about Open Source projects, and help them deliver new features. It is also a chance to engage the students beyond just one summer, and grow them into active contributors.
In GSoC, students first learn about the Open Source organization, and either pick a summer project from the list provided by the org, or submit their own idea for review. After a “community bonding” period, the students have time to implement their idea, under supervision of mentors from the Open Source organization. There is also an incentive: first, Google pays the students for their work on improving Open Source projects. And second, having a completed GSoC project in a CV is well recognized.
Continue reading "Google Summer of Code 2019 - PostgreSQL participates with 5 projects"
At FOSDEM someone asked how long 64 bit Transaction-IDs will last.
To refresh: PostgreSQL is currently using 32 bits for the TXID, and is good for around 4 billion transactions:
fosdem=# SELECT 2^32;
That will not last very long if you have a busy database, doing many writes over the day. MVCC keeps the new and old versions of a row in the table, and the TXID will increase with every transaction. At some point the 4 billion transactions are reached, the TXID will overrun, and start again at the beginning. The way transactions are working in PostgreSQL, suddenly all data in your database will become invisible. No one wants that!
To limit this problem, PostgreSQL has a number mechanism in place:
- PostgreSQL splits transaction ids into half: 2 billion in the past are visible, 2 billion in the future are not visible - all visible rows must live in the 2 billion in the past, at all times.
- Old, deleted row versions are enevtually removed by VACUUM (or Autovacuum), the XID is no longer used.
- Old row versions, which are still live, are marked as "freezed" in a table, and assigned a special XID - the previously used XID is no longer needed. The problem here is that every single table in every database must be Vacuumed before the 2 billion threshold is reached.
- PostgreSQL uses lazy XIDs, where a "real" transaction id is only assigned if the transaction changes something on disk - if a transaction is read only, and does not change anything, no transaction id is consumed.
Continue reading "How long will a 64 bit Transaction-ID last in PostgreSQL?"
The PostgreSQL Project participates in Google Code-In (GCI) 2018. This is a program which allows pre-university students to pick up tasks defined by the partnering open source projects, learn about these projects, and also win a prize (certificates, t-shirts, hoodies, but also a trip to Google HQ).
Every project creates a number different tasks, some technical, some design based, some about updating documentation, or validating bugs. Whatever is useful in order to get to know the project better. Students can select tasks and submit their work. Mentors from the project then evaluate the work, and either approve it or send it back to the student because more work is needed.
Now we are halfway into this year's competition, it's time to run the numbers.
Continue reading "Google Code-In 2018 - Halftime"
For a long time I was using a Makefile to quickly build, start, stop and then wipe a predefined PostgreSQL version. That comes handy if you just want to test something on an older version, without actually installing the software. Everything happens in a single directory, even a different port is assigned.
When I needed that setup recently, I ran into unrelated build errors:
relpath.c:21:10: fatal error: catalog/pg_tablespace_d.h: No such file or directory
Can't be - pg_tablespace_d.h is included in the tarball I'm using.
Continue reading "Using Makefiles to build PostgreSQL"
The PostgreSQL Project was assigned a devroom at FOSDEM again this year. We got a room with approx. 200 seats, and as usual every last seat was taken for many of the talks. This requires that we manage the room very effectively and carefully, both to avoid that people walk in and out at all times and disturb the speaker, but also in order to keep the exits free at all times.
For a speaker it is quite disturbing if the door in the back opens, someone walks in, down all the way to the front, tries to find a seat, walk around to the other side, maybe leaves again, or just stays somewhere and blocks the exit. The worst case is when there is a free seat somewhere in the middle of a row, and everybody has to get up to let the new attendee in.
The FOSDEM team requires that we keep the exits free, in case the room needs to be evacuated. If people are staying on the steps along the wall, it is not possible to clear the room effectively and quickly.
Continue reading "How we manage the PostgreSQL FOSDEM devroom"