Wöchentlicher PostgreSQL Newsletter - 02. März 2008

== Angewandte Patches ==

Peter Eisentraut committed:

- Link postgres from all object files at once, to avoid the
error-prone SUBSYS.o step and allow for better optimization by the
linker. Instead of partial linking into SUBSYS.o, the list of
object files is assembled in objfiles.txt files that are expanded
when the final linking is done. Because we are not yet sure how
long command lines different platforms can handle, the old way of
linking is still available, by defining the make variable
PARTIAL_LINKING (e.g., make all PARTIAL_LINKING=1). If we determine
that this is necessary for some platforms, then we will document
this in a more prominent place.

- Refactor the code that creates the shared library export files to
appear only once in Makefile.shlib and not in four copies.

- In pgsql/src/backend/common.mk, we don't need to rebuild
objfiles.txt every time an object file changes. So only rebuild
when a makefile changes (which presumably defines the file list
somewhere), and only touch the file if an object changed. The touch
is necessary so the parent make knows something changed and
ultimately rebuilds postgres.

- In pgsql/src/backend/common.mk, need more dependencies to get the
build order right when objfiles.txt doesn't exist yet.

- In pgsql/src/Makefile.shlib, escape # character in variable
assignment.

- Reorganize some of the exports list generation code. It seems that
this has been reinvented about four different times throughout
history (aix, cygwin, win32, darwin/linux) and a lot of the concepts
are actually shared, which the code now shows better.

- Create two separate libpq.rc's: One that is built at build time, and
one that is shipped in the distribution, named libpq-dist.rc. This
way the build system doesn't get upset when a distributed file is
forcibly overwritten by during a normal build.

- In pgsql/src/interfaces/ecpg/include/Makefile, fix uninstall target.

- Don't build the win32 support files in the all target, only in
distprep and when they are actually needed as prerequisites.

- In pgsql/doc/src/sgml/func.sgml, add information about format
modifiers that apply to numeric formats. These were previously only
documented in the context of date/time formats.

- In pgsql/doc/src/sgml/runtime.sgml, change SSH setup instructions to
use localhost, which is allowed by default, and a port number above
the reserved range with explanations of each. Per Faheem Mitha.

- In pgsql/doc/src/sgml/runtime.sgml, add more clarification about SSH
tunnels from Faheem Mitha.

- In pgsql/src/backend/common.mk, change expand_subsys function so
that it preserves the relative order of the files passed as
argument. This is desirable so that the dtrace rule in
src/backend/Makefile works.

- In pgsql/src/backend/common.mk, write the objfiles.txt rules in a
way that is compatible with GNU make 3.78, and simpler, too.

Tom Lane committed:

- Fix datetime input to behave correctly for Feb 29 in years BC.
Formerly, DecodeDate attempted to verify the day-of-the-month
exactly, but it was under the misapprehension that it would know
whether we were looking at a BC year or not. In reality this check
can't be made until the calling function (eg DecodeDateTime) has
processed all the fields. So, split the BC adjustment and validity
checks out into a new function ValidateDate that is called only
after processing all the fields. In passing, this patch makes
DecodeTimeOnly work for BC inputs, which it never did before. (The
historical veracity of all this is nonexistent, of course, but if
we're going to say we support proleptic Gregorian calendar then we
should do it correctly. In any case the unpatched code is broken
because it could emit dates that it would then reject on
re-inputting.) Per report from Bernd Helmle. Back-patch as far as
8.0; in 7.x we were not using our own calendar support and so this
seems a bit too risky to put into 7.4.

- In pgsql/src/backend/utils/adt/datetime.c, reject year zero during
datetime input, except when it's a 2-digit year (then it means 2000
AD). Formerly we silently interpreted this as 1 BC, which at best
is unwarranted familiarity with the implementation. It's barely
possible that some app somewhere expects the old behavior, though,
so we won't back-patch this into existing release branches.

- In pgsql/src/backend/utils/adt/encode.c, fix encode(...bytea...,
'escape') so that it converts all high-bit-set byte values into \nnn
octal escape sequences. When the database encoding is multibyte
this is necessary to avoid generating invalidly encoded text.
Even in a single-byte encoding, the old behavior seems very
hazardous --- consider for example what happens if the text is
transferred to another database with a different encoding. Decoding
would then yield some other bytea value than what was encoded, which
is surely undesirable. Per gripe from Hernan Gonzalez. Backpatch
to 8.3, but not further. This is a bit of a judgment call, but I
make it on these grounds: pre-8.3 we don't really have much encoding
safety anyway because of the convert() function family, and we would
also have much higher risk of breaking existing apps that may not be
expecting this behavior. 8.3 is still new enough that we can
probably get away with making this change in the function's
behavior.

- In pgsql/src/backend/utils/cache/relcache.c, if RelationBuildDesc()
fails to open a critical system index, PANIC with a relevant error
message instead of just dumping core. Odd that nobody reported this
before Darren Reed.

- In pgsql/src/backend/access/heap/tuptoaster.c, reducing the assumed
alignment of struct varlena means that the compiler is also licensed
to put a local variable declared that way at an unaligned address.
Which will not work if the variable is then manipulated with
SET_VARSIZE or other macros that assume alignment. So the previous
patch is not an unalloyed good, but on balance I think it's still a
win, since we have very few places that do that sort of thing. Fix
the one place in tuptoaster.c that does it. Per buildfarm results
from gypsy_moth (I'm a bit surprised that only one machine showed a
failure).

- In pgsql/src/backend/utils/adt/xml.c, disable the undocumented
xmlvalidate() function, which was unintentionally left in the code
though it was not meant to be provided. It represents a security
hole because unprivileged users could use it to look at (at least
the first line of) any file readable by the backend. Fortunately,
this is only possible if the backend was built with XML support, so
the damage is at least mitigated; and 8.3 probably hasn't propagated
into any security-critical uses yet anyway. Per report from Sergey
Burladyan.

- Fix unportable usages of tolower(). On signed-char machines, it is
necessary to explicitly cast the output back to char before
comparing it to a char value, else we get the wrong result for
high-bit-set characters. Found by Rolf Jentsch. Also, fix several
places where <ctype.h> functions were being called without casting
the argument to unsigned char; this is likewise unportable, but we
keep making that mistake :-(. These found by buildfarm member
salamander, which I will desperately miss if it ever goes belly-up.

- In pgsql/src/backend/storage/large_object/inv_api.c, fix another
place that was assuming that a local variable declared as "struct
varlena" would be at least word-aligned. Per buildfarm results from
gypsy_moth. I did a little bit of trawling for other instances of
this coding pattern, and didn't find any; but if we turn up any more
of them I think we'd better revert the "char [4]" patch and find
another way of making tuptoaster.c alignment-safe.

- Venezuela Time now means UTC-4:30, not UTC-4:00. Adjust our
treatment of "VET" accordingly. Per bug #3997 from Aaron Mizrachi.

Magnus Hagander committed:

- Support for building contrib/uuid-ossp with MSVC. Original patch
from Hiroshi Saito, modified by me.

- Fix handling of restricted processes for Windows Vista (mainly), by
explicitly adding back the user to the DACL of the new process.
This fixes the failure case when executing as the Administrator
user, which had no permissions left at all after we dropped the
Administrators group. Dave Page with some modifications from me

Neil Conway committed:

- Fix several memory leaks when rescanning SRFs. Arrange for an SRF's
"multi_call_ctx" to be a distinct sub-context of the EState's
per-query context, and delete the multi_call_ctx as soon as the SRF
finishes execution. This avoids leaking SRF memory until the end of
the current query, which is particularly egregious when the SRF is
scanned multiple times. This change also fixes a leak of the fields
of the AttInMetadata struct in shutdown_MultiFuncCall(). Also fix a
leak of the SRF result TupleDesc when rescanning a FunctionScan
node. The TupleDesc is allocated in the per-query context for every
call to ExecMakeTableFunctionResult(), so we should free it after
calling that function. Since the SRF might choose to return a
non-expendable TupleDesc, we only free the TupleDesc if it is not
being reference-counted. Backpatch to 8.3 and 8.2 stable branches.

Bruce Momjian committed:

- Update TODO wording to read: "Add features of Oracle-style packages
(Pavel)."

Alvaro Herrera committed:

- Remove long-unused and broken TCL_ARRAYS.

Andrew Dunstan committed:

- Don't call AddUserToDacl on Cygwin.

Michael Meskes committed:

- In ecpg, fixed bug that caused arrays of varchar to be output with
incomplete name. In the process expanded one test case.

== Abgelehnte Patches (bis jetzt) ==

Simon Riggs's patch to tune bulk inserts, as it doesn't clean up
nicely after a subtransaction abort.

== Eingesandte Patches ==

Hiroshi Saito sent in a patch to fix MinGW builds which Peter
Eisentraut's patch moving the build of win32 support files from the
"all" target to the "distprep" target broke.

Heikki Linnakangas sent in a revision of his patch to make
CopyReadLineText run faster.

Euler Taveira de Oliveira sent in a patch which makes to_char() pay
attention to lc_time.

Gevik Babakhani sent in an update to Euler Taveira de Oliveira's patch
above.

Kenneth D'Souza sent in a patch to psql which shows incoming foreign
key constraints along with the existing out-going foreign key
constraints when people invoke \d table_name.

Markus Bertheau sent in a patch intended to fix holes in the
backup.sgml docs.

Peter Eisentraut sent in a patch which factors out four nearly
identical copies of multiplatform shared library exports file
generation code, placing it in one spot.

Rober Lor sent in three revisions of a patch intended to add DTrace
probes for OS/X Leopard.

Neil Conway sent in a patch which allows an array of Datums + nulls to
be inserted into a tuplestore without first creating a HeapTuple,
eliminating an unneeded copy.

ITAGAKI Takahiro sent in a patch to log conflicted queries on
deadlocks.

Alvaro Herrera sent in a patch which modifies the interactions in
sinval.c and sinvaladt.c.

Zdenek Kotala sent in a patch to remove FATAL from pg_lzdecompress by
adding some boundary checks and memory overwriting protection when
compressed data is corrupted.

Heikki Linnakangas sent in a patch to speed up CopyReadAttributesCSV.

Mark Wong sent in a patch intended to correct posix_fadvise() in
xlog.c.