This blog posting compares the performance of the pg_tde by Percona extension for PostgreSQL with the performance of an unmodified PostgreSQL. The pg_tde by Percona extension provides encryption for both data pages on disk as well as WAL files on disk - Encryption at Rest. The extension is currently in the rc stage (release candidate). This test does not compare pg_tde by Percona to other products. It tests the extension performance and compares with unmodified PostgreSQL, and shows how much overhead it is to encrypt data at rest, and encrypt WAL.
Executive Summary
The performance impact of pg_tde by Percona for encrypted data pages is measurable across all tests, but not very large. The performance impact of encrypting WAL pages is about 20% for write-heavy tests. The tests were run with an extension RC (Release Candidate), however the WAL encryption feature is still in Beta stage.
What is Transparent Data Encryption
Transparent Data Encryption (TDE) is a mechanism that encrypts data „at rest“, when stored on a storage system („on disk“). This protects sensitive data while it is safely stored in a database.
Encrypting data while it is transported (Transport Layer Security - TLS) is very common nowadays, not only in PostgreSQL. Many websites also use it, that is shown by using https as protocol, instead of http. A large part of emails are transported using TLS these days. In short, TLS is abundant. PostgreSQL supports encrypted connections since at least version 7.1.
There is however no built-in encryption for data while it is stored in a database, at least not in PostgreSQL. The database can be stored on an encrypted disk or an encrypted filesystem, which provides some security against stolen or lost disks. This does not prevent accessing data while the operating system has decrypted the filesystem or disk. The pgcrypto extension provides basic support and digital signatures for encrypting and decrypting some data on disk, but the handling is somewhat intricate, plus the data can’t be safely indexed.
Encryption in any form does not come for free, it is CPU intensive, both encryption and decryption require additional CPU cycles. Depending on how the encryption is implemented, this additional overhead can be substantial.
Test Methodology
The test uses pgbench, and runs a number of tests:
- Number of transactions is always
1000(default for pgbench is10). - Tests are run in select only mode as well as in read/write mode.
- Number of clients range from
1to70(1,2,5,10,20,30,40,50,60,70). - Scale factor ranges from
5to5000(5,10,50,100,400,600,1000,2000,4000,5000). - Each test is repeated
5times, the best value is selected.
The test is explained in detail in the blog posting: Test Methodology for TDE Performance Tests. This other blog posting also includes a larger number of graphs with more details.
The number of transitions was increased to 1000 for all tests, from the default of 10. Initial results have shown fewer fluctuations in numbers when this number is increased.
Summary of Performance Tests
For practical purposes, the performance of pg_tde by Percona for data pages, compared to native PostgreSQL, is almost equal. Numbers and graphs show a minimal better performance for native PostgreSQL with no encryption, but the differences are often negligible. This pattern is consistent for a large number of clients, as well as across different test data sizes.
This is explained by equal I/O necessary for encrypted and unencrypted data pages, as well as use of hardware-accelerated encryption in the CPU. pg_tde by Percona is using OpenSSL (version ‘3.0.13’) for encryption operations. The CPU used in this test has the „aes“ flag set.
The overhead for WAL encryption is measurable in heavy read/write tests, and can be up to 20%. This is explained by constant encryption of even small data blocks before writing the WAL blocks to disk. Read workloads are not affected by WAL encryption. The performance impact of decrypting WAL data on a replica or during restore was not tested.
Handling pg_tde by Percona as a DBA
One aspect I paid particular attention to through the tests is how easy or complicated it is to use pg_tde by Percona.
As a starter, right now if you want to use pg_tde by Percona, you have to use the Percona PostgreSQL package. That is easy to use, a repository (for Debian/Ubuntu in my case) allows to select the distribution channel (experimental, testing, release) and install the package. The package works similar to the native APT packages. It is currently not possible to use pg_tde by Percona as an extension in an unmodified PostgreSQL.
The key handling allows local keys, but that is reserved for testing. Keeping the keys local on the database server is not secure - anyone with access to the server also has access to the keys and can decrypt the data using said keys. The key handling can also be managed by using a key management storage. This approach is recommended. I did use local keys for the test.
Once the key handling is activated, all the DBA or user has to do is create tables using the tde_heap access method. Or alter a table and set the access method, the table will be rewritten and this will enable encryption:
| |
Ideally this is done before writing any data to the disk. The SET ACCESS METHOD syntax has been supported in PostgreSQL since version 15, the specific tde_heap method however is not available in regular PostgreSQL. This is part of the pg_tde by Percona version.
Other than that, pg_tde by Percona behaves like a regular PostgreSQL database in my tests.
Future of TDE
Bruce blogged about TDE in PostgreSQL a short while ago. It is unlikely that TDE itself will be a core feature short to mid-term. There are too many open questions. However it is an often demanded feature for compliance reasons, which require that sensitive data is encrypted at rest.
For the time being, if TDE in PostgreSQL is something you need for compliance, one of the vendors is your best choice.
Transparency
Percona approached me last year and asked for an independent performance review of their pg_tde by Percona extension for PostgreSQL. I agreed under the following conditions:
- I have full autonomy over the test and the resulting blog posting (this posting).
- Percona has no influence in the test and over the content of this posting.
- Percona has the option to cancel the test, in which case the contract is still fulfilled.
- Percona pays for the test system for the duration of the test.
- This is not a security review, only a performance review.
I did find a couple of problems during the test phase, which I reported back to Percona. Details can be found near the end of this blog posting.
Comments
With an account on the Fediverse or Mastodon, you can respond to this post. Known replies are displayed below:
Note: This will load data from mastodon.social.
Learn how this is implemented here.
Share: