Schedule - PGConf.EU 2015

Access method extendability in PostgreSQL or back to origin

Date: 2015-10-28
Time: 14:00–14:50
Room: Palais I-III
Level: Intermediate

Postgres was initially designed to support access methods extendability. Well known citation about access method in Postgres claims: "It is imperative that a user be able to construct new access methods to provide efficient access to instances of nontraditional base types" Michael Stonebraker, Jeff Anton, Michael Hirohama. Extendability in POSTGRES, IEEE Data Eng. Bull. 10 (2) pp.16-23, 1987

Such extendability basically remains nowadays in PostgreSQL: access methods are defined in pg_am table of system catalog and have well-documented interface. However, it's impossible to create access methods in external (non-core) module, since * There is no "CREATE ACCESS METHOD" command, one has to insert am definition directly to pg_am. * There is no way to define new WAL-records types in extension. Indexes of this new access method wouldn't be WAL-logged.

This breaks extendability of PostgreSQL, which we consider important feature and motivate us to fix this ancient problem.

The first problem is mostly mechanical. We just need to implement "CREATE ACCESS METHOD" command like many other "CREATE" command.

Problem of WAL is a bit more complex. Actually, we don't want extensions to define their own WAL-records, because implementing of corresponding redo functions is a challenging task (even for skilled core programers) and there is a chance to break both recovery and replication, which is not acceptable.

Another approach is to define generic WAL-records, that specify a difference between pages in generalized way. The important property of such WAL-records is that they should be bullet-proof: error in user code shoudn't produce broken WAL-records which fails to replay.

Custom access methods is not the only consumer of generic WAL-records. For example, PostgreSQL now has powerful extendability mechanisms: FDW and custom nodes, which can be used to implement custom storage engines in extensions and they has to be WAL-logged somehow.

In this talk we will present a generic WAL and an example of extension, which creates custom WAL-logged access method.

Speaker