Schedule - PGConf.EU 2019
Beyond the pushdowns – distributed query planning and execution
Date: 2019-10-16
Time: 13:00–13:50
Room: Manzoni
Level: Advanced
Nowadays, it's generally agreed that community way to sharding is combination of partitioning and foreign data wrappers (FDW). Naturally, one could turn a local database to distributed one by moving some data partitions to foreign servers.
A lot of efforts were spent to provide FDW a bunch of query optimization techniques including pushdowns of aggregates, joins and orderings. These optimizations allow to making more computations locally. That in turn improves scalability of query processing.
However, pushdowns are not enough for real scalability of OLAP queries, which requires complex processing of large amounts of data. Such queries requires so-called "re-partition", runtime redistribution of data between cluster nodes.
In this talk we present our work on distributed query planning/execution. It provides full-fledged OLAP query optimization including "re-partition". Our distributed query planner and executor require only few core modification, while being built in pluggable way using hooks and custom nodes. The efficiency of proposed approach will be shown on both syntactical and real-life benchmarks.
While being essential part of sharding, we hope distributed planner/executor will become part of community sharding roadmap.
Slides
The following slides have been made available for this session: