Schedule - PGConf.DE 2025

Breaking Down Monoliths: Automating PostgreSQL Cluster Isolation at Scale

Date: 2025-05-08
Time: 13:45–14:30
Room: Ballsaal 2
Level: Intermediate

Operating at Datadog scale presents unique challenges, particularly when managing a monolithic PostgreSQL database that spans multiple cloud providers and regions, with versions ranging from 12 to 15. Last year, we embarked on a project to break this monolith—comprising over 300 tables—into separate Kubernetes-hosted clusters to improve workload isolation, simplify maintenance, and reduce operational risks.

During this talk, we will share how we tackled these challenges with a focus on automation; leveraging Temporal workflows to orchestrate complex database migrations. Will cover the following topics:

• Navigating nuances of managed PostgreSQL services (AWS, Azure, GCP).
• The difficulties of operating on multiple different Postgres major versions.
• How we use PostgreSQL’s logical replication and a custom-built connection pooler to minimize downtime during traffic cutovers.
• The automation of every step, from replication setup to operator confirmation, with a custom saga pattern for safe rollbacks.

Beyond the technical details, we’ll also discuss the importance of internal customer collaboration to ensure migrations align with the diverse needs and priorities of different teams. Additionally, we’ll explore unexpected benefits, such as repurposing this migration framework for multitenancy and dynamic workload redistribution.

Whether you’re planning to modernize your PostgreSQL infrastructure or looking for practical automation strategies, this talk will provide actionable insights and hard-learned lessons on automating at scale, all while keeping customer needs front and center. Regardless of the scale you operate, by the end of the session you’ll walk away with a deeper understanding of PostgreSQL migration best practices, how to build automation tools, and how to manage migrations from complex cloud environments effectively.

Speaker