Schedule - PGConf.EU 2015

Integrating PostgreSQL with Logstash for real-time monitoring

Date: 2015-10-29
Time: 12:50–13:40
Room: Ballroom C+D
Level: Intermediate

The Logstash pipeline is a very common framework that allows system administrators to monitor their distributed infrastructures using ElasticSearch and Kibana. In such cases, it is important that PostgreSQL, as a relevant and crucial component of such systems, could fit in this wider architecture.

In a very simplicistic scenario, log entries can be post-processed by Logstash through standard text log files, automatically produced by PostgreSQL.

However, through the 'redislog' extension, PostgreSQL can ship log entries directly to a Redis server in JSON format. The Redis server can act as a broker in a typical Logstash architecture. Once data is pulled out from Redis, entries are immediately pushed into ElasticSearch as JSON objects and made available to end users through the Kibana web interface.

In this presentation we will be exploring some future evolution of such an approach, including, for example, a generic implementation around AMQP.

Speaker