Quick Tour of Postgres Logical Replication - Failures, Recovery, and Monitoring (Part 2)

August 12, 2026

Okay, now that we know how to set up logical replication between a publisher and a subscriber.

So far, so good.

But still… something doesn’t feel right, right?

It kinda feels too magical, kinda too good to be true.

For example, what happens if either one of those instances goes down?

Database instances go down all the time. You know, crashes, infrastructure issues, a bad K8s config, database or OS updates, DBAs getting laid off and replaced by AI, etc. 💀

So if this is such a common thing, PostgreSQL must handle it pretty well, right?

Right?

Right?

Well, let’s find out.

Let’s say the subscriber goes down

Stop the subscriber

lets stop the docker container first:

1docker stop pg-subscriber

check:

1> docker ps -a
2
3CONTAINER ID   IMAGE                        COMMAND                  CREATED        STATUS                      PORTS                                         NAMES
47834bfce1f81   postgres:18                  "docker-entrypoint.s…"   44 hours ago   Exited (0) 49 seconds ago                                                 pg-subscriber
5be66572f1428   postgres:18                  "docker-entrypoint.s…"   44 hours ago   Up 44 hours                 0.0.0.0:5432->5432/tcp, [::]:5432->5432/tcp   pg-publisher

Generate changes while the subscriber is down

now that we have confirmation that the subscriber instance is stopped

lets now insert some more rows on the publisher:

 1-- on the publisher (subscriber is down)
 2
 3INSERT INTO users (name, email, password)
 4VALUES ('Miso', '[email protected]', 'miso123');
 5
 6INSERT INTO groups (name)
 7VALUES ('ferrets');
 8
 9INSERT INTO groups_users (group_id, user_id)
10VALUES (3, 5);

Check the replication slot

now before bring the subscriber back up, lets check whats going on in the table pg_replication_slots

1-- on the publisher (sub is down)
2
3SELECT slot_name, active, restart_lsn 
4FROM pg_replication_slots;
5
6 slot_name | active | restart_lsn
7-----------+--------+-------------
8 demo_sub  | f      | 0/180E1E8
9(1 row)

quick explanation: pg_replication_slots.slot_name is demo_sub obviously is the name of the subscription we created in part 1

pg_replication_slots.active is false cuz the subscribare owner of that subs (our pg-subscriber instance on docker) is down

pg_replication_slots.restart_lsn is an interesting one.
Basically, it’s the oldest WAL position that PostgreSQL still needs to retain for this replication slot.

So, while our subscriber is down, the slot’s restart_lsn stays at the position the subscriber last consumed, while the publisher keeps generating WAL. That means the gap between the current WAL position and restart_lsn grows:

1-- on the publisher (sub is down)
2
3SELECT pg_current_wal_lsn();
4
5 pg_current_wal_lsn
6--------------------
7 0/1813B20
8(1 row)

See how much WAL is being retained

We can show the distance between it and the publisher’s current WAL position:

 1-- on the publisher (sub is down)
 2
 3SELECT slot_name,
 4       pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) AS wal_retained_bytes
 5FROM pg_replication_slots;
 6
 7 slot_name | wal_retained_bytes
 8-----------+--------------------
 9 demo_sub  |              22840
10(1 row)

This tells us how much WAL has been generated since the slot’s restart_lsn.

for our example example, if this returns 22840, that’s roughly 22 KB of WAL between the slot’s restart_lsn and the current WAL position. This is WAL that the slot may still require, so PostgreSQL can’t recycle that portion of the WAL while this slot still needs it.

basically, the publisher is holding onto WAL for the downed subscriber

By default, each subscription gets its own replication slot, which is why a subscriber falling behind can cause WAL to accumulate on the publisher.

Bring the subscriber back up

1docker start pg-subscriber

Watch it catch up

 1-- on the subscriber
 2
 3SELECT count(*) FROM users;
 4
 5 count
 6-------
 7     4
 8(1 row)
 9
10SELECT name FROM users;
11
12    name
13-----------
14 Biscuit
15 Cannoli
16 Momo
17 Miso
18(4 rows)
19
20SELECT name FROM groups;
21
22   name
23-----------
24 dogs
25 cats
26 ferrets
27(3 rows)

The data that was inserted while the subscriber was down is now replicated automatically. The subscription reconnects and resumes streaming from the point it previously reached, replaying the WAL changes it missed while it was down.

Back on the publisher, the slot is active again:

1-- on the publisher
2
3SELECT slot_name, active, restart_lsn, wal_status
4FROM pg_replication_slots;
5
6 slot_name  | active | restart_lsn | wal_status
7------------+--------+-------------+------------
8 demo_sub   | t      | 0/180F128   | reserved

Notice active = t now. The restart_lsn advanced past where it was, confirming the subscriber caught up.

This is also why the restart_lsn gap matters: the larger the gap, the more WAL needs to be retained, and the more the subscriber has to replay when it reconnects.

Let’s say the publisher goes down

Stop the publisher

1docker stop pg-publisher

1> docker ps -a
2
3CONTAINER ID   IMAGE                        COMMAND                  CREATED        STATUS                      PORTS                                         NAMES
47834bfce1f81   postgres:18                  "docker-entrypoint.s…"   46 hours ago   Up 36 minutes               0.0.0.0:5433->5432/tcp, [::]:5433->5432/tcp   pg-subscriber
5be66572f1428   postgres:18                  "docker-entrypoint.s…"   46 hours ago   Exited (0) 6 seconds ago                                                  pg-publisher

pg-publisher is now Exited.

Watch the apply worker retry

Now that the publisher is no longer available, let’s see what is happening on the subscriber.

First, check pg_stat_subscription:

1-- on the subscriber
2
3SELECT subname, worker_type, pid
4FROM pg_stat_subscription;
5
6 subname  | worker_type | pid
7----------+-------------+-----
8 demo_sub |             |
9(1 row)

The subscription itself is still there, but worker_type and pid are NULL. At this particular moment, there is no apply worker running.

But is PostgreSQL just sitting there doing nothing?

Let’s run the query repeatedly:

 1SELECT subname, worker_type, pid FROM pg_stat_subscription;
 2
 3 subname  | worker_type | pid
 4----------+-------------+-----
 5 demo_sub | apply       | 210
 6(1 row)
 7
 8SELECT subname, worker_type, pid FROM pg_stat_subscription;
 9
10 subname  | worker_type | pid
11----------+-------------+-----
12 demo_sub |             |
13(1 row)
14
15SELECT subname, worker_type, pid FROM pg_stat_subscription;
16
17 subname  | worker_type | pid
18----------+-------------+-----
19 demo_sub | apply       | 211
20(1 row)
21
22SELECT subname, worker_type, pid FROM pg_stat_subscription;
23
24 subname  | worker_type | pid
25----------+-------------+-----
26 demo_sub |             |
27(1 row)

nteresting.

The apply worker keeps getting restarted. Each time PostgreSQL starts a new worker, it gets a new PID. Between workers, pid is NULL because there is no worker running at that particular moment.

Let’s check the subscriber logs:

 1❯ docker logs pg-subscriber --tail 20
 2
 32026-08-12 13:32:33.563 UTC [1] LOG:  background worker "logical replication apply worker" (PID 209) exited with exit code 1
 42026-08-12 13:32:38.510 UTC [210] LOG:  logical replication apply worker for subscription "demo_sub" has started
 52026-08-12 13:32:42.511 UTC [210] ERROR:  apply worker for subscription "demo_sub" could not connect to the publisher: could not translate host name "pg-publisher" to address: Name or service not known
 62026-08-12 13:32:42.513 UTC [1] LOG:  background worker "logical replication apply worker" (PID 210) exited with exit code 1
 72026-08-12 13:32:43.511 UTC [211] LOG:  logical replication apply worker for subscription "demo_sub" has started
 82026-08-12 13:32:47.512 UTC [211] ERROR:  apply worker for subscription "demo_sub" could not connect to the publisher: could not translate host name "pg-publisher" to address: Name or service not known
 92026-08-12 13:32:47.514 UTC [1] LOG:  background worker "logical replication apply worker" (PID 211) exited with exit code 1
10...

So pid = NULL doesn’t mean PostgreSQL has given up on the subscription. It just means that there is no apply worker running at that exact moment.

PostgreSQL will keep retrying until the publisher becomes available again.

Bring the publisher back up

1docker start pg-publisher

Wait a few seconds for the publisher to recover and the subscriber to reconnect.

Verify replication resumes

1-- on the publisher
2
3SELECT slot_name, active, restart_lsn
4FROM pg_replication_slots;
5
6 slot_name | active | restart_lsn
7-----------+--------+-------------
8 demo_sub  | t      | 0/18145B0
9(1 row)

well, looks like our slot is still there.
thank god replication slots are persistent so they survive a restart.

now that everything looks fine on the pub side, lets check if the sub worker picked up again:

1-- on the subscriber
2
3SELECT subname, worker_type, pid
4FROM pg_stat_subscription;
5
6 subname  | worker_type | pid
7----------+-------------+-----
8 demo_sub | apply       | 446
9(1 row)

yup, looks like the apply worker is back with a real pid happely applying our wal stream events again.

Let’s say everything goes down

What if both instances crash? Surely replication is screwed too, right?

Not really. As long as the PostgreSQL data and replication state survive, both databases can come back up, reconnect, and pick up replication where they left off.

Unless something went very wrong and your data is toasted too.

At that point, replication is probably the least of your problems. 🫠