This is the multi-page printable view of this section. .
Backup & Restore
- 1: Backup Mechanism
- 2: Backup Policy
- 3: Backup Repository
- 4: Admin Commands
- 5: Restore Operations
- 6: Clone a PG Cluster
Pigsty uses pgBackRest for PostgreSQL backups. It supports full, differential, and incremental backups, parallel processing, encryption, and Silo/S3 object storage. By default, Pigsty configures backup and WAL archiving for every PGSQL cluster.
This chapter is the operational manual for backup configuration, management, recovery, and drills. For design concepts and tradeoffs, see Point-in-Time Recovery.
All backup and recovery operations call pgBackRest. Pigsty provides these wrapper layers:
| Layer | Interface | Form | Scope |
|---|---|---|---|
| Cluster orchestration | pg_pitr + pgsql-pitr.yml | Ansible playbook | HA, etcd, and multi-node recovery |
| Instance orchestration | pig pitr | CLI | Local-node recovery without the admin node |
| Command primitives | pig pb, pb, and pg-backup | pgBackRest wrappers | Backup, inspection, expiry, and unmanaged restore |
| Engine | pgbackrest | Native CLI | Underlying backup, archive, and restore engine |
| Section | Content |
|---|---|
| Mechanism | Stanzas, repositories, retention, timelines, and Pigsty wrapper mapping |
| Policy | Scheduling, recovery windows, and storage planning |
| Repository | Local, Silo, and external S3 repositories; encryption, versioning, and locking |
| Administration | Backup commands, inspection, expiration, and stanza management |
| Restore | Recovery targets, staged PITR, and complete parameter reference |
| Clone | Restore production history into another cluster and perform drills |
| Tutorial | A sandbox restore using pgBackRest primitives |
Pigsty makes every effort to provide a reliable PITR solution. It accepts no liability for data loss caused by restore operations. If you need assistance, consider professional services.
Before PITR, inspect pig pg list <target-cluster> and pig pb info.
Verify a recent usable backup and its recovery window.
Have the operator state the exact target cluster and recovery point.
Then run the target-scoped ./pgsql-pitr.yml -l <target-cluster> ... command.
pgsql-pitr.yml prints a plan but does not pause for approval.
Production recovery also requires a maintenance window and an independently verified backup.
Quick Start
- Design a backup policy: declare scheduled backups in
pg_crontaband select a repository withpgbackrest_repo. - Manage backups: run
pg-backupand inspect recovery coverage withpb info. - Perform recovery: declare
pg_pitr, then runpgsql-pitr.yml.
1 - Backup Mechanism
Pigsty’s backup and restore operations execute pgBackRest commands. For safe operation, understand the pgBackRest model and how Pigsty maps its orchestration parameters to native options.
Core pgBackRest Concepts
Stanza: the Cluster’s Backup Identity
A stanza names one PostgreSQL backup configuration and isolates that cluster inside a repository.
Pigsty uses pg_cluster as the stanza name.
The pg-meta stanza stores data under backup/pg-meta/ and archive/pg-meta/.
Thus, several clusters can share one repository.
The stanza records the source system identifier and major version.
pgBackRest checks these values before a backup.
Because of this check, a cross-cluster clone needs stanza-upgrade afterward.
Pigsty creates the stanza during cluster initialization.
stanza-upgrade updates it after a major-version change or clone.
Repository: Where Backups Live
A repository stores backup files and WAL archives.
repo1-type selects POSIX, S3, Azure, GCS, or SFTP.
repo1-path, repo1-cipher-*, and repo1-retention-* define location, encryption, and retention.
Pigsty generates these settings from pgbackrest_repo.
See Backup Repository.
Backup Chains and Labels
| Type | Contents | Label suffix |
|---|---|---|
| Full | Complete database-cluster copy | F |
| Differential | Changes since the latest full | D |
| Incremental | Changes since the latest backup of any type | I |
Labels identify the backup chain.
20250715-013657F is a full backup.
20250715-013657F_20250715-013724D and ..._20250715-013730I depend on the full backup identified before the underscore.
Use --set to select the starting backup explicitly.
Otherwise, pgBackRest selects the newest usable set before the target.
Retention: Keeping the Repository Bounded
repo1-retention-full and repo1-retention-full-type (count or time) determine when full backup chains expire.
Dependent differential/incremental backups and WAL needed only by that chain expire with the full backup.
Pigsty enables expire-auto.
Use pig pb expire --plan to preview a manual run.
Time retention defines a minimum window. It does not remove every full backup older than N days. An old full backup expires only when another retained full backup has reached that age. With weekly full backups, a 14-day setting therefore commonly retains three full chains and approximately 14–21 days of history.
WAL Archiving
PostgreSQL calls archive-push when a WAL segment fills or archive_timeout elapses.
During recovery, restore_command calls archive-get.
Pigsty enables asynchronous archiving through /pg/spool.
Thus, a temporary repository delay does not block the primary directly.
Timelines
Each promotion after recovery or failover creates a new timeline.
Older timeline history remains in the repository.
--target-timeline selects the recovery branch and defaults to latest.
See the conceptual PITR mechanism.
What restore Actually Does
restore first reconstructs the data directory.
Pigsty enables --delta, so pgBackRest validates existing files and rewrites only files that differ.
It then writes the recovery state (recovery.signal, restore_command, and recovery_target_*).
WAL replay starts after PostgreSQL starts.
A successful restore command does not complete PITR.
--target-action controls what happens when replay reaches the target: pause, promote, or shutdown.
Observe a Backup Chain
Run read-only info after full, differential, and incremental backups to inspect labels, size, WAL bounds, and references:
A representative sequence looks like:
Pigsty’s Wrapper Layers
| Layer | Interface | What it does |
|---|---|---|
| Cluster orchestration | pg_pitr + pgsql-pitr.yml | Pause HA, stop nodes, render configuration, restore/replay, inspect control data, clean etcd, and rebuild HA |
| Instance orchestration | pig pitr | Preflight, keep one target offline, restore, optionally start PostgreSQL, and leave Patroni stopped for inspection |
| Command primitive | pig pb, pb, pg-backup | Supply stanza/DBSU context and call the corresponding pgBackRest command |
| Engine | pgbackrest | Read /etc/pgbackrest/pgbackrest.conf and perform backup, archive, and restore operations |
Command Primitives
pb is a login-shell function that reads the first stanza from the local configuration and forwards arguments:
pg-backup adds a primary-role check for scheduled use:
pig pb adds stanza detection, DBSU privilege handling, primary checks for backup, and plan/confirmation guards for destructive primitives. See Admin Commands.
Parameter Mapping
pg_pitr field | pig pitr option | pgBackRest option | Meaning |
|---|---|---|---|
cluster | --stanza | --stanza | Source cluster/stanza |
type plus time/xid/lsn/name | corresponding target option | --type + --target | Recovery target |
default | --default | no --type/--target | Replay to archive end |
immediate | --immediate | --type=immediate | Stop at the first consistent point |
exclusive | --exclusive / -X | --target-exclusive | Stop before the target |
action | --target-action | --target-action | pause, promote, or shutdown |
timeline | --target-timeline / -T | --target-timeline | Target timeline |
set | --set / -b | --set | Starting backup set |
db_include / db_exclude | — | --db-include / --db-exclude | Select databases in a physical restore |
link_map | — | --link-map | Remap directory or tablespace links |
process | — | process-max | Parallel restore processes |
data | --data / -D | --pg1-path | Target data directory |
repo | repository number only in pig pitr | rendered repo1-* | Override repository definition in the playbook |
A selective restore is still a physical restore.
Excluded databases receive sparse zeroed files so PostgreSQL can complete recovery.
These databases are inaccessible.
You must remove them explicitly afterward.
This is not a logical subset as with pg_dump.
How Configuration Is Rendered
pgbackrest_method selects the entry that Pigsty writes to /etc/pgbackrest/pgbackrest.conf.
Pigsty converts underscores to hyphens and adds the repo1- prefix to keys.
pgsql-pitr.yml generates a separate temporary configuration at /pg/conf/pitr.conf.
PostgreSQL recovery output goes to /pg/tmp/recovery.log.
Scheduled Backups
Pigsty installs pg_crontab entries for the postgres OS user on every cluster node.
pg-backup checks the current role, so only the primary makes backups.
After promotion, the new primary runs future scheduled backups.
See Backup Policy for frequency and retention design.
Deployment Details
The pg_backup subtask installs and configures pgBackRest, then creates the stanza.
If pgbackrest_init_backup is enabled, it attempts an initial full backup.
It writes /etc/pgbackrest/initial.done only after that backup succeeds.
| Path | Purpose |
|---|---|
/usr/bin/pgbackrest | pgBackRest binary |
/etc/pgbackrest/pgbackrest.conf | Main stanza and repository configuration |
/pg/backup | Local repository path |
/pg/spool | Asynchronous archive spool |
/pg/log/pgbackrest/ | Backup, archive, and restore logs |
/pg/conf/pitr.conf | Temporary PITR configuration |
/pg/tmp/recovery.log | PostgreSQL recovery log |
pgbackrest_exporter exports backup metrics and listens on pgbackrest_exporter_port, which defaults to 9854.
To disable it, use pgbackrest_exporter_enabled.
To customize it, use pgbackrest_exporter_options.
2 - Backup Policy
The chart below shows the “Recovery Window” and “Backup Storage Usage” on one timeline (0~108h).
Use it to compare the two metrics.
The 30-day example uses a database size of 100GB and daily writes of 10GB.
It shows both metrics with weekly full backups, daily incremental backups, and 14-day full-backup retention.
- When: Backup schedule
- Where: Backup repository
- How: Backup method
When to Backup
First, choose when to back up the database. Backup frequency affects recovery time. Recovery replays WAL from the last backup to the target point. More frequent backups reduce the amount of WAL to replay and shorten recovery.
Daily Full Backup
For production databases, we recommend a daily full backup as the simplest initial policy. This is also Pigsty’s default backup policy. Pigsty implements it through crontab.
Assume a database size of 100GB, daily writes of 10GB, and a duration of 1 hour for each full backup. The chart below shows the recovery window and storage usage for this daily full backup policy with a local repository.
The recovery window cycles between 25-49 hours, and storage usage is roughly 2 full backups plus around 2 days of WAL archives.
In practice, prepare at least 3~5 times the base database size as backup disk capacity for the default policy.
tooltip: { trigger: axis, formatter: $fn:tipMerged, axisPointer: { type: line, snap: true, label: { show: false } } }
axisPointer: { link: [ { xAxisIndex: [0, 1] } ] }
legend: { show: false, bottom: 10, itemGap: 18, data: ["Primary Backup", "Secondary Backup", "WAL Archive", "Transient Backup"] }
grid:
- { left: 82, right: "10%", top: 42, height: 218, containLabel: false }
- { left: 82, right: "10%", top: 286, height: 218, containLabel: false }
xAxis:
- type: category
gridIndex: 0
position: bottom
boundaryGap: false
data: [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108]
name: Time h
nameLocation: end
nameGap: 10
nameTextStyle: { align: left, verticalAlign: top, padding: [8, 0, 0, 0] }
axisLabel: { interval: 11, formatter: $fn:fmtHour }
axisLine: { show: true, symbol: [none, arrow], symbolSize: [10, 14], lineStyle: { width: 1.6, color: "#4b5563" } }
axisTick: { show: true, length: 6 }
splitLine: { show: true, lineStyle: { type: dashed, width: 1, opacity: 0.28, color: "#9ca3af" } }
minorTick: { show: true, splitNumber: 12, length: 3 }
minorSplitLine: { show: true, lineStyle: { type: dotted, width: 1, opacity: 0.14, color: "#9ca3af" } }
- type: category
gridIndex: 1
position: top
boundaryGap: true
data: [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108]
axisLabel: { show: false }
axisLine: { show: true, lineStyle: { width: 1.6, color: "#4b5563" } }
axisTick: { show: true, length: 6 }
splitLine: { show: true, lineStyle: { type: dashed, width: 1, opacity: 0.22, color: "#9ca3af" } }
yAxis:
- type: value
gridIndex: 0
min: 0
max: 52
interval: 5
name: Recovery Window h
nameLocation: end
nameRotate: 0
nameGap: 8
nameTextStyle: { align: left, verticalAlign: bottom, padding: [0, 0, 8, 4] }
axisLabel: { formatter: $fn:fmtWin }
axisLine: { show: true, symbol: [none, arrow], symbolSize: [10, 14], lineStyle: { width: 1.6, color: "#4b5563" } }
axisTick: { show: true, length: 6 }
splitLine: { show: true, lineStyle: { type: dashed, width: 1, opacity: 0.35, color: "#9ca3af" } }
minorTick: { show: true, splitNumber: 5, length: 3 }
minorSplitLine: { show: true, lineStyle: { type: dotted, width: 1, opacity: 0.18, color: "#9ca3af" } }
- type: value
gridIndex: 1
min: 0
max: 350
interval: 50
inverse: true
name: Backup Storage GB
nameLocation: end
nameRotate: 0
nameGap: 8
nameTextStyle: { align: left, verticalAlign: top, padding: [10, 0, 0, 4] }
axisLabel: { formatter: $fn:fmtGbTick }
axisLine: { show: true, symbol: [arrow, none], symbolSize: [10, 14], lineStyle: { width: 1.6, color: "#4b5563" } }
axisTick: { show: true, length: 6 }
splitLine: { show: true, lineStyle: { type: dashed, width: 1, opacity: 0.32, color: "#9ca3af" } }
series: [ { name: Recovery Window, type: line, smooth: false, symbol: none, showSymbol: false, xAxisIndex: 0, yAxisIndex: 0, lineStyle: { width: 3, color: "#f2a000" }, itemStyle: { color: "#f2a000" }, data: [[0,0],[1,0],[1,1],[2,2],[3,3],[4,4],[5,5],[6,6],[7,7],[8,8],[9,9],[10,10],[11,11],[12,12],[13,13],[14,14],[15,15],[16,16],[17,17],[18,18],[19,19],[20,20],[21,21],[22,22],[23,23],[24,24],[25,25],[26,26],[27,27],[28,28],[29,29],[30,30],[31,31],[32,32],[33,33],[34,34],[35,35],[36,36],[37,37],[38,38],[39,39],[40,40],[41,41],[42,42],[43,43],[44,44],[45,45],[46,46],[47,47],[48,48],[49,49],[49,25],[50,26],[51,27],[52,28],[53,29],[54,30],[55,31],[56,32],[57,33],[58,34],[59,35],[60,36],[61,37],[62,38],[63,39],[64,40],[65,41],[66,42],[67,43],[68,44],[69,45],[70,46],[71,47],[72,48],[73,49],[73,25],[74,26],[75,27],[76,28],[77,29],[78,30],[79,31],[80,32],[81,33],[82,34],[83,35],[84,36],[85,37],[86,38],[87,39],[88,40],[89,41],[90,42],[91,43],[92,44],[93,45],[94,46],[95,47],[96,48],[97,49],[97,25],[98,26],[99,27],[100,28],[101,29],[102,30],[103,31],[104,32],[105,33],[106,34],[107,35],[108,36]], markLine: { symbol: none, label: { show: false }, data: [ { xAxis: 0, lineStyle: { color: "#59a14f", type: "solid", width: 1.4, opacity: 0.75 } }, { xAxis: 24, lineStyle: { color: "#59a14f", type: "solid", width: 1.4, opacity: 0.75 } }, { xAxis: 48, lineStyle: { color: "#59a14f", type: "solid", width: 1.4, opacity: 0.75 } }, { xAxis: 72, lineStyle: { color: "#59a14f", type: "solid", width: 1.4, opacity: 0.75 } }, { xAxis: 96, lineStyle: { color: "#59a14f", type: "solid", width: 1.4, opacity: 0.75 } }, { xAxis: 1, lineStyle: { color: "#336791", type: "solid", width: 1.4, opacity: 0.8 } }, { xAxis: 25, lineStyle: { color: "#336791", type: "solid", width: 1.4, opacity: 0.8 } }, { xAxis: 49, lineStyle: { color: "#336791", type: "solid", width: 1.4, opacity: 0.8 } }, { xAxis: 73, lineStyle: { color: "#336791", type: "solid", width: 1.4, opacity: 0.8 } }, { xAxis: 97, lineStyle: { color: "#336791", type: "solid", width: 1.4, opacity: 0.8 } }, { yAxis: 25, label: { show: true, formatter: "lower 25h", position: "end", distance: 12, color: "#2563eb" }, lineStyle: { color: "#2563eb", type: "dashdot", width: 1.4, opacity: 0.75 } }, { yAxis: 49, label: { show: true, formatter: "upper 49h", position: "end", distance: 12, color: "#7c3aed" }, lineStyle: { color: "#7c3aed", type: "dashdot", width: 1.4, opacity: 0.75 } } ] } }, { name: Primary Backup, type: bar, stack: used, xAxisIndex: 1, yAxisIndex: 1, barWidth: 5, itemStyle: { color: "#59a14f" }, data: [0,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100] }, { name: Secondary Backup, type: bar, stack: used, xAxisIndex: 1, yAxisIndex: 1, barWidth: 5, itemStyle: { color: "#4e79a7" }, data: [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100] }, { name: WAL Archive, type: bar, stack: used, xAxisIndex: 1, yAxisIndex: 1, barWidth: 5, itemStyle: { color: "#edc949" }, data: [0,0,0.42,0.83,1.25,1.67,2.08,2.5,2.92,3.33,3.75,4.17,4.58,5,5.42,5.83,6.25,6.67,7.08,7.5,7.92,8.33,8.75,9.17,9.58,10,10.42,10.83,11.25,11.67,12.08,12.5,12.92,13.33,13.75,14.17,14.58,15,15.42,15.83,16.25,16.67,17.08,17.5,17.92,18.33,18.75,19.17,19.58,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20,20] }, { name: Transient Backup, type: bar, stack: used, xAxisIndex: 1, yAxisIndex: 1, barWidth: 5, itemStyle: { color: "#9ca3af", opacity: 0.75 }, data: [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,100,0,0,0,0,0,0,0,0,0,0,0,0] } ]Full + Incremental Backup
You can optimize backup space usage by adjusting these parameters.
If using Silo / S3 as a centralized backup repository, storage is no longer limited by the local disk. In this case, consider using full + incremental backup with a 2-week retention policy:
With weekly full backups and 14-day time-based retention, the steady-state recovery window is approximately 14–21 days. The exact window depends on successful backups and WAL archiving.
For a database of 100GB with daily writes of 10GB, the following chart shows backup size:
tooltip: { trigger: axis, formatter: $fn:tipMerged30, axisPointer: { type: line, snap: true, label: { show: false } } }
axisPointer: { link: [ { xAxisIndex: [0, 1] } ] }
legend: { show: false, bottom: 10, itemGap: 18, data: ["Primary Backup", "Secondary Backup", "Incremental Backup", "WAL Archive", "Transient Backup"] }
grid:
- { left: 82, right: "10%", top: 42, height: 218, containLabel: false }
- { left: 82, right: "10%", top: 302, height: 218, containLabel: false }
xAxis:
- type: value
gridIndex: 0
position: bottom
boundaryGap: false
min: 0
max: 31
interval: 1
name: Time
nameLocation: end
nameGap: 10
nameTextStyle: { align: left, verticalAlign: top, padding: [8, 0, 0, 0] }
axisLabel: { formatter: $fn:fmtDay30 }
axisLine: { show: true, symbol: [none, arrow], symbolSize: [10, 14], lineStyle: { width: 1.6, color: "#4b5563" } }
axisTick: { show: true, length: 6 }
splitLine: { show: true, lineStyle: { type: dashed, width: 1, opacity: 0.28, color: "#9ca3af" } }
minorTick: { show: true, splitNumber: 4, length: 3 }
minorSplitLine: { show: true, lineStyle: { type: dotted, width: 1, opacity: 0.14, color: "#9ca3af" } }
- type: category
gridIndex: 1
position: top
boundaryGap: true
z: 10
data: [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30]
axisLabel: { show: false }
axisLine: { show: true, lineStyle: { width: 1.6, color: "#4b5563" } }
axisTick: { show: true, alignWithLabel: true, length: 6 }
splitLine: { show: true, lineStyle: { type: dashed, width: 1, opacity: 0.22, color: "#9ca3af" } }
yAxis:
- type: value
gridIndex: 0
min: 0
max: 360
interval: 48
name: Recovery Window h
nameLocation: end
nameRotate: 0
nameGap: 8
nameTextStyle: { align: left, verticalAlign: bottom, padding: [0, 0, 8, 4] }
axisLabel: { formatter: $fn:fmtWin30 }
axisLine: { show: true, symbol: [none, arrow], symbolSize: [10, 14], lineStyle: { width: 1.6, color: "#4b5563" } }
axisTick: { show: true, length: 6 }
splitLine: { show: true, lineStyle: { type: dashed, width: 1, opacity: 0.35, color: "#9ca3af" } }
minorTick: { show: true, splitNumber: 4, length: 3 }
minorSplitLine: { show: true, lineStyle: { type: dotted, width: 1, opacity: 0.18, color: "#9ca3af" } }
- type: value
gridIndex: 1
min: 0
max: 600
interval: 50
inverse: true
z: 10
name: Storage GB
nameLocation: end
nameRotate: 0
nameGap: 8
nameTextStyle: { align: left, verticalAlign: top, padding: [10, 0, 0, 4] }
axisLabel: { formatter: $fn:fmtGbTick30 }
axisLine: { show: true, symbol: [arrow, none], symbolSize: [10, 14], lineStyle: { width: 1.6, color: "#4b5563" } }
axisTick: { show: true, length: 6 }
splitLine: { show: true, lineStyle: { type: dashed, width: 1, opacity: 0.32, color: "#9ca3af" } }
series:
- { name: Recovery Window, type: line, smooth: false, symbol: none, showSymbol: false, xAxisIndex: 0, yAxisIndex: 0, lineStyle: { width: 3, color: "#f28e2c" }, itemStyle: { color: "#f28e2c" }, data: [[1,24],[2,48],[3,72],[4,96],[5,120],[6,144],[7,168],[8,192],[9,216],[10,240],[11,264],[12,288],[13,312],[14,336],[14,168],[15,192],[16,216],[17,240],[18,264],[19,288],[20,312],[21,336],[21,168],[22,192],[23,216],[24,240],[25,264],[26,288],[27,312],[28,336],[28,168],[29,192],[30,216]], markLine: { symbol: none, label: { show: false }, data: [ { xAxis: 7, lineStyle: { color: "#336791", type: "solid", width: 1.4, opacity: 0.65 } }, { xAxis: 14, lineStyle: { color: "#336791", type: "solid", width: 1.4, opacity: 0.65 } }, { xAxis: 21, lineStyle: { color: "#336791", type: "solid", width: 1.4, opacity: 0.65 } }, { xAxis: 28, lineStyle: { color: "#336791", type: "solid", width: 1.4, opacity: 0.65 } }, { yAxis: 168, label: { show: true, formatter: "lower 7d", position: "end", distance: 12, color: "#2563eb" }, lineStyle: { color: "#2563eb", type: "dashdot", width: 1.4, opacity: 0.72 } }, { yAxis: 336, label: { show: true, formatter: "upper 14d", position: "end", distance: 12, color: "#7c3aed" }, lineStyle: { color: "#7c3aed", type: "dashdot", width: 1.4, opacity: 0.72 } } ] } }
- { name: Primary Backup, type: bar, stack: used, xAxisIndex: 1, yAxisIndex: 1, barWidth: 16, itemStyle: { color: "#59a14f" }, data: [100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100] }
- { name: Secondary Backup, type: bar, stack: used, xAxisIndex: 1, yAxisIndex: 1, barWidth: 16, itemStyle: { color: "#4e79a7" }, data: [0,0,0,0,0,0,0,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100] }
- { name: Incremental Backup, type: bar, stack: used, xAxisIndex: 1, yAxisIndex: 1, barWidth: 16, itemStyle: { color: "#76b7b2" }, data: [0,10,20,30,40,50,60,70,80,90,100,110,120,130,70,80,90,100,110,120,130,70,80,90,100,110,120,130,70,80] }
- { name: WAL Archive, type: bar, stack: used, xAxisIndex: 1, yAxisIndex: 1, barWidth: 16, itemStyle: { color: "#edc949" }, data: [10,20,30,40,50,60,70,80,90,100,110,120,130,140,80,90,100,110,120,130,140,80,90,100,110,120,130,140,80,90] }
- { name: Transient Backup, type: bar, stack: used, xAxisIndex: 1, yAxisIndex: 1, barWidth: 16, itemStyle: { color: "#9ca3af", opacity: 0.75 }, data: [0,0,0,0,0,0,0,0,0,0,0,0,0,110,0,0,0,0,0,0,110,0,0,0,0,0,0,110,0,0] }Backup Location
Pigsty provides two default backup repository definitions: local and minio.
local: Default option, uses local/pg/backupdirectory (symlink topg_fs_backup:/data/backups)minio: Uses SNSD single-node MinIO cluster (supported by Pigsty, but not enabled by default)
3 - Backup Repository
Two parameters select the backup location.
pgbackrest_repo defines candidate repositories.
pgbackrest_method selects one repository.
Pigsty converts repository keys to pgBackRest repo1-* options, so you can use any supported pgBackRest repository option directly.
Pigsty v4.5.0 writes only the entry selected by pgbackrest_method as repo1.
Multiple candidate keys do not enable multi-repository backup.
Default Repositories
Pigsty supplies two definitions: local and minio.
localis the default./pg/backuppoints atpg_fs_backup,/data/backupsby default.miniouses Silo from the MINIO module or another compatible S3 service. It is supported but not selected by default.
The presets use different settings.
local provides simple, fast restores with count-based retention and no encryption or bundling.
minio enables AES-256-CBC encryption, bundles small files, uses block incremental backup, and retains full backups by time.
For a remote repository, replace both cipher_pass and s3_key_secret.
pgBackRest and S3User.Backup are public example defaults.
If you lose the encryption passphrase, you cannot recover the repository.
Store the passphrase separately from the backups under controlled recovery procedures.
See Deployment Security.
Retention Policy
pgBackRest applies retention after each backup (expire-auto).
When a full backup expires, its dependent differential/incremental backups expire with it.
WAL needed only by that chain also expires.
retention_full_type: countwithretention_full: 2keeps the two newest full chains. A third chain can exist briefly while a new full backup completes.retention_full_type: timewithretention_full: 14defines a minimum time window. An old full backup expires only when another full backup is at least 14 days old. With weekly full backups, this usually retains three chains and approximately 14–21 days of recovery history.
See Backup Policy for recovery-window and space calculations, and Admin Commands for a safe expiration preview.
Use a Silo Repository
The MINIO module currently deploys Silo, an S3-compatible object store.
It provides an independent disaster-recovery copy only if it is outside the database host or site failure domain.
Deploy it, then select the minio preset:
By default, the preset uses the HTTPS endpoint sss.pigsty and validates it with /etc/pki/ca.crt.
MINIO initialization creates the default pgsql bucket and pgbackrest user.
For production, deploy and test a fault-tolerant object store with multiple nodes and drives.
See MINIO Configuration.
The minio preset also supports servers that Pigsty does not manage, including independent MinIO, RustFS, and other compatible implementations.
For these servers, installation, upgrades, certificates, and lifecycle management remain outside the MINIO role support scope.
Use S3 or Cloud Object Storage
For a single database node, an off-host cloud object store is often the most useful repository. Define a new entry and select it:
pgBackRest also supports Azure, GCS, and SFTP repositories.
Share a Repository Across Clusters
A centralized repository can serve several PostgreSQL clusters.
Each stanza takes its name from pg_cluster and isolates one cluster’s backup and archive history.
This also enables cross-cluster restore.
Cluster names must therefore be globally unique within a shared repository, even across otherwise separate deployment environments.
Repository Versioning
Object-store versioning can preserve earlier versions after an overwrite or deletion. These versions share the same storage system and control plane. Thus, versioning does not replace an independent off-site or offline copy. Enable versioning when you create a bucket:
pgBackRest’s repo-target-time option can read the repository as it existed at an earlier time when the backend retains those versions.
Repository Locking
Some S3-compatible services support Object Lock/WORM. You cannot change or permanently delete a retained object version until its retention period ends. A normal delete can still create a delete marker that hides the current object. Historical versions remain and continue to consume capacity.
The lock flag enables versioning and lock capability only when Pigsty creates the bucket:
The flag does not set a default retention period.
Configure GOVERNANCE or COMPLIANCE retention with mcli retention set or the storage console.
Then verify it with mcli retention info.
A sufficiently privileged principal can bypass GOVERNANCE.
Even root cannot shorten COMPLIANCE retention.
Locking changes expiration and stanza deletion. pgBackRest may expire objects logically while retained historical versions remain until their deadline. Before you enable locking, test backup, expiration, delete-marker cleanup, and version recovery on a non-production bucket.
Switch Repositories
After you change a repository definition or pgbackrest_method, generate the configuration again and initialize the stanza.
Promptly create a recovery point in the new repository:
Pigsty does not migrate existing backups automatically.
While you retain the old repository, you can select it as a restore source through pg_pitr.repo.
4 - Admin Commands
On a database node, run backup commands as the database superuser (pg_dbsu, which defaults to postgres.
Use one of these interfaces:
pig pb: the PIG CLI wrapper, with automatic stanza detection, DBSU switching, and safety checks; this is the recommended interfacepb: a login-shell function that supplies--stanzaand forwards arguments to pgBackRestpgbackrest: the native command; see the pgBackRest command reference
Command Overview
| PIG command | Alias | Native pgBackRest command | Purpose |
|---|---|---|---|
pig pb info | i | info | Show backup and archive status |
pig pb list | ls | — | List repositories, stanzas, or backup sets |
pig pb backup [full/diff/incr] | b | backup | Create a backup after checking the primary role |
pig pb restore | r | restore | Low-level restore primitive; see Restore Operations |
pig pb expire | e | expire | Expire backups according to retention (--plan previews) |
pig pb create | c | stanza-create | Create a stanza |
pig pb upgrade | u | stanza-upgrade | Upgrade a stanza after a major-version change or clone |
pig pb delete | d | stanza-delete | Delete a stanza and all of its backups |
pig pb check | ck | check | Verify configuration, repository access, and archiving |
pig pb start | up | start | Re-enable pgBackRest operations |
pig pb stop | dw | stop | Stop new pgBackRest operations |
pig pb log [list/show/tail] | l | — | Inspect pgBackRest logs |
Enable Backup
If pgbackrest_enabled is true at cluster creation, Pigsty enables backup automatically.
If backup was disabled at creation, or repository settings have changed, run the pg_backup subtask:
After cluster initialization, Pigsty attempts an initial full backup.
It writes /etc/pgbackrest/initial.done only after the backup command succeeds.
If the attempt fails, the playbook ignores the failure and leaves no marker.
This file only prevents the initialization task from repeating.
Always verify repository state with pig pb info or pgbackrest info.
Define scheduled backups with pg_crontab; see Backup Policy.
Remove Backup
To remove only a backup stanza, use pig pb delete as the preferred interface.
It asks for interactive confirmation.
With a multi-stanza configuration, you must also specify the target explicitly.
Verify the exact target first:
By default, pgsql-rm.yml also tries to delete the cluster’s backup stanza when it removes a primary instance (pg_role = primary).
Each command below changes or deletes state.
Do not run a command only because it appears in an example:
Before execution, verify a recent usable backup and record the recovery requirement.
Have the operator re-enter the exact cluster/stanza name.
To preserve backups when you remove the cluster, set pg_rm_backup to false.
pgsql-rm.yml -t pg_backup forcibly runs pgbackrest stanza-delete on the primary.
In local mode, it removes the local repository directory.
It then removes the pgBackRest configuration and initial-backup marker.
The task ignores some deletion errors.
Thus, a successful playbook result does not prove that it physically removed the repository objects.
To delete only a stanza, prefer pig pb delete, which provides a plan and requires confirmation.
With object versioning and object-lock retention, deletion may create a delete marker. Locked historical versions continue to consume storage until their retention period expires.
Deleting backups can permanently destroy recovery options.
Confirm the cluster/stanza. Verify a recent backup and an alternative recovery copy.
Keep the pig pb info output and deletion plan as an audit record.
Manual Backup
You can start a backup outside the crontab schedule.
Both pg-backup and pig pb backup check that the current instance is primary.
They exit on a replica:
Backup consumes disk I/O and network bandwidth. Pigsty limits parallelism to a small number of processes. Still, you should schedule production backups for periods with low traffic.
Inspect Backups
pb info shows backups and WAL archive status for the current stanza:
Backup labels that end in F, D, and I identify full, differential, and incremental backups.
The portion before an underscore identifies the full backup at the start of that chain.
The WAL archive range and oldest usable full backup define the recovery window.
The pgbackrest_exporter service on port 9854 continuously exports metrics such as the latest backup time, type, size, and error status.
Expire Old Backups
pgBackRest applies the retention policy automatically after backups (expire-auto).
To preview or run expiration manually, use:
Stanza Management
A stanza records a cluster’s backup identity, including its system identifier and major version. You may need to manage it manually:
The usual manual upgrade case is post-clone cleanup.
After you restore another cluster’s backup into a new cluster, update the stanza identity before you write new backups.
Check and Control
check verifies that WAL can reach the repository through the archive path.
It is not a purely local or read-only operation.
Logs
For pgsql-pitr.yml, PostgreSQL writes recovery output to /pg/tmp/recovery.log.
Alternative Backup Tools
pg-basebackup
The legacy /pg/bin/pg-basebackup script creates a single-file physical backup with native pg_basebackup.
It uses an lz4-compressed tar stream and writes to /pg/backup by default.
Use it only for a simple local copy when a pgBackRest repository is unavailable:
pg-basebackup -e uses the obsolete OpenSSL RC4 cipher. You must not treat it as confidentiality protection.
For encrypted backups, use a pgBackRest repository with AES-256 (cipher_type: aes-256-cbc).
Logical Backup
Logical backups made with pg_dump cannot provide PITR.
They are suitable for cross-major-version migration, partial exports, and long-term logical snapshots.
Production recovery plans commonly use logical and physical backups together.
See the PostgreSQL documentation.
5 - Restore Operations
Pigsty provides three restore interfaces. They use the same parameter semantics for different scopes:
| Entry point | Use case | What it controls |
|---|---|---|
pgsql-pitr.yml | Production cluster recovery | HA pause, multiple nodes, etcd cleanup, restore, and restart |
pig pitr | A local database node | Single-instance orchestration without the admin node |
pig pb restore | An instance not managed by Patroni | A direct pgBackRest restore wrapper |
For a sandbox drill, see Manual Recovery. To recover into another cluster without changes to production, see Clone a PG Cluster.
pgsql-pitr.yml pauses HA and stops Patroni/PostgreSQL.
It overwrites the target data directory with pgbackrest --force restore.
It then deletes the target cluster’s etcd prefix and rebuilds HA.
The playbook prints a plan but does not wait for confirmation.
Before a real restore, inspect the topology with pig pg list <target-cluster>.
Verify a recent usable backup and recovery window with pig pb info.
Have the operator state and confirm the exact target cluster and recovery point.
For production recovery, schedule a maintenance window and retain an independently verified backup.
Quick Start
To roll pg-meta back to an earlier time, declare pg_pitr:
Before recovery, verify the same target with these checks, then run the playbook:
You can pass the same object temporarily on the command line:
The -e value must be valid JSON: quote keys and string values, for example {"pg_pitr": {"time": "...", "archive": true}}.
Booleans are not quoted. Invalid quoting can fail parsing or silently produce the wrong value.
The playbook pauses Patroni HA, stops the cluster, and performs a delta pgBackRest restore.
It starts PostgreSQL and waits for a consistent recovery state.
It then prints control data, removes old etcd metadata, and starts the cluster under Patroni again.
The playbook first prints the source, target, and restore command, but does not ask for interactive approval.
For a one-shot targeted recovery, explicitly set action: promote.
To inspect data at the target, use step-by-step execution with action: pause.
Recovery Targets
pg_pitr supports six recovery target forms.
The four target values are mutually exclusive.
With no target, recovery replays all archived WAL to the latest available state.
Pigsty uses the internal type default for this mode.
immediate stops at the first consistent point.
Use it to get a usable instance as quickly as possible or to test a backup.
Recover by Time
Use a valid PostgreSQL TIMESTAMP.
We strongly recommend an explicit time zone:
Recover by Name
Create an unambiguous marker before a risky change with pg_create_restore_point:
Recover by Transaction ID
If monitoring or the CSVLOG TXID field identifies the offending transaction, use exclusive to stop before that transaction:
Recover by LSN
An LSN identifies a position in the WAL stream.
Pigsty’s PG LSN dashboard panel also shows it.
If the required position is on a particular timeline, set timeline.
The default is latest.
Targets are inclusive by default, so recovery replays the target transaction.
exclusive: true excludes the exact target.
It applies only to time, xid, and lsn.
It maps to PostgreSQL’s recovery_target_inclusive.
Recovery Source
By default, recovery uses the target cluster’s own backup. Use these three fields to select another source:
cluster: the source stanza, including another cluster in a shared repositoryrepo: a temporary repository definition in the same format as apgbackrest_repoentryset: a specific backup label; otherwise pgBackRest selects a suitable set
For example, recover pg-meta2 from pg-meta:
Step-by-Step Execution
During an incident, use tags to require a human decision between stages. Confirm the backup, recovery point, and exact target. Then run the stages in order:
After down, confirm that the processes have stopped.
After pitr, inspect /pg/tmp/recovery.log and query the recovery state.
Then perform the narrowly authorized checks of business data.
pg_controldata /pg/data reports checkpoint and timeline metadata.
This information alone does not prove that recovery reached a time, XID, or LSN target.
With action: pause, validate the data before promotion, then run up.
If the target is wrong, adjust pg_pitr and repeat pitr before up.
pause or shutdown provides an effective human decision point only in this staged procedure.
For a one-shot targeted recovery, explicitly set action: promote.
With backup: true, the playbook moves the current data directory to /pg/data-backup.
It deletes any existing /pg/data-backup first.
The staged procedure is supported. A restore with backup: true is not generally idempotent.
PITR Parameter Definition
Declare the target, action, and treatment of existing data explicitly:
See Parameter Mapping for the corresponding pgBackRest options.
Single Instance: pig pitr
pig pitr performs a local-node procedure without Ansible.
It validates the target, stanza, and backup, then stops Patroni/PostgreSQL and restores the data.
It can optionally start PostgreSQL.
It then prints instructions for the next steps.
Use -b/--set for a backup set, -T/--target-timeline for a timeline, and --target-action for the action after recovery reaches the target.
For a side restore, use -D/--data with --no-restart.
The command normally attempts a fast stop and aborts if that fails.
Only explicit --force-stop permits immediate shutdown and a kill fallback.
For managed PGDATA, the command leaves Patroni stopped.
Validate the instance before pig pt start.
The command does not remove etcd metadata, rebuild replicas, or rejoin the instance to an HA cluster.
Primitive: pig pb restore
Use pig pb restore for an instance that Patroni does not manage, or that you deliberately removed from management.
The command directly wraps pgbackrest restore.
It validates the environment and requires PostgreSQL to be stopped.
It then shows the plan and asks for confirmation.
It rejects a live Patroni-managed target because Patroni could restart a partially restored instance.
It also rejects any running PostgreSQL target.
Arguments after --, such as --tablespace-map or --link-all, pass through to pgBackRest.
You cannot override wrapped options, such as target, stanza, and repository, after --.
Post-Recovery
After restore:
Verify the recovery state and the smallest authorized set of application checks.
After a cross-cluster clone, complete stanza cleanup. Create a full backup on the new timeline as soon as practical:
If exploratory recovery used
archive: false, restore archiving. Thearchive_modesetting requires a restart. First confirm the maintenance window, current primary, and replication state. Then obtain explicit approval for the restart:
6 - Clone a PG Cluster
Cloning restores production history into another cluster and leaves the production cluster unchanged. This makes cloning a useful recovery method with less risk to production. You can recover accidentally deleted data from the clone, validate backups in a drill, or inspect a historical state. You can also reset a test environment to a production snapshot.
The target must have access to the source backup repository and use a compatible PostgreSQL major version. Recovery may overwrite the target. In a shared Silo/S3 repository, a stanza isolates each cluster’s backups. Targets with the required credentials can access these backups.
Inspect the target topology with pig pg list <target-cluster>.
Verify the source stanza’s recent backups and recovery window with pig pb info.
Before the restore, have the operator confirm the exact source cluster, target cluster, and recovery point.
Recovery overwrites existing data on the target.
Production work requires a maintenance window and an independently verified backup.
Clone an Existing Cluster
Assume the four-node sandbox contains pg-meta and pg-test, sharing a Silo repository.
To reset pg-test to the latest state of pg-meta, point pg_pitr to the pg-meta stanza:
Add a recovery target to clone any state inside the recovery window. For example, reset to 15:30 on December 26, 2025:
These cross-cluster examples set archive: false to keep the exploratory recovery from archiving under the target stanza.
After Patroni takes control, complete the stanza and archive cleanup below.
The target may also be a newly initialized empty cluster, such as pg-meta2.
Use the normal cluster creation procedure to create it.
Then perform cross-cluster PITR.
pgBackRest restore uses delta mode and rewrites only files that differ from the backup. Repeated drills, or a target already synchronized through a standby cluster, can therefore restore much faster than a first full restore.
For accidental deletion, validate the clone and use pg_dump to export only the affected tables or database back into production.
An in-place rollback of the entire production cluster should be the last resort, not the first response.
Post-Clone Cleanup
The clone contains the source cluster’s data, while the target stanza may still record the target’s old PostgreSQL system identifier. pgBackRest refuses a backup when the identifiers do not match, preventing the new cluster from contaminating the source history.
After you validate the clone, complete these steps. A cluster restart changes the running service. First inspect the primary and replication state, schedule the maintenance window, and obtain explicit approval.
Until you complete these steps, scheduled backups can fail the identity check.
A clone restored with archive: false produces no new WAL archive:
Rebuild Backup Identity
stanza-upgrade lets the new cluster continue writing under its existing stanza.
If the clone needs a completely independent backup history, rebuild the stanza instead.
Declarative workflow:
Equivalent low-level workflow:
Before deletion, check recent backups and retain any required independent recovery copy.
Have the operator confirm the exact pg-test stanza.
Object-locked versions can remain and continue to consume storage.
A successful deletion command does not prove that it physically erased every underlying version.
Online Copies: Standby Clusters
A PITR clone is a static snapshot. Use a streaming-replication standby cluster for an online copy that follows the source continuously. Use a delayed cluster for a fixed rollback window, such as one hour.
The three methods complement each other. Standby clusters provide a live copy. Delayed clusters preserve a fixed delay. PITR clones provide any historical state inside the recovery window and do not require an existing online replica.
Restore Drills
A clone provides an end-to-end restore drill without changes to production. It overwrites the designated drill target. Run one quarterly and after major backup changes:
- Select a point inside the production recovery window.
- Restore it into the drill cluster and record elapsed time as the measured PITR RTO.
- Validate integrity with authorized row-count checks, critical-table checks, and application connectivity.
- Complete post-clone cleanup and verify that the drill cluster can create a new backup.
- Record timing, failures, and any difference between the runbook and reality.
See Manual Recovery for a sandbox exercise using pgBackRest primitives, or Fork an Instance for an XFS snapshot-based local test copy.