Deployment

• 1: Install Pigsty for Production
• 2: Prepare Resources for Serious Deployment
• 3: Planning Architecture and Nodes
• 4: Setup Admin User and Privileges
• 5: Sandbox
• 6: Vagrant
• 7: OpenTofu
• 8: Security Considerations

1 - Install Pigsty for Production

This guide describes production deployment on multiple nodes. For single-node demonstrations or development, use Getting Started.

Summary

1. Prepare the nodes according to your architecture plan.
2. Install a supported Linux operating system.
3. Configure SSH access.
4. Use an administration account with passwordless ssh and sudo access.
5. Run the installation command:

curl -fsSL https://repo.pigsty.io/get | bash;         # International
curl -fsSL https://repo.pigsty.cc/get | bash;         # Backup Mirror

The installation script downloads and extracts Pigsty into your home directory. It also installs the dependencies. Then complete configuration and deployment.

Before you run deploy.yml, examine and edit pigsty.yml. This file is the configuration inventory.

cd ~/pigsty      # Enter Pigsty directory
./configure -g   # Generate config file (optional, skip if you know how to configure)
./deploy.yml     # Execute deployment playbook based on generated config

After deployment, open the Web UI through the IP address or domain on ports 80/443. Connect to PostgreSQL on port 5432.

Installation usually takes 3-10 minutes, depending on server resources and network speed. Offline installation can reduce this time. If you do not need monitoring, Slim Installation installs fewer components and takes less time.

Example: 20-node production simulation on Ubuntu 24.04, x86_64

demo/install-simu.cast — /demo/install-simu.cast

Prepare

Use this checklist to complete the deployment preparations:

Install

Use one of these commands to install the Pigsty source package into ~/pigsty. This method is recommended and also installs Ansible and other deployment dependencies.

curl -fsSL https://repo.pigsty.io/get | bash            # Install current default version
curl -fsSL https://repo.pigsty.io/get | bash -s v4.5.0  # Explicitly install the current public stable release

curl -fsSL https://repo.pigsty.cc/get | bash            # Install current default version
curl -fsSL https://repo.pigsty.cc/get | bash -s v4.5.0  # Explicitly install the current public stable release

You can instead download a release or clone the source repository. If you use git, select a released version before deployment:

git clone https://github.com/pgsty/pigsty; cd pigsty;
git checkout v4.5.0;  # Always checkout a released tag when using git

After a manual download or clone, run bootstrap to install Ansible and the other dependencies. You can also install them yourself.

./bootstrap           # Install ansible for subsequent deployment

Configure

The inventory defines the deployment configuration. Pigsty uses pigsty.yml as the default configuration file. Edit it to define the required deployment.

The configure script is an optional configuration wizard. It generates an inventory with defaults for your environment:

./configure -g                # Use wizard to generate config with random passwords

The default output file is ~/pigsty/pigsty.yml. Examine this file and make the necessary changes before deployment.

Use the configuration templates as examples. You can also edit pigsty.yml directly without the wizard:

./configure -c ha/full -g       # Use 4-node sandbox template
./configure -c ha/trio -g       # Use 3-node minimal HA template
./configure -c ha/dual -g -v 18 # Use 2-node semi-HA template with PG 18
./configure -c ha/simu -s       # Use 20-node production simulation, skip IP check, no random passwords

vagrant@meta:~/pigsty$ ./configure
configure pigsty v4.5.0 begin
[ OK ] region = china
[ OK ] kernel  = Linux
[ OK ] machine = x86_64
[ OK ] package = deb,apt
[ OK ] vendor  = ubuntu (Ubuntu)
[ OK ] version = 22 (22.04)
[ OK ] sudo = vagrant ok
[ OK ] ssh = [email protected] ok
[WARN] Multiple IP address candidates found:
    (1) 192.168.121.38	    inet 192.168.121.38/24 metric 100 brd 192.168.121.255 scope global dynamic eth0
    (2) 10.10.10.10	    inet 10.10.10.10/24 brd 10.10.10.255 scope global eth1
[ OK ] primary_ip = 10.10.10.10 (from demo)
[ OK ] admin = [email protected] ok
[ OK ] mode = meta (ubuntu22.04)
[ OK ] locale  = C.UTF-8
[ OK ] ansible = ready
[ OK ] pigsty configured
[WARN] don't forget to check it and change passwords!
proceed with ./deploy.yml

The wizard replaces only the current node IP address. Use -s to skip this replacement. For multiple nodes, replace the other IP addresses manually. Change default passwords, add nodes, and adjust other configuration values for your environment.

Common configure parameters:

If the node has multiple IP addresses, select its primary address with -i|--ip <ipaddr> or enter it interactively. The script replaces 10.10.10.10 with the current node’s primary IPv4 address. Use a static private address. Do not use a public IP address.

Review the generated ~/pigsty/pigsty.yml before deployment.

Deploy

The deploy.yml playbook applies the configuration to all target nodes:

./deploy.yml     # Deploy core modules on all target nodes at once

......

TASK [pgsql : pgsql init done] *************************************************
ok: [10.10.10.11] => {
    "msg": "postgres://10.10.10.11/postgres | meta  | dbuser_meta dbuser_view "
}
......

TASK [pg_monitor : load grafana datasource meta] *******************************
changed: [10.10.10.11]

PLAY RECAP *********************************************************************
10.10.10.11                : ok=302  changed=232  unreachable=0    failed=0    skipped=65   rescued=0    ignored=1
localhost                  : ok=6    changed=3    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0

Interface

The 4-node template creates this deployment structure:

The INFRA module supplies the Web UI through Nginx on ports 80/443. The PGSQL module supplies a PostgreSQL server on port 5432. You can also connect through PgBouncer or HAProxy.

For production HA clusters, use service access to route traffic automatically.

More

After installation, open the Web UI and connect to PostgreSQL on port 5432. To deploy and monitor additional clusters, add their definitions to the inventory. Then run the applicable commands:

bin/node-add   pg-test      # Add pg-test cluster's 3 nodes to Pigsty management
bin/pgsql-add  pg-test      # Initialize a 3-node pg-test HA PG cluster
bin/redis-add  redis-ms     # Initialize Redis cluster: redis-ms

Most modules require the NODE module first. See the module reference. Available modules include PGSQL, INFRA, NODE, ETCD, MINIO, REDIS, and DOCKER.

2 - Prepare Resources for Serious Deployment

Pigsty runs on nodes, which can be physical machines or virtual machines. This document describes the planning and preparation for deployment.

Node

Pigsty runs on the Linux kernel with x86_64 or aarch64 architecture. A node supplies a bare Linux operating system that you can access through SSH. It can be a physical machine, virtual machine, or container with systemd support. The node must have systemd, sudo, and sshd.

A deployment requires at least 1 node. You can deploy to multiple nodes in one playbook run, or add nodes later. The minimum resources are 1C1G; we recommend at least 1C2G. There is no upper resource limit. Pigsty automatically tunes parameters for the available resources.

Select the node count for your requirements; see Architecture Planning. A single-node deployment with external backups provides recovery protection. For production, we recommend multiple nodes. A working HA deployment requires at least 3 nodes. A 2-node deployment provides Semi-HA.

Disk

Pigsty uses /data as the default data directory. If you have a dedicated data disk, mount it at /data. Use /data1, /data2, and /dataN for additional disks.

To change a data directory, set the applicable parameter:

The native MySQL 8.4 pilot module always uses /var/lib/mysql. It does not currently expose a data-directory parameter.

Filesystem

You can use any supported Linux filesystem for data disks. For production, we recommend xfs.

xfs is a standard Linux filesystem with good performance. Its copy-on-write (CoW) support permits instant cloning of large database clusters. Silo deployments with multiple drives require xfs.

ext4 is another supported choice. It has more data recovery tools but does not support CoW. zfs supplies RAID and snapshots, but adds significant performance overhead and requires separate installation.

Select the filesystem for your requirements. Avoid NFS for database services.

Pigsty expects root:root ownership and 755 permissions on /data. Administrators can assign ownership to directories directly under /data. Each application runs with a dedicated account in its own subdirectory. See FHS for the directory structure.

Network

The default online installation requires outbound Internet access. Offline installation does not require Internet access.

Assign a fixed IPv4 address to each node for internal communication. Pigsty uses this primary address on the main network interface as the unique node identifier. If a single-node deployment has no fixed IP address, use 127.0.0.1.

VIP

Pigsty supports optional Layer 2 virtual IP addresses (L2 VIPs). NODE clusters use keepalived; PGSQL clusters use vip-manager.

To use this feature, assign an L2 VIP address to each node or database cluster. This is usually straightforward on your own hardware. Public cloud networks can make this configuration difficult.

CA

Pigsty creates a self-signed certificate authority (CA) for each deployment. The CA issues the encryption certificates.

You can instead use an existing enterprise CA or self-signed CA to issue the required certificates.

Domain

Pigsty uses the local static domain i.pigsty for the Web UI by default. You can also use the node IP address.

For production, we recommend a domain name to enable HTTPS and encrypted data transfer. Domain names also let multiple services share one port.

For Internet access, use a public DNS provider, such as Cloudflare or AWS Route53. Point the domain to the node’s public IP address. For LAN or office access, use internal DNS with the node’s internal IP address.

For local access, add this entry to /etc/hosts on each computer that accesses the Web UI:

10.10.10.10 i.pigsty    # Replace with your domain and Pigsty node IP

Linux

Pigsty runs on Linux and currently targets 16 platform combinations. These combinations cover eight distribution major versions and two architectures. See the Compatible OS List.

The recommended operating system versions are:

• Rocky Linux 9.8 / 10.2.
• Debian 12.15 / 13.6.
• Ubuntu 22.04.5 / 24.04.4 / 26.04.0.

On macOS or Windows, use a virtual machine or a Docker image with systemd.

We strongly recommend a fresh operating system installation. If the server already runs Nginx, PostgreSQL, or similar services, consider a new node.

Locale

We recommend en_US as the primary operating system language so PostgreSQL writes logs in English. At minimum, make sure that this locale is available.

Some distributions, including Debian, do not always supply en_US by default. Use these commands to enable it:

localedef -i en_US -f UTF-8 en_US.UTF-8
localectl set-locale LANG=en_US.UTF-8

For PostgreSQL 17 and later, we strongly recommend the built-in C.UTF-8 collation as the default. The configuration wizard selects C.UTF-8 when the PostgreSQL version and operating system support it.

Ansible

Pigsty uses Ansible on the administration node to control all managed nodes. See Installing Ansible.

By default, Pigsty installs Ansible on INFRA nodes. These nodes can serve as administration nodes or backup administration nodes. In a single-node deployment, one node has both roles. It is the administration node that runs Ansible and the INFRA node that hosts infrastructure.

Pigsty

Use one of these commands to install the current default Pigsty source:

curl -fsSL https://repo.pigsty.io/get | bash;

curl -fsSL https://repo.pigsty.cc/get | bash;

To install a specific version, supply -s <version>:

curl -fsSL https://repo.pigsty.io/get | bash -s <version>  # Install a specific version (current stable: v4.5.0)

curl -fsSL https://repo.pigsty.cc/get | bash -s <version>  # Install a specific version (current stable: v4.5.0)

To install the latest beta version, use one of these commands:

curl -fsSL https://repo.pigsty.io/beta | bash;

curl -fsSL https://repo.pigsty.cc/beta | bash;

For development, you can clone the repository directly:

git clone https://github.com/pgsty/pigsty.git;
cd pigsty; git checkout <tag>  # Use a released version (current stable tag: v4.5.0)

For an environment without Internet access, obtain the source tarball from GitHub Releases or a Pigsty repository:

wget https://repo.pigsty.io/src/pigsty-v<version>.tgz
wget https://repo.pigsty.cc/src/pigsty-v<version>.tgz

3 - Planning Architecture and Nodes

Pigsty uses a modular architecture. Select the modules for your requirements and define the deployment through declarative configuration.

Common Patterns

Use these common deployment patterns as a starting point. Adjust the configuration for your requirements:

Select the architecture for your reliability requirements and available resources. Production deployments with full high availability require at least 3 nodes. With 2 nodes, use the Semi-HA configuration.

Trade-offs

• Monitoring requires at least 1 INFRA node. Production typically uses 2 nodes; large deployments use 3.
• PostgreSQL HA requires at least 1 ETCD node. Production typically uses 3 nodes; large deployments use 5. An even-member ETCD cluster tolerates no more failures than an odd-member cluster with one fewer member. Prefer an odd number of members.
• Silo object storage requires at least 1 MINIO node. Production typically uses 4 or more nodes in an MNMD cluster.
• Production PostgreSQL clusters typically have at least two nodes: a primary and a replica. More demanding deployments use 3 nodes. Deployments with high read loads can have dozens of replicas.
• Advanced PostgreSQL configurations include offline instances, synchronous instances, standby clusters, and delayed clusters.

Single-Node Setup

The single-node configuration puts all components on one node. It installs four core modules by default. Use it for demonstrations, development, or testing.

An external S3/MinIO backup repository can supply the required recovery time and recovery point objectives (RTO/RPO). With this protection, a single-node configuration can serve standard production environments.

These single-node variants are available:

• Rich (rich) is a production template with local Silo storage, a local software repository, and all PostgreSQL extensions.
• Slim (slim) installs PGSQL and ETCD without the monitoring infrastructure. Slim Installation can expand to multiple nodes with HA.
• Infra-only (infra) installs only INFRA to monitor other instances. It does not install database services.
• Alternative kernels replace the standard PostgreSQL kernel with a derivative. Templates include pgsql, mssql, polar, ivory, and mysql. Other templates include pgtde, oriole, agens, and pgedge.

Two-Node Setup

The two-node configuration enables database replication and Semi-HA. It improves data redundancy and supplies limited automatic failover:

Automatic recovery depends on which node fails:

• If node-1 fails, manually promote node-2. Automatic failover is unavailable.
• If node-2 fails, node-1 is automatically promoted.

Three-Node Setup

The three-node template supplies the minimum full HA configuration. It tolerates the failure of any one node and recovers automatically.

Four-Node Setup

The Pigsty Sandbox uses the standard four-node configuration.

In the demonstration configuration, INFRA and ETCD do not have HA. You can extend the configuration as follows:

More Nodes

If your virtualization environment or hardware resources permit it, use more nodes to deploy modules separately. Dedicated nodes can improve reliability, monitoring, and performance.

4 - Setup Admin User and Privileges

Pigsty uses an operating system administration account with SSH and sudo access to every managed node. The account must be able to log in through SSH and run sudo commands on these nodes. Passwordless access is the standard configuration; password prompts are also supported.

User

Use an account name such as dba or admin. Avoid root and postgres:

• You can deploy as root, but this is not recommended for production.
• Do not use postgres (pg_dbsu) as the administration account.

Passwordless

You can enter passwords for each ssh and sudo command instead of configuring passwordless access. When you run playbooks, use -k|--ask-pass for the SSH password prompt. Use -K|--ask-become-pass for the sudo password prompt:

./deploy.yml -k -K

If your security policy prohibits passwordless ssh or sudo, use these options. You can also configure a longer sudo password cache time in sudoers to reduce repeated prompts.

Create Admin User

Your server or virtual machine provider usually creates an initial administration account. If you need a different account, the deployment playbook can create one.

Use an existing administration account or root access to create the new account:

./node.yml -k -K -t node_admin \
  -e ansible_user=[current_login_admin] \
  -e node_admin_username=[new_admin_to_create]

The existing account creates a dedicated dba account with UID 88 by default. The playbook also configures sudo and SSH access. These parameters control the account:

Sudo

Give the administration account sudo access on every managed node. Passwordless sudo is recommended.

For an account named vagrant, create or edit its sudoers file:

echo '%vagrant ALL=(ALL) NOPASSWD: ALL' | sudo tee /etc/sudoers.d/vagrant

For the dba account, use this content in /etc/sudoers.d/dba:

%dba ALL=(ALL) NOPASSWD: ALL

If your security policy prohibits passwordless sudo, remove NOPASSWD::

%dba ALL=(ALL) ALL

Ansible uses sudo to run commands with root privileges on managed nodes. If an environment has no sudo, install it first. This can apply to Docker containers.

SSH

The current local account should have passwordless SSH access to the administration account on every managed node. The local and remote account names can differ. The following procedures describe the basic SSH configuration.

Generate SSH Key

If you do not have an SSH key pair, generate one:

ssh-keygen -t rsa -b 2048 -N '' -f ~/.ssh/id_rsa -q

The bootstrap script also generates a key pair if none exists.

Copy SSH Key

Copy the public key to the administration account’s ~/.ssh/authorized_keys file on every node, including the local node. Use ssh-copy-id:

ssh-copy-id <ip>                        # Interactive password entry
sshpass -p <password> ssh-copy-id <ip>  # Non-interactive (use with caution)

Using Alias

If a node requires a jumpserver, another port, or different credentials, define an SSH alias in ~/.ssh/config:

Host meta
    HostName 10.10.10.10
    User dba                      # Different user on remote
    IdentityFile /etc/dba/id_rsa  # Non-standard key
    Port 24                       # Non-standard port

Set ansible_host to this SSH alias in the inventory:

nodes:
  hosts:          # If node `10.10.10.10` requires SSH alias `meta`
    10.10.10.10: { ansible_host: meta }  # Access via `ssh meta`

Ansible can also use SSH connection parameters directly. See the Ansible Inventory Guide.

This method supports private nodes behind jumpservers and nodes with different ports or credentials. It also lets you use a local laptop as the administration node.

Check Accessibility

From the administration node, the current account should reach every managed node through passwordless ssh. The remote administration account should also have passwordless sudo access.

Run this command from the administration node for each managed node:

ssh <ip|alias> 'sudo ls'

If the command returns without a password prompt or error, passwordless SSH and sudo access work.

Firewall

For production, configure the firewall to block unauthorized access to service ports. By default, block inbound access from office and Internet networks except these ports:

• 22 for SSH node access.
• 80 for HTTP and 443 for HTTPS Web UI access.
• 5432 for PostgreSQL access.

If clients use other PostgreSQL service ports, permit those ports as required. See the complete port list.

• 5432: PostgreSQL database.
• 6432: PgBouncer connection pooler.
• 5433: PostgreSQL primary service.
• 5434: PostgreSQL replica service.
• 5436: PostgreSQL default service.
• 5438: PostgreSQL offline service.

5 - Sandbox

Pigsty supplies a standard 4-node sandbox for learning, testing, and feature demonstrations. It uses fixed IP addresses and predefined identity values to make the demonstrations repeatable.

Description

The default sandbox has 4 nodes and uses the ha/full.yml template.

This example shows the sandbox configuration:

all:
  children:
    infra: { hosts: { 10.10.10.10: { infra_seq: 1 } } }
    etcd:  { hosts: { 10.10.10.10: { etcd_seq:  1 } }, vars: { etcd_cluster: etcd } }
    minio: { hosts: { 10.10.10.10: { minio_seq: 1 } }, vars: { minio_cluster: minio } }

    pg-meta:
      hosts: { 10.10.10.10: { pg_seq: 1, pg_role: primary } }
      vars:  { pg_cluster: pg-meta }

    pg-test:
      hosts:
        10.10.10.11: { pg_seq: 1, pg_role: primary }
        10.10.10.12: { pg_seq: 2, pg_role: replica }
        10.10.10.13: { pg_seq: 3, pg_role: replica }
      vars: { pg_cluster: pg-test }

  vars:
    version: v4.5.0
    admin_ip: 10.10.10.10
    region: default
    pg_version: 18

PostgreSQL Clusters

The meta node has a PostgreSQL cluster named pg-meta with one instance:

10.10.10.10 meta pg-meta-1
10.10.10.2  pg-meta          # Optional L2 VIP

The other three nodes host the pg-test HA cluster with 3 instances:

10.10.10.11 node-1 pg-test-1
10.10.10.12 node-2 pg-test-2
10.10.10.13 node-3 pg-test-3
10.10.10.3  pg-test          # Optional L2 VIP

The sandbox has two optional L2 VIPs. Each VIP binds to the primary instance of its cluster, pg-meta or pg-test.

Infrastructure

The meta node also hosts these services:

• A single-node etcd cluster that supplies the distributed configuration store (DCS) for PostgreSQL HA.
• A single-node Silo minio cluster that supplies S3-compatible object storage through the MINIO module.

10.10.10.10 etcd-1
10.10.10.10 minio-1

The ha/full.yml template also defines three Redis example topologies and enables Docker on the INFRA node. The standard deploy.yml playbook does not deploy Redis or Docker. When required, run ./redis.yml and ./docker.yml separately.

Creating Sandbox

Use Vagrant to create a local sandbox from the supplied templates. Use OpenTofu to create a cloud sandbox.

Local Sandbox (Vagrant)

The local sandbox uses VirtualBox or libvirt to create virtual machines on your Mac or PC. It has no cloud resource charges. For the complete 4-node sandbox, provide at least 4 CPU cores and 8GB memory.

cd ~/pigsty/vagrant
make full       # Create 4-node sandbox with default Ubuntu 24.04 image
make full9      # Create 4-node sandbox with RockyLinux 9
make full12     # Create 4-node sandbox with Debian 12
make full24     # Create 4-node sandbox with Ubuntu 24.04
make full26     # Create 4-node sandbox with Ubuntu 26.04

The current Vagrant configuration uses cloud-image/* boxes from Vagrant Cloud. See Supported Images for available images, pinned versions, and architectures. If the source does not pin a box version, Vagrant selects its currently available version.

Cloud Sandbox (Terraform)

The cloud sandbox uses public cloud APIs to create virtual machines. You can create and remove these resources for short tests and pay for their use.

Use spec/aliyun-full.tf to create a 4-node sandbox on Alibaba Cloud:

cd ~/pigsty/terraform
cp spec/aliyun-full.tf terraform.tf
terraform init
terraform apply

See the OpenTofu guide for details.

Other Specs

Pigsty also supplies other environment specifications. Run these Makefile shortcuts from ~/pigsty/vagrant:

cd ~/pigsty/vagrant

Single Node Devbox (meta)

Use the 1-node environment for the quick start, development, and testing:

make meta       # Create single-node devbox

Two Node Environment (dual)

Use the 2-node environment to test replication between a primary and a replica:

make dual       # Create 2-node environment

Three Node Environment (trio)

Use the 3-node environment to test basic high availability:

make trio       # Create 3-node environment

Production Simulation (simu)

Use the 20-node simulation to test a complete production environment:

make simu       # Create 20-node production simulation environment

This environment includes these nodes:

• 3 infrastructure nodes: meta1, meta2, and meta3.
• 2 HAProxy proxy nodes.
• 4 MINIO (Silo) nodes.
• 5 ETCD nodes.
• 6 PostgreSQL nodes in 2 clusters with 3 nodes each.

6 - Vagrant

Vagrant creates local virtual machines from declarative configuration. Pigsty requires Linux. You can use Vagrant to create local Linux virtual machines for testing.

The recommended and validated versions are:

• Rocky Linux 9.8 / 10.2.
• Debian 12.15 / 13.6.
• Ubuntu 22.04.5 / 24.04.4 / 26.04.0. Major-version aliases select pinned box versions.

Quick Start

Install Dependencies

Install Vagrant and a virtual machine provider, such as VirtualBox or libvirt. On macOS, you can install the dependencies with Homebrew:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
brew install vagrant virtualbox ansible

On Linux, use VirtualBox or vagrant-libvirt as the virtual machine provider.

Create Virtual Machines

Use the supplied make shortcuts to create virtual machines:

cd ~/pigsty/vagrant

make meta       # 1 node devbox for quick start, development, and testing
make full       # 4 node sandbox for HA testing and feature demonstration
make simu       # 20 node simubox for production environment simulation

# Other less common specs
make dual       # 2 node environment
make trio       # 3 node environment
make deci       # 10 node environment

To select a different operating system image, use a variant alias:

make meta9      # Create single node with Rocky Linux 9.8
make full12     # Create 4-node sandbox with Debian 12.15
make simu24     # Create 20-node simubox with Ubuntu 24.04.4
make full26     # Create 4-node sandbox with Ubuntu 26.04.0

These operating system suffixes are available:

• 8: EL8; 9: EL9; 10: EL10.
• 12: Debian 12.15; 13: Debian 13.6.
• 22: Ubuntu 22.04.5; 24: Ubuntu 24.04.4; 26: Ubuntu 26.04.0.

Build Environment

Use these aliases to create Pigsty build environments. These templates retain the base image:

make oss        # 7 node OSS build environment
make pro        # 7 node PRO build environment
make rpm        # 2 node EL9/10 build environment
make deb        # 5 node Debian12/13 Ubuntu22/24/26 build environment
make all        # 7 node full build environment

Spec Templates

The vagrant/spec/ directory contains predefined virtual machine specifications:

Each specification has a Specs variable that defines its nodes. For example, full.rb defines the 4-node sandbox shown below.

The current templates give every virtual machine a 32 GB primary system disk. Regular nodes also receive one data disk. Its size comes from the specification’s disk value, with a default of 128 GB if omitted.

Nodes with names that start with minio receive four 32 GB data disks instead. These disks mount at /data1 through /data4.

The disk configuration requires the experimental Vagrant disks feature. The repository Makefile automatically exports VAGRANT_EXPERIMENTAL=disks. If you run vagrant directly, set this environment variable yourself.

# full: pigsty full-featured 4-node sandbox for HA-testing & tutorial & practices

Specs = [
  { "name" => "meta"   , "ip" => "10.10.10.10" ,  "cpu" => "2" ,  "mem" => "4096" ,  "image" => "cloud-image/ubuntu-24.04" },
  { "name" => "node-1" , "ip" => "10.10.10.11" ,  "cpu" => "1" ,  "mem" => "2048" ,  "image" => "cloud-image/ubuntu-24.04" },
  { "name" => "node-2" , "ip" => "10.10.10.12" ,  "cpu" => "1" ,  "mem" => "2048" ,  "image" => "cloud-image/ubuntu-24.04" },
  { "name" => "node-3" , "ip" => "10.10.10.13" ,  "cpu" => "1" ,  "mem" => "2048" ,  "image" => "cloud-image/ubuntu-24.04" },
]

simu Spec Details

The simu.rb specification defines a 20-node production simulation:

• 3 infrastructure nodes (meta1-3): 4c16g each.
• 2 HAProxy nodes (proxy1-2): 1c2g each.
• 4 MINIO nodes (minio1-4): 1c2g each.
• 5 ETCD nodes (etcd1-5): 1c2g each.
• 6 PostgreSQL nodes (pg-src-1-3, pg-dst-1-3): 2c4g each.

Config Script

Use vagrant/config to generate the Vagrantfile from a specification and options:

cd ~/pigsty/vagrant
vagrant/config [spec] [image] [scale] [provider]

# Examples
vagrant/config meta u24            # Use 1-node spec with Ubuntu 24.04.4 image
vagrant/config dual el9            # Use 2-node spec with RockyLinux 9.7 image
vagrant/config trio d12 2          # Use 3-node spec with Debian 12.14, double resources
vagrant/config full u22 4          # Use 4-node spec with Ubuntu 22.04.5, 4x resources
vagrant/config simu u26 1 libvirt  # Use 20-node spec with Ubuntu 26.04.0, libvirt provider

Image Aliases

The configuration script supports these image aliases:

The mapping still contains the historical d11/debian11/deb11 and u20/ubuntu20/ubuntu2004 aliases. The current script explicitly rejects these aliases. They are not supported images.

Resource Scaling

Set VM_SCALE to change the resource multiplier. The default value is 1:

VM_SCALE=2 vagrant/config meta     # Double the CPU/memory resources for meta spec

For example, VM_SCALE=4 changes the meta specification from 2c4g to 8c16g:

Specs = [
  { "name" => "meta" , "ip" => "10.10.10.10", "cpu" => "8" , "mem" => "16384" , "image" => "cloud-image/ubuntu-24.04" },
]

VM Management

The vagrant/Makefile supplies shortcuts to manage virtual machines. Run these commands from the vagrant/ directory:

cd ~/pigsty/vagrant
make           # Equivalent to make start
make new       # Destroy existing VMs and create new ones
make ssh       # Write VM SSH config to ~/.ssh/ (must run after creation)
make dns       # Write VM DNS records to /etc/hosts (optional)
make start     # Start VMs and configure SSH (up + ssh)
make up        # Start VMs with vagrant up
make halt      # Shutdown VMs (alias: down, dw)
make clean     # Destroy VMs (alias: del, destroy)
make status    # Show VM status (alias: st)
make pause     # Pause VMs (alias: suspend)
make resume    # Resume VMs
make nuke      # Destroy all VMs and volumes with virsh (libvirt only)
make info      # Show libvirt info (VMs, networks, storage volumes)

SSH Keys

By default, the templates use ~/.ssh/id_rsa[.pub] as the virtual machine SSH key pair. Before deployment, make sure that this key pair exists and is valid. If necessary, generate it:

ssh-keygen -t rsa -b 2048 -N '' -f ~/.ssh/id_rsa -q

Supported Images

EL, Debian, Ubuntu, and AlmaLinux templates use cloud-image/* boxes from Vagrant Cloud. Explicit RHEL and Oracle Linux aliases use generic/* boxes.

The script uses the same cloud-image/* mapping for VirtualBox and libvirt on amd64 and arm64. Vagrant Cloud resolves actual payload availability at runtime. The vagrant/config script writes these validated versions for each supported image to make environments repeatable:

The retained Debian 11 alias pins version 20260618.2513.0. The retained Ubuntu 20.04 alias pins version 20250624.0.0. Both aliases remain unsupported.

Experimental generic/* images for RHEL, Oracle Linux, and CentOS 7 pin the final 4.3.12 release. These legacy images are outside the current support matrix.

Environment Variables

Use these environment variables to control Vagrant:

export VM_SPEC='meta'              # Spec name
export VM_IMAGE='cloud-image/rocky-9' # Image name
export VM_SCALE='1'                # Resource scaling multiplier
export VM_PROVIDER='virtualbox'    # Virtualization provider
export VAGRANT_EXPERIMENTAL=disks  # Enable disks for direct vagrant use; Makefile sets this automatically

Notes

echo "* 10.0.0.0/8" | sudo tee -a /etc/vbox/networks.conf

7 - OpenTofu

OpenTofu is an open-source infrastructure-as-code tool. It creates public cloud virtual machines from configuration files.

Pigsty uses OpenTofu by default. It supplies Terraform-compatible .tf templates for Alibaba Cloud, global AWS, AWS China, Azure, GCP, Tencent Cloud, Hetzner, Vultr, DigitalOcean, and Linode. The aliyun-s3.tf template also creates a private OSS bucket and dedicated RAM read/write credentials for S3 and pgBackRest.

Quick Start

Install OpenTofu

On macOS, install OpenTofu with Homebrew:

brew install opentofu
tofu version

For Debian, Ubuntu, RHEL, or another platform, use the OpenTofu installation guide. The package and command name is tofu.

Initialize and Apply

Enter the cloud template directory. Select a template, initialize the provider plugins, and examine the deployment plan:

cd ~/pigsty/terraform
cp spec/aliyun.tf terraform.tf         # Select template
tofu init                              # Install cloud provider plugins (first use)
tofu plan                              # Review the execution plan
tofu apply                             # Create resources after interactive confirmation

After you review the plan, run tofu apply. Enter yes to confirm. OpenTofu then creates the virtual machines and related cloud resources.

Get IP Address

After creation, print the administration node’s public IP address:

tofu output -raw meta_ip

Configure SSH Access

Global cloud templates usually supply an executable ssh_command output:

tofu output -raw ssh_command

The repository’s ./ssh script supports legacy templates whose outputs contain only IP addresses. These templates use the root password PigstyDemo4. The script treats each infrastructure-as-code output as an IP address and writes it to ~/.ssh/pigsty_config. It then distributes keys with sshpass.

Compatible templates include aliyun.tf, aliyun-full.tf, aliyun-oss.tf, and aliyun-pro.tf. Do not use this script with modern templates that output ssh_command, private IP addresses, or access keys.

For a compatible template, run:

./ssh       # Write SSH config and distribute keys
ssh meta    # Login using hostname instead of IP

Include ~/.ssh/pigsty_config

Destroy Resources

After testing, use this command to remove the cloud resources:

tofu destroy

Terraform Compatibility and Migration

OpenTofu supports the existing names terraform/, .tf, terraform.tfvars, .terraform.lock.hcl, and terraform.tfstate for compatibility. The repository keeps these names. You can explicitly select Terraform as a compatibility option:

make IAC_CLI=terraform plan

Use only one infrastructure command-line tool in a working directory at a time. Before you migrate existing state, preserve it and compare both plans:

cp -p terraform.tfstate "terraform.tfstate.pre-tofu.$(date +%Y%m%d%H%M%S)"
terraform plan
tofu init
tofu plan

Continue only if the OpenTofu plan contains the expected changes. Do not delete terraform.tfstate when you initialize providers again.

Template Specs

The terraform/spec/ directory contains predefined cloud resource templates:

Copy the selected template to terraform.tf:

cd ~/pigsty/terraform
cp spec/aliyun-full.tf terraform.tf   # Use Alibaba Cloud 4-node sandbox template
tofu init && tofu apply

Variable Configuration

Available variables depend on the template. Alibaba Cloud supports the full distribution matrix and defaults to u26. Global AWS, Azure, GCP, Tencent Cloud, and Hetzner support Debian 12/13 with AMD or ARM selection. Their defaults are generally d12 and amd64. Vultr, DigitalOcean, and Linode currently offer only AMD instance choices.

Architecture and Distribution

variable "architecture" {
  description = "Architecture type (amd64 or arm64)"
  type        = string
  default     = "amd64"    # Comment this line to use arm64
  #default     = "arm64"   # Uncomment to use arm64
}

variable "distro" {
  description = "Distribution code (the exact set depends on the template)"
  type        = string
  default     = "d12"       # Global-cloud templates usually default to Debian 12; Alibaba Cloud defaults to u26
}

Resource Configuration

Alibaba Cloud templates define these resource parameters in a locals block. Other cloud templates use provider-specific variables or local values for instances, disks, and networks. Read the selected .tf file for its parameters.

locals {
  bandwidth        = 100                    # Public bandwidth (Mbps)
  disk_size        = 40                     # System disk size (GB)
  spot_policy      = "SpotWithPriceLimit"   # Spot policy: NoSpot, SpotWithPriceLimit, SpotAsPriceGo
  spot_price_limit = 5                      # Max spot price (only effective with SpotWithPriceLimit)
}

Alibaba Cloud Configuration

Credential Setup

Set Alibaba Cloud credentials in environment variables. For example, add them to ~/.bash_profile or ~/.zshrc:

export ALICLOUD_ACCESS_KEY="<your_access_key>"
export ALICLOUD_SECRET_KEY="<your_secret_key>"
export ALICLOUD_REGION="cn-shanghai"

Supported Images

The table lists common Alibaba Cloud ECS Public OS Image prefixes. The recommended and validated versions are:

• Rocky Linux 9.8 / 10.2.
• Debian 12.15 / 13.6.
• Ubuntu 22.04.5 / 24.04.4 / 26.04.0.

OSS Storage Configuration

The aliyun-s3.tf template creates an OSS bucket and permissions for PostgreSQL PITR backups:

• A private OSS bucket named pigsty-oss.
• A dedicated RAM user named pigsty-oss-user.
• An AccessKey saved to ~/pigsty.sk.
• A RAM policy with oss:* permissions on the bucket and its objects for read/write operations.

AWS Configuration

Credential Setup

The global and China-region templates can read standard AWS environment variables or credential files:

export AWS_ACCESS_KEY_ID="<your_access_key>"
export AWS_SECRET_ACCESS_KEY="<your_secret_key>"
export AWS_REGION="us-west-2"

# ~/.aws/config
[default]
region = us-west-2

# ~/.aws/credentials
[default]
aws_access_key_id = <YOUR_AWS_ACCESS_KEY>
aws_secret_access_key = <AWS_ACCESS_SECRET>

By default, aws.tf reads ~/.ssh/id_rsa.pub. The legacy aws-cn.tf template reads this dedicated public key instead:

~/.aws/pigsty-key.pub

Tencent Cloud Configuration

Credential Setup

Set Tencent Cloud credentials in environment variables:

export TENCENTCLOUD_SECRET_ID="<your_secret_id>"
export TENCENTCLOUD_SECRET_KEY="<your_secret_key>"
export TENCENTCLOUD_REGION="ap-beijing"

Other Cloud Credentials

# Azure: az login is recommended; for a service principal, use all four
export ARM_CLIENT_ID="<client_id>"
export ARM_CLIENT_SECRET="<client_secret>"
export ARM_SUBSCRIPTION_ID="<subscription_id>"
export ARM_TENANT_ID="<tenant_id>"

# GCP: gcloud auth application-default login is also supported
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account-key.json"

# Hetzner / Vultr / DigitalOcean / Linode
export HCLOUD_TOKEN="<api_token>"
export VULTR_API_KEY="<api_key>"
export DIGITALOCEAN_TOKEN="<api_token>"
export LINODE_TOKEN="<api_token>"

The GCP template also requires project, for example tofu apply -var="project=my-project". Except for AWS China, the current templates with key authentication read ~/.ssh/id_rsa.pub by default. To use another public key, edit the path in the selected template.

Shortcut Commands

The Makefile shortcuts use OpenTofu by default. To use Terraform, explicitly set IAC_CLI=terraform.

cd ~/pigsty/terraform

make init       # tofu init
make validate   # tofu validate
make plan       # tofu plan
make u          # interactive tofu apply + legacy ./ssh (compatible templates only)
make d          # interactive tofu destroy
make apply      # interactive tofu apply
make destroy    # interactive tofu destroy
make out        # tofu output
make ssh        # Run ssh script to configure SSH access
make r          # Reset terraform.tf to repository state

For modern templates with ssh_command, private-IP, or other non-IP outputs, run tofu apply directly. Do not use make u: it runs the legacy ./ssh script after deployment. Automatic confirmation is available only through up-auto, apply-auto, and destroy-auto.

Notes

ssh root@<public_ip>

8 - Security Considerations

Pigsty defaults support development, testing, and demonstrations on a trusted intranet. For production, configure credentials, network boundaries, authentication, certificates, backups, and audit controls for your threat model.

Security and Compliance describes the mechanisms and their limits. Use the Launch Hardening Checklist for executable checks. The ha/safe template is a hardening example. Review each control for your environment before you use it.

Confidentiality

Critical Files

Protect these assets:

• pigsty.yml and other inventories. They normally contain system and application credentials.
• files/pki/ca/ca.key. This key can issue certificates that the deployment trusts.
• The administration account’s SSH private key. By default, this account can use sudo on managed nodes.
• Private keys for client certificates and backup encryption.
• Generated /pg/tmp/pg-user-*.sql files.

Restrict access to the administration node and configuration repository. Do not commit complete inventories or private keys to public repositories. Use controlled channels to back up the CA private key and recovery configuration.

Passwords

Replace every public default credential before production deployment. Start with this command:

./configure -g

This option does not replace the pgBackRest cipher_pass or user-defined credentials. It also leaves some Silo example credentials in ha/safe unchanged. Examine the result against the Default Credentials Checklist.

PostgreSQL uses SCRAM-SHA-256 for new or updated passwords by default. To enforce password complexity, preload passwordcheck through pg_libs, or configure credcheck. Set the account lifetime with expire_in or expire_at.

Before credential rotation, prepare a rollback plan. Update the database users, PgBouncer user list, component configuration, and client connection information during rotation.

Network Boundaries

IP Addresses

PostgreSQL listens on 0.0.0.0 by default. To restrict its listen addresses, set these parameters:

pg_listen: '${ip},${vip},${lo}'

The listen address supplies only part of the network boundary. Also review these controls:

• Cloud security groups or upstream firewalls.
• node_firewall_public_port.
• The trusted CIDRs in node_firewall_intranet. Make sure that these ranges are not too broad.
• PostgreSQL and PgBouncer HBA rules.
• The Patroni REST API allowlist.

The demonstration pigsty.yml also exposes 5432 to public networks. Remove that exception for production. If applications require direct database access, permit only their specific CIDRs.

Network Traffic

• PostgreSQL enables server-side TLS by default. The default intranet HBA rules do not require TLS.
• PgBouncer TLS is disabled by default. pgbouncer_sslmode controls this setting.
• HTTPS for the Patroni REST API is disabled by default. patroni_ssl_enabled controls this setting.
• Nginx and the object-storage backend selected by MINIO enable HTTPS by default. Etcd uses TLS for client and peer traffic.

HBA auth: ssl requires encryption but does not verify the server identity for the client. Clients should use sslmode=verify-full with a trusted CA to verify the database server. See Encrypted Communication.

Grafana, VictoriaMetrics, and other components can listen on node ports. The default firewall does not expose these ports directly to public networks. Prefer Nginx for external access. Restrict management pages by source address and identity.

Authentication and Access Control

• Use HBA to define the user, database, source address, and authentication method. Avoid broad world rules.
• Use auth: cert for privileged remote users. Establish a process to deliver and revoke client certificates.
• Assign application privileges through built-in roles. Do not grant superuser privileges to ordinary application accounts.
• Set revokeconn: true for shared multi-tenant clusters. Examine the effective database ACLs.
• Create objects through the declared database owner or a controlled administration role so default privileges apply.
• To isolate offline queries, explicitly set role: offline on the HBA rule for dbrole_offline.

After changes to HBA, users, or roles, compare the inventory with the effective database state.

Integrity

Pigsty enables page checksums by default to detect damage after page writes. These checksums do not detect every memory error, logical error, or incorrect application write.

The CRIT template enables Patroni strict synchronous mode and more detailed connection logs. This mode aims to preserve acknowledged transactions. The result depends on synchronous_commit, the state of synchronous replicas, and the failover conditions. Writes block when no synchronous replica is available.

CRIT sets the watchdog mode to automatic. The watchdog activates only if the system has a usable watchdog device. Decide whether required is appropriate for your hardware and availability requirements.

Availability

• Critical clusters should normally have at least three instances in independent failure domains.
• Connect through HAProxy, a VIP, or a DNS service name instead of a fixed primary address.
• Use an odd number of etcd nodes in independent failure domains.
• Remove single points of failure in INFRA, DNS, monitoring, and software repositories where your availability requirements demand it.
• Understand the configuration semantics of pg_rpo and pg_rto. Test the recovery objectives through exercises.

Replicas handle some node failures. They do not replace backups.

Backup and Recovery

• The local pgBackRest repository has no encryption by default. It shares a failure domain with the database host.
• The pgbackrest_method: minio repository uses AES-256-CBC by default. Replace the public default cipher_pass: pgBackRest.
• The pgBR.${pg_cluster} value in ha/safe is also an example. Do not use it as the final encryption key.
• Store important backups in an independent failure domain. Evaluate object locking, versioning, or offline copies for additional protection.
• Test full restores and PITR regularly. Verify the WAL, keys, recovery time, and application consistency.

See Data Security and Backup and Recovery.

Audit and Response

The default OLTP template logs DDL, slow queries, and PostgreSQL 18 connection-authorization events. CRIT also logs connection and disconnection events.

Install and preload pgaudit, then configure an audit policy. The package alone does not produce SQL audit logs. If you enable Vector and VictoriaLogs, set log retention, access, and archive policies for your requirements.

Metrics, logs, and alerts supply information for incident response. Production also requires alert classification, on-call ownership, incident determination, response, evidence collection, and a review after each incident.

Host and Software Supply Chain

• After compatibility validation, change SELinux from the default permissive mode to enforcing.
• Disable unnecessary SSH password authentication and remote root login. Consider a bastion host or multi-factor authentication.
• Review the sudo permissions of the administration and database operating system accounts.
• Keep Pigsty and upstream components on current, supported versions.
• Verify the software repository GPG key fingerprint. Enable signature verification for each package where required.

See Compliance: Supply Chain and Vulnerability Response.