ER Model

• 1: PGSQL Cluster Model
• 2: ETCD Cluster Model
• 3: MinIO Cluster Model
• 4: Redis Cluster Model
• 5: INFRA Node Model

1 - PGSQL Cluster Model

The PGSQL module organizes PostgreSQL in production as clusters—logical entities composed of a group of database instances associated by primary-replica relationships.

Each cluster is an autonomous business unit consisting of at least one primary instance, exposing capabilities through services.

There are four core entities in Pigsty’s PGSQL module:

• Cluster: An autonomous PostgreSQL business unit serving as the top-level namespace for other entities.
• Service: A named abstraction that exposes capabilities, routes traffic, and exposes services using node ports.
• Instance: A single PostgreSQL server consisting of running processes and database files on a single node.
• Node: A hardware resource abstraction running Linux + Systemd environment—can be bare metal, VM, container, or Pod.

Along with two business entities—“Database” and “Role”—these form the complete logical view as shown below:

Examples

Let’s look at two concrete examples. Using the four-node Pigsty sandbox, there’s a three-node pg-test cluster:

    pg-test:
      hosts:
        10.10.10.11: { pg_seq: 1, pg_role: primary }
        10.10.10.12: { pg_seq: 2, pg_role: replica }
        10.10.10.13: { pg_seq: 3, pg_role: replica }
      vars: { pg_cluster: pg-test }

The above config fragment defines a high-availability PostgreSQL cluster with these related entities:

Identity Parameters

Pigsty uses the PG_ID parameter group to assign deterministic identities to each PGSQL module entity. Three parameters are required:

With cluster name defined at cluster level and instance number/role assigned at instance level, Pigsty automatically generates unique identifiers for each entity based on rules:

Because Pigsty adopts a 1:1 exclusive deployment model for nodes and PG instances, by default the host node identifier borrows from the PG instance identifier (node_id_from_pg). You can also explicitly specify nodename to override, or disable nodename_overwrite to use the current default.

Sharding Identity Parameters

When using multiple PostgreSQL clusters (sharding) to serve the same business, two additional identity parameters are used: pg_shard and pg_group.

In this case, this group of PostgreSQL clusters shares the same pg_shard name with their own pg_group numbers, like this Citus cluster:

In this case, pg_cluster cluster names are typically composed of: {{ pg_shard }}{{ pg_group }}, e.g., pg-citus0, pg-citus1, etc.

all:
  children:
    pg-citus0: # citus shard 0
      hosts: { 10.10.10.10: { pg_seq: 1, pg_role: primary } }
      vars: { pg_cluster: pg-citus0 , pg_group: 0 }
    pg-citus1: # citus shard 1
      hosts: { 10.10.10.11: { pg_seq: 1, pg_role: primary } }
      vars: { pg_cluster: pg-citus1 , pg_group: 1 }
    pg-citus2: # citus shard 2
      hosts: { 10.10.10.12: { pg_seq: 1, pg_role: primary } }
      vars: { pg_cluster: pg-citus2 , pg_group: 2 }
    pg-citus3: # citus shard 3
      hosts: { 10.10.10.13: { pg_seq: 1, pg_role: primary } }
      vars: { pg_cluster: pg-citus3 , pg_group: 3 }

Pigsty provides dedicated monitoring dashboards for horizontal sharding clusters, making it easy to compare performance and load across shards, but this requires using the above entity naming convention.

There are also other identity parameters for special scenarios, such as pg_upstream for specifying backup clusters/cascading replication upstream, gp_role for Greenplum cluster identity, pg_exporters for external monitoring instances, pg_offline_query for offline query instances, etc. See PG_ID parameter docs.

Monitoring Label System

Pigsty provides an out-of-box monitoring system that uses the above identity parameters to identify various PostgreSQL entities.

pg_up{cls="pg-test", ins="pg-test-1", ip="10.10.10.11", job="pgsql"}
pg_up{cls="pg-test", ins="pg-test-2", ip="10.10.10.12", job="pgsql"}
pg_up{cls="pg-test", ins="pg-test-3", ip="10.10.10.13", job="pgsql"}

For example, the cls, ins, ip labels correspond to cluster name, instance name, and node IP—the identifiers for these three core entities. They appear along with the job label in all native monitoring metrics collected by VictoriaMetrics and VictoriaLogs log streams.

The job name for collecting PostgreSQL metrics is fixed as pgsql; The job name for monitoring remote PG instances is fixed as pgrds. The job name for collecting PostgreSQL CSV logs is fixed as postgres; The job name for collecting pgbackrest logs is fixed as pgbackrest, other PG components collect logs via job: syslog.

Additionally, some entity identity labels appear in specific entity-related monitoring metrics, such as:

• datname: Database name, if a metric belongs to a specific database.
• relname: Table name, if a metric belongs to a specific table.
• idxname: Index name, if a metric belongs to a specific index.
• funcname: Function name, if a metric belongs to a specific function.
• seqname: Sequence name, if a metric belongs to a specific sequence.
• query: Query fingerprint, if a metric belongs to a specific query.

2 - ETCD Cluster Model

The ETCD module organizes ETCD in production as clusters—logical entities composed of a group of ETCD instances associated through the Raft consensus protocol.

Each cluster is an autonomous distributed key-value storage unit consisting of at least one ETCD instance, exposing service capabilities through client ports.

There are three core entities in Pigsty’s ETCD module:

• Cluster: An autonomous ETCD service unit serving as the top-level namespace for other entities.
• Instance: A single ETCD server process running on a node, participating in Raft consensus.
• Node: A hardware resource abstraction running Linux + Systemd environment, implicitly declared.

Compared to PostgreSQL clusters, the ETCD cluster model is simpler, without Services or complex Role distinctions. All ETCD instances are functionally equivalent, electing a Leader through the Raft protocol while others become Followers. During scale-out intermediate states, non-voting Learner instance members are also allowed.

Examples

Let’s look at a concrete example with a three-node ETCD cluster:

etcd:
  hosts:
    10.10.10.10: { etcd_seq: 1 }
    10.10.10.11: { etcd_seq: 2 }
    10.10.10.12: { etcd_seq: 3 }
  vars:
    etcd_cluster: etcd

The above config fragment defines a three-node ETCD cluster with these related entities:

Identity Parameters

Pigsty uses the ETCD parameter group to assign deterministic identities to each ETCD module entity. Two parameters are required:

With cluster name defined at cluster level and instance number assigned at instance level, Pigsty automatically generates unique identifiers for each entity based on rules:

The ETCD module does not assign additional identity to host nodes; nodes are identified by their existing hostname or IP address.

Ports & Protocols

Each ETCD instance listens on the following two ports:

ETCD clusters enable TLS encrypted communication by default and use RBAC authentication mechanism. Clients need correct certificates and passwords to access ETCD services.

Cluster Size

As a distributed coordination service, ETCD cluster size directly affects availability, requiring more than half (quorum) of nodes to be alive to maintain service.

Therefore, even-numbered ETCD clusters are meaningless, and clusters over five nodes are uncommon. Typical sizes are single-node, three-node, and five-node.

Monitoring Label System

Pigsty provides an out-of-box monitoring system that uses the above identity parameters to identify various ETCD entities.

etcd_up{cls="etcd", ins="etcd-1", ip="10.10.10.10", job="etcd"}
etcd_up{cls="etcd", ins="etcd-2", ip="10.10.10.11", job="etcd"}
etcd_up{cls="etcd", ins="etcd-3", ip="10.10.10.12", job="etcd"}

For example, the cls, ins, ip labels correspond to cluster name, instance name, and node IP—the identifiers for these three core entities. They appear along with the job label in all ETCD monitoring metrics collected by VictoriaMetrics. The job name for collecting ETCD metrics is fixed as etcd.

3 - MinIO Cluster Model

The MinIO module organizes MinIO in production as clusters—logical entities composed of a group of distributed MinIO instances, collectively providing highly available object storage services.

Each cluster is an autonomous S3-compatible object storage unit consisting of at least one MinIO instance, exposing service capabilities through the S3 API port.

There are three core entities in Pigsty’s MinIO module:

• Cluster: An autonomous MinIO service unit serving as the top-level namespace for other entities.
• Instance: A single MinIO server process running on a node, managing local disk storage.
• Node: A hardware resource abstraction running Linux + Systemd environment, implicitly declared.

Additionally, MinIO has the concept of Storage Pool, used for smooth cluster scaling. A cluster can contain multiple storage pools, each composed of a group of nodes and disks.

Deployment Modes

MinIO supports three main deployment modes for different scenarios:

SNSD mode can use any directory as storage for quick experimentation; SNMD and MNMD modes require real disk mount points, otherwise startup is refused.

Examples

Let’s look at a concrete multi-node multi-drive example with a four-node MinIO cluster:

minio:
  hosts:
    10.10.10.10: { minio_seq: 1 }
    10.10.10.11: { minio_seq: 2 }
    10.10.10.12: { minio_seq: 3 }
    10.10.10.13: { minio_seq: 4 }
  vars:
    minio_cluster: minio
    minio_data: '/data{1...4}'
    minio_node: '${minio_cluster}-${minio_seq}.pigsty'

The above config fragment defines a four-node MinIO cluster with four disks per node:

Identity Parameters

Pigsty uses the MINIO parameter group to assign deterministic identities to each MinIO module entity. Two parameters are required:

With cluster name defined at cluster level and instance number assigned at instance level, Pigsty automatically generates unique identifiers for each entity based on rules:

The MinIO module does not assign additional identity to host nodes; nodes are identified by their existing hostname or IP address. The minio_node parameter generates node names for MinIO cluster internal use (written to /etc/hosts for cluster discovery), not host node identity.

Core Configuration Parameters

Beyond identity parameters, the following parameters are critical for MinIO cluster configuration:

These parameters together determine MinIO’s core config MINIO_VOLUMES:

• SNSD: Direct minio_data value, e.g., /data/minio
• SNMD: Expanded minio_data directories, e.g., /data{1...4}
• MNMD: Combined minio_node and minio_data, e.g., https://minio-{1...4}.pigsty:9000/data{1...4}

Ports & Services

Each MinIO instance listens on the following ports:

MinIO enables HTTPS encrypted communication by default (controlled by minio_https). This is required for backup tools like pgBackREST to access MinIO.

Multi-node MinIO clusters can be accessed through any node. Best practice is to use a load balancer (e.g., HAProxy + VIP) for unified access point.

Resource Provisioning

After MinIO cluster deployment, Pigsty automatically creates the following resources (controlled by minio_provision):

Default Buckets (defined by minio_buckets):

Default Users (defined by minio_users):

pgbackrest is used for PostgreSQL cluster backups; s3user_meta and s3user_data are reserved users not actively used.

Monitoring Label System

Pigsty provides an out-of-box monitoring system that uses the above identity parameters to identify various MinIO entities.

minio_up{cls="minio", ins="minio-1", ip="10.10.10.10", job="minio"}
minio_up{cls="minio", ins="minio-2", ip="10.10.10.11", job="minio"}
minio_up{cls="minio", ins="minio-3", ip="10.10.10.12", job="minio"}
minio_up{cls="minio", ins="minio-4", ip="10.10.10.13", job="minio"}

For example, the cls, ins, ip labels correspond to cluster name, instance name, and node IP—the identifiers for these three core entities. They appear along with the job label in all MinIO monitoring metrics collected by VictoriaMetrics. The job name for collecting MinIO metrics is fixed as minio.

4 - Redis Cluster Model

The Redis module organizes Redis in production as clusters—logical entities composed of a group of Redis instances deployed on one or more nodes.

Each cluster is an autonomous high-performance cache/storage unit consisting of at least one Redis instance, exposing service capabilities through ports.

There are three core entities in Pigsty’s Redis module:

• Cluster: An autonomous Redis service unit serving as the top-level namespace for other entities.
• Instance: A single Redis server process running on a specific port on a node.
• Node: A hardware resource abstraction running Linux + Systemd environment, can host multiple Redis instances, implicitly declared.

Unlike PostgreSQL, Redis uses a single-node multi-instance deployment model: one physical/virtual machine node typically deploys multiple Redis instances to fully utilize multi-core CPUs. Therefore, nodes and instances have a 1:N relationship. Additionally, production typically advises against Redis instances with memory > 12GB.

Operating Modes

Redis has three different operating modes, specified by the redis_mode parameter:

• Standalone: Default mode, replication via replica_of parameter. Requires additional Sentinel cluster for HA.
• Sentinel: Stores no business data, dedicated to monitoring standalone Redis clusters for auto-failover; multi-node itself provides HA.
• Native Cluster: Data auto-sharded across multiple primaries, each can have multiple replicas, built-in HA, no sentinel needed.

Examples

Let’s look at concrete examples for each mode:

Standalone Cluster

Classic master-replica on a single node:

redis-ms:
  hosts:
    10.10.10.10:
      redis_node: 1
      redis_instances:
        6379: { }
        6380: { replica_of: '10.10.10.10 6379' }
  vars:
    redis_cluster: redis-ms
    redis_password: 'redis.ms'
    redis_max_memory: 64MB

Sentinel Cluster

Three sentinel instances on a single node for monitoring standalone clusters. Sentinel clusters specify monitored standalone clusters via redis_sentinel_monitor:

redis-sentinel:
  hosts:
    10.10.10.11:
      redis_node: 1
      redis_instances: { 26379: {}, 26380: {}, 26381: {} }
  vars:
    redis_cluster: redis-sentinel
    redis_password: 'redis.sentinel'
    redis_mode: sentinel
    redis_max_memory: 16MB
    redis_sentinel_monitor:
      - { name: redis-ms, host: 10.10.10.10, port: 6379, password: redis.ms, quorum: 2 }

Native Cluster

A Redis native distributed cluster with two nodes and six instances (minimum spec: 3 primaries, 3 replicas):

redis-test:
  hosts:
    10.10.10.12: { redis_node: 1, redis_instances: { 6379: {}, 6380: {}, 6381: {} } }
    10.10.10.13: { redis_node: 2, redis_instances: { 6379: {}, 6380: {}, 6381: {} } }
  vars:
    redis_cluster: redis-test
    redis_password: 'redis.test'
    redis_mode: cluster
    redis_max_memory: 32MB

This creates a 3 primary 3 replica native Redis cluster.

Identity Parameters

Pigsty uses the REDIS parameter group to assign deterministic identities to each Redis module entity. Three parameters are required:

With cluster name defined at cluster level and node number/instance definition assigned at node level, Pigsty automatically generates unique identifiers for each entity:

The Redis module does not assign additional identity to host nodes; nodes are identified by their existing hostname or IP address. redis_node is used for instance naming, not host node identity.

Instance Definition

redis_instances is a JSON object with port number as key and instance config as value:

redis_instances:
  6379: { }                                      # Primary instance, no extra config
  6380: { replica_of: '10.10.10.10 6379' }       # Replica, specify upstream primary
  6381: { replica_of: '10.10.10.10 6379' }       # Replica, specify upstream primary

Each Redis instance listens on a unique port within the node. You can choose any port number, but avoid system reserved ports (< 1024) or conflicts with Pigsty used ports. The replica_of parameter sets replication relationship in standalone mode, format '<ip> <port>', specifying upstream primary address and port.

Additionally, each Redis node runs a Redis Exporter collecting metrics from all local instances:

Redis’s single-node multi-instance deployment model has some limitations:

• Node Exclusive: A node can only belong to one Redis cluster, not assigned to different clusters simultaneously.
• Port Unique: Redis instances on the same node must use different ports to avoid conflicts.
• Password Shared: Multiple instances on the same node cannot have different passwords (redis_exporter limitation).
• Manual HA: Standalone Redis clusters require additional Sentinel configuration for auto-failover.

Monitoring Label System

Pigsty provides an out-of-box monitoring system that uses the above identity parameters to identify various Redis entities.

redis_up{cls="redis-ms", ins="redis-ms-1-6379", ip="10.10.10.10", job="redis"}
redis_up{cls="redis-ms", ins="redis-ms-1-6380", ip="10.10.10.10", job="redis"}

For example, the cls, ins, ip labels correspond to cluster name, instance name, and node IP—the identifiers for these three core entities. They appear along with the job label in all Redis monitoring metrics collected by VictoriaMetrics. The job name for collecting Redis metrics is fixed as redis.

5 - INFRA Node Model

The INFRA module plays a special role in Pigsty: it’s not a traditional “cluster” but rather a management hub composed of a group of infrastructure nodes, providing core services for the entire Pigsty deployment. Each INFRA node is an autonomous infrastructure service unit running core components like Nginx, Grafana, and VictoriaMetrics, collectively providing observability and management capabilities for managed database clusters.

There are two core entities in Pigsty’s INFRA module:

• Node: A server running infrastructure components—can be bare metal, VM, container, or Pod.
• Component: Various infrastructure services running on nodes, such as Nginx, Grafana, VictoriaMetrics, etc.

INFRA nodes typically serve as Admin Nodes, the control plane of Pigsty.

Component Composition

Each INFRA node runs the following core components:

These components together form Pigsty’s observability infrastructure.

Examples

Let’s look at a concrete example with a two-node INFRA deployment:

infra:
  hosts:
    10.10.10.10: { infra_seq: 1 }
    10.10.10.11: { infra_seq: 2 }

The above config fragment defines a two-node INFRA deployment:

For production environments, deploying at least two INFRA nodes is recommended for infrastructure component redundancy.

Identity Parameters

Pigsty uses the INFRA_ID parameter group to assign deterministic identities to each INFRA module entity. One parameter is required:

With node sequence assigned at node level, Pigsty automatically generates unique identifiers for each entity based on rules:

The INFRA module assigns infra-N format identifiers to nodes for distinguishing multiple infrastructure nodes in the monitoring system. However, this doesn’t change the node’s hostname or system identity; nodes still use their existing hostname or IP address for identification.

Service Portal

INFRA nodes provide unified web service entry through Nginx. The infra_portal parameter defines services exposed through Nginx:

infra_portal:
  home         : { domain: i.pigsty }
  grafana      : { domain: g.pigsty, endpoint: "${admin_ip}:3000", websocket: true }
  prometheus   : { domain: p.pigsty, endpoint: "${admin_ip}:8428" }   # VMUI
  alertmanager : { domain: a.pigsty, endpoint: "${admin_ip}:9059" }

Users access different domains, and Nginx routes requests to corresponding backend services:

Accessing Pigsty services via domain names is recommended over direct IP + port.

Deployment Scale

The number of INFRA nodes depends on deployment scale and HA requirements:

In singleton deployment, INFRA components share the same node with PGSQL, ETCD, etc. In small-scale deployments, INFRA nodes typically also serve as “Admin Node” / backup admin node and local software repository (/www/pigsty). In larger deployments, these responsibilities can be separated to dedicated nodes.

Monitoring Label System

Pigsty’s monitoring system collects metrics from INFRA components themselves. Unlike database modules, each component in the INFRA module is treated as an independent monitoring object, distinguished by the cls (class) label.

Using a two-node INFRA deployment (infra_seq: 1 and infra_seq: 2) as example, component monitoring labels are:

All INFRA component metrics use a unified job="infra" label, distinguished by the cls label:

nginx_up{cls="nginx", ins="nginx-1", ip="10.10.10.10", job="infra"}
grafana_info{cls="grafana", ins="grafana-1", ip="10.10.10.10", job="infra"}
vm_app_version{cls="vmetrics", ins="vmetrics-1", ip="10.10.10.10", job="infra"}
vlogs_rows_ingested_total{cls="vlogs", ins="vlogs-1", ip="10.10.10.10", job="infra"}
alertmanager_alerts{cls="alertmanager", ins="alertmanager-1", ip="10.10.10.10", job="infra"}