Skip to content

Monitoring and Alerting

How Pigsty monitors Silo, including its Metrics V3 endpoint, Grafana dashboards, and alert rules.

Administration UI

Silo provides an administration UI through minio_admin_port, which defaults to 9001. Access it directly at https://<node-ip>:9001.

Some configuration templates also expose the administration entry point at m.pigsty. Login credentials come from minio_access_key and minio_secret_key.

HTTPS and Certificate Trust

Object storage uses HTTPS certificates issued by the Pigsty CA by default. Browsers and container clients must trust that CA. Do not substitute disabled certificate verification for a correct trust configuration in production.


Collection Paths

Silo retains the stable job="minio", cls, ins, ip, and instance identity labels and uses flavor="silo":

BackendMetric PathTarget and Labels
SiloVictoriaMetrics scrapes https://<instance>:9000/minio/metrics/v3job=minio, flavor=silo

Each instance’s FileSD target is written to /infra/targets/minio/<minio_cluster>-<minio_seq>.yml.

Silo registers one Metrics V3 root endpoint, which exposes cluster, system, API, and aggregated usage metrics. Pigsty drops samples with a non-empty bucket label and does not register separate per-bucket or replication endpoints, keeping time-series cardinality under control.


Grafana Dashboards

Pigsty provides the compatibility-named MinIO Overview / MinIO Instance dashboards for Silo Metrics V3 data, system logs, and instance state.

minio-overview.jpg


Alert Rules

The current files/victoria/rules/minio.yml defines five alerts for Silo:

AlertCondition SummarySeverity
MinioServerDownminio_up < 1 for 1 minuteCRIT
MinioNodeOfflineFive-minute average offline-node count above 0 for 3 minutesWARN
MinioDiskOfflineFive-minute average offline-drive count above 0 for 3 minutesWARN
MinioErasureSetUnhealthyAny erasure set’s overall health below 1 for 1 minuteCRIT
MinioClusterCapacityHighUsable capacity utilization above 90% for 15 minutesWARN

Key expressions use Metrics V3 names:

minio_up < 1

max by (cls) (
  avg_over_time(minio_cluster_health_nodes_offline_count{job="minio"}[5m])
) > 0

max by (cls) (
  avg_over_time(minio_cluster_health_drives_offline_count{job="minio"}[5m])
) > 0

min by (cls) (
  minio_cluster_erasure_set_overall_health{job="minio"}
  or (minio_cluster_erasure_set_overall_write_quorum{job="minio"} * 0)
) < 1

max by (cls) (
  1 - (
    (minio_cluster_health_capacity_usable_free_bytes{job="minio"}
     or (minio_cluster_health_capacity_usable_total_bytes{job="minio"} * 0))
    / minio_cluster_health_capacity_usable_total_bytes{job="minio"}
  )
) > 0.90

Was this page helpful?