Pigsty Blog Articles

• 1: Featured
  • 1.1: Scaling Postgres to the Next Level at OpenAI
  • 1.2: Self-Hosting Supabase on PostgreSQL
  • 1.3: PostgreSQL is Eating the Database World
  • 1.4: Database in K8S: Pros & Cons
• 2: Pigsty
  • 2.1: Pigsty v4.5: 575 Extensions, Kafka, MySQL, Valkey, and Silo
  • 2.2: Pigsty v4.3: 510 Extensions & Ubuntu 26
  • 2.3: Pigsty v4.2: 12 Kernels
  • 2.4: Pigsty v4.1: Day-Zero 18.2 Support
  • 2.5: Pigsty v4.0: Victoria Stack + Security Hardening
  • 2.6: Pigsty v3.7: PostgreSQL Magneto Award, PG18 Deep Support
  • 2.7: Pigsty v3.6: The Ultimate PostgreSQL Distribution
  • 2.8: Pigsty v3.5: 4K Stars, PG18 Beta, 421 Extensions
  • 2.9: Pigsty v3.4: PITR Enhancement, Locale Best Practices, Auto Certificates
  • 2.10: Pigsty v3.3: 404 Extensions, Turnkey Apps, New Website
  • 2.11: Pigsty v3.2: The pig CLI, Full ARM Support, Supabase & Grafana Enhancements
  • 2.12: Pigsty v3.1: One-Click Supabase, PG17 Default, ARM & Ubuntu 24
  • 2.13: Pigsty v3.0: Pluggable Kernels & 340 Extensions
  • 2.14: Pigsty v2.7: The Extension Superpack
  • 2.15: Pigsty v2.6: PostgreSQL Crashes the OLAP Party
  • 2.16: Pigsty v2.5: Ubuntu & PG16
  • 2.17: Pigsty v2.4: Monitor Cloud RDS
  • 2.18: Pigsty v2.3: Richer App Ecosystem
  • 2.19: Pigsty v2.2: Monitoring System Reborn
  • 2.20: Pigsty v2.1: Vector + Full PG Version Support!
  • 2.21: Pigsty v2.0: Open-Source RDS PostgreSQL Alternative
  • 2.22: Pigsty v1.5: Docker Application Support, Infrastructure Self-Monitoring
  • 2.23: Pigsty v1.4: Modular Architecture, MatrixDB Data Warehouse Support
  • 2.24: Pigsty v1.3: Redis Support, PGCAT Overhaul, PGSQL Enhancements
  • 2.25: Pigsty v1.2: PG14 Default, Monitor Existing PG
  • 2.26: Pigsty v1.1: Homepage, Jupyter, Pev2, PgBadger
  • 2.27: Pigsty v1.0: GA Release with Monitoring Overhaul
  • 2.28: Pigsty v0.9: CLI + Logs
  • 2.29: Pigsty v0.8: Service Provisioning
  • 2.30: Pigsty v0.7: Monitor-Only Deployments
  • 2.31: Pigsty v0.6: Provisioning Upgrades
  • 2.32: Pigsty v0.5: Declarative DB Templates
  • 2.33: Pigsty v0.4: PG13 and Better Docs
  • 2.34: Pigsty v0.3: First Public Beta
• 3: Cloud
  • 3.1: Don't Run AI Assistant on Cloud
  • 3.2: Did RedNote Exit the Cloud?
  • 3.3: Alipay, Taobao, Xianyu Went Dark. Smells Like a Message Queue Meltdown.
  • 3.4: Cloudflare’s Nov 18 Outage, Translated and Dissected
  • 3.5: Alicloud “Borrowed” Supabase. This Is What Happens When Giants Strip-Mine Open-Source.
  • 3.6: AWS’s Official DynamoDB Outage Postmortem
  • 3.7: How One AWS DNS Failure Cascaded Across Half the Internet
  • 3.8: Column: Cloud-Exit
  • 3.9: KubeSphere: Trust Crisis Behind Open-Source Supply Cut
  • 3.10: Alicloud’s rds_duckdb: Tribute or Rip-Off?
  • 3.11: Escaping Cloud Computing Scam Mills: The Big Fool Paying for Pain
  • 3.12: OpenAI Global Outage Postmortem: K8S Circular Dependencies
  • 3.13: WordPress Community Civil War: On Community Boundary Demarcation
  • 3.14: Cloud Database: Michelin Prices for Cafeteria Pre-made Meals
  • 3.15: Alibaba-Cloud: High Availability Disaster Recovery Myth Shattered
  • 3.16: Amateur Hour Opera: Alibaba-Cloud PostgreSQL Disaster Chronicle
  • 3.17: What Can We Learn from NetEase Cloud Music's Outage?
  • 3.18: Blue Screen Friday: Amateur Hour on Both Sides
  • 3.19: How Ahrefs Saved US$400M by NOT Going to the Cloud
  • 3.20: Database Deletion Supreme - Google Cloud Nuked a Major Fund's Entire Cloud Account
  • 3.21: Cloud Dark Forest: Exploding Cloud Bills with Just S3 Bucket Names
  • 3.22: Cloudflare Roundtable Interview and Q&A Record
  • 3.23: What Can We Learn from Tencent Cloud's Major Outage?
  • 3.24: Cloudflare - The Cyber Buddha That Destroys Public Cloud
  • 3.25: Can Luo Yonghao Save Toothpaste Cloud?
  • 3.26: Analyzing Alibaba-Cloud Server Computing Cost
  • 3.27: Will DBAs Be Eliminated by Cloud?
  • 3.28: Cloud-Exit High Availability Secret: Rejecting Complexity Masturbation
  • 3.29: S3: Elite to Mediocre
  • 3.30: Cloud Exit FAQ: DHH Saves Millions
  • 3.31: From Cost-Reduction Jokes to Real Cost Reduction and Efficiency
  • 3.32: Reclaim Hardware Bonus from the Cloud
  • 3.33: What Can We Learn from Alibaba-Cloud's Global Outage?
  • 3.34: Harvesting Alibaba-Cloud Wool, Building Your Digital Homestead
  • 3.35: Cloud Computing Mudslide: Deconstructing Public Cloud with Data
  • 3.36: Cloud Exit Odyssey: Time to Leave Cloud?
  • 3.37: DHH: Cloud-Exit Saves Over Ten Million, More Than Expected!
  • 3.38: FinOps: Endgame Cloud-Exit
  • 3.39: Why Isn't Cloud Computing More Profitable Than Sand Mining?
  • 3.40: SLA: Placebo or Insurance?
  • 3.41: EBS: Pig Slaughter Scam
  • 3.42: Garbage QCloud CDN: From Getting Started to Giving Up?
  • 3.43: Refuting "Why You Still Shouldn't Hire a DBA"
  • 3.44: Paradigm Shift: From Cloud to Local-First
  • 3.45: Are Cloud Databases an IQ Tax?
  • 3.46: Cloud RDS: From Database Drop to Exit
  • 3.47: Is DBA Still a Good Job?
• 4: PostgreSQL
  • 4.1: Happy 30th Birthday, PostgreSQL
  • 4.2: Extensions for Everyone
  • 4.3: 504 Extensions: Expand the PostgreSQL Landscape
  • 4.4: From AGPL to Apache: Why I Changed Pigsty's License
  • 4.5: Git for Data: Instant PostgreSQL Database Cloning
  • 4.6: Why PostgreSQL Will Dominate the AI Era
  • 4.7: Forging a China-Rooted, Global PostgreSQL Distro
  • 4.8: PG Extension Cloud: Unlocking PostgreSQL’s Entire Ecosystem
  • 4.9: The PostgreSQL 'Supply Cut' and Trust Issues in Software Supply Chain
  • 4.10: PostgreSQL Dominates Database World, but Who Will Devour PG?
  • 4.11: PostgreSQL Has Dominated the Database World
  • 4.12: PGDG Cuts Off Mirror Sync Channel
  • 4.13: Postgres Extension Day - See You There!
  • 4.14: OrioleDB is Coming! 4x Performance, Eliminates Pain Points, Storage-Compute Separation
  • 4.15: OpenHalo: MySQL Wire-Compatible PostgreSQL is Here!
  • 4.16: PGFS: Using Database as a Filesystem
  • 4.17: PostgreSQL Ecosystem Frontier Developments
  • 4.18: Pig, The Postgres Extension Wizard
  • 4.19: Don't Upgrade! Released and Immediately Pulled - Even PostgreSQL Isn't Immune to Epic Fails
  • 4.20: PostgreSQL 12 End-of-Life, PG 17 Takes the Throne
  • 4.21: The ideal way to deliver PostgreSQL Extensions
  • 4.22: PostgreSQL Convention 2024
  • 4.23: PostgreSQL 17 Released: No More Pretending!
  • 4.24: Can PostgreSQL Replace Microsoft SQL Server?
  • 4.25: Whoever Integrates DuckDB Best Wins the OLAP World
  • 4.26: StackOverflow 2024 Survey: PostgreSQL Has Gone Completely Berserk
  • 4.27: Self-Hosting Dify with PG, PGVector, and Pigsty
  • 4.28: PGCon.Dev 2024, The conf that shutdown PG for a week
  • 4.29: PostgreSQL 17 Beta1 Released!
  • 4.30: Why PostgreSQL is the Future Standard?
  • 4.31: Will PostgreSQL Change Its License?
  • 4.32: Postgres is eating the database world
  • 4.33: Technical Minimalism: Just Use PostgreSQL for Everything
  • 4.34: New PostgreSQL Ecosystem Player: ParadeDB
  • 4.35: PostgreSQL's Impressive Scalability
  • 4.36: PostgreSQL Outlook for 2024
  • 4.37: PostgreSQL Wins 2024 Database of the Year Award! (Fifth Time)
  • 4.38: PostgreSQL Macro Query Optimization with pg_stat_statements
  • 4.39: FerretDB: PostgreSQL Disguised as MongoDB
  • 4.40: How to Use pg_filedump for Data Recovery?
  • 4.41: Vector is the New JSON
  • 4.42: PostgreSQL, The most successful database
  • 4.43: AI Large Models and Vector Database PGVector
  • 4.44: How Powerful is PostgreSQL Really?
  • 4.45: Why PostgreSQL is the Most Successful Database?
  • 4.46: Ready-to-Use PostgreSQL Distribution: Pigsty
  • 4.47: Why Does PostgreSQL Have a Bright Future?
  • 4.48: Implementing Advanced Fuzzy Search
  • 4.49: Localization and Collation Rules in PostgreSQL
  • 4.50: PG Replica Identity Explained
  • 4.51: PostgreSQL Logical Replication Deep Dive
  • 4.52: A Methodology for Diagnosing PostgreSQL Slow Queries
  • 4.53: Incident-Report: Patroni Failure Due to Time Travel
  • 4.54: Online Primary Key Column Type Change
  • 4.55: Golden Monitoring Metrics: Errors, Latency, Throughput, Saturation
  • 4.56: Database Cluster Management Concepts and Entity Naming Conventions
  • 4.57: PostgreSQL's KPI
  • 4.58: Online PostgreSQL Column Type Migration
  • 4.59: Frontend-Backend Communication Wire Protocol
  • 4.60: Transaction Isolation Level Considerations
  • 4.61: Incident: PostgreSQL Extension Installation Causes Connection Failure
  • 4.62: CDC Change Data Capture Mechanisms
  • 4.63: Locks in PostgreSQL
  • 4.64: O(n2) Complexity of GIN Search
  • 4.65: PostgreSQL Common Replication Topology Plans
  • 4.66: Warm Standby: Using pg_receivewal
  • 4.67: Incident-Report: Connection-Pool Contamination Caused by pg_dump
  • 4.68: PostgreSQL Data Page Corruption Repair
  • 4.69: Relation Bloat Monitoring and Management
  • 4.70: Getting Started with PipelineDB
  • 4.71: TimescaleDB Quick Start
  • 4.72: Incident-Report: Integer Overflow from Rapid Sequence Number Consumption
  • 4.73: Incident-Report: PostgreSQL Transaction ID Wraparound
  • 4.74: GeoIP Geographic Reverse Lookup Optimization
  • 4.75: PostgreSQL Trigger Usage Considerations
  • 4.76: PostgreSQL Development Convention (2018 Edition)
  • 4.77: What Are PostgreSQL's Advantages?
  • 4.78: Efficient Administrative Region Lookup with PostGIS
  • 4.79: KNN Ultimate Optimization: From RDS to PostGIS
  • 4.80: Monitoring Table Size in PostgreSQL
  • 4.81: PgAdmin Installation and Configuration
  • 4.82: Incident-Report: Uneven Load Avalanche
  • 4.83: Bash and psql Tips
  • 4.84: Distinct On: Remove Duplicate Data
  • 4.85: Function Volatility Classification Levels
  • 4.86: Implementing Mutual Exclusion Constraints with Exclude
  • 4.87: PostgreSQL Routine Maintenance
  • 4.88: Backup and Recovery Methods Overview
  • 4.89: PgBackRest2 Documentation
  • 4.90: Pgbouncer Quick Start
  • 4.91: PostgreSQL Server Log Regular Configuration
  • 4.92: Testing Disk Performance with FIO
  • 4.93: Using sysbench to Test PostgreSQL Performance
  • 4.94: Changing Engines Mid-Flight — PostgreSQL Zero-Downtime Data Migration
  • 4.95: Finding Unused Indexes
  • 4.96: Batch Configure SSH Passwordless Login
  • 4.97: Wireshark Packet Capture Protocol Analysis
  • 4.98: The Versatile file_fdw — Reading System Information from Your Database
  • 4.99: Common Linux Statistics CLI Tools
  • 4.100: Installing PostGIS from Source
  • 4.101: Go Database Tutorial: database/sql
  • 4.102: Implementing Cache Synchronization with Go and PostgreSQL
  • 4.103: Auditing Data Changes with Triggers
  • 4.104: Building an ItemCF Recommender in Pure SQL
  • 4.105: UUID Properties, Principles and Applications
  • 4.106: PostgreSQL MongoFDW Installation and Deployment
• 5: AI
  • 5.1: Cyber Dharma: A New Engineering Answer to Ancient Questions
  • 5.2: Burning Hundreds of Millions of Tokens a Day. Then What?
  • 5.3: Why PostgreSQL Won in the AI Era
  • 5.4: AGI Is Here. Do You Have a Ticket?
  • 5.5: Can You Distill an Expert?
  • 5.6: Local AI's Inflection Point: 2027
  • 5.7: Yes, I Use AI to Write
  • 5.8: LLMs Have Emotions: Claude's Internals Reveal Steerable Emotion Vectors
• 6: Database
  • 6.1: Two Months into Maintaining a MinIO Fork
  • 6.2: Palantir's 'Ontology' Hustle
  • 6.3: MinIO Is Dead, Long Live MinIO
  • 6.4: When Coding Becomes Cheap, What Still Matters?
  • 6.5: The Agent Moat: Runtime
  • 6.6: AI Ripped the Skin Off Software
  • 6.7: New Programmers in the AI Era: Where Do You Go?
  • 6.8: The Great Software Meltdown: When Translation Layers Get Squashed
  • 6.9: Agent OS: We're Building DOS Again
  • 6.10: Claude Code Observability
  • 6.11: Claude Code Quick Start: Using Alternative LLMs at 1/10 the Cost
  • 6.12: Data 2025: Year in Review with Mike Stonebraker
  • 6.13: What Database Does AI Agent Need?
  • 6.14: MySQL and Baijiu: The Internet’s Obedience Test
  • 6.15: Victoria: The Observability Stack That Slaps the Industry
  • 6.16: MinIO Is Dead. Who Picks Up the Pieces?
  • 6.17: MinIO is Dead
  • 6.18: When Answers Become Abundant, Questions Become the New Currency
  • 6.19: On Trusting Open-Source Supply Chains
  • 6.20: Don't Run Docker Postgres for Production!
  • 6.21: DDIA 2nd Edition, Chinese Translation
  • 6.22: Column: Database Guru
  • 6.23: Dongchedi Just Exposed “Smart Driving.” Where’s Our Dongku-Di?
  • 6.24: Google AI Toolbox: Production-Ready Database MCP is Here?
  • 6.25: Where Will Databases and DBAs Go in the AI Era?
  • 6.26: Stop Arguing, The AI Era Database Has Been Settled
  • 6.27: Open Data Standards: Postgres, OTel, and Iceberg
  • 6.28: The Lost Decade of Small Data
  • 6.29: Scaling Postgres to the Next Level at OpenAI
  • 6.30: How Many Shops Has etcd Torched?
  • 6.31: In the AI Era, Software Starts at the Database
  • 6.32: MySQL vs. PostgreSQL @ 2025
  • 6.33: Database Planet Collision: When PG Falls for DuckDB
  • 6.34: Comparing Oracle and PostgreSQL Transaction Systems
  • 6.35: Database as Business Architecture
  • 6.36: 7 Databases in 7 Weeks (2025)
  • 6.37: Solving Poker 24 with a Single SQL Query
  • 6.38: Self-Hosting Supabase on PostgreSQL
  • 6.39: Modern Hardware for Future Databases
  • 6.40: Can MySQL Still Catch Up with PostgreSQL?
  • 6.41: Open-Source "Tyrant" Linus's Purge
  • 6.42: Optimize Bio Cores First, CPU Cores Second
  • 6.43: MongoDB Has No Future: Good Marketing Can't Save a Rotten Mango
  • 6.44: MongoDB: Now Powered by PostgreSQL?
  • 6.45: Switzerland Mandates Open-Source for Government Software
  • 6.46: MySQL is dead, Long live PostgreSQL!
  • 6.47: CVE-2024-6387 SSH Vulnerability Fix
  • 6.48: Can Oracle Still Save MySQL?
  • 6.49: Oracle Finally Killed MySQL
  • 6.50: MySQL Performance Declining: Where is Sakila Going?
  • 6.51: Can Chinese Domestic Databases Really Compete?
  • 6.52: The $20 Brother PolarDB: What Should Databases Actually Cost?
  • 6.53: Redis Going Non-Open-Source is a Disgrace to "Open-Source" and Public Cloud
  • 6.54: How Can MySQL's Correctness Be This Garbage?
  • 6.55: Database in K8S: Pros & Cons
  • 6.56: Are Specialized Vector Databases Dead?
  • 6.57: Are Databases Really Being Strangled?
  • 6.58: Which EL-Series OS Distribution Is Best?
  • 6.59: What Kind of Self-Reliance Do Infrastructure Software Need?
  • 6.60: Back to Basics: Tech Reflection Chronicles
  • 6.61: Database Demand Hierarchy Pyramid
  • 6.62: Are Microservices a Bad Idea?
  • 6.63: NewSQL: Distributive Nonsense
  • 6.64: Time to Say Goodbye to GPL
  • 6.65: Is running postgres in docker a good idea?
  • 6.66: Understanding Time - Leap Years, Leap Seconds, Time and Time Zones
  • 6.67: Understanding Character Encoding Principles
  • 6.68: Concurrency Anomalies Explained
  • 6.69: Blockchain and Distributed Databases
  • 6.70: Consistency: An Overloaded Term
  • 6.71: Why Study Database Principles

1 - Featured

1.1 - Scaling Postgres to the Next Level at OpenAI

At PGConf.Dev 2025, Bohan Zhang from OpenAI shared how they scale PostgreSQL to millions of QPS using a single-primary, multi-replica architecture—proving that PostgreSQL can handle massive read workloads without sharding. Read more

1.2 - Self-Hosting Supabase on PostgreSQL

Supabase is great, owning your own Supabase is even better! This tutorial covers how to self-host production-grade Supabase on local/cloud VMs using Pigsty. Read more

1.3 - PostgreSQL is Eating the Database World

PostgreSQL is not just a simple relational database, but an abstract framework for data management with the power to devour the entire database world. “Just use Postgres for everything” has become a mainstream best practice. Read more

1.4 - Database in K8S: Pros & Cons

Whether databases should be housed in Kubernetes/Docker remains highly controversial. While K8s excels in managing stateless applications, it has fundamental drawbacks with stateful services like databases. Read more

2 - Pigsty

2.1 - Pigsty v4.5: 575 Extensions, Kafka, MySQL, Valkey, and Silo

Pigsty v4.5.0 is a feature release spanning data services, the PostgreSQL ecosystem, automation safety, and the software supply chain. It adds Kafka KRaft and MySQL 8.4 pilot modules, adds a Valkey engine to REDIS, converges the MINIO module on Silo, and grows the packaged PostgreSQL extension catalog from 531 to 575.

This full review covers 100 commits, 416 files, 42,058 insertions, and 15,984 deletions from v4.4.0 (2026-07-10) to v4.5.0 (dab5dba3, 2026-08-14). See the complete source diff at v4.4.0...v4.5.0. A substantial share of the structured diff comes from re-exporting the entire Grafana dashboard set to Dashboard API v2.

Release Highlights

• 575 PostgreSQL extensions: 46 entries added and 2 removed, for a net gain of 44. Many Rust extensions move to pgrx 0.19.1, with synchronized RPM/DEB coverage and package-name mappings.
• Kafka KRaft module: Native Kafka orchestration with multi-cluster support, dynamic enrollment and retirement, SCRAM-SHA-512/TLS, credential rotation, health gates, monitoring, alerting, and four Grafana dashboards.
• MySQL 8.4 module: Standalone and three-node InnoDB Cluster deployments with MySQL Router, XtraBackup, account and database provisioning, parameter protection, monitoring, alerting, and conservative retirement workflows.
• Valkey and Silo: REDIS adds redis_type: valkey; MINIO now accepts only minio_type: silo. The RustFS backend developed during this cycle was withdrawn before the final baseline.
• Safer cluster orchestration: PGSQL, REDIS, MINIO, KAFKA, and MYSQL select members by explicit cluster identity. etcd delegation, DBSU key exchange, PITR, backup markers, and removal workflows are tightened as well.
• Observability upgrades: New Kafka and MySQL dashboards, MinIO/Silo Metrics V3, and pg_exporter 1.4-series configuration with PG19, transaction-age, lock, replication, and cleanup-pressure coverage.
• Repository and supply-chain upgrades: SOW now generates local repositories atomically without fabricated ModuleMD. Offline bundles can carry versioned source archives. Infra repackaging standardizes SHA-256-pinned inputs, SPDX licenses, and the 1PGSTY release suffix.
• 51 standalone configuration templates: Adds demo/kafka, demo/mysql, and the eight-node ha/octo simulation, while updating ha/trio to a compact three-node Silo HA topology.

Kafka KRaft Pilot Module

The new Kafka module uses node state as the source of truth and manages brokers and controllers with dynamic KRaft. It supports multiple Kafka clusters in one inventory and unbounded kafka.yml runs; incomplete --limit selections are rejected to avoid updating only part of the quorum.

• Nodes persist authoritative manifests and secrets, supporting dynamic controller joins, broker admission, member retirement, and three-stage failed-member replacement.
• SCRAM-SHA-512, TLS, certificate rotation, and credential rotation are supported, followed by partition-health and protocol self-tests.
• kafka-rm.yml requires a non-empty -l/--limit and validates absolute data paths, target identity, and the surviving anchors needed for partial retirement before stopping services.
• Adds JMX Exporter, Kafka Exporter, Victoria scrape and alert rules, plus Overview, Instance, Topic, and Consumer Grafana dashboards.

Kafka remains a pilot module. Clients must resolve and reach every broker directly; do not put the Kafka data plane behind HAProxy, a VIP, or a conventional L4 load balancer.

MySQL 8.4 Pilot Module

The new MySQL module targets a fixed MySQL 8.4 LTS platform. It supports a standalone instance or a three-node InnoDB Cluster and includes MySQL Shell, MySQL Router, TLS, user/database provisioning, scheduled full XtraBackup backups, and unified monitoring.

• Initialization validates exactly one or three members, creates the cluster, coordinates Router, and handles existing members and partial states idempotently.
• mysql_parameters protects replication, TLS, and platform-reserved settings; loose_, skip_, disable_, and enable_ prefixes cannot bypass the guardrails.
• mysql_databases accepts only name, encoding, and collate; databases are created with DEFAULT ENCRYPTION='N'.
• Removal validates the target, quorum, and member state before isolation, shutdown, and cleanup. Unlike other modules, mysql-rm.yml fails closed when a selected host has no MySQL identity.
• Adds Overview, Cluster, Instance, Replication, and Alert Grafana dashboards plus mysqld_exporter alerts.

Valkey, Silo, and Mongo Mode

Redis / Valkey

REDIS keeps Redis as the default engine and can select Valkey explicitly with redis_type: valkey. Valkey uses the valkey-server and valkey-cli packages, while configuration paths, monitoring jobs, service interfaces, and module parameters retain redis names for compatibility with existing inventories and dashboards.

Redis/Valkey systemd units now use Type=notify with a 1,800-second startup timeout. Topology validation, primary/replica relationships, password handling, tag-scoped removal, and rebuild protection are also strengthened.

MINIO / Silo

MINIO remains the compatibility module name, but the v4.5 role deploys Silo and only Silo; silo is the sole valid minio_type. Silo preserves the S3/Admin APIs, /minio/* routes, MINIO_* environment variables, and existing disk format, while the package, binary, and systemd service are now named silo.

• Startup checks the systemd Invocation ID, ActiveState=active, and Silo cluster health, waiting up to roughly 600 seconds so stale process state is not mistaken for a successful restart.
• Object-storage members are grouped by minio_cluster, independent of the inventory group name. Multi-cluster inventories should use distinct minio_alias, minio_domain, and minio_endpoint values.
• ha/trio changes from single-node object storage to three single-disk Silo members with EC:1, exposed on port 9002 through VIP and HAProxy. The template declares all three default users explicitly so configure -g produces consistent S3 credentials.
• Distributed Silo requires /data/minio to be a separate filesystem; Silo rejects an ordinary directory on the root filesystem.
• A RustFS backend was added during development and fully withdrawn from the final source. Infra repositories may still carry RustFS or MinIO packages, but neither is a supported v4.5 MINIO backend.

PostgreSQL Mongo Mode

The standalone FERRET role and mongo.yml playbook are removed. Their function is split between PostgreSQL Mongo mode and the FerretDB Docker APP: PostgreSQL/DocumentDB provides the data layer, and FerretDB in Docker Compose provides the MongoDB protocol layer. Pigsty no longer ships the old FerretDB systemd service, dedicated scrape configuration, or Mongo dashboards.

Orchestration, Security, and Lifecycle

• deploy.yml, slim.yml, and the PGSQL, REDIS, MINIO, KAFKA, and MYSQL initialization playbooks skip unrelated hosts by their respective *_cluster identities. PGSQL, REDIS, MINIO, and KAFKA removal playbooks apply the same rule.
• PGSQL configuration, PITR, and removal delegate only when a canonical, non-empty etcd group exists. DBSU SSH keys are exchanged among actual pg_cluster_members, including cross-inventory-group topologies such as Citus.
• PITR and removal delete only etcd subtrees bounded by /<cluster>/, avoiding accidental deletion when one cluster name prefixes another. The initial pgBackRest backup writes initial.done only after the command succeeds.
• Removal workflows stop services before cleaning data. Kafka, MySQL, and object storage add directory, quorum, surviving-member, and cluster-identity checks.
• A full pgsql.yml run is now explicitly described as initialization-only: it restarts Patroni/PostgreSQL and reapplies managed configuration and bootstrap SQL, so it is not a routine convergence command for initialized clusters.
• Pigsty-rendered systemd units live under /etc/systemd/system; permissions are tightened for sensitive configuration, environment files, keys, and privileged scripts. Debian/Ubuntu package installation suppresses premature starts of Silo, Redis/Valkey, and legacy logging services.
• HAProxy uses a fixed /etc/haproxy/haproxy.cfg plus /etc/haproxy/conf.d layout, upstream master-worker mode, a master socket, and Type=notify. dnsmasq uses dynamic binding, answers private PTR locally, and handles node addresses that appear only after INFRA initialization.
• Debian/Ubuntu time sync writes the native /etc/chrony/chrony.conf, while EL continues to use /etc/chrony.conf. During initial synchronization, offsets greater than ten seconds may step within the first 120 updates.

PostgreSQL Kernels and Extensions

• Default PostgreSQL updates to 18.6 with the repositories. The four standard Patroni templates add an output_plugin_libraries allowlist for pgoutput, test_decoding, and wal2json; Patroni filters unsupported settings on older versions.
• The PostgreSQL 19 beta template gains pgBackRest 2.59 and scheduled backup support. It still installs no extensions by default and remains a PG19 trial template.
• Percona PostgreSQL 18 TDE uses cluster mode and keeps Pigsty’s private installation prefix to avoid collisions with native PostgreSQL.
• IvorySQL gains default-database initialization and compatible WAL compression in the standard workload templates.
• The PostgreSQL fact loader, per-platform package_map, and default extension groups are refreshed together, fixing PGDG/YUM names and missing packages.

Comparing the embedded catalog snapshots in Pig v1.5.1 (531 entries) and v1.7.0 (575 entries), this cycle adds 46 extensions and removes 2, for a net gain of 44:

• 32 primary extensions added: argm, cat_tools, cron_utils, fbsql, oidc_validator, online_advisor, pg_cjk_parser, pg_column_tetris, pg_describe, pg_disorder, pg_fts, pg_jieba, pg_kpart, pg_lake, pg_local_cache, pg_mentat, pg_oidc_validator, pg_policy, pg_roast, pg_tiktoken_c, pg_turbovec, pg_vault_tde, pgcontext, pgfr_record, pgmemento, pgmonitor, pgsqlmock, pgwasm, plruby, plx, postbis, and qdgc.
• 13 subextensions added: hstore_plruby, jsonb_plruby, ltree_plruby, pg_extension_base, pg_extension_updater, pg_lake_copy, pg_lake_engine, pg_lake_iceberg, pg_lake_table, pg_map, pgcontext_pgvector, pgfr_analyze, and qdgc_postgis.
• One PGDG extension added: pg_statviz. It remains in the online catalog but is not part of the default installation groups.
• pg_analytics and spat removed: the former is archived, and the latter is a deprecated alpha project.

In addition, emailaddr, explain_ui, the Rust oidc_validator, pg_summarize, and smlar leave the default installation groups because upstream does not provide a distributable license; packages and catalog records may remain. pg_relation_sql is a standalone SQL toolkit without CREATE EXTENSION support, so it appears in the packaging table but is not counted among the 575 extensions.

Extension Package Update Matrix

The table consolidates RPM and DEB batches after v4.4.0 into one final “old version → final version” row per extension: 207 packaged entries plus two catalog removals. Where RPM and DEB differ, both are shown. An unchanged version can still represent a rebuild or a change in package naming, license metadata, or platform coverage.

The first packaging log after July 10 covers July 7–24 without per-entry dates, so the whole batch is included to avoid omitting post-release pgrx 0.19.1 rebuilds and matrix fixes. The final repository indexes remain authoritative for versions and platform availability.

Observability

• The complete Grafana dashboard set is re-exported through the Pig/Grafana tooling as Dashboard API v2. Four Kafka and five MySQL dashboards are added, and links, variables, and layouts are refreshed for Node, PostgreSQL, Redis, and Infra dashboards.
• MinIO/Silo Overview and Instance dashboards move to Metrics V3. Victoria scrapes /minio/metrics/v3 and drops high-cardinality samples with a non-empty bucket label.
• pg_exporter configuration updates to 1.4.0 and fixes the duplicate time series produced by the 1.4.1 pg_subrel query.
• PG19 gains pg_sub_19, pg_recovery_state, pg_wal_19, pg_lock_stat, and pg_vacuum_score collectors. PG10+ gains a pg_xact_age transaction-age histogram, replication-slot idle_timeout, and WAL Receiver connecting state encoding.
• Kafka JMX/protocol exporters and the MySQL exporter join unified service discovery, Victoria scraping, and alert rules. Journald records without a PRIORITY field are handled safely.

Repositories, Offline Bundles, and Supply Chain

• REPO and CACHE use sow create --pigsty to generate DNF/APT metadata and the SHA-256 repo_complete marker atomically, without injecting fabricated ModuleMD.
• module_hotfixes becomes an explicit per-repository choice and is enabled only for EL repositories that must replace system module streams, avoiding indiscriminate DNF module-filter bypasses.
• If dist/<version>/pigsty-<version>.tgz already exists, cache.yml includes it in the offline repository. It does not implicitly package the current worktree or download source code from the internet.
• RPM exporter package names change from underscores to hyphens, for example node_exporter → node-exporter; unversioned Provides/Obsoletes entries bridge the old names.
• Local Infra repackaging standardizes the PGSTY vendor, SPDX license expressions, 1PGSTY release suffix, /etc/default/<service> environment files, vendor units under /usr/lib/systemd/system, and legal files under /usr/share/doc/<package>.
• Simple binary packages use reproducible upstream archives pinned by SHA-256; vendor-native RPM/DEB files retain their original bytes and metadata. Direct artifacts such as Silo, Grafana, Code, Code Server, and Vector gain pinned URLs and checksums.
• China-region routing is refreshed systematically: operating systems, Docker, Grafana, Percona, MongoDB APT, and uv/PyPI prefer Tencent Cloud; EL/Docker retain Huawei Cloud and Alibaba Cloud fallbacks; MySQL/Kubernetes use USTC; ClickHouse uses Huawei Cloud.

Infra Package Update Matrix

The table consolidates this cycle’s Infra batches into one row per package, showing only the v4.4.0 value, final v4.5.0 value, and essential notes. Intermediate versions are omitted. “Built” or “verified” describes packaging records only; the signed repository index at release time remains authoritative for installable versions.

Platforms, Templates, and Release Engineering

• Recommended platform baselines update to Rocky Linux 9.8/10.2, Debian 12.15/13.6, and Ubuntu 22.04.5/24.04.4/26.04.0. Terraform cloud images and pinned Vagrant box versions are refreshed with them.
• Vagrant’s root disk becomes configurable through root_disk: 64 GiB by default and 128 GiB for the all, rpm, deb, oss, and pro build matrices. The existing disk setting still means the extra /data disk and defaults to 128 GiB. The vagrant user’s login shell is standardized on Bash.
• Adds the eight-node ha/octo simulation. ha/trio becomes a three-node Silo deployment behind VIP/HAProxy; demo/minio explicitly selects Silo and narrows its local repository package groups.
• The safe and demo templates use valid extension aliases. Tuned-profile paths adapt to the operating system, cluster-size comparisons remain compatible with Ansible versions before 2.19, and unused internal flags plus MINIO/INFRA variables are removed.
• The Docker base image updates to Debian 13.6 and is marked v4.5.0. Image publishing becomes manually triggered and builds from tags.
• docker/Makefile fixes its data directory at repository-local ./data; make purge no longer accepts an external DATA override and has no countdown.
• GitHub Actions updates checkout, CodeQL, Docker build/login, and Cosign in bulk. Release, bootstrap, installation, checksum, and build templates tighten parameter and file handling.
• Release archives generate top-level pigsty.yml from conf/meta.yml, include Kafka/MySQL playbooks, and remove the old Mongo playbook. The signing workflow creates Cosign signatures only for the pigsty-<tag>.tgz source archive, not multi-gigabyte offline bundles.
• README, SECURITY, CONTRIBUTING, AGENTS, and NOTICE align on v4.5, 575 extensions, Silo, Redis/Valkey, and Mongo mode, with updated 2026 copyright and trademark language.

Compatibility Changes and Upgrade Notes

Before upgrading from v4.4, review these changes carefully:

1. FERRET split: Remove the old ferretdb systemd service. Deploy the DocumentDB data layer through Mongo configuration mode, then deploy the FerretDB protocol layer with docker.yml / app.yml. The old mongo.yml, mongo_* parameters, and dedicated dashboards are gone.
2. Silo replaces the MinIO server: minio_type accepts only silo. Protocol and disk-format compatibility do not constitute migration acceptance; validate backups, in-place compatibility, rollback, and real read/write traffic before switching existing object storage.
3. ha/trio object-storage topology changes: The new template uses three single-disk Silo members. Do not expand an existing single-node pool in place by simply adding two members; create a new cluster and migrate objects.
4. HAProxy unit changes: Pigsty no longer renders /etc/default/haproxy, though the unit can read it when present. Use EXTRAOPTS only, keep -S /run/haproxy-master.sock, and do not add -f.
5. Explicit cluster identity: Custom inventories must define pg_cluster, redis_cluster, minio_cluster, kafka_cluster, or mysql_cluster for target hosts. Older inventories that relied only on fixed group names must add these identities.
6. Full PGSQL playbook is initialization-only: Do not run the complete pgsql.yml against initialized nodes. Use precise tags for routine maintenance; for a full rebuild, validate backups and remove the target member through the formal workflow first.
7. Valkey is opt-in: Valkey is installed only with redis_type: valkey; module parameters, configuration paths, and monitoring interfaces keep Redis names.
8. SOW is a core dependency: REPO/CACHE requires SOW 0.3.0. If an old offline bundle or local repository lacks sow, add it from the Pigsty Infra repository first.
9. Exporter RPM renames: External automation and private repositories should move from old names such as node_exporter and redis_exporter to node-exporter and redis-exporter.
10. Default extension groups tighten licensing: Environments that depend on emailaddr, explain_ui, oidc_validator, pg_summarize, or smlar must install them explicitly and assess licensing and redistribution boundaries independently.
11. Docker purge semantics change: make purge directly deletes repository-local ./data; confirm that it contains no container data you need to retain.
12. Vagrant disk semantics change: root_disk controls the system disk, while disk controls the extra /data disk. Distributed Silo requires the latter to be a real independent mount.

Get v4.5.0

Install on a fresh, supported Linux node:

curl -fsSL https://repo.pigsty.io/get | bash -s v4.5.0
cd ~/pigsty
./bootstrap
./configure
./install.yml

A production upgrade is not an overwrite installation. Read the compatibility changes above, resolve the exact target cluster, backup, inventory identity, and live state, then use an explicit -l scope for check mode or maintenance operations.

2.2 - Pigsty v4.3: 510 Extensions & Ubuntu 26

Pigsty v4.3 adds 50 PostgreSQL extensions, bringing the total to 510. It also adds Ubuntu 26.04 x86_64/arm64 support, refreshes Supabase, pgEdge, PolarDB, Grafana, MinIO, and a batch of infra packages. Read more

2.3 - Pigsty v4.2: 12 Kernels

GitHub Release | Release Note

v4.2.0 Release Note

Highlights

• Aligned with PostgreSQL out-of-band minor updates: 18.3, 17.9, 16.13, 15.17, 14.22.
• Total PostgreSQL extension coverage reaches 461 packages.
• Kernel updates across Babelfish, AgensGraph, pgEdge, OriolePG, OpenHalo, and Cloudberry.
• Babelfish template now uses a Pigsty-maintained PG17-compatible build, with no WiltonDB repo dependency.
• Supabase images and self-hosted templates are refreshed to the latest stack, using Pigsty-maintained pgsty/minio.

Major Changes

• mssql now defaults to Babelfish PG17 (pg_version: 17, pg_packages: [babelfish, pgsql-common, sqlcmd]) and no longer requires an extra mssql repo.
• Kernel install paths are normalized in pg_home_map: mssql -> /usr/babelfish-$v/, gpsql -> /usr/local/cloudberry.
• package_map adds a dedicated cloudberry mapping and fixes babelfish* aliases to versioned RPM/DEB package names.
• Redis data root default changes from /data to /data/redis; deployment blocks legacy defaults, while redis_remove keeps backward-compatible cleanup.
• configure now supports absolute -o output paths with auto-created parent directories, tri-state region detection (CN/global/offline fallback), and a fix for behind_gfw() hangs.
• Debian/Ubuntu default repo URL mappings (updates/backports/security) and China mirror components are corrected to prevent bootstrap package failures.
• Supabase stack is updated (including PostgREST 14.5 and Vector 0.53.0) and now includes missing S3 protocol credential variables.
• Rich/Sample templates explicitly define dbuser_meta defaults; node.sh systemd completion is simplified.
• pgbackrest stanza initialization now retries (2 attempts, 5-second interval) to reduce lock contention with archive-push.
• Vibe template now ships @anthropic-ai/claude-code, @openai/codex, and happy-coder, and includes age in the default example.

PG Software Updates

• PostgreSQL 18.3, 17.9, 16.13, 15.17, 14.22
• RPM Changelog 2026-02-27
• DEB Changelog 2026-02-27
• Core upgrades: timescaledb 2.25.0 -> 2.25.1, citus 14.0.0-3 -> 14.0.0-4, pg_search -> 0.21.9
• New/rebuilt: pgedge 17.9, spock 5.0.5, lolor 1.2.2, snowflake 2.4, babelfish 5.5.0, cloudberry 2.0.0
• Kernel-side updates: oriolepg 17.11 -> 17.16, orioledb beta12 -> beta14, openhalo 14.10 -> 1.0(14.18)

Infrastructure Software Updates

API Changes

• pg_mode now includes agens and pgedge.
• mssql defaults are updated to pg_version: 17 and pg_packages: [babelfish, pgsql-common, sqlcmd].
• Kernel/package alias mappings are updated in pg_home_map and package_map (Babelfish, OpenHalo, IvorySQL, Cloudberry, pgEdge family).
• redis_fs_main now defaults to /data/redis, with deployment guardrails and backward-compatible cleanup behavior.
• configure output path handling and region detection logic are updated, with offline fallback warnings and unified SSH probe timeouts.
• grafana.ini.j2 is updated for Grafana 12.4 config changes and deprecations.

Compatibility Notes

• If existing Redis configs still use redis_fs_main: /data, migrate to /data/redis before deployment.
• Grafana 12.4 changes data link merge behavior. This release moves key links into field overrides; review custom dashboards accordingly.

26 commits, 122 files changed, +2,116 / -2,215 lines (v4.1.0..v4.2.0, 2026-02-15 ~ 2026-02-28)

Checksums

24a90427a7e7351ca1a43a7d53289970  pigsty-v4.2.0.tgz
d980edf5eeb0419d4f1aa7feb0100e14  pigsty-pkg-v4.2.0.d12.aarch64.tgz
24bc237d841457fbdcc899e1d0a3f87e  pigsty-pkg-v4.2.0.d12.x86_64.tgz
e395b38685e2ecbe9c3a2850876d9b7b  pigsty-pkg-v4.2.0.d13.aarch64.tgz
c5c8776f9bead9f29528b26058801f83  pigsty-pkg-v4.2.0.d13.x86_64.tgz
28ea40434bd06135fc8adc0df1c8407d  pigsty-pkg-v4.2.0.el10.aarch64.tgz
58ad715ac20dc1717d1687daecfcf625  pigsty-pkg-v4.2.0.el10.x86_64.tgz
008f955439ea311581dd0ebcf5b8bd34  pigsty-pkg-v4.2.0.el8.aarch64.tgz
2acfd127a517b09f07540f808fe9547a  pigsty-pkg-v4.2.0.el8.x86_64.tgz
58e62a92f35291a40e3f05839a1b6bc4  pigsty-pkg-v4.2.0.el9.aarch64.tgz
d311bfdf5d5f60df5fe6cb3d4ced4f9c  pigsty-pkg-v4.2.0.el9.x86_64.tgz
c98972fe9226657ac1faa7b72a22498b  pigsty-pkg-v4.2.0.u22.aarch64.tgz
44a174ee9ba030ac1ea386cf0b85f6e7  pigsty-pkg-v4.2.0.u22.x86_64.tgz
143e404f4681c7d0bbd78ef7982cd652  pigsty-pkg-v4.2.0.u24.aarch64.tgz
00dfa86f477f3adff984906211ab3190  pigsty-pkg-v4.2.0.u24.x86_64.tgz

2.4 - Pigsty v4.1: Day-Zero 18.2 Support

Same-day production support for new PG minors is the core message of Pigsty v4.1. Read more

2.5 - Pigsty v4.0: Victoria Stack + Security Hardening

VictoriaMetrics/Logs replace Prometheus/Loki for 10x observability performance, Vector handles logs, unified UI, firewall/SELinux/credential hardening. Read more

2.6 - Pigsty v3.7: PostgreSQL Magneto Award, PG18 Deep Support

PostgreSQL 18 becomes the default version, EL10 and Debian 13 support added, extensions reach 437, and Pigsty wins the PostgreSQL Magneto Award. Read more

2.7 - Pigsty v3.6: The Ultimate PostgreSQL Distribution

New doc site, PITR playbook, Percona PG TDE kernel support, and Supabase self-hosting optimization make v3.6 the last major release before 4.0. Read more

2.8 - Pigsty v3.5: 4K Stars, PG18 Beta, 421 Extensions

Pigsty crosses 4K GitHub stars, adds PG18 beta support, pushes extensions to 421, ships new doc site, and completes OrioleDB/OpenHalo full-platform support. Read more

2.9 - Pigsty v3.4: PITR Enhancement, Locale Best Practices, Auto Certificates

Pigsty v3.4 adds pgBackRest backup monitoring, cross-cluster PITR restore, automated HTTPS certificates, locale best practices, and full-platform IvorySQL and Apache AGE support. Read more

2.10 - Pigsty v3.3: 404 Extensions, Turnkey Apps, New Website

Pigsty v3.3 pushes available extensions to 404, adds turnkey app deployment with app.yml, delivers Certbot integration for automated HTTPS, and launches a redesigned website. Read more

2.11 - Pigsty v3.2: The pig CLI, Full ARM Support, Supabase & Grafana Enhancements

Pigsty v3.2 introduces the pig CLI for PostgreSQL package management, complete ARM64 extension repository support, and Supabase & Grafana enhancements. Read more

2.12 - Pigsty v3.1: One-Click Supabase, PG17 Default, ARM & Ubuntu 24

Pigsty v3.1 makes PostgreSQL 17 the default, delivers one-click Supabase self-hosting, adds ARM64 and Ubuntu 24.04 support, and simplifies configuration management. Read more

2.13 - Pigsty v3.0: Pluggable Kernels & 340 Extensions

Pigsty v3.0 ships 340 extensions across EL/Deb with full parity, adds pluggable kernels (Babelfish, IvorySQL, PolarDB) for MSSQL/Oracle compatibility, and delivers a local-first state-of-the-art RDS experience. Read more

2.14 - Pigsty v2.7: The Extension Superpack

Pigsty v2.7 bundles 255 PostgreSQL extensions, plus Docker templates for Odoo, Supabase, PolarDB, and Jupyter, with new PITR dashboards. Read more

2.15 - Pigsty v2.6: PostgreSQL Crashes the OLAP Party

Pigsty v2.6 makes PostgreSQL 16.2 the default, introduces ParadeDB and DuckDB support, and brings epic-level OLAP improvements. Read more

2.16 - Pigsty v2.5: Ubuntu & PG16

Pigsty v2.5 adds Ubuntu/Debian support (bullseye, bookworm, jammy, focal), new extensions including pointcloud and imgsmlr, and redesigned monitoring dashboards. Read more

2.17 - Pigsty v2.4: Monitor Cloud RDS

Pigsty v2.4 delivers PostgreSQL 16 GA support, RDS/PolarDB monitoring, Redis Sentinel HA, and a wave of new extensions including Apache AGE, zhparser, and pg_embedding. Read more

2.18 - Pigsty v2.3: Richer App Ecosystem

Pigsty v2.3 adds FerretDB MongoDB support, NocoDB integration, L2 VIP for node clusters, PostgreSQL security patches, and Redis 7.2. Read more

2.19 - Pigsty v2.2: Monitoring System Reborn

Pigsty v2.2 delivers a complete monitoring dashboard overhaul built on Grafana 10, a 42-node production simulation sandbox, Pigsty’s own RPM repos, and UOS compatibility. Read more

2.20 - Pigsty v2.1: Vector + Full PG Version Support!

Pigsty v2.1 provides support for PostgreSQL 12 through 16, with PGVector for AI embeddings. Read more

2.21 - Pigsty v2.0: Open-Source RDS PostgreSQL Alternative

Pigsty v2.0 delivers major improvements in security, compatibility, and feature integration — truly becoming a local open-source RDS alternative. Read more

2.22 - Pigsty v1.5: Docker Application Support, Infrastructure Self-Monitoring

Complete Docker support, infrastructure self-monitoring, ETCD as DCS, better cold backup support, and CMDB improvements. Read more

2.23 - Pigsty v1.4: Modular Architecture, MatrixDB Data Warehouse Support

Pigsty v1.4 introduces a modular architecture with four independent modules, adds MatrixDB time-series data warehouse support, and delivers global CDN acceleration. Read more

2.24 - Pigsty v1.3: Redis Support, PGCAT Overhaul, PGSQL Enhancements

Pigsty v1.3 adds Redis support with three deployment modes, rebuilds the PGCAT catalog explorer, and enhances PGSQL monitoring dashboards. Read more

2.25 - Pigsty v1.2: PG14 Default, Monitor Existing PG

Pigsty v1.2 makes PostgreSQL 14 the default version and adds support for monitoring existing database instances independently. Read more

2.26 - Pigsty v1.1: Homepage, Jupyter, Pev2, PgBadger

Pigsty v1.1.0 ships with a redesigned homepage, plus JupyterLab, PGWeb, Pev2 & PgBadger integrations. Read more

2.27 - Pigsty v1.0: GA Release with Monitoring Overhaul

Pigsty v1.0.0 GA is here — a batteries-included, open-source PostgreSQL distribution ready for production. Read more

2.28 - Pigsty v0.9: CLI + Logs

One-click installs, a beta CLI, and Loki-based logging make Pigsty easier to land. Read more

2.29 - Pigsty v0.8: Service Provisioning

Services are now first-class objects, so you can define any routing policy—built-in HAProxy, L4 VIPs, or your own balancer. Read more

2.30 - Pigsty v0.7: Monitor-Only Deployments

Monitor-only deployments unlock hybrid fleets, while DB/user provisioning APIs get a serious cleanup. Read more

2.31 - Pigsty v0.6: Provisioning Upgrades

v0.6 reworks the provisioning flow, adds exporter toggles, and makes the monitoring stack portable across environments. Read more

2.32 - Pigsty v0.5: Declarative DB Templates

Pigsty v0.5 introduces declarative database templates so roles, schemas, extensions, and ACLs can be described entirely in YAML. Read more

2.33 - Pigsty v0.4: PG13 and Better Docs

Pigsty v0.4 ships PG13 support, a Grafana 7.3 refresh, and a cleaned-up docs site for the second public beta. Read more

2.34 - Pigsty v0.3: First Public Beta

Pigsty v0.3.0, the first public beta, lands with eight battle-tested dashboards and an offline bundle. Read more

3 - Cloud

3.1 - Don't Run AI Assistant on Cloud

When you launch AI assistant on the cloud, you’re handing over cognitive data to the vendor. There’s a reason why people buy Mac mini rather than running it on the cloud. Read more

3.2 - Did RedNote Exit the Cloud?

When a company that was “born on the cloud” goes “self-host first,” does that count as cloud exit? A repost of a deleted piece on the infrastructure coming-of-age for Chinas internet giants. Read more

3.3 - Alipay, Taobao, Xianyu Went Dark. Smells Like a Message Queue Meltdown.

Dec 4, 2025, Taobao, Alipay, and Xianyu all cratered. Users got charged while orders still showed “unpaid,” a carbon copy of the 2024 Double-11 fiasco. Read more

3.4 - Cloudflare’s Nov 18 Outage, Translated and Dissected

A ClickHouse permission tweak doubled a feature file, tripped a Rust hard limit, and froze Cloudflare’s core traffic for six hours—their worst outage since 2019. Here’s the full translation plus commentary. Read more

3.5 - Alicloud “Borrowed” Supabase. This Is What Happens When Giants Strip-Mine Open-Source.

Founders here get asked the same question over and over: what if Alibaba builds the same thing? Alicloud RDS just launched Supabase as a managed service. Exhibit A. Read more

3.6 - AWS’s Official DynamoDB Outage Postmortem

AWS finally published the Oct 20 us-east-1 postmortem. I translated the key parts and added commentary on how one DNS bug toppled half the internet. Read more

3.7 - How One AWS DNS Failure Cascaded Across Half the Internet

us-east-1’s DNS control plane faceplanted for 15 hours and dragged 142 AWS services—and a good chunk of the public internet—down with it. Here’s the forensic tour. Read more

3.8 - Column: Cloud-Exit

A whole generation of developers has been told “cloud-first.” This column collects data, case studies, and analysis on the cloud exit movement. Read more

3.9 - KubeSphere: Trust Crisis Behind Open-Source Supply Cut

Deleting images and running away - this isn’t about commercial closed-source issues, but supply cut problems that directly destroy years of accumulated community trust. Read more

3.10 - Alicloud’s rds_duckdb: Tribute or Rip-Off?

Does bolting DuckDB onto RDS suddenly make open-source Postgres ‘trash’? Business and open source should be symbiotic. If a vendor only extracts without giving back, the community will spit it out." Read more

3.11 - Escaping Cloud Computing Scam Mills: The Big Fool Paying for Pain

A user consulted about distributed databases, but he wasn’t dealing with data bursting through server cabinet doors—rather, he’d fallen into another cloud computing pig-butchering scam. Read more

3.12 - OpenAI Global Outage Postmortem: K8S Circular Dependencies

Even trillion-dollar unicorns can be a house of cards when operating outside their core expertise. Read more

3.13 - WordPress Community Civil War: On Community Boundary Demarcation

When open source ideals meet commercial conflicts, what insights can this conflict between open source software communities and cloud vendors bring? On the importance of community boundary demarcation. Read more

3.14 - Cloud Database: Michelin Prices for Cafeteria Pre-made Meals

The paradigm shift brought by RDS, whether cloud databases are overpriced cafeteria meals. Quality, security, efficiency, and cost analysis, cloud exit database self-building: how to implement in practice! Read more

3.15 - Alibaba-Cloud: High Availability Disaster Recovery Myth Shattered

Seven days after Singapore Zone C failure, availability not even reaching 8, let alone multiple 9s. But compared to data loss, availability is just a minor issue Read more

3.16 - Amateur Hour Opera: Alibaba-Cloud PostgreSQL Disaster Chronicle

A customer experienced an outrageous cascade of failures on cloud database last week: a high-availability PG RDS cluster went down completely - both primary and replica servers - after attempting a simple memory expansion, troubleshooting until dawn. Poor recommendations abounded during the incident, and the postmortem was equally perfunctory. I share this case study here for reference and review. Read more

3.17 - What Can We Learn from NetEase Cloud Music's Outage?

NetEase Cloud Music experienced a two-and-a-half-hour outage this afternoon. Based on circulating online clues, we can deduce that the real cause behind this incident was… Read more

3.18 - Blue Screen Friday: Amateur Hour on Both Sides

Both client and vendor failed to control blast radius, leading to this epic global security incident that will greatly benefit local-first software philosophy. Read more

3.19 - How Ahrefs Saved US$400M by NOT Going to the Cloud

After Alibaba-Cloud’s epic global outage on Double 11, setting industry records, how should we evaluate this incident and what lessons can we learn from it? Read more

3.20 - Database Deletion Supreme - Google Cloud Nuked a Major Fund's Entire Cloud Account

Due to an “unprecedented configuration error,” Google Cloud mistakenly deleted trillion-RMB fund giant UniSuper’s entire cloud account, cloud environment and all off-site backups, setting a new record in cloud computing history! Read more

3.21 - Cloud Dark Forest: Exploding Cloud Bills with Just S3 Bucket Names

The dark forest law has emerged on public cloud: Anyone who knows your S3 object storage bucket name can explode your cloud bill. Read more

3.22 - Cloudflare Roundtable Interview and Q&A Record

As a roundtable guest, I was invited to participate in Cloudflare’s Immerse conference in Shenzhen. During the dinner, I had in-depth discussions with Cloudflare’s APAC CMO, Greater China Technical Director, and front-line engineers about many questions of interest to netizens. Read more

3.23 - What Can We Learn from Tencent Cloud's Major Outage?

Tencent Cloud’s epic global outage after Double 11 set industry records. How should we evaluate and view this failure, and what lessons can we learn from it? Read more

3.24 - Cloudflare - The Cyber Buddha That Destroys Public Cloud

While I’ve always advocated for cloud exit, if it’s about adopting a cyber bodhisattva cloud like Cloudflare, I’m all in with both hands raised. Read more

3.25 - Can Luo Yonghao Save Toothpaste Cloud?

Luo Yonghao’s livestream first spent half an hour selling robot vacuums, then Luo himself belatedly appeared to read scripts selling “cloud computing” for forty minutes — before seamlessly transitioning to selling Colgate enzyme-free toothpaste — leaving viewers bewildered between toothpaste and cloud computing. Read more

3.26 - Analyzing Alibaba-Cloud Server Computing Cost

Alibaba-Cloud claimed major price cuts, but a detailed analysis of cloud server costs reveals that cloud computing and storage remain outrageously expensive. Read more

3.27 - Will DBAs Be Eliminated by Cloud?

Two days ago, the ninth episode of Open-Source Talks had the theme “Will DBAs Be Eliminated by Cloud?” As the host, I restrained myself from jumping into the debate throughout, so I’m writing this article to discuss this question: Will DBAs be eliminated by cloud? Read more

3.28 - Cloud-Exit High Availability Secret: Rejecting Complexity Masturbation

Programmers are drawn to complexity like moths to flame. The more complex the system architecture diagram, the greater the intellectual masturbation high. Steadfast resistance to this behavior is a key reason for DHH’s success in cloud-free availability. Read more

3.29 - S3: Elite to Mediocre

S3 is no longer “cheap” with the evolution of hardware, and other challengers such as cloudflare R2. Read more

3.30 - Cloud Exit FAQ: DHH Saves Millions

Author: DHH (David Heinemeier Hansson) | Original: Cloud Exit FAQ

DHH’s cloud exit journey has reached a new stage, saving nearly a million dollars so far with potential savings of nearly ten million over the next five years. Read more

3.31 - From Cost-Reduction Jokes to Real Cost Reduction and Efficiency

Alibaba-Cloud and Didi had major outages one after another. This article discusses how to move from cost-reduction jokes to real cost reduction and efficiency — what costs should we really reduce, what efficiency should we improve? Read more

3.32 - Reclaim Hardware Bonus from the Cloud

Hardware is interesting again, developments in CPUs and SSDs remain largely unnoticed by the majority of devs. A whole generation of developers is obscured by cloud hype and marketing noise. Read more

3.33 - What Can We Learn from Alibaba-Cloud's Global Outage?

Alibaba-Cloud’s epic global outage after Double 11 set an industry record. How should we evaluate this incident, and what lessons can we learn from it? Read more

3.34 - Harvesting Alibaba-Cloud Wool, Building Your Digital Homestead

Alibaba-Cloud’s Double 11 offered a great deal: 2C2G3M ECS servers for ¥99/year, low price for three years. This article shows how to use this decent ECS to build your own digital homestead. Read more

3.35 - Cloud Computing Mudslide: Deconstructing Public Cloud with Data

Once upon a time, “going to cloud” was almost politically correct in tech circles, but few people use hard data to analyze the trade-offs involved. I’m willing to be this skeptic: let me use hard data and personal stories to explain the traps and value of public cloud rental models. Read more

3.36 - Cloud Exit Odyssey: Time to Leave Cloud?

Author: DHH (David Heinemeier Hansson) | Original: DHH’s Hey Blog

This article chronicles the complete journey of 37Signals moving off the cloud, led by DHH. Valuable reference for both cloud-bound and cloud-native enterprises. Read more

3.37 - DHH: Cloud-Exit Saves Over Ten Million, More Than Expected!

DHH migrated their seven cloud applications from AWS to their own hardware. 2024 is the first year of full savings realization. They’re delighted to find the savings exceed initial estimates. Read more

3.38 - FinOps: Endgame Cloud-Exit

At the SACC 2023 FinOps session, I fiercely criticized cloud vendors. This is a transcript of my speech, introducing the ultimate FinOps concept — Cloud-Exit and its implementation path. Read more

3.39 - Why Isn't Cloud Computing More Profitable Than Sand Mining?

Public cloud margins worse than sand mining—why are pig-butchering schemes losing money? Resource-selling models heading toward price wars, open source alternatives breaking monopoly dreams! Service competitiveness gradually neutralized—where is the cloud computing industry heading? How did domestic cloud vendors make a business with 30-40% pure profit less profitable than sand mining? Read more

3.40 - SLA: Placebo or Insurance?

SLA is a marketing tool rather than insurance. In the worst-case scenario, it’s an unavoidable loss; at best, it provides emotional comfort. Read more

3.41 - EBS: Pig Slaughter Scam

The real business model of cloud: “Cheap” EC2/S3 to attract customers, and fleece with “Expensive” EBS/RDS Read more

3.42 - Garbage QCloud CDN: From Getting Started to Giving Up?

I originally believed that at least in IaaS fundamentals — storage, compute, and networking — public cloud vendors could still make significant contributions. However, my personal experience with Tencent Cloud CDN shook that belief: domestic cloud vendors’ products and services are truly unbearable. Read more

3.43 - Refuting "Why You Still Shouldn't Hire a DBA"

Guo Degang has a comedy routine: “Say I tell a rocket scientist, your rocket is no good, the fuel is wrong. I think it should burn wood, better yet coal, and it has to be premium coal, not washed coal. If that scientist takes me seriously, he loses.” Read more

3.44 - Paradigm Shift: From Cloud to Local-First

Cloud databases’ exorbitant markups—sometimes 10x or more—are undoubtedly a scam for users outside the applicable spectrum. But we can dig deeper: why are public clouds, especially cloud databases, like this? And based on their underlying logic, make predictions about the industry’s future. Read more

3.45 - Are Cloud Databases an IQ Tax?

Winter is coming, tech giants are laying off workers entering cost-reduction mode. Can cloud databases, the number one public cloud cash cow, still tell their story? The money you spend on cloud databases for one year is enough to buy several or even dozens of higher-performing servers. Are you paying an IQ tax by using cloud databases? Read more

3.46 - Cloud RDS: From Database Drop to Exit

I recently witnessed a live cloud database drop-and-run incident. This article discusses how to handle accidentally deleted data when using PostgreSQL in production environments. Read more

3.47 - Is DBA Still a Good Job?

Ant Financial had a self-deprecating joke: besides regulation, only DBAs could bring down Alipay. Although DBA sounds like a profession with glorious history and dim prospects, who knows if it might become trendy again after a few terrifying major cloud database incidents? Read more

4 - PostgreSQL

4.1 - Happy 30th Birthday, PostgreSQL

On July 8, 1996, the PostgreSQL community picked up the flame from Postgres95. Thirty years later, it has grown from a Berkeley research project into a default foundation of the global database ecosystem. Read more

4.2 - Extensions for Everyone

A field report on the PostgreSQL extension ecosystem: 1,617 discovered projects, 511 deliverable extensions, and the shared delivery layer needed to make extensibility work for users, authors, vendors, and PostgreSQL hackers. Read more

4.3 - 504 Extensions: Expand the PostgreSQL Landscape

One GitHub issue turned into an extension sprint. 32 new additions, 504 in total, say a lot about where PostgreSQL is headed. Read more

4.4 - From AGPL to Apache: Why I Changed Pigsty's License

Pigsty switched from AGPLv3 to Apache 2.0. Aren’t you worried about freeloaders? Freeloaders welcome — if you want to become the Debian of databases, a permissive license is table stakes. Read more

4.5 - Git for Data: Instant PostgreSQL Database Cloning

How to instantly clone a massive PostgreSQL database without consuming extra storage? PostgreSQL 18 and XFS can spark some serious magic. Read more

4.6 - Why PostgreSQL Will Dominate the AI Era

Context window economics, the polyglot persistence problem, and the triumph of zero-glue architecture make PostgreSQL the database king of the AI era. Read more

4.7 - Forging a China-Rooted, Global PostgreSQL Distro

PostgreSQL already won. The real battle is the distro layer. Will Chinese developers watch from the sideline or craft a PG “Ubuntu” for the world? Read more

4.8 - PG Extension Cloud: Unlocking PostgreSQL’s Entire Ecosystem

Free and open. Install PostgreSQL and 575 packaged extensions on 16 Linux system and architecture combinations × 5 supported PG major versions via native RPM/DEB—and a tiny CLI. Read more

4.9 - The PostgreSQL 'Supply Cut' and Trust Issues in Software Supply Chain

PostgreSQL official repos cut off global mirror sync channels, open-source binaries supply disrupted, revealing the true colors of various database and cloud vendors. Read more

4.10 - PostgreSQL Dominates Database World, but Who Will Devour PG?

The same forces that once led MongoDB and MySQL toward closure are now at work in the PostgreSQL ecosystem. The PG world needs a distribution that represents “software freedom” values. Read more

4.11 - PostgreSQL Has Dominated the Database World

The 2025 SO global developer survey results are fresh out, and PostgreSQL has become the most popular, most loved, and most wanted database for the third consecutive year. Nothing can stop PostgreSQL from consolidating the entire database world! Read more

4.12 - PGDG Cuts Off Mirror Sync Channel

PGDG cuts off FTP rsync sync channels, global mirror sites universally disconnected - this time they really strangled global users’ supply chain. Read more

4.13 - Postgres Extension Day - See You There!

The annual PostgreSQL developer conference will be held in Montreal in May. Like the first PG Con.Dev, there’s also an additional dedicated event - Postgres Extensions Day Read more

4.14 - OrioleDB is Coming! 4x Performance, Eliminates Pain Points, Storage-Compute Separation

A PG kernel fork acquired by Supabase, claiming to solve PG’s XID wraparound problem, eliminate table bloat issues, improve performance by 4x, and support cloud-native storage. Now part of the Pigsty family. Read more

4.15 - OpenHalo: MySQL Wire-Compatible PostgreSQL is Here!

What? PostgreSQL can now be accessed using MySQL clients? That’s right, openHalo, which was open-sourced on April Fool’s Day, provides exactly this capability and has now joined the Pigsty kernel family. Read more

4.16 - PGFS: Using Database as a Filesystem

Leverage JuiceFS to turn PostgreSQL into a filesystem with PITR capabilities! Read more

4.17 - PostgreSQL Ecosystem Frontier Developments

Sharing some interesting recent developments in the PG ecosystem. Read more

4.18 - Pig, The Postgres Extension Wizard

Why would we need yet another package manager for PostgreSQL & extensions? Read more

4.19 - Don't Upgrade! Released and Immediately Pulled - Even PostgreSQL Isn't Immune to Epic Fails

Never deploy on Friday, or you’ll be working all weekend! PostgreSQL minor releases were pulled on the day of release, requiring emergency rollback. Read more

4.20 - PostgreSQL 12 End-of-Life, PG 17 Takes the Throne

PG17 achieved extension ecosystem adaptation in half the time of PG16, with 300 available extensions ready for production use. PG 12 officially exits support lifecycle. Read more

4.21 - The ideal way to deliver PostgreSQL Extensions

PostgreSQL Is Eating the Database World through the power of extensibility. With 575 packaged extensions powering PG, we may not say it’s invincible, but it’s definitely getting much closer. Read more

4.22 - PostgreSQL Convention 2024

No rules, no standards. Some developer conventions for PostgreSQL 16. Read more

4.23 - PostgreSQL 17 Released: No More Pretending!

PostgreSQL is now the world’s most advanced open-source database and has become the preferred open-source database for organizations of all sizes, matching or exceeding top commercial databases. Read more

4.24 - Can PostgreSQL Replace Microsoft SQL Server?

PostgreSQL can directly replace Oracle, SQL Server, and MongoDB at the kernel level. Of course, the most thorough replacement is SQL Server - AWS’s Babelfish provides wire-protocol-level compatibility. Read more

4.25 - Whoever Integrates DuckDB Best Wins the OLAP World

Just like the vector database extension race two years ago, the current PostgreSQL ecosystem extension competition has begun revolving around DuckDB. MotherDuck’s official entry into the PostgreSQL extension space undoubtedly signals that competition has entered white-hot territory. Read more

4.26 - StackOverflow 2024 Survey: PostgreSQL Has Gone Completely Berserk

The 2024 StackOverflow Global Developer Survey results are fresh out, and PostgreSQL has become the most popular, most loved, and most wanted database globally for the second consecutive year. Nothing can stop PostgreSQL from devouring the entire database world anymore! Read more

4.27 - Self-Hosting Dify with PG, PGVector, and Pigsty

Dify is an open-source LLM app development platform. This article explains how to self-host Dify using Pigsty. Read more

4.28 - PGCon.Dev 2024, The conf that shutdown PG for a week

Experience & Feeling on the PGCon.Dev 2024 Read more

4.29 - PostgreSQL 17 Beta1 Released!

The PostgreSQL Global Development Group announces PostgreSQL 17’s first Beta version is now available. This time, PostgreSQL has truly burst the toothpaste tube! Read more

4.30 - Why PostgreSQL is the Future Standard?

Author: Ajay Kulkarni (TimescaleDB CEO) | Original: Why PostgreSQL Is the Bedrock for the Future of Data

One of the biggest trends in software development today is PostgreSQL becoming the de facto database standard. This article explains why. Read more

4.31 - Will PostgreSQL Change Its License?

Author: Jonathan Katz (PostgreSQL Core Team) | Original: PostgreSQL Will Not Change Its License

PostgreSQL will not change its license. This article is a response from PostgreSQL core team members on this question. Read more

4.32 - Postgres is eating the database world

PostgreSQL isn’t just a simple relational database; it’s a data management framework with the potential to engulf the entire database realm. The trend of “Using Postgres for Everything” is no longer limited to a few elite teams but is becoming a mainstream best practice.

OLAP’s New Challenger

In a 2016 database meetup, I argued that a significant gap in the PostgreSQL ecosystem was the lack of a sufficiently good columnar storage engine for OLAP workloads. While PostgreSQL itself offers lots of analysis features, its performance in full-scale analysis on larger datasets doesn’t quite measure up to dedicated real-time data warehouses.

Consider ClickBench, an analytics performance benchmark, where we’ve documented the performance of PostgreSQL, its ecosystem extensions, and derivative databases. The untuned PostgreSQL performs poorly (x1050), but it can reach (x47) with optimization. Additionally, there are three analysis-related extensions: columnar store Hydra (x42), time-series TimescaleDB (x103), and distributed Citus (x262).

ClickBench c6a.4xlarge, 500gb gp2 results in relative time

This performance can’t be considered bad, especially compared to pure OLTP databases like MySQL and MariaDB (x3065, x19700); however, its third-tier performance is not “good enough,” lagging behind the first-tier OLAP components like Umbra, ClickHouse, Databend, SelectDB (x3~x4) by an order of magnitude. It’s a tough spot - not satisfying enough to use, but too good to discard.

However, the arrival of ParadeDB and DuckDB changed the game!

ParadeDB’s native PG extension pg_analytics achieves second-tier performance (x10), narrowing the gap to the top tier to just 3–4x. Given the additional benefits, this level of performance discrepancy is often acceptable - ACID, freshness and real-time data without ETL, no additional learning curve, no maintenance of separate services, not to mention its ElasticSearch grade full-text search capabilities.

DuckDB focuses on pure OLAP, pushing analysis performance to the extreme (x3.2) — excluding the academically focused, closed-source database Umbra, DuckDB is arguably the fastest for practical OLAP performance. It’s not a PG extension, but PostgreSQL can fully leverage DuckDB’s analysis performance boost as an embedded file database through projects like DuckDB FDW and pg_quack.

The emergence of ParadeDB and DuckDB propels PostgreSQL’s analysis capabilities to the top tier of OLAP, filling the last crucial gap in its analytic performance.

The Pendulum of Database Realm

The distinction between OLTP and OLAP didn’t exist at the inception of databases. The separation of OLAP data warehouses from databases emerged in the 1990s due to traditional OLTP databases struggling to support analytics scenarios’ query patterns and performance demands.

For a long time, best practice in data processing involved using MySQL/PostgreSQL for OLTP workloads and syncing data to specialized OLAP systems like Greenplum, ClickHouse, Doris, Snowflake, etc., through ETL processes.

DDIA, Martin Kleppmann, ch3, The republic of OLTP & Kingdom of OLAP

Like many “specialized databases,” the strength of dedicated OLAP systems often lies in performance — achieving 1-3 orders of magnitude improvement over native PG or MySQL. The cost, however, is redundant data, excessive data movement, lack of agreement on data values among distributed components, extra labor expense for specialized skills, extra licensing costs, limited query language power, programmability and extensibility, limited tool integration, poor data integrity and availability compared with a complete DMBS.

However, as the saying goes, “What goes around comes around”. With hardware improving over thirty years following Moore’s Law, performance has increased exponentially while costs have plummeted. In 2024, a single x86 machine can have hundreds of cores (512 vCPU EPYC 9754 x2), several TBs of RAM, a single NVMe SSD can hold up to 64TB, and a single all-flash rack can reach 2PB; object storage like S3 offers virtually unlimited storage.

Hardware advancements have solved the data volume and performance issue, while database software developments (PostgreSQL, ParadeDB, DuckDB) have addressed access method challenges. This puts the fundamental assumptions of the analytics sector — the so-called “big data” industry — under scrutiny.

As DuckDB’s manifesto "Big Data is Dead" suggests, the era of big data is over. Most people don’t have that much data, and most data is seldom queried. The frontier of big data recedes as hardware and software evolve, rendering “big data” unnecessary for 99% of scenarios.

If 99% of use cases can now be handled on a single machine with standalone DuckDB or PostgreSQL (and its replicas), what’s the point of using dedicated analytics components? If every smartphone can send and receive texts freely, what’s the point of pagers? (With the caveat that North American hospitals still use pagers, indicating that maybe less than 1% of scenarios might genuinely need “big data.”)

The shift in fundamental assumptions is steering the database world from a phase of diversification back to convergence, from a big bang to a mass extinction. In this process, a new era of unified, multi-modeled, super-converged databases will emerge, reuniting OLTP and OLAP. But who will lead this monumental task of reconsolidating the database field?

PostgreSQL: The Database World Eater

There are a plethora of niches in the database realm: time-series, geospatial, document, search, graph, vector databases, message queues, and object databases. PostgreSQL makes its presence felt across all these domains.

A case in point is the PostGIS extension, which sets the de facto standard in geospatial databases; the TimescaleDB extension awkwardly positions “generic” time-series databases; and the vector extension, PGVector, turns the dedicated vector database niche into a punchline.

This isn’t the first time; we’re witnessing it again in the oldest and largest subdomain: OLAP analytics. But PostgreSQL’s ambition doesn’t stop at OLAP; it’s eyeing the entire database world!

What makes PostgreSQL so capable? Sure, it’s advanced, but so is Oracle; it’s open-source, as is MySQL. PostgreSQL’s edge comes from being both advanced and open-source, allowing it to compete with Oracle/MySQL. But its true uniqueness lies in its extreme extensibility and thriving extension ecosystem.

TimescaleDB survey: what is the main reason you choose to use PostgreSQL

PostgreSQL isn’t just a relational database; it’s a data management framework capable of engulfing the entire database galaxy. Besides being open-source and advanced, its core competitiveness stems from extensibility, i.e., its infra’s reusability and extension’s composability.

The Magic of Extreme Extensibility

PostgreSQL allows users to develop extensions, leveraging the database’s common infra to deliver features at minimal cost. For instance, the vector database extension pgvector, with just several thousand lines of code, is negligible in complexity compared to PostgreSQL’s millions of lines. Yet, this “insignificant” extension achieves complete vector data types and indexing capabilities, outperforming lots of specialized vector databases.

Why? Because pgvector’s creators didn’t need to worry about the database’s general additional complexities: ACID, recovery, backup & PITR, high availability, access control, monitoring, deployment, 3rd-party ecosystem tools, client drivers, etc., which require millions of lines of code to solve well. They only focused on the essential complexity of their problem.

For example, ElasticSearch was developed on the Lucene search library, while the Rust ecosystem has an improved next-gen full-text search library, Tantivy, as a Lucene alternative. ParadeDB only needs to wrap and connect it to PostgreSQL’s interface to offer search services comparable to ElasticSearch. More importantly, it can stand on the shoulders of PostgreSQL, leveraging the entire PG ecosystem’s united strength (e.g., mixed searches with PG Vector) to “unfairly” compete with another dedicated database.

Pigsty currently packages 575 extensions for supported platforms. There are many more projects in the broader PostgreSQL extension ecosystem.

The extensibility brings another huge advantage: the composability of extensions, allowing different extensions to work together, creating a synergistic effect where 1+1 » 2. For instance, TimescaleDB can be combined with PostGIS for spatio-temporal data support; the BM25 extension for full-text search can be combined with the PGVector extension, providing hybrid search capabilities.

Furthermore, the distributive extension Citus can transparently transform a standalone cluster into a horizontally partitioned distributed database cluster. This capability can be orthogonally combined with other features, making PostGIS a distributed geospatial database, PGVector a distributed vector database, ParadeDB a distributed full-text search database, and so on.

What’s more powerful is that extensions evolve independently, without the cumbersome need for main branch merges and coordination. This allows for scaling — PG’s extensibility lets numerous teams explore database possibilities in parallel, with all extensions being optional, not affecting the core functionality’s reliability. Those features that are mature and robust have the chance to be stably integrated into the main branch.

PostgreSQL achieves both foundational reliability and agile functionality through the magic of extreme extensibility, making it an outlier in the database world and changing the game rules of the database landscape.

Game Changer in the DB Arena

The emergence of PostgreSQL has shifted the paradigms in the database domain: Teams endeavoring to craft a “new database kernel” now face a formidable trial — how to stand out against the open-source, feature-rich Postgres. What’s their unique value proposition?

Until a revolutionary hardware breakthrough occurs, the advent of practical, new, general-purpose database kernels seems unlikely. No singular database can match the overall prowess of PG, bolstered by all its extensions — not even Oracle, given PG’s ace of being open-source and free.

A niche database product might carve out a space for itself if it can outperform PostgreSQL by an order of magnitude in specific aspects (typically performance). However, it usually doesn’t take long before the PostgreSQL ecosystem spawns open-source extension alternatives. Opting to develop a PG extension rather than a whole new database gives teams a crushing speed advantage in playing catch-up!

Following this logic, the PostgreSQL ecosystem is poised to snowball, accruing advantages and inevitably moving towards a monopoly, mirroring the Linux kernel’s status in server OS within a few years. Developer surveys and database trend reports confirm this trajectory.

StackOverflow 2023 Survey: PostgreSQL, the Decathlete

StackOverflow’s Database Trends Over the Past 7 Years

PostgreSQL has long been the favorite database in HackerNews & StackOverflow. Many new open-source projects default to PostgreSQL as their primary, if not only, database choice. And many new-gen companies are going All in PostgreSQL.

As “Radical Simplicity: Just Use Postgres” says, Simplifying tech stacks, reducing components, accelerating development, lowering risks, and adding more features can be achieved by “Just Use Postgres.” Postgres can replace many backend technologies, including MySQL, Kafka, RabbitMQ, ElasticSearch, Mongo, and Redis, effortlessly serving millions of users. Just Use Postgres is no longer limited to a few elite teams but becoming a mainstream best practice.

What Else Can Be Done?

The endgame for the database domain seems predictable. But what can we do, and what should we do?

PostgreSQL is already a near-perfect database kernel for the vast majority of scenarios, making the idea of a kernel “bottleneck” absurd. Forks of PostgreSQL and MySQL that tout kernel modifications as selling points are essentially going nowhere.

This is similar to the situation with the Linux OS kernel today; despite the plethora of Linux distros, everyone opts for the same kernel. Forking the Linux kernel is seen as creating unnecessary difficulties, and the industry frowns upon it.

Accordingly, the main conflict is no longer the database kernel itself but two directions— database extensions and services! The former pertains to internal extensibility, while the latter relates to external composability. Much like the OS ecosystem, the competitive landscape will concentrate on database distributions. In the database domain, only those distributions centered around extensions and services stand a chance for ultimate success.

Kernel remains lukewarm, with MariaDB, the fork of MySQL’s parent, nearing delisting, while AWS, profiting from offering services and extensions on top of the free kernel, thrives. Investment has flowed into numerous PG ecosystem extensions and service distributions: Citus, TimescaleDB, Hydra, PostgresML, ParadeDB, FerretDB, StackGres, Aiven, Neon, Supabase, Tembo, PostgresAI, and our own PG distro — — Pigsty.

A dilemma within the PostgreSQL ecosystem is the independent evolution of many extensions and tools, lacking a unifier to synergize them. For instance, Hydra releases its own package and Docker image, and so does PostgresML, each distributing PostgreSQL images with their own extensions and only their own. These images and packages are far from comprehensive database services like AWS RDS.

Even service providers and ecosystem integrators like AWS fall short in front of numerous extensions, unable to include many due to various reasons (AGPLv3 license, security challenges with multi-tenancy), thus failing to leverage the synergistic amplification potential of PostgreSQL ecosystem extensions.

Extesion Category	Pigsty RDS & PGDG	AWS RDS PG	Aliyun RDS PG
Add Extension	Free to Install	Not Allowed	Not Allowed
Geo Spatial	PostGIS 3.4.2	PostGIS 3.4.1	PostGIS 3.3.4
Time Series	TimescaleDB 2.14.2
Distributive	Citus 12.1
AI / ML	PostgresML 2.8.1
Columnar	Hydra 1.1.1
Vector	PGVector 0.6	PGVector 0.6	pase 0.0.1
Sparse Vector	PG Sparse 0.5.6
Full-Text Search	pg_bm25 0.5.6
Graph	Apache AGE 1.5.0
GraphQL	PG GraphQL 1.5.0
Message Queue	pgq 3.5.0
OLAP	pg_analytics 0.5.6
DuckDB	duckdb_fdw 1.1
CDC	wal2json 2.5.3	wal2json 2.5
Bloat Control	pg_repack 1.5.0	pg_repack 1.5.0	pg_repack 1.4.8
Point Cloud	PG PointCloud 1.2.5		Ganos PointCloud 6.1

Many important extensions are not available on Cloud RDS (PG 16, 2024-02-29)

Extensions are the soul of PostgreSQL. A Postgres without the freedom to use extensions is like cooking without salt, a giant constrained.

Addressing this issue is one of our primary goals.

Our Resolution: Pigsty

Despite earlier exposure to MySQL Oracle, and MSSQL, when I first used PostgreSQL in 2015, I was convinced of its future dominance in the database realm. Nearly a decade later, I’ve transitioned from a user and administrator to a contributor and developer, witnessing PG’s march toward that goal.

Interactions with diverse users revealed that the database field’s shortcoming isn’t the kernel anymore — PostgreSQL is already sufficient. The real issue is leveraging the kernel’s capabilities, which is the reason behind RDS’s booming success.

However, I believe this capability should be as accessible as free software, like the PostgreSQL kernel itself — available to every user, not just renting from cyber feudal lords.

Thus, I created Pigsty, a battery-included, local-first PostgreSQL distribution as an open-source RDS Alternative, which aims to harness the collective power of PostgreSQL ecosystem extensions and democratize access to production-grade database services.

Pigsty stands for PostgreSQL in Great STYle, representing the zenith of PostgreSQL.

We’ve defined six core propositions addressing the central issues in PostgreSQL database services:

Extensible Postgres, Reliable Infras, Observable Graphics, Available Services, Maintainable Toolbox, and Composable Modules.

The initials of these value propositions offer another acronym for Pigsty:

Postgres, Infras, Graphics, Service, Toolbox, Yours.

Your graphical Postgres infrastructure service toolbox.

Extensible PostgreSQL is the linchpin of this distribution. In the recently launched Pigsty v2.6, we integrated DuckDB FDW and ParadeDB extensions, massively boosting PostgreSQL’s analytical capabilities and ensuring every user can easily harness this power.

Our aim is to integrate the strengths within the PostgreSQL ecosystem, creating a synergistic force akin to the Ubuntu of the database world. I believe the kernel debate is settled, and the real competitive frontier lies here.

• PostGIS: Provides geospatial data types and indexes, the de facto standard for GIS (& pgPointCloud, pgRouting).
• TimescaleDB: Adds time-series, continuous aggregates, distributed, columnar storage, and automatic compression capabilities.
• PGVector: Support AI vectors/embeddings and ivfflat, hnsw vector indexes (& pg_sparse for sparse vectors).
• Citus: Transforms classic master-slave PG clusters into horizontally partitioned distributed database clusters.
• Hydra: Adds columnar storage and analytics, rivaling ClickHouse’s analytic capabilities.
• ParadeDB: Elevates full-text search and mixed retrieval to ElasticSearch levels (& zhparser for Chinese tokenization).
• Apache AGE: Graph database extension, adding Neo4J-like OpenCypher query support to PostgreSQL.
• PG GraphQL: Adds native built-in GraphQL query language support to PostgreSQL.
• DuckDB FDW: Enables direct access to DuckDB’s powerful embedded analytic database files through PostgreSQL (& DuckDB CLI).
• Supabase: An open-source Firebase alternative based on PostgreSQL, providing a complete app development storage solution.
• FerretDB: An open-source MongoDB alternative based on PostgreSQL, compatible with MongoDB APIs/drivers.
• PostgresML: Facilitates classic machine learning algorithms, calling, deploying, and training AI models with SQL.

Developers, your choices will shape the future of the database world. I hope my work helps you better utilize the world’s most advanced open-source database kernel: PostgreSQL.

Read in Pigsty’s Blog | GitHub Repo: Pigsty | Official Website

4.33 - Technical Minimalism: Just Use PostgreSQL for Everything

Whether production databases should be containerized remains a controversial topic. From a DBA’s perspective, I believe that currently, putting production databases in Docker is still a bad idea. Read more

4.34 - New PostgreSQL Ecosystem Player: ParadeDB

ParadeDB aims to be an Elasticsearch alternative: “Modern Elasticsearch Alternative built on Postgres” — PostgreSQL for search and analytics. Read more

4.35 - PostgreSQL's Impressive Scalability

This article describes how Cloudflare scaled to support 55 million requests per second using 15 PostgreSQL clusters, and PostgreSQL’s scalability performance. Read more

4.36 - PostgreSQL Outlook for 2024

Author: Jonathan Katz (PostgreSQL Core Team) | Original: PostgreSQL 2024

PostgreSQL core team member Jonathan Katz’s outlook for PostgreSQL in 2024, reviewing the progress made over the past few years. Read more

4.37 - PostgreSQL Wins 2024 Database of the Year Award! (Fifth Time)

DB-Engines officially announced today that PostgreSQL has once again been crowned “Database of the Year.” This is the fifth time PG has received this honor in the past seven years. If not for Snowflake stealing the spotlight for two years, the database world would have almost become a PostgreSQL solo show. Read more

4.38 - PostgreSQL Macro Query Optimization with pg_stat_statements

Query optimization is one of the core responsibilities of DBAs. This article introduces how to use metrics provided by pg_stat_statements for macro-level PostgreSQL query optimization. Read more

4.39 - FerretDB: PostgreSQL Disguised as MongoDB

FerretDB aims to provide a truly open-source MongoDB alternative based on PostgreSQL. Read more

4.40 - How to Use pg_filedump for Data Recovery?

Backups are a DBA’s lifeline — but what if your PostgreSQL database has already exploded and you have no backups? Maybe pg_filedump can help you! Read more

4.41 - Vector is the New JSON

Author: Jonathan Katz (PostgreSQL Core Team) | Original: Vectors are the new JSON in PostgreSQL

Vectors will become a key element in building applications, just like JSON historically. PostgreSQL leads the AI era with vector extensions. Read more

4.42 - PostgreSQL, The most successful database

StackOverflow 2023 Survey shows PostgreSQL is the most popular, loved, and wanted database, solidifying its status as the ‘Linux of Database’. Read more

4.43 - AI Large Models and Vector Database PGVector

This article focuses on vector databases hyped by AI, introduces the basic principles of AI embeddings and vector storage/retrieval, and demonstrates the functionality, performance, acquisition, and application of the vector database extension PGVECTOR through a concrete knowledge base retrieval case study. Read more

4.44 - How Powerful is PostgreSQL Really?

Let performance data speak: Why PostgreSQL is the world’s most advanced open-source relational database, aka the world’s most successful database. MySQL vs PostgreSQL performance showdown and distributed database reality check. Read more

4.45 - Why PostgreSQL is the Most Successful Database?

Database users are developers, but what about developers’ preferences, likes, and choices? Looking at StackOverflow survey results over the past six years, it’s clear that in 2022, PostgreSQL has won all three categories, becoming literally the “most successful database” Read more

4.46 - Ready-to-Use PostgreSQL Distribution: Pigsty

Yesterday I gave a live presentation in the PostgreSQL Chinese community, introducing the open-source PostgreSQL full-stack solution: Pigsty. Read more

4.47 - Why Does PostgreSQL Have a Bright Future?

Databases are the core component of information systems, relational databases are the absolute backbone of databases, and PostgreSQL is the world’s most advanced open source relational database. With such favorable timing and positioning, how can it not achieve great success? Read more

4.48 - Implementing Advanced Fuzzy Search

How to implement relatively complex fuzzy search logic in PostgreSQL? Read more

4.49 - Localization and Collation Rules in PostgreSQL

What? Don’t know what COLLATION is? Remember one thing: using C COLLATE is always the right choice! Read more

4.50 - PG Replica Identity Explained

Replica identity is important - it determines the success or failure of logical replication Read more

4.51 - PostgreSQL Logical Replication Deep Dive

This article introduces the principles and best practices of logical replication in PostgreSQL 13. Read more

4.52 - A Methodology for Diagnosing PostgreSQL Slow Queries

Slow queries are the sworn enemy of OLTP databases. Here’s how to identify, analyze, and fix them using metrics (Pigsty dashboards), pg_stat_statements, and logs. Read more

4.53 - Incident-Report: Patroni Failure Due to Time Travel

Machine restarted due to failure, NTP service corrected PG time after PG startup, causing Patroni to fail to start. Read more

4.54 - Online Primary Key Column Type Change

How to change column types online, such as upgrading from INT to BIGINT? Read more

4.55 - Golden Monitoring Metrics: Errors, Latency, Throughput, Saturation

Understanding the golden monitoring metrics in PostgreSQL Read more

4.56 - Database Cluster Management Concepts and Entity Naming Conventions

Concepts and their naming are very important. Naming style reflects an engineer’s understanding of system architecture. Poorly defined concepts lead to communication confusion, while carelessly set names create unexpected additional burden. Therefore, they need careful design. Read more

4.57 - PostgreSQL's KPI

Managing databases is similar to managing people - both need KPIs (Key Performance Indicators). So what are database KPIs? This article introduces a way to measure PostgreSQL load: using a single horizontally comparable metric that is basically independent of workload type and machine type, called PG Load. Read more

4.58 - Online PostgreSQL Column Type Migration

How to modify PostgreSQL column types online? A general approach Read more

4.59 - Frontend-Backend Communication Wire Protocol

Understanding the TCP protocol used for communication between PostgreSQL server and client, and printing messages using Go Read more

4.60 - Transaction Isolation Level Considerations

PostgreSQL actually has only two transaction isolation levels: Read Committed and Serializable Read more

4.61 - Incident: PostgreSQL Extension Installation Causes Connection Failure

Today encountered an interesting case where a customer reported database connection issues caused by extensions. Read more

4.62 - CDC Change Data Capture Mechanisms

Change Data Capture is an interesting ETL alternative solution. Read more

4.63 - Locks in PostgreSQL

Snapshot isolation does most of the heavy lifting in PG, but locks still matter. Here’s a practical guide to table locks, row locks, intention locks, and pg_locks. Read more

4.64 - O(n2) Complexity of GIN Search

When GIN indexes are used to search with very long keyword lists, performance degrades significantly. This article explains why GIN index keyword search has O(n^2) time complexity. Read more

4.65 - PostgreSQL Common Replication Topology Plans

Replication is one of the core issues in system architecture. Read more

4.66 - Warm Standby: Using pg_receivewal

There are various backup strategies. Physical backups can usually be divided into four types. Read more

4.67 - Incident-Report: Connection-Pool Contamination Caused by pg_dump

Sometimes, interactions between components manifest in subtle ways. For example, using pg_dump to export data from a connection pool can cause connection pool contamination issues. Read more

4.68 - PostgreSQL Data Page Corruption Repair

Using binary editing to repair PostgreSQL data pages, and how to make a primary key query return two records. Read more

4.69 - Relation Bloat Monitoring and Management

PostgreSQL uses MVCC as its primary concurrency control technology. While it has many benefits, it also brings other effects, such as relation bloat. Read more

4.70 - Getting Started with PipelineDB

PipelineDB is a PostgreSQL extension for streaming analytics. Here’s how to install it and build continuous views over live data. Read more

4.71 - TimescaleDB Quick Start

TimescaleDB is a PostgreSQL extension plugin that provides time-series database functionality. Read more

4.72 - Incident-Report: Integer Overflow from Rapid Sequence Number Consumption

If you use Integer sequences on tables, you should consider potential overflow scenarios. Read more

4.73 - Incident-Report: PostgreSQL Transaction ID Wraparound

XID WrapAround is perhaps a unique type of failure specific to PostgreSQL Read more

4.74 - GeoIP Geographic Reverse Lookup Optimization

A common requirement in application development is GeoIP conversion - converting source IP addresses to geographic coordinates or administrative divisions (country-state-city-county-town-village) Read more

4.75 - PostgreSQL Trigger Usage Considerations

Detailed understanding of trigger management and usage in PostgreSQL Read more

4.76 - PostgreSQL Development Convention (2018 Edition)

Without rules, there can be no order. This article compiles a development specification for PostgreSQL database principles and features, which can reduce confusion encountered when using PostgreSQL. Read more

4.77 - What Are PostgreSQL's Advantages?

PostgreSQL’s slogan is “The World’s Most Advanced Open-Source Relational Database,” but I think the most vivid characterization should be: The Full-Stack Database That Does It All - one tool to rule them all. Read more

4.78 - Efficient Administrative Region Lookup with PostGIS

How to efficiently solve the typical reverse geocoding problem: determining administrative regions based on user coordinates. Read more

4.79 - KNN Ultimate Optimization: From RDS to PostGIS

Ultimate optimization of KNN problems, from traditional relational design to PostGIS Read more

4.80 - Monitoring Table Size in PostgreSQL

Tables in PostgreSQL correspond to many physical files. This article explains how to calculate the actual size of a table in PostgreSQL. Read more

4.81 - PgAdmin Installation and Configuration

PgAdmin is a GUI program for managing PostgreSQL, written in Python, but it’s quite dated and requires some additional configuration. Read more

4.82 - Incident-Report: Uneven Load Avalanche

Recently there was a perplexing incident where a database had half its data volume and load migrated away, but ended up being overwhelmed due to increased load. Read more

4.83 - Bash and psql Tips

Some tips for interacting between PostgreSQL and Bash. Read more

4.84 - Distinct On: Remove Duplicate Data

Use Distinct On extension clause to quickly find records with maximum/minimum values within groups Read more

4.85 - Function Volatility Classification Levels

PostgreSQL functions have three volatility levels by default. Proper use can significantly improve performance. Read more

4.86 - Implementing Mutual Exclusion Constraints with Exclude

Exclude constraint is a PostgreSQL extension that can implement more advanced and sophisticated database constraints. Read more

4.87 - PostgreSQL Routine Maintenance

Cars need oil changes, databases need maintenance. For PG, three important maintenance tasks: backup, repack, vacuum Read more

4.88 - Backup and Recovery Methods Overview

Backup is the foundation of a DBA’s livelihood. With backups, there’s no need to panic. Read more

4.89 - PgBackRest2 Documentation

PgBackRest is a set of PostgreSQL backup tools written in Perl Read more

4.90 - Pgbouncer Quick Start

Pgbouncer is a lightweight database connection pool. This guide covers basic Pgbouncer configuration, management, and usage. Read more

4.91 - PostgreSQL Server Log Regular Configuration

It’s recommended to configure PostgreSQL’s log format as CSV for easy analysis, and it can be directly imported into PostgreSQL data tables. Read more

4.92 - Testing Disk Performance with FIO

FIO is a convenient tool for testing disk I/O performance Read more

4.93 - Using sysbench to Test PostgreSQL Performance

Although PostgreSQL provides pgbench, sometimes you need sysbench to outperform MySQL. Read more

4.94 - Changing Engines Mid-Flight — PostgreSQL Zero-Downtime Data Migration

Data migration typically involves stopping services for updates. Zero-downtime data migration is a relatively advanced operation. Read more

4.95 - Finding Unused Indexes

Indexes are useful, but they’re not free. Unused indexes are a waste. Use these methods to identify unused indexes. Read more

4.96 - Batch Configure SSH Passwordless Login

Quick configuration for passwordless login to all machines Read more

4.97 - Wireshark Packet Capture Protocol Analysis

Wireshark is a very useful tool, especially suitable for analyzing network protocols. Here’s a simple introduction to using Wireshark for packet capture and PostgreSQL protocol analysis. Read more

4.98 - The Versatile file_fdw — Reading System Information from Your Database

With file_fdw, you can easily view operating system information, fetch network data, and feed various data sources into your database for unified viewing and management. Read more

4.99 - Common Linux Statistics CLI Tools

top, free, vmstat, iostat: Quick reference for four commonly used CLI tools Read more

4.100 - Installing PostGIS from Source

PostGIS is PostgreSQL’s killer extension, but compiling and installing it isn’t easy. Read more

4.101 - Go Database Tutorial: database/sql

Similar to JDBC, Go also has a standard database access interface. This article details how to use database/sql in Go and important considerations. Read more

4.102 - Implementing Cache Synchronization with Go and PostgreSQL

Cleverly utilizing PostgreSQL’s Notify feature, you can conveniently notify applications of metadata changes and implement trigger-based logical replication. Read more

4.103 - Auditing Data Changes with Triggers

Sometimes we want to record important metadata changes for audit purposes. PostgreSQL triggers can conveniently solve this need automatically. Read more

4.104 - Building an ItemCF Recommender in Pure SQL

Five minutes, PostgreSQL, and the MovieLens dataset—that’s all you need to implement a classic item-based collaborative filtering recommender. Read more

4.105 - UUID Properties, Principles and Applications

UUID properties, principles and applications, and how to manipulate UUIDs using PostgreSQL stored procedures. Read more

4.106 - PostgreSQL MongoFDW Installation and Deployment

Recently had business requirements to access MongoDB through PostgreSQL FDW, but compiling MongoDB FDW is really a nightmare. Read more

5 - AI

5.1 - Cyber Dharma: A New Engineering Answer to Ancient Questions

A project manifesto: why build Cyber Dharma, and what it is not. Read more

5.2 - Burning Hundreds of Millions of Tokens a Day. Then What?

Once token burn turns from usage exhaust into a KPI and leaderboard, it quickly mutates into theater. Don’t post fuel burn. Post where you got to. Read more

5.3 - Why PostgreSQL Won in the AI Era

Boring technology won the wildest era. A look at extensibility, agent choice, database cloning, and the future of the DBA. Read more

5.4 - AGI Is Here. Do You Have a Ticket?

When the strongest AI is not expensive but simply unavailable, the world starts converging on digital feudalism. And the window to act is narrowing. Read more

5.5 - Can You Distill an Expert?

Polanyi’s tacit knowledge explains the 70% ceiling of AI agents: real intuition, feel, and judgment do not serialize cleanly. They grow, if at all, through practice. Read more

5.6 - Local AI's Inflection Point: 2027

When subsidies fade, hardware catches up, and open models mature, all three lines cross in 2027. “Build your own AI” goes from idea to reality. Read more

5.7 - Yes, I Use AI to Write

AI is a multiplier. It amplifies depth and mediocrity alike. In an age where answers are cheap, questions are the real currency. There is nothing to hide about writing with AI. Read more

5.8 - LLMs Have Emotions: Claude's Internals Reveal Steerable Emotion Vectors

Anthropic’s new research gives us the first direct look at causally steerable emotion vectors inside a large language model. That should change how we think about AI. Read more

6 - Database

6.1 - Two Months into Maintaining a MinIO Fork

Two months after forking MinIO, pgsty/minio ships patches for four CVEs and related security issues. No new features, just working builds, a restored console, and timely security fixes. Read more

6.2 - Palantir's 'Ontology' Hustle

Ontology is database modeling. The word’s only function is to make people who don’t understand databases think they’re looking at something new — and happily pay a thousand times more for something old. Read more

6.3 - MinIO Is Dead, Long Live MinIO

MinIO’s repo is officially archived and abandoned. And how AI Agents helped bring MinIO back from the dead. Read more

6.4 - When Coding Becomes Cheap, What Still Matters?

Codex 5.3 xHigh pushed my workflow past a tipping point: writing code is no longer the scarce resource. The real leverage is design quality and engineering acceptance. This is the practical loop I use to ship reliable software with AI agents. Read more

6.5 - The Agent Moat: Runtime

A mediocre local who knows the terrain beats a genius parachuted into unknown territory. Intelligence without context is idle. An agent without a runtime is vapor. Read more

6.6 - AI Ripped the Skin Off Software

Software stocks are melting down. Who survives? Who rises? AI stripped away software’s skin, exposing the database skeleton underneath. The market isn’t panic-selling — it’s repricing. Read more

6.7 - New Programmers in the AI Era: Where Do You Go?

Should we still hire fresh grads? Squeezed between AI and senior devs, what’s the play for new programmers? Master the right tools, take initiative, find the right mentor. Read more

6.8 - The Great Software Meltdown: When Translation Layers Get Squashed

SaaS and workflow software are dead. From APPs & GUIs to Agents, Databases, and CLI. Read more

6.9 - Agent OS: We're Building DOS Again

LLM = CPU. Context = RAM. Database = Disk. Agent = App. The mapping is surprisingly clean. And if OS history is any guide, we may know what comes next — and what’s still missing. Read more

6.10 - Claude Code Observability

Export Claude Code’s OTEL logs and metrics to Victoria stack, visualize with Grafana dashboards. Read more

6.11 - Claude Code Quick Start: Using Alternative LLMs at 1/10 the Cost

How to install and use Claude Code? How to achieve similar results at 1/10 of Claude’s cost with alternative models? A one-liner to get CC up and running! Read more

6.12 - Data 2025: Year in Review with Mike Stonebraker

A conversation between Mike Stonebraker (Turing Award Winner, Creator of PostgreSQL), Andy Pavlo (Carnegie Mellon), and the DBOS team. Read more

6.13 - What Database Does AI Agent Need?

The bottleneck for AI agents isn’t in database engines but in upper-layer integration. Muscle memory, associative memory, and trial-and-error courage will be key. Read more

6.14 - MySQL and Baijiu: The Internet’s Obedience Test

MySQL is to the internet what baijiu is to China: harsh, hard to swallow, yet worshipped because culture demands obedience. Both are loyalty tests—will you endure discomfort to fit in? Read more

6.15 - Victoria: The Observability Stack That Slaps the Industry

VictoriaMetrics is brutally efficient—using a fraction of Prometheus + Loki’s resources for multiples of the performance. Pigsty v4 swaps to the Victoria stack; here’s the beta for anyone eager to try it. Read more

6.16 - MinIO Is Dead. Who Picks Up the Pieces?

MinIO just entered maintenance mode. What replaces it? Can RustFS step in? I tested the contenders so you don’t have to. Read more

6.17 - MinIO is Dead

MinIO announces that it is entering maintenance mode: the dragon-slayer has become the dragon. This is how MinIO transformed from an open-source S3 alternative into just another commercial software company. Read more

6.18 - When Answers Become Abundant, Questions Become the New Currency

Your ability to ask questions—and your taste in what to ask—determines your position in the AI era. When answers become commodities, good questions become the new wealth. We are living in the moment this prophecy comes true. Read more

6.19 - On Trusting Open-Source Supply Chains

In serious production you can’t rely on an upstream that explicitly says “no guarantees.” When someone says “don’t count on me,” the right answer is “then I’ll run it myself.” Read more

6.20 - Don't Run Docker Postgres for Production!

Tons of users running the official docker postgres image got burned during recent minor version upgrades. A friendly reminder: think twice before containerizing production databases. Read more

6.21 - DDIA 2nd Edition, Chinese Translation

The second edition of Designing Data-Intensive Applications has released ten chapters. I translated them into Chinese and rebuilt a clean Hugo/Hextra web version for the community. Read more

6.22 - Column: Database Guru

The database world is full of hype and marketing fog. This column cuts through it with blunt commentary on industry trends and product reality. Read more

6.23 - Dongchedi Just Exposed “Smart Driving.” Where’s Our Dongku-Di?

Imagine a “closed-course” shootout for domestic databases and clouds, the way Dongchedi just humiliated 30+ autonomous cars. This industry needs its own stress test. Read more

6.24 - Google AI Toolbox: Production-Ready Database MCP is Here?

Google recently launched a database MCP toolbox, perhaps the first production-ready solution. Read more

6.25 - Where Will Databases and DBAs Go in the AI Era?

Who will be revolutionized first - OLTP or OLAP? Integration vs specialization, how to choose? Where will DBAs go in the AI era? Feng’s views from the HOW 2025 conference roundtable, organized and published. Read more

6.26 - Stop Arguing, The AI Era Database Has Been Settled

The database for the AI era has been settled. Capital markets are making intensive moves on PostgreSQL targets, with PG having become the default database for the AI era. Read more

6.27 - Open Data Standards: Postgres, OTel, and Iceberg

Author: Paul Copplestone (Supabase CEO) | Original: Open Data Standards: Postgres, OTel, and Iceberg

Three emerging standards in the data world: Postgres, OpenTelemetry, and Iceberg. Postgres is already the de facto standard. Read more

6.28 - The Lost Decade of Small Data

Author: Hannes Mühleisen (DuckDB Labs) | Original: The Lost Decade of Small Data

If DuckDB had launched in 2012, the great migration to distributed analytics might never have happened. Data isn’t that big after all. Read more

6.29 - Scaling Postgres to the Next Level at OpenAI

At PGConf.Dev 2025, Bohan Zhang from OpenAI shared a session titled Scaling Postgres to the next level at OpenAI, giving us a peek into the database usage of a top-tier unicorn.

“At OpenAl, we’ve proven that PostgreSQL can scale to support massive read-heavy workloads - even without sharding - using a single primary writer”

—— Bohan Zhang from OpenAI, PGConf.Dev 2025

Bohan Zhang is a member of the OpenAI Infra team, student of Andy Pavlo, and co-found OtterTune with him.

This article is based on Bohan’s presentation at the conference. with chinese translation/commentary by Ruohang Feng (Vonng): Author of Pigsty. The original chinese version is available on WeChat Column and Pigsty CN Blog.

Hacker News Discussion: OpenAI: Scaling Postgres to the Next Level

Background

Postgres is the backbone of our most critical systems at OpenAl. If Postgres goes down, many of OpenAI’s key features go down with it — and there’s plenty of precedent for this. PostgreSQL-related failures have caused several ChatGPT outages in the past.

OpenAI uses managed PostgreSQL databases on Azure, without sharding. Instead, they employ a classic primary-replica replication architecture with one primary and over dozens of read replicas. For a service with several hundred million active users like OpenAI, scalability is a major concern.

Challenges

In OpenAI’s PostgreSQL architecture, read scalability is excellent, but “write requests” have become the primary bottleneck. OpenAI has already made many optimizations here, such as offloading write workloads wherever possible and avoiding placing new business logic into the main database.

PostgreSQL’s MVCC design has some known issues, such as table and index bloat. Tuning autovacuum is complex, and every write generates a completely new version of a row. Index access might also require additional heap fetches for visibility checks. These design choices create challenges for scaling read replicas: for instance, more WAL typically leads to greater replication lag, and as the number of replicas grows, network bandwidth can become the new bottleneck.

Measures

To tackle these issues, we’ve made efforts on multiple fronts:

Reduce Load on Primary

The first optimization is to smooth out write spikes on the primary and minimize its load as much as possible, for example:

• Offloading all possible writes.
• Avoiding unnecessary writes at the application level.
• Using lazy writes to smooth out write bursts.
• Controlling the rate of data backfilling.

Additionally, OpenAI offloads as many read requests as possible to replicas. The few read requests that cannot be moved from the primary because they are part of read-write transactions are required to be as efficient as possible.

Query Optimization

The second area is query-level optimization. Since long-running transactions can block garbage collection and consume resources, they use timeout settings to prevent long “idle in transaction” states and set session, statement, and client-level timeouts. They also optimized some multi-way JOIN queries (e.g., joining 12 tables at once). The talk specifically mentioned that using ORMs can easily lead to inefficient queries and should be used with caution.

Mitigating Single Points of Failure

The primary is a single point of failure; if it goes down, writes are blocked. In contrast, we have many read-only replicas. If one fails, applications can still read from others. In fact, many critical requests are read-only, so even if the primary goes down, they can continue to serve reads.

Furthermore, we’ve distinguished between low-priority and high-priority requests. For high-priority requests, OpenAI allocates dedicated read-only replicas to prevent them from being impacted by low-priority ones.

Schema Management

The fourth measure is to allow only lightweight schema changes on this cluster. This means:

• Creating new tables or adding new workloads to it is not allowed.
• Adding or removing columns is allowed (with a 5-second timeout), but any operation that requires a full table rewrite is forbidden.
• Creating or removing indexes is allowed, but must be done using CONCURRENTLY.

Another issue mentioned was that persistent long-running queries (>1s) would continuously block schema changes, eventually causing them to fail. The solution was to have the application optimize or move these slow queries to replicas.

Results

• Scaled PostgreSQL on Azure to millions of QPS, supporting OpenAI’s critical services.
• Added dozens of replicas without increasing replication lag.
• Deployed read-only replicas to different geographical regions while maintaining low latency.
• Only one SEV0 incident related to PostgreSQL in the past nine months.
• Still have plenty of room for future growth.

“At OpenAl, we’ve proven that PostgreSQL can scale to support massive read-heavy workloads - even without sharding - using a single primary writer”

Case Studies

OpenAI also shared a few case studies of failures they’ve faced. The first was a cascading failure caused by a redis outage.

The second incident was more interesting: extremely high CPU usage triggered a bug where the WALSender process kept spin-looping instead of sending WAL to replicas, even after CPU levels returned to normal. This led to increased replication lag.

Feature Suggestions

Finally, Bohan raised some questions and feature suggestions to the PostgreSQL developer community:

First, regarding disabling indexes. Unused indexes cause write amplification and extra maintenance overhead. They want to remove useless indexes, but to minimize risk, they wish for a feature to “disable” an index. This would allow them to monitor performance metrics to ensure everything is fine before actually dropping it.

Second is about RT observability. Currently, pg_stat_statement only provides the average response time for each query type, but doesn’t directly offer latency metrics like p95 or p99. They hope for more histogram-like and percentile latency metrics.

The third point is about schema changes. They want PostgreSQL to record a history of schema change events, such as adding/removing columns and other DDL operations.

The fourth case is about the semantics of monitoring views. They found a session with state = 'active' and wait_event = 'ClientRead' that lasted for over two hours. This means a connection remained active long after query_start, and such connections can’t be killed by the idle_in_transaction_timeout. They wanted to know if this is a bug and how to resolve it.

Finally, a suggestion for optimizing PostgreSQL’s default parameters. The default values are too conservative. Could better defaults be used, or perhaps a heuristic-based configuration rule?

Vonng’s Commentary

Although PGConf.Dev 2025 is primarily focused on development, you often see use case presentations from users, like this one from OpenAI on their PostgreSQL scaling practices. These topics are actually quite interesting for core developers, as many of them don’t have a clear picture of how PostgreSQL is used in extreme scenarios, and these talks are very helpful.

Since late 2017, I managed dozens of PostgreSQL clusters at Tantan, which was one of the largest and most complex PG deployments in the Chinese internet scene: dozens of PG clusters with around 2.5 million QPS. Back then, our largest core primary had a 1-primary-33-replica setup, with a single cluster handling around 400K QPS. The bottleneck was also on single-database writes, which we eventually solved with application-side sharding, similar to Instagram’s approach.

You could say I’ve encountered all the problems and used all the solutions OpenAI mentioned in their talk. Of course, the difference is that today’s top-tier hardware is orders of magnitude better than it was eight years ago. This allows a startup like OpenAI to serve its entire business with a single PostgreSQL cluster without sharding. This is undoubtedly another powerful piece of evidence for the argument that “Distributed Databases Are a False Need”.

During the Q&A, I learned that OpenAI uses managed PostgreSQL on Azure with the highest available server hardware specs. They have dozens of replicas, including some in different geographical regions, and this behemoth cluster handles a total of about millions QPS. They use Datadog for monitoring, and the services access the RDS cluster from Kubernetes through a business-side PgBouncer connection pool.

As a strategic customer, the Azure PostgreSQL team provides them with dedicated support. But it’s clear that even with top-tier cloud database services, the customer needs to have sufficient knowledge and skill on the application and operations side. Even with the brainpower of OpenAI, they still stumble on some of the practical driving lessons of PostgreSQL.

During the social event after the conference, I had a great chat with Bohan and two other database founders until the wee hours. The off-the-record discussions were fascinating, but I can’t disclose more here, haha.

Vonng’s Q&A

Regarding the questions and feature requests Bohan raised, I can offer some answers here.

Most of the features OpenAI wants already exist in the PostgreSQL ecosystem, they just might not be available in the vanilla PG kernel or in a managed cloud database environment.

On Disabling Indexes

PostgreSQL actually has a “feature” to disable indexes. You just need to update the indisvalid field in the pg_index system catalog to false. The planner will then stop using the index, but it will continue to be maintained during DML operations. In principle, there’s nothing wrong with this, as concurrent index creation uses these two flags (isready, isvalid). It’s not black magic.

However, I can understand why OpenAI can’t use this method: it’s an undocumented “internal detail” rather than a formal feature. But more importantly, cloud databases usually don’t grant superuser privileges, so you just can’t update the system catalog like this.

But back to the original need — fear of accidentally deleting an index. There’s a simpler solution: just confirm from monitoring view (pg_stat_all_indexes) that the index isn’t being used on either the primary or the replicas. If you know an index hasn’t been used for a long time, you can safely delete it.

Monitoring index switch with Pigsty PGSQL TABLES Dashboard

-- Create a new index
CREATE UNIQUE INDEX CONCURRENTLY pgbench_accounts_pkey2 ON pgbench_accounts USING BTREE(aid);

-- Mark the original index as invalid (not used), but still maintained. planner will not use it.
UPDATE pg_index SET indisvalid = false WHERE indexrelid = 'pgbench_accounts_pkey'::regclass;

On Observability

Actually, pg_stat_statements provides the mean and stddev metrics, which you can use with properties of the normal distribution to estimate percentile metrics. But this is only a rough estimate, and you need to reset the counters periodically, otherwise the effectiveness of the full historical statistics will degrade over time.

RT Distribution with PGSQL QUERY Dashboard from PGSS

PGSS is unlikely to provide P95, P99 RT percentile metrics anytime soon, because it would increase the extension’s memory footprint by several dozen times. While that’s not a big deal for modern servers, it could be an issue in extremely conservative environments. I asked the maintainer of PGSS about this at the Unconference, and it’s unlikely to happen in the short term. I also asked Jelte, the maintainer of Pgbouncer, if this could be solved at the connection pool level, and a feature like that is not coming soon either.

However, there are other solutions to this problem. First, the pg_stat_monitor extension explicitly provides detailed percentile RT metrics, but you have to consider the performance impact of collecting these metrics on the cluster. A universal, non-intrusive method with no database performance overhead is to add query RT monitoring directly at the application’s Data Access Layer (DAL), but this requires cooperation and effort from the application side.

Also, using eBPF for side-channel collection of RT metrics is a great idea, but considering they’re using managed PostgreSQL on Azure, they won’t have server access, so that path is likely blocked.

On Schema Change History

Actually, PostgreSQL’s logging already provides this option. You just need to set log_statement to ddl (or the more advanced mod or all), and all DDL logs will be preserved. The pgaudit extension also provides similar functionality.

But I suspect what they really want isn’t DDL logs, but something like a system view that can be queried via SQL. In that case, another option is CREATE EVENT TRIGGER. You can use an event trigger to log DDL events directly into a data table. The pg_ddl_historization extension provides a more convenient way to do this, and I’ve compiled and packaged this extension as well.

Creating an event trigger also requires superuser privileges. AWS RDS has some special handling to allow this, but it seems that PostgreSQL on Azure does not support it.

On Monitoring View Semantics

In OpenAI’s example, pg_stat_activity.state = active means the backend process is still within the lifecycle of a single SQL statement. The WaitEvent = ClientRead means the process is on the CPU waiting for data from the client. When both appear together, a typical example is an idle COPY FROM STDIN, but it could also be TCP blocking or being stuck between BIND / EXECUTE. So it’s hard to say if it’s a bug without knowing what the connection is actually doing.

Some might argue that waiting for client I/O should be considered “idle” from a CPU perspective. But state tracks the execution state of the statement itself, not whether the CPU is busy. state = 'active' means the PostgreSQL backend considers “this statement is not yet finished.” Resources like row locks, buffer pins, snapshots, and file handles are considered “in use.” This doesn’t mean it’s running on the CPU. When the process is running on the CPU in a loop waiting for client data, the wait event is ClientRead. When it yields the CPU and “waits” in the background, the wait event is NULL.

But back to the problem itself, there are other solutions. For example, in Pigsty, when accessing PostgreSQL through HAProxy, we set a connection timeout at the LB level for the primary service, defaulting to 24 hours. More stringent environments would have a shorter timeout, like 1 hour. This means any connection lasting over an hour would be terminated. Of course, this also needs to be configured with a corresponding max lifetime in the application-side connection pool, to proactively close connections rather than having them be cut off. For offline, read-only services, this parameter can be omitted to allow for ultra-long queries that might run for two or three days. This provides a safety net for these active-but-waiting-on-I/O situations.

But I also doubt whether Azure PostgreSQL offers this kind of control.

On Default Parameters

PostgreSQL’s default parameters are quite conservative. For example, it defaults to using 128 MB of memory (the minimum can be set to 128 KB!). On the bright side, this allows its default configuration to run in almost any environment. On the downside, I’ve actually seen a case of a production system with 1TB of physical memory running with the 128 MB default… (thanks to double buffering, it actually ran for a long time).

But overall, I think conservative defaults aren’t a bad thing. This issue can be solved in a more flexible, dynamic configuration process. RDS and Pigsty both provide pretty good initial parameter heuristic config rules, which fully address this problem. But this feature could indeed be added to the PG command-line tools, for example, having initdb automatically detect CPU/memory count, disk size, and storage type and set optimized parameter values accordingly.

Self-hosted PostgreSQL?

The challenges OpenAI raised are not really from PostgreSQL itself, but from the additional limitations of managed cloud services. One solution is to use the IaaS layer and self-host a PostgreSQL cluster on instances with local NVMe SSD storage to bypass these restrictions.

In fact, my project Pigsty built for ourselves to solve PostgreSQL challenges at a similar scale. It scales well, having supported Tantan’s 25K vCPU PostgreSQL cluster and 2.5M QPS. It includes solutions for all the problems mentioned above, and even for many that OpenAI hasn’t encountered yet. And in a self-hosting manner, open-source, free, and ready to use out of the box.

If OpenAI is interested, I’d certainly be happy to provide some help. But I think when you’re in a phase of hyper-growth, fiddling with database infra is probably not a high-priority item. Fortunately, they still have excellent PostgreSQL DBAs who can continue to forge these paths.

References

[1] HackerNews OpenAI: Scaling Postgres to the Next Level: https://news.ycombinator.com/item?id=44071418#44072781

[2] PostgreSQL is eating the database world: https://pigsty.io/blog/pg/pg-eat-db-world/

[3] Chinese: Scaling Postgres to the Next Level at OpenAI https://pigsty.cc/db/openai-pg/

[4] The part of PostgreSQL we hate the most: https://www.cs.cmu.edu/~pavlo/blog/2023/04/the-part-of-postgresql-we-hate-the-most.html

[5] PGConf.Dev 2025: https://2025.pgconf.dev/schedule.html

[6] Schedule: Scaling Postgres to the next level at OpenAI: https://www.pgevents.ca/events/pgconfdev2025/schedule/session/433-scaling-postgres-to-the-next-level-at-openai/

[7] Bohan Zhang: https://www.linkedin.com/in/bohan-zhang-52b17714b

[8] Ruohang Feng / Vonng: https://github.com/Vonng/

[9] Pigsty: https://pigsty.io

[10] Instagram’s Sharding IDs: https://instagram-engineering.com/sharding-ids-at-instagram-1cf5a71e5a5c

[11] Reclaim hardware bonus: https://pigsty.io/blog/cloud/bonus/

[12] Distributed Databases Are a False Need: https://pigsty.io/blog/db/distributive-bullshit/

6.30 - How Many Shops Has etcd Torched?

Plenty. If you’re rolling your own Kubernetes, odds are you’ll crash because etcd ships with a 2 GB time bomb. Read more

6.31 - In the AI Era, Software Starts at the Database

Future software = Agent + Database. No middle tiers, just agents issuing CRUD. Database skills age well, and PostgreSQL is poised to be the agent-era default. Read more

6.32 - MySQL vs. PostgreSQL @ 2025

A 2025 reality check on where PostgreSQL stands relative to MySQL across features, performance, quality, and ecosystem. Read more

6.33 - Database Planet Collision: When PG Falls for DuckDB

If you ask me, we’re on the brink of a cosmic collision in database-land, and Postgres + DuckDB is the meteor we should all be watching. Read more

6.34 - Comparing Oracle and PostgreSQL Transaction Systems

The PG community has started punching up: Cybertec’s Laurenz Albe breaks down how Oracle’s transaction system stacks against PostgreSQL. Read more

6.35 - Database as Business Architecture

Databases are the core of business architecture, but what happens if we go further and let databases become the business architecture itself? Read more

6.36 - 7 Databases in 7 Weeks (2025)

Is PostgreSQL the king of boring databases? Here are seven databases worth studying in 2025: PostgreSQL, SQLite, DuckDB, ClickHouse, FoundationDB, TigerBeetle, and CockroachDB—each deserving a week of deep exploration. Read more

6.37 - Solving Poker 24 with a Single SQL Query

An interesting but tricky puzzle: solving the 24 game with SQL. The PostgreSQL solution. Read more

6.38 - Self-Hosting Supabase on PostgreSQL

Supabase is an open-source PostgreSQL-based BaaS. It provides authentication, Realtime, Edge Functions, object storage, and REST APIs generated from the database schema; enabling pg_graphql on demand also provides GraphQL APIs.

Self-hosting can make sense when you need control over data and infrastructure, isolated or compliant operation, or freedom to choose PostgreSQL versions and extensions. The official Supabase self-hosting guide recommends Docker and states that self-hosters are responsible for servers, security hardening, database maintenance, high availability, backups, and monitoring. Pigsty is an independent community integration and is not currently listed among the community projects on that page.

Pigsty v4.5’s supabase template moves stateful components under Pigsty management: the PGSQL module manages PostgreSQL, while the MINIO module’s current Silo implementation provides object storage. Auth, Storage, Realtime, Studio, and the other stateless services run through Docker Compose. The template supports PostgreSQL 15-18 (18 by default) and can use the 576 PostgreSQL extensions in the current Pigsty catalog.

This article was originally published on 25 November 2024 and revalidated against the current Pigsty v4.5 source on 14 August 2026. The continuously maintained Enterprise Self-Hosted Supabase reference is the single source of truth for deployment parameters and procedures.

Quick Start

The complete Supabase stack needs at least 2 CPU cores and 4 GB of RAM; 4 cores and 8 GB or more are recommended. Prepare a supported Linux system, download the current public stable Pigsty release, generate the inventory, and change every domain, password, and key before deployment:

curl -fsSL https://repo.pigsty.io/get | bash; cd ~/pigsty
./configure -c supabase
vi pigsty.yml
./deploy.yml
./docker.yml -l supabase
./app.yml -l supabase

deploy.yml is the standard single-node deployment entry point. The -l supabase limits the Docker and application playbooks to the intended host group. After installation, Supabase Studio is available at http://<node-ip>:8000. The template’s supabase / pigsty credentials are demonstration defaults and must never remain in production.

Before running the deployment, verify that:

• JWT_SECRET, ANON_KEY, SERVICE_ROLE_KEY, Dashboard, Logflare, Realtime, and S3 credentials were regenerated;
• API_EXTERNAL_URL retains the /auth/v1 suffix, while SITE_URL and SUPABASE_PUBLIC_URL use the site root URL;
• PostgreSQL role passwords match POSTGRES_PASSWORD, and object-storage users match the S3 configuration;
• public or OAuth deployments use a real domain and HTTPS;
• pig pb info confirms a usable backup and a restore drill has succeeded.

See Security Hardening for exact variables, length constraints, source paths, and reload commands.

Architecture and Boundaries

The Pigsty template does not start the upstream Compose db or supavisor containers. Stateless services connect directly to the PostgreSQL service managed by Pigsty. The single-node template uses service port 5436, which always routes to the current primary.

Logflare / Analytics uses a dedicated _supabase database and its _analytics schema. Studio Query Performance reads pg_stat_statements compatibility objects in the extensions schema, while the actual extension remains in the monitor schema for Pigsty monitoring compatibility.

The default single-node template is suitable for evaluation, small workloads, and initial deployment, but it is not an HA topology. With only one server, use external S3 for both Supabase Storage and PostgreSQL backups to reduce whole-node failure risk. Actual RPO depends on recoverable backups, WAL archiving, and object-storage availability; it is not a fixed amount of data implied by the topology.

A production HA design normally needs:

• at least three ETCD nodes for the DCS;
• multi-node PostgreSQL with an explicitly selected synchronization policy;
• multi-node Silo or an independent highly available S3 service;
• multiple stateless Supabase replicas plus DNS, VIP, or HAProxy access;
• periodic restore, failover, and certificate-renewal drills.

The default norm preset targets RTO below 45 seconds, and asynchronous replication does not promise RPO=0. A target below 30 seconds while preserving acknowledged transactions requires the fast RTO preset plus the crit.yml strict-synchronous policy and must be demonstrated by failure tests. See High Availability and Deployment Planning.

Domains, Object Storage, and Mail

For production, configure the domain, reverse proxy, and certificate under infra_portal.supa, and set the three external URLs under apps.supabase.conf. After obtaining a certificate with make cert, update only the Supabase application configuration and containers with:

./app.yml -l supabase -t app_config,app_launch

Supabase Storage can use Silo, cloud S3, or another S3-compatible service. Switching the PostgreSQL pgBackRest repository is a stateful change: check existing backups, run --check, and only after explicit authorization run ./pgsql.yml -t pg_backup -l <cluster>. Immediately create and verify a full backup in the new repository. Existing backups are not migrated automatically; see Switching Repositories.

Auth email requires a production SMTP provider. Put only the hostname in SMTP_HOST and the port in SMTP_PORT, then apply the change with the target-limited app.yml command.

The complete configuration example, Alibaba Cloud OSS fields, and SMTP parameters are in Enterprise Self-Hosted Supabase.

Why Pigsty

Supabase currently promises more than 50 preconfigured extensions and changes the exact list as the platform evolves. Pigsty v4.5 catalogs 576 extensions and packages Supabase dependencies including pg_graphql, pg_jsonschema, wrappers, index_advisor, pg_net, supabase_vault, pgjwt, pgsodium, supautils, and plan_filter.

Self-hosting does not mean “zero operations.” It returns control over versions, extensions, data, backups, and availability policy to the operator. It also makes that operator responsible for server maintenance, security updates, secret management, capacity planning, backup validation, and failure drills. Before starting, read the Supabase reference, security guide, and backup documentation.

6.39 - Modern Hardware for Future Databases

Author: Alex Miller (Snowflake, Apple, Google) | Original: Modern Hardware for Future Databases

A survey on how hardware developments affect database design, covering key advancements in networking, storage, and computing. Read more

6.40 - Can MySQL Still Catch Up with PostgreSQL?

Author: Peter Zaitsev (Percona Founder) | Original: Can MySQL Catch Up with PostgreSQL?

Percona founder Peter Zaitsev discusses whether MySQL can still keep up with PostgreSQL. His views largely represent the MySQL community’s perspective. Read more

6.41 - Open-Source "Tyrant" Linus's Purge

The Linux community is essentially imperial — and Linus himself is the earliest and most successful technical dictator. People are used to Linus’s generosity but forget this point. Read more

6.42 - Optimize Bio Cores First, CPU Cores Second

Programmers are expensive, scarce biological computing cores, the anchor point of software costs — please prioritize optimizing biological cores before optimizing CPU cores. Read more

6.43 - MongoDB Has No Future: Good Marketing Can't Save a Rotten Mango

MongoDB has a terrible track record on integrity, lackluster products and technology, gets beaten by PG in correctness, performance, and functionality, with collapsing developer reputation, declining popularity, stock price halving, and expanding losses. Provocative marketing against PG can’t save it with “good marketing.” Read more

6.44 - MongoDB: Now Powered by PostgreSQL?

MongoDB 3.2’s analytics subsystem turned out to be an embedded PostgreSQL database? A whistleblowing story from MongoDB’s partner about betrayal and disillusionment. Read more

6.45 - Switzerland Mandates Open-Source for Government Software

Switzerland’s government leads the way with open source legislation, showing IT developing countries how to ensure software sovereignty and control. True autonomy and control stem from “open source communities,” not some “nationalist” style “domestic software.” Read more

6.46 - MySQL is dead, Long live PostgreSQL!

This July, MySQL 9.0 was finally released—a full eight years after its last major version, 8.0 (@2016-09). Read more

6.47 - CVE-2024-6387 SSH Vulnerability Fix

This vulnerability affects EL9, Ubuntu 22.04, Debian 12. Users should promptly update OpenSSH to fix this vulnerability. Read more

6.48 - Can Oracle Still Save MySQL?

Author: Peter Zaitsev (Percona Founder) | Original: Can Oracle Save MySQL?

Percona founder Peter Zaitsev publicly expressed disappointment with MySQL and Oracle, criticizing the declining performance with newer versions. Read more

6.49 - Oracle Finally Killed MySQL

Author: Peter Zaitsev (Percona Founder) | Original: Is Oracle Finally Killing MySQL?

Peter Zaitsev, founder of Percona, criticizes how Oracle’s actions and inactions have killed MySQL. About 15 years after acquiring Sun and MySQL. Read more

6.50 - MySQL Performance Declining: Where is Sakila Going?

Author: Marco Tusa (Percona) | Original: Sakila, Where Are You Going?

Higher MySQL versions mean worse performance? Percona monitoring shows slow migration from 5.7 to 8.x. PostgreSQL is pulling ahead. Read more

6.51 - Can Chinese Domestic Databases Really Compete?

Friends often ask me, can Chinese domestic databases really compete? To be honest, it’s a question that offends people. So let’s try speaking with data - I hope the charts provided in this article can help readers understand the database ecosystem landscape and establish more accurate proportional awareness. Read more

6.52 - The $20 Brother PolarDB: What Should Databases Actually Cost?

Today we discuss the fair pricing of commercial databases, open-source databases, cloud databases, and domestic Chinese databases. Read more

6.53 - Redis Going Non-Open-Source is a Disgrace to "Open-Source" and Public Cloud

Redis “going non-open source” is not a disgrace to Redis, but a disgrace to “open source/OSI” and even more so to public cloud. What truly matters has always been software freedom, while open source is just one means to achieve software freedom. Read more

6.54 - How Can MySQL's Correctness Be This Garbage?

MySQL’s transaction ACID has flaws and doesn’t match documentation promises. This may lead to serious correctness issues - use with caution. Read more

6.55 - Database in K8S: Pros & Cons

Whether databases should be housed in Kubernetes/Docker remains highly controversial. While Kubernetes (k8s) excels in managing stateless applications, it has fundamental drawbacks with stateful services, especially databases like PostgreSQL and MySQL.

In the previous article, “Databases in Docker: Good or Bad,” we discussed the pros and cons of containerizing databases. Today, let’s delve into the trade-offs in orchestrating databases in K8S and explore why it’s not a wise decision.

Summary

Kubernetes (k8s) is an exceptional container orchestration tool aimed at helping developers better manage a vast array of complex stateless applications. Despite its offerings like StatefulSet, PV, PVC, and LocalhostPV for supporting stateful services (i.e., databases), these features are still insufficient for running production-level databases that demand higher reliability.

Databases are more like “pets” than “cattle” and require careful nurturing. Treating databases as “cattle” in K8S essentially turns external disk/file system/storage services into new “database pets.” Running databases on EBS/network storage presents significant disadvantages in reliability and performance. However, using high-performance local NVMe disks will make the database bound to nodes and non-schedulable, negating the primary purpose of putting them in K8S.

Placing databases in K8S results in a “lose-lose” situation - K8S loses its simplicity in statelessness, lacking the flexibility to quickly relocate, schedule, destroy, and rebuild like purely stateless use. On the other hand, databases suffer several crucial attributes: reliability, security, performance, and complexity costs, in exchange for limited “elasticity” and utilization - something virtual machines can also achieve. For users outside public cloud vendors, the disadvantages far outweigh the benefits.

The “cloud-native frenzy,” exemplified by K8S, has become a distorted phenomenon: adopting k8s for the sake of k8s. Engineers add extra complexity to increase their irreplaceability, while managers fear being left behind by the industry and getting caught up in deployment races. Using tanks for tasks that could be done with bicycles, to gain experience or prove oneself, without considering if the problem needs such “dragon-slaying” techniques - this kind of architectural juggling will eventually lead to adverse outcomes.

Until the reliability and performance of the network storage surpass local storage, placing databases in K8S is an unwise choice. There are other ways to seal the complexity of database management, such as RDS and open-source RDS solutions like Pigsty, which are based on bare Metal or bare OS. Users should make wise decisions based on their situations and needs, carefully weighing the pros and cons.

The Status Quo

K8S excels in orchestrating stateless application services but was initially limited to stateful services. Despite not being the intended purpose of K8S and Docker, the community’s zeal for expansion has been unstoppable. Evangelists depict K8S as the next-generation cloud operating system, asserting that databases will inevitably become regular applications within Kubernetes. Various abstractions have emerged to support stateful services: StatefulSet, PV, PVC, and LocalhostPV.

Countless cloud-native enthusiasts have attempted to migrate existing databases into K8S, resulting in a proliferation of CRDs and Operators for databases. Taking PostgreSQL as an example, there are already more than ten different K8S deployment solutions available: PGO, StackGres, CloudNativePG, PostgresOperator, PerconaOperator, CYBERTEC-pg-operator, TemboOperator, Kubegres, KubeDB, KubeBlocks, and so on. The CNCF landscape rapidly expands, turning into a playground of complexity.

However, complexity is a cost. With “cost reduction” becoming mainstream, voices of reflection have begun to emerge. Could-Exit Pioneers like DHH, who deeply utilized K8S in public clouds, abandoned it due to its excessive complexity during the transition to self-hosted open-source solutions, relying only on Docker and a Ruby tool named Kamal as alternatives. Many began to question whether stateful services like databases suit Kubernetes.

K8S itself, in its effort to support stateful applications, has become increasingly complex, straying from its original intention as a container orchestration platform. Tim Hockin, a co-founder of Kubernetes, also voiced his rare concerns at this year’s KubeCon in “K8s is Cannibalizing Itself!”: “Kubernetes has become too complex; it needs to learn restraint, or it will stop innovating and lose its base.”

Lose-Lose Situation

In the cloud-native realm, the analogy of “pets” versus “cattle” is often used for illustrating stateful services. “Pets,” like databases, need careful and individual care, while “cattle” represent disposable, stateless applications (Disposability).

Cloud Native Applications 12 Factors: Disposability

One of the leading architectural goals of K8S is to treat what can be treated as cattle as cattle. The attempt to “separate storage from computation” in databases follows this strategy: splitting stateful database services into state storage outside K8S and pure computation inside K8S. The state is stored on the EBS/cloud disk/distributed storage service, allowing the “stateless” database part to be freely created, destroyed, and scheduled in K8S.

Unfortunately, databases, especially OLTP databases, heavily depend on disk hardware, and network storage’s reliability and performance still lag behind local disks by orders of magnitude. Thus, K8S offers the LocalhostPV option, allowing containers to use data volumes directly lies on the host operating system, utilizing high-performance/high-reliability local NVMe disk storage.

However, this presents a dilemma: should one use subpar cloud disks and tolerate poor database reliability/performance for K8S’s scheduling and orchestration capabilities? Or use high-performance local disks tied to host nodes, virtually losing all flexible scheduling abilities? The former is like stuffing an anchor into K8S’s small boat, slowing overall speed and agility; the latter is like anchoring and pinning the ship to a specific point.

Running a stateless K8S cluster is simple and reliable, as is running a stateful database on a physical machine’s bare operating system. Mixing the two, however, results in a lose-lose situation: K8S loses its stateless flexibility and casual scheduling abilities, while the database sacrifices core attributes like reliability, security, efficiency, and simplicity in exchange for elasticity, resource utilization, and Day1 delivery speed that are not fundamentally important to databases.

A vivid example of the former is the performance optimization of PostgreSQL@K8S, which KubeBlocks contributed. K8S experts employed various advanced methods to solve performance issues that did not exist on bare metal/bare OS at all. A fresh case of the latter is Didi’s K8S architecture juggling disaster; if it weren’t for putting the stateful MySQL in K8S, would rebuilding a stateless K8S cluster and redeploying applications take 12 hours to recover?

Pros and Cons

For serious technology decisions, the most crucial aspect is weighing the pros and cons. Here, in the order of “quality, security, performance, cost,” let’s discuss the technical trade-offs of placing databases in K8S versus classic bare metal/VM deployments. I don’t want to write a comprehensive paper that covers everything. Instead, I’ll throw some specific questions for consideration and discussion.

Quality

K8S, compared to physical deployments, introduces additional failure points and architectural complexity, increasing the blast radius and significantly prolonging the average recovery time of failures. In “Is it a Good Idea to Put Databases into Docker?”, we provided an argument about reliability, which can also apply to Kubernetes — K8S and Docker introduce additional and unnecessary dependencies and failure points to databases, lacking community failure knowledge accumulation and reliability track record (MTTR/MTBF).

In the cloud vendor classification system, K8S belongs to PaaS, while RDS belongs to a more fundamental layer, IaaS. Database services have higher reliability requirements than K8S; for instance, many companies’ cloud management platforms rely on an additional CMDB database. Where should this database be placed? You shouldn’t let K8S manage things it depends on, nor should you add unnecessary extra dependencies. The Alibaba Cloud global epic failure and Didi’s K8S architecture juggling disaster have taught us this lesson. Moreover, maintaining a separate database system inside K8S when there’s already one outside is even more unjustifiable.

Security

The database in a multi-tenant environment introduces additional attack surfaces, bringing higher risks and more complex audit compliance challenges. Does K8S make your database more secure? Maybe the complexity of K8S architecture juggling will deter script kiddies unfamiliar with K8S, but for real attackers, more components and dependencies often mean a broader attack surface.

In “BrokenSesame Alibaba Cloud PostgreSQL Vulnerability Technical Details”, security personnel escaped to the K8S host node using their own PostgreSQL container and accessed the K8S API and other tenants’ containers and data. This is clearly a K8S-specific issue — the risk is real, such attacks have occurred, and even Alibaba Cloud, a local cloud industry leader, has been compromised.

Source: The Attacker Perspective - Insights From Hacking Alibaba Cloud

Performance

As stated in “Is it a Good Idea to Put Databases into Docker?”, whether it’s additional network overhead, Ingress bottlenecks, or underperforming cloud disks, all negatively impact database performance. For example, as revealed in “PostgreSQL@K8s Performance Optimization” — you need a considerable level of technical prowess to make database performance in K8S barely match that on bare metal.

Note: Latency is measured in ms, not µs — a significant performance difference.

Another misconception about efficiency is resource utilization. Unlike offline analytical businesses, critical online OLTP databases should not aim to increase resource utilization but rather deliberately lower it to enhance system reliability and user experience. If there are many fragmented businesses, resource utilization can be improved through PDB/shared database clusters. K8S’s advocated elasticity efficiency is not unique to it — KVM/EC2 can also effectively address this issue.

In terms of cost, K8S and various Operators provide a decent abstraction, encapsulating some of the complexity of database management, which is attractive for teams without DBAs. However, the complexity reduced by using it to manage databases pales in comparison to the complexity introduced by using K8S itself. For instance, random IP address drifts and automatic Pod restarts may not be a big issue for stateless applications, but for databases, they are intolerable — many companies have had to attempt to modify kubelet to avoid this behavior, thereby introducing more complexity and maintenance costs.

As stated in “From Reducing Costs and Smiles to Reducing Costs and Efficiency” “Reducing Complexity Costs” section: Intellectual power is hard to accumulate spatially: when a database encounters problems, it needs database experts to solve them; when Kubernetes has problems, it needs K8S experts to look into them; however, when you put a database into Kubernetes, complexities combine, the state space explodes, but the intellectual bandwidth of individual database experts and K8S experts is hard to stack — you need a dual expert to solve the problem, and such experts are undoubtedly much rarer and more expensive than pure database experts. Such architectural juggling is enough to cause major setbacks for most teams, including top public clouds/big companies, in the event of a failure.

The Cloud-Native Frenzy

An interesting question arises: if K8S is unsuitable for stateful databases, why are so many companies, including big players, rushing to do this? The reasons are not technical.

Google open-sourced its K8S battleship, modeled after its internal Borg spaceship, and managers, fearing being left behind, rushed to adopt it, thinking using K8S would put them on par with Google. Ironically, Google doesn’t use K8S; it was more likely to disrupt AWS and mislead the industry. However, most companies don’t have the manpower like Google to operate such a battleship. More importantly, their problems might need a simple vessel. Running MySQL + PHP, PostgreSQL + Go/Python on bare metal has already taken many companies to IPO.

Under modern hardware conditions, the complexity of most applications throughout their lifecycle doesn’t justify using K8S. Yet, the “cloud-native” frenzy, epitomized by K8S, has become a distorted phenomenon: adopting k8s just for the sake of k8s. Some engineers are looking for “advanced” and “cool” technologies used by big companies to fulfill their personal goals like job hopping or promotions or to increase their job security by adding complexity, not considering if these “dragon-slaying” techniques are necessary for solving their problems.

The cloud-native landscape is filled with fancy projects. Every new development team wants to introduce something new: Helm today, Kubevela tomorrow. They talk big about bright futures and peak efficiency, but in reality, they create a mountain of architectural complexities and a playground for “YAML Boys” - tinkering with the latest tech, inventing concepts, earning experience and reputation at the expense of users who bear the complexity and maintenance costs.

CNCF Landscape: A sprawling ecosystem of complexity

The cloud-native movement’s philosophy is compelling - democratizing the elastic scheduling capabilities of public clouds for every user. K8S indeed excels in stateless applications. However, excessive enthusiasm has led K8S astray from its original intent and direction - simply doing well in orchestrating stateless applications, burdened by the ill-conceived support for stateful applications.

Making Wise Decisions

Years ago, when I first encountered K8S, I too was fervent —— It was at TanTan. We had over twenty thousand cores and hundreds of database clusters, and I was eager to try putting databases in Kubernetes and testing all the available Operators. However, after two to three years of extensive research and architectural design, I calmed down and abandoned this madness. Instead, I architected our database service based on bare metal/operating systems. For us, the benefits K8S brought to databases were negligible compared to the problems and hassles it introduced.

Should databases be put into K8S? It depends: for public cloud vendors who thrive on overselling resources, elasticity and utilization are crucial, which are directly linked to revenue and profit, While reliability and performance take a back seat - after all, an availability below three nines means compensating 25% monthly credit. But for most user, including ourselves, these trade-offs hold different: One-time Day1 Setup, elasticity, and resource utilization aren’t their primary concerns; reliability, performance, Day2 Operation costs, these core database attributes are what matter most.

We open-sourced our database service architecture — an out-of-the-box PostgreSQL distribution and a local-first RDS alternative: Pigsty. We didn’t choose the so-called “build once, run anywhere” approach of K8S and Docker. Instead, we adapted to different OS distros & major versions, and used Ansible to achieve a K8S CRD IaC-like API to seal management complexity. This was arduous, but it was the right thing to do - the world does not need another clumsy attempt at putting PostgreSQL into K8S. Still, it does need a production database service architecture that maximizes hardware performance and reliability.

Pigsty vs StackGres: Different approaches to PostgreSQL deployment

Perhaps one day, when the reliability and performance of distributed network storage surpass local storage and mainstream databases have some native support for storage-computation separation, things might change again — K8S might become suitable for databases. But for now, I believe putting serious production OLTP databases into K8S is immature and inappropriate. I hope readers will make wise choices on this matter.

Reference

Database in Docker: Is that a good idea?

《Kubernetes is Rotten!》

《Curse of Docker?》

《What can we learn from DiDi’s Epic k8s Failure》

《PostgreSQL@K8s Performance Optimization》

《Running Database on Kubernetes》

6.56 - Are Specialized Vector Databases Dead?

Vector storage and retrieval is a real need, but specialized vector databases are already dead. Small needs are solved by OpenAI directly, standard needs are captured by existing mature databases with vector extensions. The ecological niche left for specialized vector databases might support one company, but trying to build an industry around AI stories is impossible. Read more

6.57 - Are Databases Really Being Strangled?

Many “domestic databases” are just shoddy, inferior products that can’t be helped. Xinchuang domestic OS/databases are essentially IT pre-made meals in schools. Users hold their noses while migrating, developers pretend to work hard, and everyone plays along with leaders who neither understand nor care about technology. The infrastructure software industry isn’t being strangled by anyone - the real chokehold comes from the so-called “insiders.” Read more

6.58 - Which EL-Series OS Distribution Is Best?

RHEL-series OS distribution compatibility level: RHEL = Rocky ≈ Anolis > Alma > Oracle » Euler. Recommend using RockyLinux 8.8, or Anolis 8.8 for domestic requirements. Read more

6.59 - What Kind of Self-Reliance Do Infrastructure Software Need?

When we talk about self-reliance and control, what are we really talking about? Operational self-reliance vs. R&D self-reliance - what nations/users truly need is the former, not flashy “self-research”. Read more

6.60 - Back to Basics: Tech Reflection Chronicles

The cost-cutting imperative has triggered a reevaluation of all technologies, including databases. This series critiques hot DB technologies and poses fundamental questions about their trade-offs: Are cloud databases, distributed databases, microservices, and containerization real needs or false hype? Read more

6.61 - Database Demand Hierarchy Pyramid

Similar to Maslow’s hierarchy of needs, user demands for databases also have a progressive hierarchy: physiological needs, safety needs, belonging needs, esteem needs, cognitive needs, aesthetic needs, self-actualization needs, and transcendence needs. Read more

6.62 - Are Microservices a Bad Idea?

Author: DHH (David Heinemeier Hansson) | Original: Even Amazon can’t make sense of serverless or microservices

Even Amazon’s SOA paradigm team admits microservices and Serverless have problems. Prime Video team switched to monolith, saving 90% operational costs. Read more

6.63 - NewSQL: Distributive Nonsense

As hardware technology advances, the capacity and performance of standalone databases have reached unprecedented heights, which makes distributed (TP) databases appear utterly powerless, much like the “data middle platform” wearing the emperor’s new clothes. Read more

6.64 - Time to Say Goodbye to GPL

Author: Martin Kleppmann (DDIA Author) | Original: It’s time to say goodbye to the GPL

DDIA author Martin Kleppmann argues we should move away from GPL licenses. In the 2020s, the enemy of computing freedom is cloud software. Read more

6.65 - Is running postgres in docker a good idea?

Thou shalt not run a prod database inside a container Read more

6.66 - Understanding Time - Leap Years, Leap Seconds, Time and Time Zones

A proper understanding of time is very helpful for correctly handling time-related issues in work and life. For example, time representation and processing in computers, as well as time handling in databases and programming languages. Read more

6.67 - Understanding Character Encoding Principles

Without understanding the basic principles of character encoding, even simple string operations like comparison, sorting, and random access can easily lead you into pitfalls. This article attempts to clarify these issues through a comprehensive explanation. Read more

6.68 - Concurrency Anomalies Explained

Concurrent programs are hard to write correctly and even harder to write well. Many programmers simply throw these problems at the database… But even the most sophisticated databases won’t help if you don’t understand concurrency anomalies and isolation levels. Read more

6.69 - Blockchain and Distributed Databases

The technical essence, functionality, and evolution of blockchain is distributed databases. Specifically, it’s a Byzantine Fault Tolerant (resistant to malicious node attacks) distributed (leaderless replication) database. Read more

6.70 - Consistency: An Overloaded Term

The term “consistency” is heavily overloaded, representing different concepts in different contexts. For example, the C in ACID and the C in CAP actually refer to different concepts. Read more

6.71 - Why Study Database Principles

Those who only know how to code are just programmers; learn databases well, and you can at least make a living; but for excellent engineers, merely using databases is far from enough. Read more