Module: JUICE

• 1: Configuration
• 2: Parameters
• 3: Playbook
• 4: Administration
• 5: Monitoring
• 6: FAQ

flowchart LR
    subgraph Client["App/User"]
        app["POSIX Access"]
    end

    subgraph JUICE["JUICE"]
        jfs["JuiceFS Mount"]
    end

    subgraph PGSQL["PGSQL"]
        meta["Metadata DB"]
        blob["Data DB / jfs_blob (optional)"]
    end

    subgraph Object["Object Storage (optional)"]
        s3["Silo / S3"]
    end

    subgraph INFRA["INFRA (optional)"]
        vm["VictoriaMetrics"]
    end

    app --> jfs
    jfs --> meta
    jfs -.->|alternative data backend| blob
    jfs -.->|alternative data backend| s3
    jfs -->|/metrics| vm

    style JUICE fill:#5B9CD5,stroke:#4178a8,color:#fff
    style PGSQL fill:#3E668F,stroke:#2d4a66,color:#fff
    style Object fill:#FCDB72,stroke:#d4b85e,color:#333
    style INFRA fill:#999,stroke:#666,color:#fff

juice_instances:
  jfs:
    path: /fs
    meta: postgres://dbuser_meta:[email protected]:5432/meta
    data: --storage postgres --bucket 10.10.10.10:5432/meta --access-key dbuser_meta --secret-key DBUser.Meta
    port: 9567

./juice.yml -l <host>

1 - Configuration

Concepts and Implementation

JuiceFS consists of a metadata engine and data storage. The current version passes meta to juicefs as the metadata engine URL. Production deployments typically use PostgreSQL. Data storage is defined by data options passed to juicefs format.

JUICE module core commands:

# Format (only effective on first creation)
juicefs format --no-update <data> "<meta>" "<name>"

# Mount
juicefs mount <mount_opts> --cache-dir <juice_cache> --metrics 0.0.0.0:<port> <meta> <path>

Notes:

• --no-update ensures existing filesystems are not overwritten.
• data applies only to the initial format. It does not affect existing filesystems.
• mount applies only during mounting. Use it to pass cache and concurrency options.

Module Parameters

JUICE module has only two parameters:

• juice_cache: shared local cache directory for all instances, default /data/juice
• juice_instances: Instance-level dictionary. Each key is a filesystem name. An empty dictionary means that the module manages no instances.

Instance Configuration

Each entry in juice_instances represents a JuiceFS instance:

Example:

juice_instances:
  jfs:
    path: /fs
    meta: postgres://dbuser_meta:[email protected]:5432/meta
    data: --storage postgres --bucket 10.10.10.10:5432/meta --access-key dbuser_meta --secret-key DBUser.Meta
    port: 9567

Storage Backends

The module appends data to juicefs format. You can use any supported backend. Common examples follow.

PostgreSQL Data Backend

juice_instances:
  jfs:
    path: /fs
    meta: postgres://dbuser_meta:[email protected]:5432/meta
    data: --storage postgres --bucket 10.10.10.10:5432/meta --access-key dbuser_meta --secret-key DBUser.Meta

JuiceFS creates a jfs_blob table in the database selected by --bucket for file data. The PostgreSQL data backend and the meta metadata engine have separate roles. They can use one database or separate deployments. The database and a user with read/write privileges must already exist.

Silo / MinIO-Compatible Object Storage

juice_instances:
  jfs:
    path: /fs
    meta: postgres://dbuser_meta:[email protected]:5432/meta
    data: --storage minio --bucket https://sss.pigsty:9000/juice --access-key <s3_access_key> --secret-key <s3_secret_key>

S3-Compatible Storage

juice_instances:
  jfs:
    path: /fs
    meta: postgres://dbuser_meta:[email protected]:5432/meta
    data: --storage s3 --bucket https://s3.amazonaws.com/my-bucket --access-key AKIAXXXXXXXX --secret-key XXXXXXXXXX

Typical Configurations

Multi-Instance (Same Node)

juice_instances:
  pgfs:
    path: /pgfs
    meta: postgres://dbuser_meta:[email protected]:5432/meta
    data: --storage postgres --bucket 10.10.10.10:5432/meta --access-key dbuser_meta --secret-key DBUser.Meta
    port: 9567
  shared:
    path: /shared
    meta: postgres://dbuser_meta:[email protected]:5432/shared
    data: --storage minio --bucket https://sss.pigsty:9000/shared --access-key <s3_access_key> --secret-key <s3_secret_key>
    port: 9568
    owner: postgres
    group: postgres

Shared Mount Across Nodes

Mount the same JuiceFS on multiple nodes:

app:
  hosts:
    10.10.10.11: { juice_instances: { shared: { path: /shared, meta: "postgres://...", port: 9567 } } }
    10.10.10.12: { juice_instances: { shared: { path: /shared, meta: "postgres://...", port: 9567 } } }

Only one node needs to format the filesystem. Other nodes skip this operation because of --no-update.

Notes

• port is exposed on 0.0.0.0. Use firewall or security group to restrict access.
• A change to data does not update an existing filesystem. Migrate the data manually.
• meta and data can contain database or object-storage credentials. Restrict access to pigsty.yml. Use dedicated accounts with the minimum required privileges. Never keep example passwords in production.

2 - Parameters

The JUICE module has 2 parameters:

• juice_cache: shared cache directory
• juice_instances: instance definition dict

Parameter Overview

Level: C = cluster level, I = instance level.

Default Parameters

Defined in roles/juice/defaults/main.yml:

#-----------------------------------------------------------------
# JUICE
#-----------------------------------------------------------------
juice_cache: /data/juice
juice_instances: {}

juice_cache

Parameter: juice_cache, type: path, level: C

Shared local cache directory for all JuiceFS instances, default /data/juice. JuiceFS isolates caches by filesystem UUID under this directory.

juice_cache: /data/juice

juice_instances

Parameter: juice_instances, type: dict, level: I

Dictionary of instance definitions, usually set at the instance level. The default is an empty dictionary, which deploys no instances. Each key is a filesystem name. Each value is an instance configuration object.

juice_instances:
  jfs:
    path: /fs
    meta: postgres://u:p@h:5432/db
    data: --storage postgres --bucket ...
    port: 9567

Instance fields:

3 - Playbook

The JUICE module provides the juice.yml playbook to deploy and remove JuiceFS instances.

juice.yml

Task structure in juice.yml:

juice_id        : validate config, check port conflicts
juice_install   : install juicefs package
juice_cache     : create shared cache dir
juice_clean     : remove instance (state=absent)
juice_instance  : create instance (state=create)
  - juice_init  : format filesystem (--no-update)
  - juice_dir   : create mount dir
  - juice_config: render env file and systemd unit
  - juice_launch: start service and wait for metrics port
juice_register  : register to VictoriaMetrics targets

Scope

Examples:

./juice.yml -l 10.10.10.10                 # deploy all instances on the node
./juice.yml -l 10.10.10.10 -e fsname=jfs   # only deploy jfs instance

Common Tags

Config Updates

Render config only (no restart):

./juice.yml -l <host> -t juice_config

Update config and ensure service is online (without force restart):

./juice.yml -l <host> -t juice_config,juice_launch

If you need new mount options to take effect immediately, manually restart the instance service:

systemctl restart juicefs-<name>

Remove Instance

Removal flow:

1. Set instance state to absent
2. Run juice_clean

juice_instances:
  jfs:
    path: /fs
    meta: postgres://...
    state: absent

./juice.yml -l <host> -t juice_clean,juice_register
./juice.yml -l <host> -e fsname=jfs -t juice_clean,juice_register

Removal stops the service and performs a lazy unmount. It removes the systemd unit and environment files, then reloads systemd. The juice_register task then rewrites the node’s target file and removes stale scrape endpoints. If you run only juice_clean, monitoring targets remain unchanged. PostgreSQL metadata, PostgreSQL jfs_blob data tables, and object-storage data are not deleted.

Monitoring Registration

juice_register writes the target file on the infra node:

/infra/targets/juice/<hostname>.yml

To re-register manually:

./juice.yml -l <host> -t juice_register

4 - Administration

Common operations:

• Initialize Instance
• Reconfigure
• Remove Instance
• Add New Instance
• Shared Mount Across Nodes
• PITR Recovery
• Troubleshooting
• Performance Tuning

See FAQ for more.

Initialize Instance

./juice.yml -l <host>
./juice.yml -l <host> -e fsname=<name>

Initialization steps:

• Install juicefs package
• Create shared cache dir (default /data/juice)
• Run juicefs format --no-update (only effective on first creation)
• Create mount point and set permissions
• Render systemd unit and env files
• Start service and wait for metrics port
• Register to VictoriaMetrics (if infra node exists)

Reconfigure

After configuration changes, we recommend this command to update the configuration and make sure the service is running:

./juice.yml -l <host> -t juice_config,juice_launch

To generate the configuration without changing the service state:

./juice.yml -l <host> -t juice_config

Notes:

• juice_config,juice_launch sets the service state to started. It does not force a restart of a running instance.
• data only takes effect on the first format
• After changing mount options, manually restart the instance service (systemctl restart juicefs-<name>)

Remove Instance

1. Set instance state to absent
2. Run juice_clean

juice_instances:
  jfs:
    path: /fs
    meta: postgres://...
    state: absent

./juice.yml -l <host> -t juice_clean,juice_register
./juice.yml -l <host> -e fsname=jfs -t juice_clean,juice_register

Removal actions:

• Stop systemd service
• umount -l lazy unmount
• Remove unit and env files
• Reload systemd
• Rewrite this node’s VictoriaMetrics target file, removing instances with state=absent

PostgreSQL metadata, PostgreSQL jfs_blob data tables, and object-storage data are not deleted.

If you run only -t juice_clean, monitoring targets remain unchanged. Removed instances temporarily retain stale scrape endpoints. The commands above also run juice_register to update these targets.

Add New Instance

Add a new instance to the configuration. Give it a unique port:

juice_instances:
  newfs:
    path: /newfs
    meta: postgres://...
    data: --storage minio --bucket https://sss.pigsty:9000/newfs --access-key <s3_access_key> --secret-key <s3_secret_key>
    port: 9568

Deploy:

./juice.yml -l <host> -e fsname=newfs

Shared Mount Across Nodes

Configure the same meta and instance name on multiple nodes:

app:
  hosts:
    10.10.10.11: { juice_instances: { shared: { path: /shared, meta: "postgres://...", port: 9567 } } }
    10.10.10.12: { juice_instances: { shared: { path: /shared, meta: "postgres://...", port: 9567 } } }

Only one node needs to format the filesystem. Other nodes skip this operation because of --no-update.

PITR Recovery

Restore JuiceFS metadata and data to a mutually consistent state. Before any restore, stop every writer. Unmount or stop the corresponding JuiceFS service on every client. Identify the exact PostgreSQL cluster and target time. Confirm that a backup is available:

# Inspect Patroni members; inspect the target stanza's backup on a database node
pig pt list <cluster>
pig pb info -s <stanza>

# On the target database node, run the restore command as postgres
sudo -iu postgres pg-pitr -s <stanza> -t "2026-08-14 10:30:00+08"

If metadata and the --storage postgres jfs_blob table share the restored PostgreSQL database, PITR can restore both to the same time. If they reside in different databases or clusters, design a coordinated recovery point for both.

If file data resides in Silo/S3, PostgreSQL PITR rolls back metadata only. It does not restore objects. Newer objects can remain. Older objects can be unavailable if they were deleted or removed by garbage collection.

Recovery depends on object versioning, trash, and lifecycle policies. Do not run garbage collection until validation is complete.

Troubleshooting

Mount Fails

systemctl status juicefs-jfs
journalctl -u juicefs-jfs -f
mountpoint /fs

Metadata Connection Issues

psql "postgres://dbuser_meta:[email protected]:5432/meta" -c "SELECT 1"

Metrics Port Check

ss -tlnp | grep 9567
curl http://localhost:9567/metrics

Performance Tuning

Pass juicefs mount options via mount:

juice_instances:
  jfs:
    path: /fs
    meta: postgres://...
    mount: --cache-size 102400 --prefetch 3 --max-uploads 50

Key metrics to watch:

• juicefs_blockcache_hits/juicefs_blockcache_miss: cache hit ratio
• juicefs_object_request_durations_histogram_seconds: object storage latency
• juicefs_transaction_durations_histogram_seconds: metadata transaction latency

5 - Monitoring

JuiceFS instances expose Prometheus metrics via juicefs mount --metrics. In JUICE, the metrics endpoint listens on 0.0.0.0:<port>. The default port is 9567.

Monitoring Architecture

JuiceFS Mount (metrics: 0.0.0.0:<port>)
    ↓
VictoriaMetrics (scrape)
    ↓
Grafana Dashboard

If INFRA is deployed, juice_register writes scrape targets to:

/infra/targets/juice/<hostname>.yml

The current source includes the Node JuiceFS dashboard (UID: node-juice). It shows capacity, cache, object-storage, metadata-transaction, and client-resource metrics for JuiceFS mounts on a node.

Target File Example

- labels: { ip: 10.10.10.10, ins: "node-jfs", cls: "jfs" }
  targets: [ 10.10.10.10:9567 ]

To register manually:

./juice.yml -l <host> -t juice_register

Key Metrics

Object Storage

Cache

Metadata Transactions

Common PromQL

Cache hit ratio:

rate(juicefs_blockcache_hits[5m]) /
(rate(juicefs_blockcache_hits[5m]) + rate(juicefs_blockcache_miss[5m]))

Object storage P99 latency:

histogram_quantile(0.99, rate(juicefs_object_request_durations_histogram_seconds_bucket[5m]))

6 - FAQ

Port Conflicts?

Multiple instances on the same node must use different port values. Example:

juice_instances:
  fs1:
    path: /fs1
    meta: postgres://...
    port: 9567
  fs2:
    path: /fs2
    meta: postgres://...
    port: 9568

Why does changing data not take effect?

Only juicefs format --no-update uses data. This setting does not change the filesystem after creation. To change the backend, migrate the data and reformat manually.

How to add a new instance?

1. Add instance definition in config
2. Run:

./juice.yml -l <host> -e fsname=<name>

How to remove an instance?

1. Set instance state to absent
2. Run:

./juice.yml -l <host> -t juice_clean,juice_register

Removal does not delete PostgreSQL metadata or object storage data. juice_register updates the target file. If you run only juice_clean, a stale monitoring scrape endpoint remains.

Where is file data stored?

Depends on data:

• --storage postgres: JuiceFS creates a jfs_blob table in the PostgreSQL database selected by --bucket
• --storage minio/s3: data is stored in a Silo/S3-compatible object-storage bucket

meta specifies the metadata engine. Pigsty production deployments usually use PostgreSQL for this role.

Multi-node mount notes?

• Use the same meta and instance name on all nodes
• Only one node needs to format the filesystem. Other nodes skip this operation.
• Ensure port does not conflict on each node

Monitoring target not generated?

juice_register only writes /infra/targets/juice/ when infra group exists. You can run manually:

./juice.yml -l <host> -t juice_register

How to change mount options?

After you update mount in the instance, refresh the configuration. Then restart the service manually:

./juice.yml -l <host> -t juice_config,juice_launch
systemctl restart juicefs-<name>