Binary content storage

The storage-aware VFS driver reads and writes the storage identifier and content hash stored with each content row. New installations configure this driver automatically.

db.vfs.driver=org.opencms.db.mysql.CmsStorageVfsDriver
db.vfs.pool=opencms:default
db.vfs.sqlmanager=org.opencms.db.mysql.CmsSqlManager

Use the matching package for MySQL, PostgreSQL, Oracle, MSSQL, DB2, AS/400 or HSQLDB. Only the VFS driver uses the storage variant; the project, user, subscription and history drivers keep their normal SQL manager classes.

Existing installations may continue to use the classic CmsVfsDriver. In that mode no storage schema migration is required, but storage-aware content handling is not available.

The storage policy in WEB-INF/config/opencms-vfs.xml, below <vfs>/<resources>, decides which resource contents stay in the VFS content tables and which are written deduplicated to a storage backend. New installations from OpenCms 22 configure CmsDefaultStoragePolicy without parameters.

The following configuration makes its defaults explicit:

<storage-policy class="org.opencms.db.storage.policy.CmsDefaultStoragePolicy">
  <param name="threshold">0</param>
  <param name="resourceTypeIds">2,3</param>
</storage-policy>

threshold is specified in bytes. Content is stored deduplicated only when its size is strictly greater than the threshold; 0 therefore stores every non-empty matching resource deduplicated. The default resource type IDs 2,3 cover binary files and images. Other resource types and empty files remain in the regular VFS content columns. A higher threshold can keep small files in those columns.

A custom policy can implement org.opencms.db.storage.policy.I_CmsStoragePolicy.

To keep new writes in the regular VFS content columns, configure:

<storage-policy class="org.opencms.db.storage.policy.CmsNoExternalStoragePolicy" />

If the <storage-policy> element is absent, the storage-aware VFS driver uses CmsNoExternalStoragePolicy for backward compatibility.

storage.active names the backend used for new external writes. storage.legacy is an optional comma-separated list of older backends which remain readable during migration.

The database backend is built in and always uses the reserved ID db. It does not have a storage.backend.db.* configuration block.

storage.active=db

This is also the implicit default when storage.active is not set. Together with CmsDefaultStoragePolicy, qualifying blobs are stored deduplicated in CMS_STORAGE.

storage.active=s3main
storage.legacy=db

storage.backend.s3main.type=s3
storage.backend.s3main.endpoint=http://localhost:9000
storage.backend.s3main.bucket=opencms-data
storage.backend.s3main.accessKey=YOUR_ACCESS_KEY
storage.backend.s3main.secretKey=YOUR_SECRET_KEY
storage.backend.s3main.pathStyle=true
storage.backend.s3main.region=aws-global

The backend uses org.opencms.db.storage.s3.CmsGenericS3Client; the client implementation is not selected through opencms.properties. The access and secret keys are passed through the configured OpenCms credentials resolver. The default resolver uses the configured values unchanged, while a custom resolver can resolve placeholders or external secret references at startup. Use pathStyle=true for services which require path-style bucket access.

The S3 client supports configuration for connection acquisition, connection establishment, socket, API-call and retry limits. The most relevant settings are:

storage.backend.s3main.connectionTimeout=5000
storage.backend.s3main.connectionAcquisitionTimeout=10000
storage.backend.s3main.maxConnections=50
storage.backend.s3main.socketTimeout=30000
storage.backend.s3main.apiCallAttemptTimeout=30000
storage.backend.s3main.apiCallTimeout=60000
storage.backend.s3main.maxRetries=2

maxConnections defaults to 50 and connectionAcquisitionTimeout defaults to 10000 milliseconds. Size the pool for the expected concurrent storage access on one OpenCms node. If the pool is exhausted, the acquisition timeout limits how long another request waits for a connection.

storage.active=fs1
storage.legacy=db

storage.backend.fs1.type=fs
storage.backend.fs1.path=/var/opencms/storage/fs1

The path must be durable and writable by OpenCms. In a cluster it must be mounted by all nodes with semantics that preserve atomic moves and consistent visibility.

Content can remain readable after several backend changes by listing all previous backend IDs:

storage.active=s3main
storage.legacy=db,fs1

Legacy backends are not used for new writes. A rewrite or migration reads the old blob and writes the result to the active backend.

When switching a default installation from database storage to file system or S3 storage, keep db in storage.legacy while any content rows still reference it.

The OpenCms updater can add the CMS_STORAGE table, storage metadata columns and required indexes. Configure the updater in WEB-INF/config/opencms.properties:

setup.storage.schema.update=auto

auto updates a missing or incomplete schema. true enables the update explicitly, while false skips it and is the default when the property is absent. The schema update is idempotent and skips objects which already exist. It does not change db.vfs.driver, modify the remaining OpenCms configuration or move existing content.

For large installations or DBA-controlled deployments, apply the database-specific SQL while OpenCms is stopped. Back up the database first and verify which schema objects already exist.

opencms-core/src-setup/org/opencms/setup/db/update21to22/<db>/README.md

Creating indexes on large CMS_CONTENTS and CMS_OFFLINE_CONTENTS tables can require substantial time, I/O and locking. Database statistics updates remain the responsibility of the DBA.

Switch from CmsVfsDriver to the corresponding CmsStorageVfsDriver only after the storage-aware schema is present. Existing rows without a storage identifier and hash remain readable from their normal FILE_CONTENT columns.

1. Back up the OpenCms database and every involved storage backend.
2. Back up WEB-INF/config/opencms.properties and WEB-INF/config/opencms-vfs.xml.
3. Stop all writing OpenCms nodes.
4. Verify that the storage-aware schema is present.
5. Configure the final active and legacy backends.
6. Configure the final storage policy.
7. Keep every source backend readable through storage.legacy.

Read-only frontend nodes may remain online only in a controlled cluster where they cannot write and can read both the source and target backends.

The migration tool is a standalone JAR that is part of the OpenCms distribution. In the extracted ZIP file, it can be found under /tools/storage/.

Alternatively, you can build the standalone tool from the OpenCms source tree:

./gradlew storageMigrationJar

Without --execute, the tool performs a dry-run. Start with:

java -jar opencms-storage-migration.jar --webinf /path/to/opencms/WEB-INF --dry-run

Execute the migration and verify the result:

java -jar opencms-storage-migration.jar --webinf /path/to/opencms/WEB-INF --execute --verify

Use --properties or --vfs-config to select explicit configuration files, --tables offline,contents to restrict the scan, --batch-size to control fetch and commit size and --driver-jar or --driver-dir to supply JDBC drivers.

The tool evaluates every candidate against the configured storage policy. It migrates local content that should be deduplicated, content stored in a non-active backend and external content that should return to local table storage.

For local-to-deduplicated migration, the tool writes the blob to the target backend, empties FILE_CONTENT and stores the new storage ID and hash reference. For deduplicated-to-deduplicated migration, it writes the target and updates the rows but intentionally leaves the source blob in place.

To roll back to local table storage, configure CmsNoExternalStoragePolicy and run the migration tool with --execute --verify.

The candidate scan benefits from indexes whose leading columns are (STORAGE, HASH) on both content tables. Run a dry-run before the maintenance window and inspect database load and execution plans on large installations.

The maintenance tool is a standalone JAR that is part of the OpenCms distribution. In the extracted ZIP file, it can be found under /tools/storage/.

Alternatively, you can build the maintenance tool from the sources with:

./gradlew storageMaintenanceJar

Run all read-only checks:

java -jar opencms-storage-maintenance.jar --webinf /path/to/opencms/WEB-INF --mode all

• validate-backends checks read/write availability.
• verify-references reports referenced blobs that cannot be loaded.
• scan-orphans lists unreferenced blobs.
• all runs all read-only checks.

The tool enumerates file system and S3 backends and scans the database backend through CMS_STORAGE.

Deletion is a separate mode and remains a dry-run unless --execute is present:

java -jar opencms-storage-maintenance.jar --webinf /path/to/opencms/WEB-INF --mode delete-orphans --execute

The default limit is 1000 blobs per run. Set --delete-limit to another number or to 0 for an unlimited run. The tool checks references again immediately before deletion. S3 backends require bucket-listing permission in addition to read, write and delete access.

Use CmsDefaultStoragePolicy with storage.active=db. Matching resources are written to CMS_STORAGE, while the installation continues to use one database service.

storage.active=fs1
storage.legacy=db
storage.backend.fs1.type=fs
storage.backend.fs1.path=/var/opencms/storage/fs1

Validate the path from every OpenCms node, then use the migration tool to move existing matching content.

storage.active=s3main
storage.legacy=db
storage.backend.s3main.type=s3
storage.backend.s3main.endpoint=https://storage.example.com
storage.backend.s3main.bucket=opencms-data

Add credentials and service-specific client settings, validate the backend and run a dry-run before migration.

Set the destination as storage.active and retain the source ID under storage.legacy. After migration and verification, use the maintenance tool to identify and remove source blobs that are no longer referenced.