Package org.opencms.db.storage

Class CmsStorageManager

java.lang.Object

org.opencms.db.storage.CmsStorageManager

All Implemented Interfaces:

AutoCloseable

public class CmsStorageManager
extends Object
implements AutoCloseable

Central coordinator of the OpenCms deduplicating media storage engine (DMSE).

The DMSE stores binary resource content in database, file system or S3-compatible backends, using SHA-512 content hashes for deduplication. Storage policies determine which contents remain in the regular VFS content columns and which are stored in a backend.

This manager evaluates policies, coordinates backend access and checks content references before deleting stored blobs.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	CmsStorageManager.StorageResult	Container for storage evaluation results, used by VFS drivers to determine which database columns to update.

Field Summary

Fields

Modifier and Type	Field	Description
static final String	PARAM_STORAGE_ACTIVE	Prefix for storage properties.
static final String	PARAM_STORAGE_BACKEND_PREFIX	Prefix for backend-specific configuration.
static final String	PARAM_STORAGE_LEGACY	Prefix for the legacy storage list.

Constructor Summary

Constructors

Constructor	Description
CmsStorageManager(CmsSqlManager sqlManager, CmsParameterConfiguration configuration)	Creates a new storage manager.
CmsStorageManager(CmsSqlManager sqlManager, CmsParameterConfiguration configuration, CmsStoragePolicyConfiguration storagePolicyConfiguration)	Creates a new storage manager.

Method Summary

Modifier and Type	Method	Description
void	close()	Closes all configured storage backends.
static CmsS3ClientConfiguration	createS3ClientConfiguration(CmsParameterConfiguration configuration, String prefix)	Creates an S3 client configuration from properties below the given prefix.
static CmsS3ClientConfiguration	createS3ClientConfiguration(CmsParameterConfiguration configuration, String backendId, String bucketOverride)	Creates an S3 client configuration from a configured storage backend.
void	deleteContent(CmsDbContext dbc, String storage, String hash)	Deletes content from external storage if it is no longer referenced in any content table.
static String	getActiveStorageId(CmsParameterConfiguration configuration)	Returns the active storage backend id from the runtime property configuration.
static String	getActiveStorageType(CmsParameterConfiguration configuration)	Returns the active storage backend type from the runtime property configuration.
static List<String>	getDataStorageIds(CmsParameterConfiguration configuration)	Returns the backend identifiers used for VFS data storage.
I_CmsStorageDelivery	getDeliveryStorage(String storage)	Returns a delivery-capable storage backend by its stable storage identifier.
static String	getFileSystemStoragePath(CmsParameterConfiguration configuration, String storageId)	Returns the configured file system repository path.
I_CmsStorage	getStorage(String storage)	Returns a configured storage backend by its stable storage identifier.
static String	getStorageType(CmsParameterConfiguration configuration, String storageId)	Returns the type for a configured storage backend.
boolean	isActiveStorage(String storage)	Returns if the given stable storage identifier belongs to the active storage backend.
byte[]	loadContent(CmsDbContext dbc, byte[] contents, String storage, String hash)	Loads the content either from the provided local bytes or from external storage.
void	loadContentFrom(CmsDbContext dbc, byte[] contents, String storage, String hash, I_CmsFileContentStreamHandler handler)	Loads content either from the provided local bytes or from external storage and passes it to an input stream handler.
void	loadContentTo(CmsDbContext dbc, byte[] contents, String storage, String hash, OutputStream out)	Loads content either from the provided local bytes or from external storage into an output stream.
CmsStorageManager.StorageResult	prepareContent(CmsDbContext dbc, CmsStoragePolicyContext context)	Prepares content for storage by checking the policy and offloading to external storage if required.
void	validateStorages(CmsDbContext dbc)	Validates all configured storage backends.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

PARAM_STORAGE_ACTIVE

public static final String PARAM_STORAGE_ACTIVE

Prefix for storage properties.

See Also:

• Constant Field Values

PARAM_STORAGE_LEGACY

public static final String PARAM_STORAGE_LEGACY

Prefix for the legacy storage list.

See Also:

• Constant Field Values

PARAM_STORAGE_BACKEND_PREFIX

public static final String PARAM_STORAGE_BACKEND_PREFIX

Prefix for backend-specific configuration.

See Also:

• Constant Field Values

Constructor Details

CmsStorageManager

public CmsStorageManager(CmsSqlManager sqlManager,
 CmsParameterConfiguration configuration)

Creates a new storage manager.

Parameters:

sqlManager - the SQL manager

configuration - the runtime property configuration

CmsStorageManager

public CmsStorageManager(CmsSqlManager sqlManager,
 CmsParameterConfiguration configuration,
 CmsStoragePolicyConfiguration storagePolicyConfiguration)

Creates a new storage manager.

Parameters:

sqlManager - the SQL manager

configuration - the runtime property configuration

storagePolicyConfiguration - the storage policy configuration

Method Details

createS3ClientConfiguration

public static CmsS3ClientConfiguration createS3ClientConfiguration(CmsParameterConfiguration configuration,
 String prefix)

Creates an S3 client configuration from properties below the given prefix.

Parameters:

configuration - the runtime property configuration

prefix - the property prefix, including its trailing dot

Returns:

the S3 client configuration

createS3ClientConfiguration

public static CmsS3ClientConfiguration createS3ClientConfiguration(CmsParameterConfiguration configuration,
 String backendId,
 String bucketOverride)

Creates an S3 client configuration from a configured storage backend.

Parameters:

configuration - the runtime property configuration

backendId - the configured storage backend id, or null to use the active backend

bucketOverride - the bucket override, or null to use the backend bucket

Returns:

the S3 client configuration

getActiveStorageId

public static String getActiveStorageId(CmsParameterConfiguration configuration)

Returns the active storage backend id from the runtime property configuration.

Parameters:

configuration - the runtime property configuration

Returns:

the active storage backend id

getActiveStorageType

public static String getActiveStorageType(CmsParameterConfiguration configuration)

Returns the active storage backend type from the runtime property configuration.

Parameters:

configuration - the runtime property configuration

Returns:

the active storage backend type

getDataStorageIds

public static List<String> getDataStorageIds(CmsParameterConfiguration configuration)

Returns the backend identifiers used for VFS data storage.

Parameters:

configuration - the runtime property configuration

Returns:

the active backend followed by all distinct legacy backends

getFileSystemStoragePath

public static String getFileSystemStoragePath(CmsParameterConfiguration configuration,
 String storageId)

Returns the configured file system repository path.

Parameters:

configuration - the runtime property configuration

storageId - the storage backend id

Returns:

the configured repository path

getStorageType

public static String getStorageType(CmsParameterConfiguration configuration,
 String storageId)

Returns the type for a configured storage backend.

Parameters:

configuration - the runtime property configuration

storageId - the storage backend id

Returns:

the storage backend type

close

public void close()
           throws Exception

Closes all configured storage backends.

Specified by:

close in interface AutoCloseable

Throws:

Exception - if closing one or more storage backends fails

deleteContent

public void deleteContent(CmsDbContext dbc,
 String storage,
 String hash)

Deletes content from external storage if it is no longer referenced in any content table.

Parameters:

dbc - the database context

storage - the stable storage identifier stored in the STORAGE column

hash - the hash of the content to delete

getDeliveryStorage

public I_CmsStorageDelivery getDeliveryStorage(String storage)
                                        throws CmsStorageException

Returns a delivery-capable storage backend by its stable storage identifier.

Parameters:

storage - the stable storage identifier

Returns:

the delivery-capable storage backend, or null if the backend does not support delivery

Throws:

CmsStorageException - if no backend is configured for the storage identifier

getStorage

public I_CmsStorage getStorage(String storage)

Returns a configured storage backend by its stable storage identifier.

Parameters:

storage - the stable storage identifier

Returns:

the configured storage backend, or null if no such backend is configured

isActiveStorage

public boolean isActiveStorage(String storage)

Returns if the given stable storage identifier belongs to the active storage backend.

Parameters:

storage - the stable storage identifier

Returns:

true if the storage identifier belongs to the active backend

loadContent

public byte[] loadContent(CmsDbContext dbc,
 byte[] contents,
 String storage,
 String hash)
                   throws CmsStorageException

Loads the content either from the provided local bytes or from external storage.

Parameters:

dbc - the database context

contents - the local bytes (from FILE_CONTENT column)

storage - the stable storage identifier stored in the STORAGE column

hash - the SHA-512 content hash stored in the HASH column

Returns:

the raw bytes of the file

Throws:

CmsStorageException - if external storage content can not be loaded

loadContentFrom

public void loadContentFrom(CmsDbContext dbc,
 byte[] contents,
 String storage,
 String hash,
 I_CmsFileContentStreamHandler handler)
                     throws CmsStorageException

Loads content either from the provided local bytes or from external storage and passes it to an input stream handler.

Parameters:

dbc - the database context

contents - the local bytes (from FILE_CONTENT column)

storage - the stable storage identifier stored in the STORAGE column

hash - the SHA-512 content hash stored in the HASH column

handler - the stream handler

Throws:

CmsStorageException - if external storage content can not be loaded or handled

loadContentTo

public void loadContentTo(CmsDbContext dbc,
 byte[] contents,
 String storage,
 String hash,
 OutputStream out)
                   throws CmsStorageException

Loads content either from the provided local bytes or from external storage into an output stream.

Parameters:

dbc - the database context

contents - the local bytes (from FILE_CONTENT column)

storage - the stable storage identifier stored in the STORAGE column

hash - the SHA-512 content hash stored in the HASH column

out - the output stream to write to

Throws:

CmsStorageException - if external storage content can not be loaded or written

prepareContent

public CmsStorageManager.StorageResult prepareContent(CmsDbContext dbc,
 CmsStoragePolicyContext context)
                                               throws CmsDataAccessException

Prepares content for storage by checking the policy and offloading to external storage if required.

Parameters:

dbc - the database context

context - the storage policy context

Returns:

the storage result containing either the bytes or the hash

Throws:

CmsDataAccessException - if storing required external content fails

validateStorages

public void validateStorages(CmsDbContext dbc)
                      throws Exception

Validates all configured storage backends.

Parameters:

dbc - the database context

Throws:

Exception - if a configured storage backend is not available