Package org.opencms.db.storage

Class CmsS3Storage

java.lang.Object

org.opencms.db.storage.A_CmsStorage

org.opencms.db.storage.CmsS3Storage

All Implemented Interfaces:

AutoCloseable, I_CmsEnumerableStorage, I_CmsStorage, I_CmsStorageDelivery

public class CmsS3Storage
extends A_CmsStorage
implements I_CmsEnumerableStorage, I_CmsStorageDelivery

S3-compatible object storage backend for the OpenCms deduplicating media storage engine (DMSE).

This implementation stores binary content in an S3-compatible object storage. To ensure high performance for read and write operations, it is highly recommended to use a local object storage (e.g., RustFS or an on-premise S3 appliance) within the same network as the OpenCms application server. Using remote cloud storage may introduce latency that significantly impacts VFS performance.

Content is addressed by its unique hash to support deduplication across different resources.

See Also:

• CmsStorageManager

Nested Class Summary

Nested classes/interfaces inherited from interface org.opencms.db.storage.I_CmsEnumerableStorage

I_CmsEnumerableStorage.I_CmsContentHashVisitor

Field Summary

Fields

Modifier and Type	Field	Description
static final String	STORAGE_TYPE	The type name of the storage implementation.

Constructor Summary

Constructors

Constructor	Description
CmsS3Storage(String name, String endpoint, String bucketName, String accessKey, String secretKey, boolean pathStyle)	Creates a new S3 storage.
CmsS3Storage(String name, String endpoint, String bucketName, String accessKey, String secretKey, boolean pathStyle, int connectionTimeout, int socketTimeout, int apiCallAttemptTimeout, int apiCallTimeout, int maxRetries)	Creates a new S3 storage.
CmsS3Storage(String name, String endpoint, String bucketName, String accessKey, String secretKey, boolean pathStyle, String region, int connectionTimeout, int socketTimeout, int apiCallAttemptTimeout, int apiCallTimeout, int maxRetries)	Creates a new S3 storage.
CmsS3Storage(String name, CmsS3ClientConfiguration configuration)	Creates a new S3 storage.

Method Summary

Modifier and Type	Method	Description
void	close()	Closes this storage backend and releases associated resources.
void	deleteContent(CmsDbContext dbc, String hash)	Deletes the content from the storage if applicable.
String	getStorageIdentifier()	Returns the stable storage identifier stored in the VFS content tables.
boolean	isPathStyle()	Returns whether path-style access is configured.
byte[]	loadContent(CmsDbContext dbc, String hash)	Loads the actual binary content.
void	loadContentFrom(CmsDbContext dbc, String hash, I_CmsFileContentStreamHandler handler)	Loads the actual binary content and passes it to the given input stream handler.
void	loadContentTo(CmsDbContext dbc, String hash, OutputStream out)	Loads the actual binary content into the given output stream.
void	storeContent(CmsDbContext dbc, String hash, byte[] content)	Stores the given content.
void	streamRangeTo(CmsDbContext dbc, String hash, long start, long length, OutputStream out)	Streams a byte range to the given output stream.
void	streamTo(CmsDbContext dbc, String hash, OutputStream out)	Streams the content to the given output stream.
boolean	supportsRangeDelivery()	Returns if range delivery is supported.
void	validateAvailable(CmsDbContext dbc)	Validates that the storage backend is available for read and write operations.
void	visitContentHashes(CmsDbContext dbc, I_CmsEnumerableStorage.I_CmsContentHashVisitor visitor)	Visits all content hashes stored by this backend.

Methods inherited from class org.opencms.db.storage.A_CmsStorage

getHashedPath, validateHash

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

STORAGE_TYPE

public static final String STORAGE_TYPE

The type name of the storage implementation.

See Also:

• Constant Field Values

Constructor Details

CmsS3Storage

public CmsS3Storage(String name,
 CmsS3ClientConfiguration configuration)

Creates a new S3 storage.

Parameters:

name - the configured backend id used as storage identifier

configuration - the S3 client configuration

CmsS3Storage

public CmsS3Storage(String name,
 String endpoint,
 String bucketName,
 String accessKey,
 String secretKey,
 boolean pathStyle)

Creates a new S3 storage.

Parameters:

name - the configured backend id used as storage identifier

endpoint - the S3 endpoint

bucketName - the bucket name

accessKey - the access key

secretKey - the secret key

pathStyle - whether path-style access should be used

CmsS3Storage

public CmsS3Storage(String name,
 String endpoint,
 String bucketName,
 String accessKey,
 String secretKey,
 boolean pathStyle,
 int connectionTimeout,
 int socketTimeout,
 int apiCallAttemptTimeout,
 int apiCallTimeout,
 int maxRetries)

Creates a new S3 storage.

Parameters:

name - the configured backend id used as storage identifier

endpoint - the S3 endpoint

bucketName - the bucket name

accessKey - the access key

secretKey - the secret key

pathStyle - whether path-style access should be used

connectionTimeout - the connection timeout in milliseconds

socketTimeout - the socket timeout in milliseconds

apiCallAttemptTimeout - the timeout for a single API call attempt in milliseconds

apiCallTimeout - the timeout for the complete API call in milliseconds

maxRetries - the maximum number of retries

CmsS3Storage

public CmsS3Storage(String name,
 String endpoint,
 String bucketName,
 String accessKey,
 String secretKey,
 boolean pathStyle,
 String region,
 int connectionTimeout,
 int socketTimeout,
 int apiCallAttemptTimeout,
 int apiCallTimeout,
 int maxRetries)

Creates a new S3 storage.

Parameters:

name - the configured backend id used as storage identifier

endpoint - the S3 endpoint

bucketName - the bucket name

accessKey - the access key

secretKey - the secret key

pathStyle - whether path-style access should be used

region - the AWS region

connectionTimeout - the connection timeout in milliseconds

socketTimeout - the socket timeout in milliseconds

apiCallAttemptTimeout - the timeout for a single API call attempt in milliseconds

apiCallTimeout - the timeout for the complete API call in milliseconds

maxRetries - the maximum number of retries

Method Details

close

public void close()
           throws Exception

Description copied from interface: I_CmsStorage

Closes this storage backend and releases associated resources.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface I_CmsStorage

Throws:

Exception - if closing the storage backend fails

See Also:

• I_CmsStorage.close()

deleteContent

public void deleteContent(CmsDbContext dbc,
 String hash)
                   throws Exception

Description copied from interface: I_CmsStorage

Deletes the content from the storage if applicable.

Specified by:

deleteContent in interface I_CmsStorage

Parameters:

dbc - the database context

hash - the SHA-512 content hash

Throws:

Exception - if the deletion in the external storage fails

See Also:

• I_CmsStorage.deleteContent(org.opencms.db.CmsDbContext, java.lang.String)

getStorageIdentifier

public String getStorageIdentifier()

Description copied from interface: I_CmsStorage

Returns the stable storage identifier stored in the VFS content tables.

The identifier is the configured backend id, for example media or archive. The database storage uses the reserved backend id db. The identifier must not depend on mutable connection details such as paths, endpoints, or bucket names.

Specified by:

getStorageIdentifier in interface I_CmsStorage

Returns:

the stable storage identifier

See Also:

• I_CmsStorage.getStorageIdentifier()

isPathStyle

public boolean isPathStyle()

Returns whether path-style access is configured.

Returns:

true if path-style access is configured

loadContent

public byte[] loadContent(CmsDbContext dbc,
 String hash)
                   throws Exception

Description copied from interface: I_CmsStorage

Loads the actual binary content.

Specified by:

loadContent in interface I_CmsStorage

Parameters:

dbc - the database context

hash - the SHA-512 content hash

Returns:

the actual binary file content

Throws:

Exception - if the content cannot be retrieved

See Also:

• I_CmsStorage.loadContent(org.opencms.db.CmsDbContext, java.lang.String)

loadContentFrom

public void loadContentFrom(CmsDbContext dbc,
 String hash,
 I_CmsFileContentStreamHandler handler)
                     throws Exception

Description copied from interface: I_CmsStorage

Loads the actual binary content and passes it to the given input stream handler.

This default implementation keeps custom storage implementations source compatible by falling back to I_CmsStorage.loadContent(CmsDbContext, String).

Specified by:

loadContentFrom in interface I_CmsStorage

Parameters:

dbc - the database context

hash - the SHA-512 content hash

handler - the stream handler

Throws:

Exception - if the content cannot be retrieved or handled

See Also:

• I_CmsStorage.loadContentFrom(CmsDbContext, String, I_CmsFileContentStreamHandler)

loadContentTo

public void loadContentTo(CmsDbContext dbc,
 String hash,
 OutputStream out)
                   throws Exception

Description copied from interface: I_CmsStorage

Loads the actual binary content into the given output stream.

This default implementation keeps custom storage implementations source compatible by falling back to I_CmsStorage.loadContent(CmsDbContext, String).

Specified by:

loadContentTo in interface I_CmsStorage

Parameters:

dbc - the database context

hash - the SHA-512 content hash

out - the output stream to write to

Throws:

Exception - if the content cannot be retrieved or written

See Also:

• I_CmsStorage.loadContentTo(CmsDbContext, String, OutputStream)

storeContent

public void storeContent(CmsDbContext dbc,
 String hash,
 byte[] content)
                  throws Exception

Description copied from interface: I_CmsStorage

Stores the given content.

Specified by:

storeContent in interface I_CmsStorage

Parameters:

dbc - the database context

hash - the SHA-512 content hash

content - the actual binary data to store

Throws:

Exception - if the storage operation fails

See Also:

• I_CmsStorage.storeContent(org.opencms.db.CmsDbContext, java.lang.String, byte[])

streamRangeTo

public void streamRangeTo(CmsDbContext dbc,
 String hash,
 long start,
 long length,
 OutputStream out)
                   throws Exception

Description copied from interface: I_CmsStorageDelivery

Streams a byte range to the given output stream.

Specified by:

streamRangeTo in interface I_CmsStorageDelivery

Parameters:

dbc - the database context

hash - the SHA-512 content hash

start - the first byte to stream

length - the number of bytes to stream

out - the output stream to write to

Throws:

Exception - if streaming fails

See Also:

• I_CmsStorageDelivery.streamRangeTo(org.opencms.db.CmsDbContext, java.lang.String, long, long, java.io.OutputStream)

streamTo

public void streamTo(CmsDbContext dbc,
 String hash,
 OutputStream out)
              throws Exception

Description copied from interface: I_CmsStorageDelivery

Streams the content to the given output stream.

Specified by:

streamTo in interface I_CmsStorageDelivery

Parameters:

dbc - the database context

hash - the SHA-512 content hash

out - the output stream to write to

Throws:

Exception - if streaming fails

See Also:

• I_CmsStorageDelivery.streamTo(org.opencms.db.CmsDbContext, java.lang.String, java.io.OutputStream)

supportsRangeDelivery

public boolean supportsRangeDelivery()

Description copied from interface: I_CmsStorageDelivery

Returns if range delivery is supported.

Specified by:

supportsRangeDelivery in interface I_CmsStorageDelivery

Returns:

true if byte range delivery is supported

See Also:

• I_CmsStorageDelivery.supportsRangeDelivery()

validateAvailable

public void validateAvailable(CmsDbContext dbc)
                       throws Exception

Description copied from interface: I_CmsStorage

Validates that the storage backend is available for read and write operations.

This method is called during startup for configured storage backends. Implementations should perform a lightweight roundtrip which verifies the required prerequisites for storing content.

Specified by:

validateAvailable in interface I_CmsStorage

Parameters:

dbc - the database context

Throws:

Exception - if the storage backend is not available

See Also:

• I_CmsStorage.validateAvailable(org.opencms.db.CmsDbContext)

visitContentHashes

public void visitContentHashes(CmsDbContext dbc,
 I_CmsEnumerableStorage.I_CmsContentHashVisitor visitor)
                        throws Exception

Description copied from interface: I_CmsEnumerableStorage

Visits all content hashes stored by this backend.

Specified by:

visitContentHashes in interface I_CmsEnumerableStorage

Parameters:

dbc - the database context

visitor - the content hash visitor

Throws:

Exception - if enumeration fails

See Also:

• I_CmsEnumerableStorage.visitContentHashes(org.opencms.db.CmsDbContext, org.opencms.db.storage.I_CmsEnumerableStorage.I_CmsContentHashVisitor)