Package org.opencms.db.storage.s3

Class CmsGenericS3Client

java.lang.Object

org.opencms.db.storage.s3.CmsGenericS3Client

All Implemented Interfaces:

AutoCloseable, I_CmsS3Client

public class CmsGenericS3Client
extends Object
implements I_CmsS3Client

S3 client using AWS SDK v2 optimized for a local object storage.

Should work with most products such as SeaweedFS, MinIO, Garage, Ceph.

Nested Class Summary

Nested classes/interfaces inherited from interface org.opencms.db.storage.s3.I_CmsS3Client

I_CmsS3Client.I_CmsS3ObjectKeyVisitor, I_CmsS3Client.I_CmsS3ObjectMetadataVisitor, I_CmsS3Client.I_CmsS3ObjectVisitor

Field Summary

Fields

Modifier and Type	Field	Description
static final int	DEFAULT_API_CALL_ATTEMPT_TIMEOUT	Default single API call attempt timeout in milliseconds.
static final int	DEFAULT_API_CALL_TIMEOUT	Default complete API call timeout in milliseconds.
static final int	DEFAULT_CONNECTION_ACQUISITION_TIMEOUT	Default timeout for acquiring a pooled connection in milliseconds.
static final int	DEFAULT_CONNECTION_TIMEOUT	Default connection timeout in milliseconds.
static final int	DEFAULT_MAX_CONNECTIONS	Default maximum number of pooled connections.
static final int	DEFAULT_MAX_RETRIES	Default maximum number of retries.
static final String	DEFAULT_REGION	Default AWS region.
static final int	DEFAULT_SOCKET_TIMEOUT	Default socket timeout in milliseconds.
static final int	MAX_DELETE_OBJECTS	Maximum number of keys accepted by the S3 DeleteObjects API.

Constructor Summary

Constructors

Constructor	Description
CmsGenericS3Client(String endpoint, String bucketName, String accessKey, String secretKey, boolean pathStyle)	Creates a new local S3 client.
CmsGenericS3Client(String endpoint, String bucketName, String accessKey, String secretKey, boolean pathStyle, int connectionTimeout, int socketTimeout, int apiCallAttemptTimeout, int apiCallTimeout, int maxRetries)	Creates a new local S3 client.
CmsGenericS3Client(CmsS3ClientConfiguration configuration)	Creates a new local S3 client.

Method Summary

Modifier and Type	Method	Description
void	close()	Closes the client and releases associated resources.
void	deleteObject(String key)	Deletes an object from the specified bucket.
CmsS3DeleteResult	deleteObjects(List<String> keys)	Deletes multiple objects.
boolean	exists(String key)	Checks if an object exists in the specified bucket.
byte[]	getObject(String key)	Retrieves an object's content from the specified bucket.
long	getObjectLength(String key)	Returns the length of an object's content.
long	getObjectLengthIfExists(String key)	Returns the length of an object's content if the object exists.
CmsS3ObjectMetadata	getObjectMetadata(String key)	Returns an object's metadata.
void	putObject(String key, byte[] content)	Uploads an object to the specified bucket.
void	readObjectFrom(String key, I_CmsFileContentStreamHandler handler)	Passes an object's content from the specified bucket to the given input stream handler.
boolean	renewObject(String key, String expectedRevision, Instant renewalTime)	Renews an object by conditionally copying it onto itself.
static CmsS3ClientConfiguration	resolveCredentials(CmsS3ClientConfiguration configuration)	Resolves credentials in the S3 client configuration with the OpenCms credentials resolver.
static CmsS3ClientConfiguration	resolveCredentials(CmsS3ClientConfiguration configuration, I_CmsCredentialsResolver credentialsResolver)	Resolves credentials in the S3 client configuration.
void	validateBucketAccess()	Validates that the configured bucket is accessible.
void	visitObjectKeys(I_CmsS3Client.I_CmsS3ObjectKeyVisitor visitor)	Visits all object keys in the configured bucket.
void	visitObjects(String prefix, I_CmsS3Client.I_CmsS3ObjectMetadataVisitor visitor)	Visits object metadata for objects matching an optional prefix.
void	visitObjects(I_CmsS3Client.I_CmsS3ObjectVisitor visitor)	Visits all objects in the configured bucket.
void	writeObjectRangeTo(String key, long start, long length, OutputStream out)	Writes an object byte range from the specified bucket to the given output stream.
CmsS3ObjectMetadata	writeObjectRangeToWithMetadata(String key, long start, long length, OutputStream out)	Writes an object byte range and returns the metadata received with the object response.
void	writeObjectTo(String key, OutputStream out)	Writes an object's content from the specified bucket to the given output stream.
CmsS3ObjectMetadata	writeObjectToWithMetadata(String key, OutputStream out)	Writes an object's content and returns the metadata received with the object response.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

MAX_DELETE_OBJECTS

public static final int MAX_DELETE_OBJECTS

Maximum number of keys accepted by the S3 DeleteObjects API.

See Also:

• Constant Field Values

DEFAULT_API_CALL_TIMEOUT

public static final int DEFAULT_API_CALL_TIMEOUT

Default complete API call timeout in milliseconds.

See Also:

• Constant Field Values

DEFAULT_API_CALL_ATTEMPT_TIMEOUT

public static final int DEFAULT_API_CALL_ATTEMPT_TIMEOUT

Default single API call attempt timeout in milliseconds.

See Also:

• Constant Field Values

DEFAULT_CONNECTION_TIMEOUT

public static final int DEFAULT_CONNECTION_TIMEOUT

Default connection timeout in milliseconds.

See Also:

• Constant Field Values

DEFAULT_CONNECTION_ACQUISITION_TIMEOUT

public static final int DEFAULT_CONNECTION_ACQUISITION_TIMEOUT

Default timeout for acquiring a pooled connection in milliseconds.

See Also:

• Constant Field Values

DEFAULT_MAX_CONNECTIONS

public static final int DEFAULT_MAX_CONNECTIONS

Default maximum number of pooled connections.

See Also:

• Constant Field Values

DEFAULT_MAX_RETRIES

public static final int DEFAULT_MAX_RETRIES

Default maximum number of retries.

See Also:

• Constant Field Values

DEFAULT_REGION

public static final String DEFAULT_REGION

Default AWS region.

DEFAULT_SOCKET_TIMEOUT

public static final int DEFAULT_SOCKET_TIMEOUT

Default socket timeout in milliseconds.

See Also:

• Constant Field Values

Constructor Details

CmsGenericS3Client

public CmsGenericS3Client(CmsS3ClientConfiguration configuration)

Creates a new local S3 client.

Parameters:

configuration - the S3 client configuration

CmsGenericS3Client

public CmsGenericS3Client(String endpoint,
 String bucketName,
 String accessKey,
 String secretKey,
 boolean pathStyle)

Creates a new local S3 client.

Parameters:

endpoint - the URL of the S3 server (e.g., "http://localhost:8080")

bucketName - the bucket name

accessKey - the access key

secretKey - the secret key

pathStyle - whether path-style access should be used

CmsGenericS3Client

public CmsGenericS3Client(String endpoint,
 String bucketName,
 String accessKey,
 String secretKey,
 boolean pathStyle,
 int connectionTimeout,
 int socketTimeout,
 int apiCallAttemptTimeout,
 int apiCallTimeout,
 int maxRetries)

Creates a new local S3 client.

Parameters:

endpoint - the URL of the S3 server (e.g., "http://localhost:8080")

bucketName - the bucket name

accessKey - the access key

secretKey - the secret key

pathStyle - whether path-style access should be used

connectionTimeout - the connection timeout in milliseconds

socketTimeout - the socket timeout in milliseconds

apiCallAttemptTimeout - the timeout for a single API call attempt in milliseconds

apiCallTimeout - the timeout for the complete API call in milliseconds

maxRetries - the maximum number of retries

Method Details

resolveCredentials

public static CmsS3ClientConfiguration resolveCredentials(CmsS3ClientConfiguration configuration)

Resolves credentials in the S3 client configuration with the OpenCms credentials resolver.

Parameters:

configuration - the original S3 client configuration

Returns:

the S3 client configuration with resolved credentials

resolveCredentials

public static CmsS3ClientConfiguration resolveCredentials(CmsS3ClientConfiguration configuration,
 I_CmsCredentialsResolver credentialsResolver)

Resolves credentials in the S3 client configuration.

Parameters:

configuration - the original S3 client configuration

credentialsResolver - the credentials resolver

Returns:

the S3 client configuration with resolved credentials

close

public void close()
           throws Exception

Description copied from interface: I_CmsS3Client

Closes the client and releases associated resources.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface I_CmsS3Client

Throws:

Exception - if closing the client fails

deleteObject

public void deleteObject(String key)
                  throws Exception

Description copied from interface: I_CmsS3Client

Deletes an object from the specified bucket.

Specified by:

deleteObject in interface I_CmsS3Client

Parameters:

key - the identifier for the object to delete

Throws:

Exception - if the deletion fails

deleteObjects

public CmsS3DeleteResult deleteObjects(List<String> keys)
                                throws Exception

Description copied from interface: I_CmsS3Client

Deletes multiple objects.

The optimized S3 implementation supports at most 1000 keys per call. This default implementation preserves compatibility for other clients.

Specified by:

deleteObjects in interface I_CmsS3Client

Parameters:

keys - the object keys

Returns:

the per-object delete result

Throws:

Exception - if the complete request fails

See Also:

• I_CmsS3Client.deleteObjects(java.util.List)

exists

public boolean exists(String key)
               throws Exception

Description copied from interface: I_CmsS3Client

Checks if an object exists in the specified bucket.

Specified by:

exists in interface I_CmsS3Client

Parameters:

key - the identifier for the object

Returns:

true if the object exists, false otherwise

Throws:

Exception - if the existence check fails

getObject

public byte[] getObject(String key)
                 throws Exception

Description copied from interface: I_CmsS3Client

Retrieves an object's content from the specified bucket.

Specified by:

getObject in interface I_CmsS3Client

Parameters:

key - the identifier for the object

Returns:

the binary data of the object

Throws:

Exception - if the object cannot be retrieved or does not exist

getObjectLength

public long getObjectLength(String key)
                     throws Exception

Description copied from interface: I_CmsS3Client

Returns the length of an object's content.

Specified by:

getObjectLength in interface I_CmsS3Client

Parameters:

key - the identifier for the object

Returns:

the object content length

Throws:

Exception - if the object can not be accessed

getObjectLengthIfExists

public long getObjectLengthIfExists(String key)
                             throws Exception

Description copied from interface: I_CmsS3Client

Returns the length of an object's content if the object exists.

Specified by:

getObjectLengthIfExists in interface I_CmsS3Client

Parameters:

key - the identifier for the object

Returns:

the object content length, or -1 if the object does not exist

Throws:

Exception - if the object can not be accessed

getObjectMetadata

public CmsS3ObjectMetadata getObjectMetadata(String key)
                                      throws Exception

Description copied from interface: I_CmsS3Client

Returns an object's metadata.

Specified by:

getObjectMetadata in interface I_CmsS3Client

Parameters:

key - the identifier for the object

Returns:

the object metadata

Throws:

Exception - if the object can not be accessed

See Also:

• I_CmsS3Client.getObjectMetadata(java.lang.String)

putObject

public void putObject(String key,
 byte[] content)
               throws Exception

Description copied from interface: I_CmsS3Client

Uploads an object to the specified bucket.

Specified by:

putObject in interface I_CmsS3Client

Parameters:

key - the unique identifier (path) for the object

content - the binary data to upload

Throws:

Exception - if the upload fails

readObjectFrom

public void readObjectFrom(String key,
 I_CmsFileContentStreamHandler handler)
                    throws Exception

Description copied from interface: I_CmsS3Client

Passes an object's content from the specified bucket to the given input stream handler.

Specified by:

readObjectFrom in interface I_CmsS3Client

Parameters:

key - the identifier for the object

handler - the stream handler

Throws:

Exception - if the object cannot be retrieved or handled

See Also:

• I_CmsS3Client.readObjectFrom(String, I_CmsFileContentStreamHandler)

renewObject

public boolean renewObject(String key,
 String expectedRevision,
 Instant renewalTime)
                    throws Exception

Description copied from interface: I_CmsS3Client

Renews an object by conditionally copying it onto itself.

The copy must only be applied when the source still has the expected revision. A missing object or revision mismatch is not an error and is reported by returning false. The S3 last-modified time is always assigned by the storage server.

Specified by:

renewObject in interface I_CmsS3Client

Parameters:

key - the object key

expectedRevision - the expected source revision

renewalTime - the requested renewal time

Returns:

true if the object was renewed, or false if it no longer matched

Throws:

Exception - if the copy fails

See Also:

• I_CmsS3Client.renewObject(java.lang.String, java.lang.String, java.time.Instant)

validateBucketAccess

public void validateBucketAccess()
                          throws Exception

Description copied from interface: I_CmsS3Client

Validates that the configured bucket is accessible.

Specified by:

validateBucketAccess in interface I_CmsS3Client

Throws:

Exception - if the bucket can not be accessed

visitObjectKeys

public void visitObjectKeys(I_CmsS3Client.I_CmsS3ObjectKeyVisitor visitor)
                     throws Exception

Description copied from interface: I_CmsS3Client

Visits all object keys in the configured bucket.

Specified by:

visitObjectKeys in interface I_CmsS3Client

Parameters:

visitor - the object key visitor

Throws:

Exception - if listing fails

visitObjects

public void visitObjects(I_CmsS3Client.I_CmsS3ObjectVisitor visitor)
                  throws Exception

Description copied from interface: I_CmsS3Client

Visits all objects in the configured bucket.

Specified by:

visitObjects in interface I_CmsS3Client

Parameters:

visitor - the object visitor

Throws:

Exception - if listing fails

visitObjects

public void visitObjects(String prefix,
 I_CmsS3Client.I_CmsS3ObjectMetadataVisitor visitor)
                  throws Exception

Description copied from interface: I_CmsS3Client

Visits object metadata for objects matching an optional prefix.

This compatibility implementation filters client-side and can not provide timestamps or revisions. Optimized clients should override it.

Specified by:

visitObjects in interface I_CmsS3Client

Parameters:

prefix - the object key prefix, or null

visitor - the metadata visitor

Throws:

Exception - if listing fails

See Also:

• I_CmsS3Client.visitObjects(java.lang.String, org.opencms.db.storage.s3.I_CmsS3Client.I_CmsS3ObjectMetadataVisitor)

writeObjectRangeTo

public void writeObjectRangeTo(String key,
 long start,
 long length,
 OutputStream out)
                        throws Exception

Description copied from interface: I_CmsS3Client

Writes an object byte range from the specified bucket to the given output stream.

Specified by:

writeObjectRangeTo in interface I_CmsS3Client

Parameters:

key - the identifier for the object

start - the first byte to write

length - the number of bytes to write

out - the output stream to write to

Throws:

Exception - if the object cannot be retrieved or written

See Also:

• I_CmsS3Client.writeObjectRangeTo(String, long, long, OutputStream)

writeObjectRangeToWithMetadata

public CmsS3ObjectMetadata writeObjectRangeToWithMetadata(String key,
 long start,
 long length,
 OutputStream out)
                                                   throws Exception

Description copied from interface: I_CmsS3Client

Writes an object byte range and returns the metadata received with the object response.

Optimized clients should override this method so that no additional metadata request is needed. The default implementation preserves compatibility by reading the range first and loading the metadata afterwards.

Specified by:

writeObjectRangeToWithMetadata in interface I_CmsS3Client

Parameters:

key - the identifier for the object

start - the first byte to write

length - the number of bytes to write

out - the output stream to write to

Returns:

the object metadata

Throws:

Exception - if the object cannot be retrieved or written

See Also:

• I_CmsS3Client.writeObjectRangeToWithMetadata(String, long, long, OutputStream)

writeObjectTo

public void writeObjectTo(String key,
 OutputStream out)
                   throws Exception

Description copied from interface: I_CmsS3Client

Writes an object's content from the specified bucket to the given output stream.

Specified by:

writeObjectTo in interface I_CmsS3Client

Parameters:

key - the identifier for the object

out - the output stream to write to

Throws:

Exception - if the object cannot be retrieved or written

writeObjectToWithMetadata

public CmsS3ObjectMetadata writeObjectToWithMetadata(String key,
 OutputStream out)
                                              throws Exception

Description copied from interface: I_CmsS3Client

Writes an object's content and returns the metadata received with the object response.

Optimized clients should override this method so that no additional metadata request is needed. The default implementation preserves compatibility by reading the object first and loading the metadata afterwards.

Specified by:

writeObjectToWithMetadata in interface I_CmsS3Client

Parameters:

key - the identifier for the object

out - the output stream to write to

Returns:

the object metadata

Throws:

Exception - if the object cannot be retrieved or written

See Also:

• I_CmsS3Client.writeObjectToWithMetadata(String, OutputStream)