OneFS S3 Object Lock and Bucket Lock Configuration and Management

As we saw in the previous article in this series, OneFS 9.12 introduces support for S3 Object Locks. This feature enables write-once-read-many (WORM) protection for objects, ensuring critical data remains immutable and safeguarded against accidental or malicious deletion. By applying Object Locks, organizations can prevent object deletion or modification for a specified duration, or indefinitely, helping maintain data integrity and meet regulatory retention requirements. This capability is particularly valuable for industries such as finance services, where compliance and secure data preservation are essential.

The S3 bucket and object locking features become available after the OneFS 9.12 release has been committed, and no additional licensing is required. To enable them, the pertinent access zone is configured to support bucket or object locks, during which, if not already active, the compliance clock is automatically enabled in support of locking. Within the S3 zone configuration, three new options are introduced.

Name Value Description
Object lock support boolean Allows Object Lock or Immutable Buckets to be created in the zone
Default lock protection mode ObjectLock | BucketLock Default object lock mode for Buckets with object lock enabled.
Compliance clock support NULL Automatically enables compliance clock if not already enabled

‘Object-lock-support’ is disabled by default and must be enabled to allow the creation of locked buckets, while ‘default-lock-protection-mode’ determines the default lock type when none is specified in an API call. ‘Compliance-clock-support’ enables the compliance clock if it is not already active, and these are all represented in the OneFS 9.12 S3 CLI as follows:

# isi s3 settings zone modify -h | grep -i lock

[--object-lock-support <boolean>]

[--default-lock-protection-mode (ObjectLock | BucketLock)]

[--compliance-clock-support]

For example:

# isi s3 zone settings modify –object-lock-support 1

Similarly, the new S3 bucket settings include:

Name Value Description
Object lock enabled boolean Enable Locking on the bucket. Cannot be disabled afterwards.
Lock protection mode ObjectLock | BucketLock Bucket Lock Mode
Default Retention Mode GOVERNANCE | COMPLIANCE Only Governance is supported in OneFS 9.12
Default Retention Days Int Retention period in days. Mutually exclusive with years.
Default Retention Years Int Retention period in years. Mutually exclusive with days.

These can be configured in the S3 CLI with the following arguments:

# isi s3 buckets modify <bucket>

        [--object-lock-enabled <boolean>]

        [--lock-protection-mode (ObjectLock | BucketLock)]

        [--default-retention-mode (GOVERNANCE | COMPLIANCE)]

        [--default-retention-days <integer>]

        [--default-retention-years <integer>]

The OneFS S3 per-access zone configuration settings can be viewed from the WebUI under Protocols > Object Storage (S3) > Zone Settings, as follows:

Or from the CLI:

# isi s3 settings zone view

                   Root Path: /ifs/data

                 Base Domain: tme.isilon.com

           Object ACL Policy: replace

Bucket Directory Create Mode: 0777

            Use Md5 For Etag: Yes

        Validate Content Md5: Yes

         Object Lock Support: Yes

       Syslog Access Logging: No

Default Lock Protection Mode: Object Lock

These settings can modified, for example to change the default lock protection mode from object to bucket:

# isi s3 settings zone view | grep -i protection

Default Lock Protection Mode: Object Lock

# isi s3 settings zone modify --default-lock-protection-mode=BucketLock

# isi s3 settings zone view | grep -i protection

Default Lock Protection Mode: Bucket Lock

At the bucket level, new settings include ‘object-lock-enabled’, which, once set, cannot be disabled. The ‘lock-protection-mode’ setting specifies whether Object Lock or Bucket Lock is used. The default-retention-mode is currently limited to ‘Governance’ mode, since ‘Compliance’ mode is not currently supported in the OneFS 9.12 release. Retention periods can be specified in either days or years, with a maximum duration of 100 years.

From the WebUI, object or bucket locks and their retention period can be configured on buckets as follows:

Followed by a warning that locking cannot be disabled and confirmation:

Then confirmation of creation:

Or from the CLI:

# isi s3 buckets create test-bkt /ifs/data/zone-a --owner rlm --object-lock-enabled 1 --lock-protection-mode BucketLock

The S3 buckets and their lock status can be reported as follows:

# isi s3 buckets list

Bucket Name  Path                                    Owner  Object ACL Policy  Object Lock Enabled  Lock Protection Mode  Description

--------------------------------------------------------------------------------------------------------------------------------------

test-bkt     /ifs/data/zone-a                        rlm    replace            Yes                  Bucket Lock 

test-bkt-b   /ifs/data/zone-b                        rlm    replace            Yes                  Bucket Lock

--------------------------------------------------------------------------------------------------------------------------------------

Total: 4

And to view the details of a specific bucket:

# isi s3 buckets view test-bkt

         Bucket Name: test-bkt

                Path: /ifs/data/zone-a

               Owner: rlm

   Object ACL Policy: replace

 Object Lock Enabled: Yes

Lock Protection Mode: Bucket Lock

         Description:

Default Retention

                 Mode: GOVERNANCE

                Years: -

                 Days: 1

Access Logging Enabled: Yes

         Target Bucket: test-bkt

            Log Prefix:

Additionally, the following CLI syntax can be used to confirm the presence of a lock on the bucket:

# isi get -DDD /ifs/data/zone-a | grep -i ObjectLock

 IFS Domain IDs:     {2.0100(Snapshot), 1d.0900(ObjectLock) }

The following error will be displayed if attempting to create a bucket lock on a target bucket:

There are a couple of caveats and proclivities to bear in mind with S3 bucket and object locking in OneFS 9.12. Specifically, certain S3 clients may require additional configuration to support custom headers for BucketLock buckets.

Header Description
x-amz-bypass-governance-retention PutObjectLockConfiguration when lowering retention for BucketLock bucket
x-isi-lock-protection-mode Specifying the lock-protection-mode of a bucket

Standard tools might require modification in order to support these special headers. For example, with the ubiquitous Boto3 client, callback handlers may need to be registered using the Boto toolkit to include these headers in requests.

If a client HTTP request is invalid, or goes awry, OneFS follows the general AWS S3 error codes format – albeit with modifications to remove any AWS-specific info. The OneFS S3 implementation also includes some additional error codes for its intrinsic behaviors. These include:

Since OneFS S3 Object Lock does not support legal hold and compliance retention mode, any attempts to use it will return an HTTP 501 ‘Not Implemented’ error. Additionally, due to the current absence of versioning support, attempts to overwrite a locked object will result in an HTTP 403 ‘Access Denied’ error.

For investigative and troubleshooting purposes, all relevant operations are logged in the standard S3 server logs, located at /var/log/s3.log.

In summary, the general behavior of the new OneFS S3 object locking feature in OneFS 9.12 includes the following:

OneFS S3 Specific Behavior AWS S3-compliant Behavior
·         Object locks may not be enabled for older buckets or non-empty buckets.

·         Only a bucket’s owner can enable object lock for the bucket. The owner must match the file system directory owner of the directory bucket since there’s no concept of directory ownership in AWS S3.

·         Unlike AWS S3 where lock modes (i.e. Governance and Compliance) are directly linked to S3 objects, in OneFS they are bucket-specific, with the same mode applying to all objects within a bucket.

·         Compliance mode for buckets is not supported when the system is in enterprise state.

·         Compliance mode buckets are not supported in OneFS 9.12.

·         Object locks will be enabled at “Bucket” level.

·         Once Object locks are enabled, they CANNOT be disabled.

·         Every S3 bucket will have a default retention that will apply to all objects inside the bucket.

·         Objects inside a bucket can have different retention periods.

·         Retention period can be extended or lowered (privileged action) for S3 objects.

 

As a further level of Bucket lock security, OneFS multi-party authorization (MPA) includes the following S3 privileged actions that, if enabled, require an additional administrator to approve the reduction of an immutable bucket’s retention time configuration and/or changes to a bucket’s access logging configuration.

Service /

Component

Action Description 
S3 reduce_immutable_bucket_retention Reduce bucket retention for an immutable bucket.
S3 modify_server_access_logging_config Change a bucket’s access logging configuration.

MPA helps to mitigate the risk of data loss or system configuration damage from critical actions, by vastly reducing the likelihood of accidental or malicious execution of consequential operations. MPA enforces a security control mechanism wherein operations involving critical or sensitive systems or data require explicit approval from multiple authorized entities. This ensures that no single actor can execute high-impact actions unilaterally, thereby mitigating risk and enhancing operational integrity through enforced oversight. As such, many enterprises require MPA in order to meet industry data protection requirements, in addition to internal security mandates and best practices.

MPA S3 privileged actions supporting immutable bucket retention and logging are available for selection in the MPA configuration, for example from the WebUI MPA Requests dropdown menu, located under Access > Multi-Party Authorization > Requests:

Introduced in OneFS 9.12, MPA can be configured and managed from the CLI, WebUI or platform API, but the feature is not enabled by default. Once MPA is enabled and configured, all predefined ’privileged actions’ require additional approval from an authorizing user. So this is markedly different from earlier OneFS releases, where the original requester would likely have had the basic rights and been able to execute that action in its entirety themselves.

Note that MPA is incompatible with, and therefore not supported on, a PowerScale cluster that is already running in Compliance Mode.

Leave a Reply

Your email address will not be published. Required fields are marked *