As we saw in the previous article, OneFS 9.14 adds S3 Lifecycle Management, which allows administrators to define policies that automate object management within PowerScale S3 buckets. These policies enable the automatic deletion of objects based on criteria such as age, size, or key prefix and are applied uniformly to both existing and newly created objects in a bucket. Lifecycle processing is handled by the OneFS Job Engine, which runs daily to evaluate configured rules and generates per‑bucket tasks that traverse bucket directories and remove objects that meet the defined conditions.
To support this new functionality, the S3 API support in OneFS 9.14 and later now includes the following endpoints:
| API Endpoint | Description |
| PutBucketLifecycleConfiguration | Sets the lifecycle configuration for the bucket and replaces any existing one. User must be the bucket owner to create the lifecycle configuration. |
| GetBucketLifecycleConfiguration | Returns the current lifecycle configuration for the bucket. User must be the bucket owner to get the lifecycle configuration. Will return a ‘NoSuchLifecycleConfiguration’ error if a configuration is not found. |
| DeleteBucketLifecycle | Deletes the lifecycle configuration for the bucket. User must be the bucket owner to delete the lifecycle configuration. |
| AbortIncompleteMultipartUpload | |
| · DaysAfterInitiation | Number of days after the system aborts an incomplete MPU. |
Plus, the following S3 endpoints are also updated in 9.14 and require the following read and write permissions:
| S3 Endpoint | Read Permission | Write Permission |
| CompleteMultiPartUpload | x | x |
| CopyObject | x | x |
| GetObject | x | |
| HeadObject | x | |
| PutObject | x | x |
In this second article in the series, we’ll walk through a simple example demonstrating how to configure and validate S3 bucket lifecycle management in OneFS 9.14 and later releases.
To configure the feature, an S3 command must be sent to the cluster. A simple way to accomplish this without writing code is by using a utility like the ‘s3cmd’ tool. This tool provides a Python script which can be executed directly on a PowerScale cluster.
- The s3cmd tool’s zip file can be downloaded (or copied) to a directory on the cluster and unpacked with the following CLI command:
# unzip s3cmd-2.4.0.zip
Once unzipped, a new subdirectory named ‘s3cmd-2.4.0’ (with the corresponding version-specific suffix) is created. The working directory should be changed to this new subdirectory so that the ‘s3cmd’ script itself can be executed. The contents of the directory are as follows:
# ls S3cmd-2.4.0 s3cmd-.2.4.0.zip #cd s3cmd-2.4.0 # ls INSTALL.md NEWS S3 s3cmd.egg-info LICENSE PKG-INFO s3cmd setup.cfg MANIFEST.in README.md s3cmd.1 setup.py
- Next, a test bucket is configured on the cluster, and an access key and secret are generated for that bucket. In this example, the ‘root’ user and the ‘System’ multi-tenant access zone are used for access. This process begins by creating the test directory, verifying that the S3 service is enabled, and disabling HTTPS-only access.
# mkdir -p /ifs/s3lifecycle # isi s3 settings global modify --service=true --https-only=false
Next, the bucket is created, in conjunction with the access key and secret. For example:
# isi s3 buckets create --name=s3life --path=/ifs/s3lifecycle --owner=root # isi s3 keys create --user=root --force --show-key > /ifs/root-s3.keys # isi s3 buckets list Bucket Name Path Owner Object ACL Policy Object Lock Enabled Lock Protection Mode Description --------------------------------------------------------------------------------------------------------------- s3life /ifs/s3lifecycle root replace No - --------------------------------------------------------------------------------------------------------------- Total: 1 # cat /ifs/root-s3.keys Access ID: 1_root_accid Secret Key: 0t1O0URz0H5pef6Wn6P6L9BKc8Ad Timestamp: 2026-05-14T14:27:36 Old Secret Key: **************************** Old Key Timestamp: 2026-05-12T17:25:02 Old Key Expiry: 2026-05-14T14:37:36
- After obtaining the access ID and secret, s3cmd can be configured with these credentials and the appropriate endpoint parameters to simplify command execution. Site-specific configuration parameters that will need to be specified include:
- Access Key
- Secret Key
- S3 Endpoint
- DNS-style bucket+hostname:port
If HTTPS is preferred, the S3 endpoint port should be changed from 9020 to 9021, and Y should be selected when prompted to use the HTTPS protocol. HTTP may be used instead when packet‑level debugging is required, as it allows network sniffing tools such as Wireshark to capture traffic more efficiently and comprehensibly.
# python3 s3cmd --configure Enter new values or accept defaults in brackets with Enter. Refer to user manual for detailed description of all options. Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables. Access Key: 1_root_accid Secret Key: 0t1O0URz0H5pef6Wn6P6L9BKc8Ad Default Region [US]: Use "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3. S3 Endpoint [s3.amazonaws.com]: 127.0.0.1:9020 Use "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be used if the target S3 system supports dns based buckets. DNS-style bucket+hostname:port template for accessing a bucket [%(bucket)s.s3.amazonaws.com]: s3://%(bucket) Encryption password is used to protect your files from reading by unauthorized persons while in transfer to S3 Encryption password: Path to GPG program: When using secure HTTPS protocol all communication with Amazon S3 servers is protected from 3rd party eavesdropping. This method is slower than plain HTTP, and can only be proxied with Python 2.7 or newer Use HTTPS protocol [Yes]: no On some networks all internet access must go through a HTTP proxy. Try setting it here if you can't connect to S3 directly HTTP Proxy server name: New settings: Access Key: 1_root_accid Secret Key: 0t1O0URz0H5pef6Wn6P6L9BKc8Ad Default Region: US S3 Endpoint: 127.0.0.1:9020 DNS-style bucket+hostname:port template for accessing a bucket: s3://%(bucket) Encryption password: Path to GPG program: None Use HTTPS protocol: False HTTP Proxy server name: HTTP Proxy server port: 0 Test access with supplied credentials? [Y/n] y Please wait, attempting to list all buckets... Success. Your access key and secret key worked fine :-) Now verifying that encryption works... Not configured. Never mind. Save settings? [y/N] y Configuration saved to '/root/.s3cfg'
- Utilities such as the ubiquitous ‘dd’ CLI command are useful for easily and rapidly generating some test data files. Similarly, the ‘touch’ CLI command can be used to alter the ‘atime’ (last access) and ‘mtime’ (last modified) timestamps of these files. For example:
# dd if=/dev/zero of=/ifs/s3lifecycle/smlfl1 bs=1k count=1 # dd if=/dev/zero of=/ifs/s3lifecycle/smlfl2 bs=1k count=1 # touch -A -400000 /ifs/s3lifecycle/smlfl2* # dd if=/dev/zero of=/ifs/s3lifecycle/bigfl1 bs=1M count=1 # dd if=/dev/zero of=/ifs/s3lifecycle/bigfl2 bs=1M count=1 # touch -A -250000 /ifs/s3lifecycle/big* # dd if=/dev/zero of=/ifs/s3lifecycle/notrmv1 bs=1M count=1 # dd if=/dev/zero of=/ifs/s3lifecycle/notrmv2 bs=1M count=1 # ls -l /ifs/s3lifecycle/ total 4147 -rw------- 1 root wheel 1048576 May 13 13:48 bigfl1 -rw------- 1 root wheel 1048576 May 13 13:48 bigfl2 -rw------- 1 root wheel 1048576 May 14 14:48 notrmv1 -rw------- 1 root wheel 1048576 May 14 14:48 notrmv2 -rw------- 1 root wheel 1024 May 12 22:48 smlfl1 -rw------- 1 root wheel 1024 May 12 22:48 smlfl2 # date Thu May 14 14:49:26 GMT 2026 A total of six test files are created, two of which are 1 MB in size and have a modified time more than 1 day prior, two files that are 1 MiB in size but modified within 1 day, and two small 1 KB files with modification timestamps from 2 days prior.
- Next, copy the following text into a text file on the cluster. In the following example, the file is named /ifs/lifecycle.xml
<LifecycleConfiguration> <Rule> <Filter> <ObjectSizeGreaterThan>10000</ObjectSizeGreaterThan> </Filter> <Status>Enabled</Status> <Expiration> <Days>1</Days> </Expiration> </Rule> </LifecycleConfiguration>
Once the above XML request body has been added to the lifecycle.xml file, the ‘s3cmd’ can be run to ‘put’ the policy on the bucket. Once done, run a get lifecycle command to confirm that the policy has been applied correctly.
# python s3cmd setlifecycle /ifs/lifecycle.xml s3://s3life s3://s3life/: Lifecycle Policy updated onefs914-1# python s3cmd getlifecycle s3://s3life <?xml version="1.0" ?> <LifecycleConfiguration> <Rule> <Status>Enabled</Status> <Expiration> <Days>1</Days> </Expiration> <Filter> <ObjectSizeGreaterThan>10000</ObjectSizeGreaterThan> </Filter> </Rule> </LifecycleConfiguration>
- Next, compare the data from pre and post S3Lifecycle job runs:
# ls /ifs/s3lifecycle bigfl1 bigfl2 notrmv1 notrmv2 smlfl1 smlfl2 # isi job start S3Lifecycle Started job [25] # ls /ifs/s3lifecycle notrmv 1 notrmv l2 smlfl1 smlfl2
After the S3Lifecycle job is initiated and allowed to run for a short period, the results show that two of the larger files have been deleted while the remaining files remain unchanged. This behavior is expected and reflects the configured expiration and filter criteria defined in the XML policy, under which only objects larger than 10,000 bytes and older than one day qualify for automatic deletion.
If and when it comes to investigating and troubleshooting S3 lifecycle, the following issues and possible resolutions may be useful:
| Issue | Background Potential Resolution |
| Unable to create lifecycle policy on bucket | Bucket owner must be the same as directory owner |
| Objects marked for deletion have not been deleted | Ensure object or bucket does not have lock protection enabled |
| | Job for deletion has not been run |
| | Job for deletion is still in progress |
Beyond this, S3 operations are logged in the S3 log file ‘/var/log/s3.log’. Similarly, S3 Job Engine job operations and deleted objects are logged in ‘/var/log/isi_job_d_s3_lifecycle.log’.