Tag: Cluster

As part of Dell’s broader common architecture initiative, OneFS 9.13 introduces Dynamic Licensing for PowerScale, providing a unified and modernized licensing framework. Dynamic Licensing standardizes licensing mechanisms across Dell’s storage portfolio, streamlining customer interactions and reducing operational overhead through an automated, frictionless licensing experience. This approach is especially advantageous for customers operating multiple Dell Technologies products.

In OneFS 9.13, the legacy key‑based licensing model used in prior releases is replaced by Dynamic Licensing. The new system supports both direct‑connected and offline modes, allowing customers to manage license registration and entitlement reservations according to their connectivity capabilities while enabling real‑time entitlement updates and improved operational intelligence.

Key advantages of Dynamic Licensing include a shared licensing model across appliance, hybrid, and cloud deployments through a centralized reservation pool; near‑instant time‑to‑value with a cloud‑like experience; seamless mobility of licensed assets across clusters; reduced friction throughout the licensing lifecycle; and minimized serviceability requirements or support interventions through increased automation.

Dynamic Licensing supports two operational modes to accommodate varying customer environments:

The direct, or connected, model serves customers who can establish connectivity to Dell Technologies through Dell Technologies Connectivity Services (DTCS), enabling automated entitlement updates and dynamic allocation of software assets. The Offline model supports environments without external connectivity by allowing customers to manually register PowerScale instances through registration files uploaded directly to the system.

The current licensing method can be viewed from the WebUI under Cluster management > Dynamic licensing > Settings. For example, the following cluster which is still configured for ‘Offline’ licensing:

PowerScale’s integration with Dell Dynamic Licensing relies on several core operations exposed through backend API workflows. The Create Registration operation establishes a Common Software ID (cSWID) and associates the system with the appropriate reservation pool and entitlements. Update Registration adjusts licensing data in response to environmental changes such as cluster expansion, entitlement extensions, reductions, or try‑and‑buy scenarios. Get Registration performs periodic checks with the Dell Dynamic Licensing backend for entitlement updates. Send Consumption transmits usage telemetry to Dell and processes compliance notifications.

Deregistration removes a PowerScale instance from the reservation pool during cluster deprovisioning or uninstallation, ensuring proper lifecycle management of the associated components. As such, the focus of Dynamic Licensing is on delivering a frictionless licensing experience and reducing serviceability overhead and support requests through an automated, streamlined licensing workflow. Dynamic Licensing delivers the ability to activate licensing during connectivity provisioning for appliance-based deployments, and through License Activation Codes (LACs) for PowerScale for public cloud virtual instances. When a PowerScale cluster is upgraded to OneFS 9.13 or later, the system will automatically migrate to Dynamic Licensing if external connectivity is available. If the cluster becomes disconnected, XML‑based licensing continues to function as a fallback during the upgrade process.

The Dynamic Licensing framework supports entitlement expansion and reduction by registering or deregistering nodes as they are added to or removed from the cluster, and it also supports software capacity expansion through additional LACs. In the Direct model, the system automatically generates alerts whenever Dell Technologies Connectivity Services (ESE/DTCS) become unavailable. The cluster also generates alerts in cases of licensing compliance failures. Administrators have the ability to view complete license information—including expiration dates, licensed capacity, and associated entitlements—directly within the cluster interface.

The prerequisites for Dynamic Licensing require that the Direct model establishes connectivity through Dell Technologies Connectivity Services. Software‑only deployments require a License Authorization Code or valid order number for registration, while hardware‑based installations require node serial numbers to complete registration.

Architecturally, the OneFS Dynamic Licensing workflow is as follows:

Whenever a PowerScale cluster receives a dynamic licensing request from the OneFS WebUI or CLI, it is quarterbacked by its corresponding platform API (pAPI) method. In this way, dynamic licensing tasks are added in the transaction journal. The OneFS isi_rice_d connectivity service, which is part of the secure remote services (SRS), is responsible for picking up the task and assigning it to a task manager. Internally, the licensing state machine performs the job of registration, consumption update, registration and deregistration. This licensing state machine talks to Dell’s backend dynamic licensing library which is implemented on top of the dynamic licensing SDK. If the cluster is already connected via the Dell Technologies Connectivity Services (DTCS), then all requests to the dynamic licensing backend are routed through SRS. Alternatively, if the cluster is offline then OneFS makes the pAPI call directly. Once the response is received back from the dynamic licensing back end, the data is stored in the licensing section of OneFS gconfig.

OneFS 9.13 and later support switching between Dynamic Licensing modes, allowing transitions from connected (Direct) to disconnected (Offline) operation in the initial phase. Once Direct mode is established, the administrator may switch to Offline mode, at which point OneFS relies on the licensing data stored in gconfig—populated by the Direct workflows—and begins executing the Offline Dynamic Licensing processes. In this state, the system bases alerting on the local configuration and stops transmitting consumption information to the Dynamic Licensing backend. Transitioning back from the disconnected Offline mode to the connected Direct mode is also supported; once connected, the system resumes sending consumption telemetry and generating alerts based on backend compliance responses.

When changing the licensing model, a PowerScale cluster that maintains DTCS connectivity automatically converts to Dynamic Licensing during the upgrade workflow, specifically within the post‑upgrade commit phase. Disconnected clusters continue using the legacy XML‑based licensing model and are not migrated until connectivity is enabled. If ESE connectivity is present, Dynamic Licensing is automatically activated and cannot be disabled; if ESE connectivity is absent, the system continues using XML‑based licensing. License violation reporting remains consistent with current behavior: Direct‑mode systems generate alerts based on backend compliance reports, while Offline systems rely on the most recently uploaded static license file. Dynamic Licensing becomes fully active only after all nodes in the cluster complete the upgrade commit process; until that point, key‑based licensing continues to function.

The transition path to Dynamic Licensing varies by deployment type. Greenfield environments activate Dynamic Licensing immediately upon enabling ESE connectivity, or, in the absence of connectivity, operate in Offline mode using a manually uploaded JSON license file. Brownfield clusters with existing entitlements convert them as part of the Dynamic Licensing transition. Connected systems convert automatically once ESE connectivity becomes available, while disconnected systems continue using XML‑based licensing until connectivity is established or the administrator manually selects the Offline Dynamic Licensing mode through the WebUI or CLI. If Dynamic Licensing activation fails, the system notifies the administrator while maintaining a consistent user experience. In‑product activation through LACs continues to operate as it does today, and trial activation remains local to the cluster. If registration or backend communication fails at any point, the user receives a notification.

When upgrading to the latest OneFS release, connected PowerScale clusters automatically transition to Dynamic Licensing. Newly deployed connected clusters running OneFS 9.13 or later enroll in Dynamic Licensing without additional steps. Administrators retain visibility into trial and licensing data consistent with earlier releases, including expiration details, licensed capacity, and pre‑upgrade entitlement information. However, because of Dynamic Licensing’s design, per‑feature expiration dates no longer appear in the WebUI; instead, features are labeled ‘Licensed’ or ‘Unlicensed.’ Direct‑mode compliance continues to rely on backend feature‑specific expiration data, even though the WebUI abstracts this into license state indicators. Offline systems also hide per‑feature expiration dates and base compliance solely on the static license file or the entitlement information stored locally.

Customers may still view detailed per‑feature expiration data through the Software Licensing Central (SLC) portal. For disconnected environments, the licensing workflow generally mirrors earlier OneFS versions, except for the removal of visible per‑feature expiration dates. Trial functionality continues to operate independently of Dynamic Licensing, and in‑product activation via LACs behaves the same from a user‑experience perspective, with the addition of backend telemetry when connectivity is available. When customers add nodes or capacity, Dynamic Licensing ensures that new nodes are licensed automatically, assuming the appropriate entitlements exist, maintaining the experience of current LAC‑based workflows.

Additionally, admins are notified if the cluster becomes disconnected, prompting them either to restore connectivity or manage entitlements manually through the ‘offline mode. For cloud deployments, customers must provide a LAC and related parameters to activate licensing through a two‑step sequence in which connectivity is first established without full registration, followed by registration of the cluster with the Dynamic Licensing backend using the LAC.

Alerting behavior differs between modes. In Direct mode, alerts are driven by backend compliance responses for both feature term and capacity. In Offline mode, alerts rely solely on the capacity and entitlements defined in the static registration file uploaded to the cluster.

At a high level, the new functionality in OneFS 9.13 that allows PowerScale to take advantage of Dell Dynamic Licensing includes:

The Dell Licensing backend provides a Common Licensing Platform (CLP) for integrating licensing into each product, enabling a standard licensing solution across the Dell portfolio. This SDK relies on third-party utilities and libraries such as cjson, curl, libxml, openssl, and xmlsec which, with the exception of cjson, are available by default in OneFS, located under /usr/local/lib/.

Product registration registers the installed product instance with Licensing to get a unique Software ID (cSWID) by passing the product’s ‘model-id’ plus additional information.

Introduced as part of Dynamic Licensing, the cSWID is a combination of the legacy iSWID and eSWID. As such, it draws benefits from both, with its ability to automatically register with Dell licensing like an iSWID as part of product registration, and also link to commerce entitlements like an eSWID.

The type of identifier used for product registration depends on the cluster type:

The product registration process validates whether the customer has existing entitlements against the identifiers provided (i.e., Service Tag, Order Number, LAC) and then attempt to create a Reservation between the product’s Software ID and the entitlements related to the identifiers provided.

As mentioned previously, there are two product registration modes available: Direct and Offline. Each mode offers a distinct approach to registering products, catering to different customer needs and environments. Product registration is facilitated using the licensing library built on top of the Dynamic Licensing SDK.

With Direct mode, product registration occurs automatically through ESE as soon as the customer enables connectivity. In contrast, Offline Mode requires the customer to manually accesses the Dell Software Licensing Central (SLC) portal, select the product line, enter the LAC (TLA Agreement ID), and provide additional information (e.g., late-binding HW serial, node locking, etc.), then click ‘Register’ to obtain a file. The customer must upload the Dynamic Licensing file to activate the license on the PowerScale cluster, making this mode suitable for dark sites, and other environments with limited internet connectivity.

Upon completing the registration process, the registration response, including the entitled state, is stored in the gconfig under the licensing tree, along with the licensing topology and other pertinent info.

# isi_gconfig -t licensing
[root] {version:1}
(empty dir licensing.features)
licensing.swid (char*) = <null>
ignore_signature (bool) = true
last_upgrade_commit_epoch (int64) = -1
# Additional Attributes
license_topology = "direct"
license_technology = "dynamic"

The product registration process migrates existing key-based installations to dynamic licensing for customers who wish to upgrade to the latest version of PowerScale. By leveraging the registration process, customers can seamlessly transition from traditional key-based licensing to the more flexible and scalable dynamic licensing model.

A Transformational License Agreement (TLA) is an enterprise-level licensing model that allows customers to purchase large volumes of capacity for an extended term, typically five years. Under a TLA, customers receive software-only entitlements that are not associated with specific hardware at the time of purchase. Instead, hardware binding occurs later in the process during license registration and activation, when Service Tags are applied. A single registration request can support multiple TLA License Activation Codes (LACs).

TLA orders are characterized by their software-only nature, meaning entitlements are not tied to particular systems at the point of sale. Additionally, the late-binding model ensures that license entitlements are associated with hardware only during the registration and activation phase rather than at purchase time.

To register a TLA order, specific information is required, including the TLA License Activation Code, which uniquely identifies the license for activation purposes, and the relevant Service Tags, which are used to bind the entitlements to the appropriate hardware during the late-binding process.

The Dynamic Licensing system periodically conducts a License Assessment to compare a customer’s usage of Dell products with their allocated entitlements. If the assessment reveals any Compliance Failures, it will trigger an event/alert. Once registered with the Dell Dynamic Licensing backend, the PowerScale cluster transmits consumption data to the Dell backend at regular intervals. Specifically:

Consumption data is collected hourly and transmitted to the Dell Dynamic Licensing backend on a daily basis (for hardware clusters). For capacity-based features associated with a tier, the reported used capacity represents the aggregate of all nodes linked to that tier. In contrast, for features licensed on a base-only model, such as CloudPools, OneFS hardening, and HDFS, the quantity reported is the actual number of nodes. The license consumption call response provides the current state of the reservation pool, including existing entitlements. PowerScale processes compliance events based on consumption responses from Dynamic Licensing (DL), generating alerts for licensing violations and clearing them once violations are resolved. After processing compliance results, PowerScale acknowledges the audit back to the Dynamic Licensing APIs, completing the audit cycle.

Cluster that are running in Offline mode have their license assessment conducted against a customer-uploaded JSON license file. The License Assessment is evaluated based on capacity usage, not on a term basis.

So, as we’ve seen, Dynamic Licensing in OneFS 9.13 replaces the legacy, key‑based licensing model used in prior releases. In the next article in this series, we’ll turn our attention to its configuration and use.

Among the array of features and functionality which debut in the OneFS 9.12 release lies One-Click Upgrade. This feature simplifies OneFS software lifecycle management on a PowerScale cluster by enabling automated package discovery, download, and installation via the WebUI, providing a seamless upgrade experience.

When Dell Connectivity Services (DTCS) are enabled and the cluster registered via its unique software ID, telemetry data is securely transmitted to the Dell support backend. This enables the Cluster Update Platform (CUP) to determine the appropriate upgrade packages for each cluster.

Based on this telemetry, CUP generates a manifest containing package metadata, which is distributed by the Converged Management System (CMS) via Dell Managed File Transfer (MFT) to the Embedded Service Enabler (ESE) component on the target cluster.

Under the hood, the one-click upgrade feature’s architecture and workflow is as follows:

The upgrade package is staged on the MFT locker for retrieval. Upon receiving the manifest, ESE invokes the OneFS platform API (pAPI) to forward the data to the ‘Rice’ service. The rice daemon parses the manifest, stores package details in the local database, and exposes them to the WebUI.

As such, cluster admins can view available packages, including version, release stage, and firmware options, and initiate downloads or installations directly from the WebUI.

OneFS 9.12 supports optional auto-download policies, allowing customers to select preferences such as latest release or LTS versions. Installation is triggered with a single click, leveraging the isi upgrade workflow to orchestrate node reboots and apply updates. This process is as follows:

1. Configure Dell Connectivity Services (DTCS) using the WebUI wizard located under Cluster Management > General Settings > Connectivity Services:

2. Configure the download settings, located under Cluster Management > Upgrade > Settings:

3. Enable or disable one-click upgrade functionality:

# isi upgrade package setting modify --enable-subscription <true|false>

Once enabled, one-click upgrade can be easily invoked by selecting the desired release version and clicking on the ‘Start Update’ button under Cluster Management > Upgrade > Overview:

4. Upgrade status will be visually communicated in the WebUI via the completion status bar under Cluster Management > Upgrade > Overview:

5. Once completed, a banner similar to the following will be displayed by the WebUI under Cluster Management > Upgrade > Overview, indicating that the upgrade was successful. When ready, the upgrade can then be committed by clicking on the ‘Commit Upgrade’ button:

Post-upgrade, users can commit or roll back changes through the same interface. If there’s an anomaly or issue, investigation and troubleshooting can be performed from the CLI with the following commands.

To verify the Connectivity service status:

# isi connectivity view

To check the rice service log file:

# cat /var/log/isi_rice_d.log

To view the check upgrade log:

# isi upgrade view

# isi_upgrade_logs

The following tables presents some potential solutions for common issue resolution.

In summary, the one-click upgrade architecture in OneFS 9.12 and later ensures secure, telemetry-driven package delivery, minimizes manual intervention, helping to accelerate upgrade cycles while maintaining cluster integrity.

In the spirit of seamless cluster upgrade experience, OneFS 9.12 and later releases enable automatic migration of customized FTP settings during upgrade. This helps avoid potential customer impact caused by lost custom configurations when up-revving a PowerScale cluster.

Prior to OneFS 9.12, edits to the FTP daemon’s template file, vsftpd.conf, were not persistent across OneFS upgrades. As such, any custom configuration changes made by customers to PowerScale’s default supported FTP configuration would be overwritten during a cluster code upgrade operation. This necessitated the manual migration and replacement of the vstftpd.conf file post-cluster upgrade.

Such a situation could have potentially resulted in disruptions to SFTP workflows for a period while the old custom configuration file was retrieved and copied back into place.

This new durable configuration functionality in OneFS 9.12 and later is designed to enhance the customer upgrade experience by ensuring that customized FTP settings are automatically migrated during system upgrades. The primary objective is to prevent disruptions that could occur if custom configuration data is overwritten.

Currently, any changes made to temporary configuration files—such as those supporting customized SFTP settings—are lost during an upgrade. These settings are often essential for workflows that involve scaled content support.

Under the hood, a dedicated template file is introduced to store customized settings, ensuring they are not overwritten during upgrades. Additionally, new migration logic is implemented which transfers existing custom configurations into this new template during the upgrade process.

Architecturally, upgrade hooks detect any customized settings and migrate them into the new template file. Additionally, the configuration generation process is enhanced such that the final VSFTP configuration is an amalgam of the original template plus the new custom template.

The FTP upgrade workflow in OneFS 9.12 and later consists of the following components:

First, the pre-upgrade phase identifies any customized settings. Next, during the post-upgrade phase, these settings are saved into the new template file and applied. As a precaution, a backup of all relevant files is also created. The pre-check phase prevents customers from adding unsupported files to the preserved configuration set. Furthermore, a health check is introduced to detect modifications to temporary files and issue appropriate warnings if changes could potentially be lost during downgrade or upgrade. Details of this new FTP configuration healthcheck include:

The above healthcheck can be viewed in the WebUI under Cluster management > HealthCheck > HealthChecks:

Or to run the healthcheck:

Similarly, from the CLI:

# isi healthcheck items view check_vsftpd_template_changes

              Name: check_vsftpd_template_changes

           Summary: Checks whether /etc/mcp/templates/vsftpd.conf is synchronized across all nodes in the cluster or no customized changes are found in it. And vsftpd.conf should not be

                    included in the user_preserve_files.xml on OneFS version 9.12.0.x and above.

             Scope: Per node

         Freshness: Now

        Parameters: -

       Description:   The possible outputs of this item are:

                    * OK: The /etc/mcp/templates/vsftpd.conf file is synchronized on all nodes in the cluster and no customized changes are found.

                    * WARNING: File /etc/mcp/templates/vsftpd.conf is included in the /etc/mcp/override/user_preserve_files.xml unexpectedly, The upgrade process will fail if target version is 9.12.0.x or above. Please consider to remove it from the /etc/mcp/override/user_preserve_files.xml file.

                    * CRITICAL: Please ensure /etc/mcp/templates/vsftpd.conf is synchronized and avoid modifying template files. File /etc/mcp/templates/vsftpd.conf should not be included in the /etc/mcp/override/user_preserve_files.xml if cluster version is 9.12.0.x or above. Please consider to remove it from the /etc/mcp/override/user_preserve_files.xml file.

                    * UNSUPPORTED: Unsupported OneFS version to check template conf file: /etc/mcp/templates/vsftpd.conf.

        Resolution:   Customized changes may be lost if the cluster are upgrading to 9.11.0.x or lower. Please consider preserve this file and re-apply the customized changes after upgrade. If upgrade target version is 9.12.0.x or above, consider saving customized changes in /etc/mcp/templates/vsftpd_custom.conf

Repair Description: -

    Repair enabled: No

   Repair behavior: -

       Repair risk: -

     Repair script: -

Repair script type: -

For issue investigation and troubleshooting, first review the upgrade logs, which include details of hook execution. Additionally, check the VSFTP logs for configuration generation and template comparison, as well as the project upgrade logs to identify any migration anomalies.

One of the object protocol enhancements introduced in OneFS 9.12 is the S3 Data Protection Mode (DPM). This feature integrates OneFS Multi-Party Authorization (MPA) with S3 data services to deliver stronger security controls, reducing risks associated with both accidental errors and malicious actions.

In the previous article in this series, we looked at the configuration and management of OneFS S3 Data Protection Mode from the WebUI. Now, we’ll turn our attention to the usage of DPM from the S3 API, plus some issue investigation and troubleshooting options.

In the following example, ‘dpmtest’ has been created and configured as an immutable S3 bucket, with its lock protection mode configured to ‘Bucket Lock and its retention period currently set to ‘10 days’:

Or from the CLI:

# isi s3 buckets list

Bucket Name  Path                Owner  Object ACL Policy  Object Lock Enabled  Lock Protection Mode  Description

-------------------------------------------------------------------------------------------------

s3bucket     /ifs/data/dpmtest   root   replace            Yes                  Bucket Lock

s3buckettgt  /ifs/data/target    root   replace            No                   -

-------------------------------------------------------------------------------------------------

Using the S3 API endpoints, the following example shows the DPM auto-creation method, with a ‘PUT’ request to reconfigure the access logging for a bucket, creative named named ‘dpmtest’.

PUT  http:// {{S3_ENDPOINT}} /dpmtest/?logging

<BucketLoggingStatus>

      <LoggingEnabled>

            <TargetBucket>target2</TargetBucket>

            <TargetPrefix>newlog</TargetPrefix>

      </LoggingEnabled>

</BucketLoggingStatus>

The response includes an HTTP 403 error code notifying that the ‘modify_server_access_login_config’ action request is pending approval, plus the unique ID number of the associated MPA request:

403 Forbidden

<?xml version=”1.0” encoding=”UTF-8”?>

<Error>

      <Code>AccessDenied</Code>

      <Message>AccessDenied</Message>

      <Resource></Resource>

      <RequestId>675969624</RequestId>

      <MpaRequestId>paareqf2c58360e0ce84cd</MpaRequestId>

      <DpmMessage>Action  ‘modify_server_access_login_config’  pending approval  -  request [paareqf2c58360e0ce84cd]  submitted.</DpmMessage>

Similarly, an S3 auto-creation method to reduce retention for the ‘dpmtest’ immutable bucket from 10 days to 2 days:

PUT  http:// {{S3_ENDPOINT}} /dpmtest/?object_lock

<ObjectLockConfiguration>

      <ObjectLockEnabled>Enabled</ObjectLockEnabled>

      <Rule>

            <DefaultRetention>

                  <Mode>GOVERNANCE</Mode>

                  <days>2</Days>

            </DefaultRetention>

</Rule>

</ObjectLockConfiguration>

Followed by a similar MPA approval pending HTTP 403 error response:

403 Forbidden

<?xml version=”1.0” encoding=”UTF-8”?>

<Error>

      <Code>AccessDenied</Code>

      <Message>AccessDenied</Message>

      <Resource></Resource>

      <RequestId>675969625</RequestId>

      <MpaRequestId>paareqc00c7e920e0ce8413</MpaRequestId>

      <DpmMessage>Action  ‘reduce_immutable_bucket_retention’  pending approval  -  request [paareqc00c7e920e0ce8413]  submitted.</DpmMessage>

After logging in as an MPA approver, pending MPA requests can be viewed and approved via the WebUI under Access > Multi-Party Authorization > Requests, and clicking on the pertinent ‘Approve’ buttons. For example:

The approver(s) uses their preferred TOTP authenticator application to generate a security code:

This security code is then added to the approval request, plus and additional comment and expiration time, if desired:

Once approved, the WebUI displays the success banner, and reports the current request status. In this case, one request approved and one still pending:

One caveat to be aware of is that MPA requests are service-sensitive. This means that, for, MPA treats the ‘S3’ and ‘platform’ services as separate requests, even if all their other fields are identical. For example:

This distinction is important when users create MPA requests manually. For example, if a user creates a request and selects the ‘S3’ service, approval of that request will only allow privileged actions through the S3 API. The user will not be able to perform those actions via the platform API (or OneFS WebUI or CLI). The same applies in reverse, where requests created for platform services will only work through their respective APIs.

When investigating or troubleshooting S3 DPM issues, the preferred course of action is typically to enable and increase the verbosity of both the platform API log and S3 log. This can be done via the CLI with the following command syntax:

# isi_ilog -a papi --level debug+

# /usr/likewise/bin/lwsm set-log-level s3 - debug

Once done, check both isi_papi.log and s3.log for pertinent MPA and DPM debug messages.

One of the object protocol enhancements introduced in OneFS 9.12 is the S3 Data Protection Mode (DPM). This feature integrates OneFS Multi-Party Authorization (MPA) with S3 data services to deliver stronger security controls, reducing risks associated with both accidental errors and malicious actions.

In the previous article in this series, we looked at the architecture and underpinnings of OneFS S3 Data Protection Mode. Now, we’ll turn our attention to the configuration and management of DPM.

From the OneFS WebUI, the S3 bucket configuration can be found under Protocols > S3 > Buckets. In the following example, ‘dpmtest’ has been created and configured as an immutable S3 bucket, with its lock protection mode configured to ‘Bucket Lock’:

Clicking on the adjacent ‘View/Edit’ button reveals that its retention period is currently set to ‘100 days’, and access logging is ‘enabled’:

Next, the retention period is reduced to 50 days, and access logging is disabled:

Or from the CLI:

# isi s3 buckets modify dpmtest –-default-retention-days 50 –-access-loging 0

Since there are both S3 protected actions, the WebUI displays the following warning banner, notifying that the actions are paused pending approval, and recommending checking the MPA request status:

After logging into the WebUI with an MPA approver account and navigating to Access > Multi-Party Authorization > Requests, the two above S3 privileged action requests are displayed, one for reducing the immutable bucket retention period and the other for disabling server access logging:

Or from the CLI:

# isi mpa requests list

The approver(s) then use their time-based one-time password (TOTP) authenticator application to generate a security code: 

This TOTP code is then entered into the approval request’s ‘security code’, plus an additional comment and expiration time, if desired:

Or via the CLI:

# isi mpa requests list

# isi mpa requests approve <id> <comment> <approved> --totp-code <******> --approval-valid-before <timestamp> --zone <zone>

Once approved, the WebUI displays the success banner, and reports the current request status. In this case, one request approved and one still pending:

Or from the CLI:

# isi mpa requests list

Moving on to the pending ‘disable access logging’ request:

Or from the CLI:

# isi mpa requests view <id>

After approving the above request in the same way, the MPA status shows both requests successfully approved:

Or from the CLI:

# isi mpa requests list

Reviewing the request details from the S3 bucket status page under Protocols > S3 > Buckets > Requests, reveals that the bucket retention period has been successfully reduced to ’50 days’ and access logging disabled, as expected:

Or from the CLI:

# isi s3 buckets view dpmtest

In the next article in this series, we’ll turn our attention to the usage of DPM from the S3 API, plus some issue investigation and troubleshooting options.

Tucked among the object protocol enhancements that were introduced in OneFS 9.12 lies S3 data protection mode (DPM). DPM integrates OneFS Multi-Party Authorization (MPA) into the S3 data services to provide enhanced security control, mitigating risks caused either by mistakes or by malicious intent.

In OneFS 9.12 and later, the DPM functionality is not independently configurable. Rather, it is implicitly enabled when MPA is activated. As such, once MPA is enabled, all S3 privileged actions need to be approved before executed. The MPA activation and configuration process is covered in-depth within the following article:   http://www.unstructureddatatips.com/onefs-multi-party-authorization-configuration

The two primary DPM capabilities that are introduced in OneFS 9.12 are:

Reducing the retention period for an immutable bucket is only supported when the bucket is in governance mode. An immutable bucket is defined as a bucket with its lock-protection mode set to ‘BucketLock’.

For S3 server logs, disabling or reconfiguring the target bucket or target prefix is protected by DPM. All such reconfiguration operations are treated as a single MPA request and are not distinguished individually.

The MPA level should be set to the bucket protection level. This is because any modification to an object is governed by the immutable bucket’s retention period, so there is no need for separate MPA approval.

S3 DPM takes effect when both MPA and S3 service are enabled on a cluster running OneFS 9.12 or later, and the specific S3-related MPA privileged actions that are supported include:

Under the hood, the S3 DPM workflow can be represented as follows:

As such, the basic flow involves the following steps:

1. First, the client initiates a protected action, either from posting an S3 API or platform API request, or a CLI or WebUI action.
2. Next, OneFS checks whether MPA is enabled.
3. If MPA is disabled, the privileged action executes directly without the protection of DPM.
4. If MPA is activated, a request is auto-generated via ‘isi_mpa_common’.
5. If the request is approved, the operation proceeds. If not approved or pending, the requesting user will receive a notification with an HTTP 403 response code.

DPM auto-creation privileged actions can be configured through WebUI under Protocols > Object Storage (S3) > Buckets.

When a privileged action is not approved for a s3 user. MPA requests can be created manually through editing the bucket configuration. The MPA requests will be created automatically if the internal system found there is not an approved request in system.

Users can also generate a privileged action request manually via the WebUI under Access > Multi-Party Authorization > Requests:

For example, to create a ‘Platform’ service request to reduce the retention for an immutable bucket to 2 days:

In the next article in this series, we’ll take a closer look at the configuration and management of S3 Data Protection Mode.

Dell PowerScale ushers in the holiday season with the release of OneFS 9.13, launched on December 16, 2025. This latest version introduces comprehensive enhancements across security, serviceability, synchronization, and protocol support, reinforcing PowerScale’s position as a unified software platform for both on-premises and cloud deployments.

OneFS 9.13 is designed for a broad range of workloads, including traditional file shares, home directories, and vertical applications such as media and entertainment, healthcare, life sciences, and financial services, as well as emerging use cases in generative AI, machine learning, deep learning, and advanced analytics.

PowerScale’s scale-out architecture offers deployment flexibility across on-site environments, co-location facilities, and customer-managed instances in AWS and Microsoft Azure, delivering core-to-edge-to-cloud scalability and performance for unstructured data workflows.

Recognizing the critical importance of security and resilience in today’s threat landscape, OneFS 9.13 introduces advanced features to enhance data protection, monitoring, and availability.

Protocol Enhancements: The release includes significant improvements to the S3 object protocol, featuring fast-path functionality for optimized multipart upload completion times. Administrators can now reconfigure the S3 HTTPS port from the default TCP 443 to any port within the range of 1024–65,535 via WebUI or CLI, with warnings provided if port 443 is already in use by the PowerScale HTTP service.

Data Management: OneFS 9.13 delivers SmartSync enhancements for incremental file-to-object replication, supporting multiple policies per bucket and improved telemetry for cloud replication jobs. Cloud tiering capabilities are expanded through CloudPools URI support for Amazon VPC endpoints, enabling more secure and efficient cloud integration.

Security: To strengthen ransomware protection and secure data paths, OneFS 9.13 introduces TLS 1.3 support for HTTP transport within Apache-based services, including RESTful Access to Namespace (RAN), WebDAV, and WebHDFS components. TLS 1.3 is now enabled by default across these interfaces.

Usability: The release provides official support for the PowerScale SDK, offering tools, documentation, and code samples for developers to build applications that interact with PowerScale clusters. The SDK includes Python bindings for programmatic access to OneFS APIs, simplifying integration with both the Platform API (pAPI) and RESTful Access to Namespace (RAN).

Support and Licensing: OneFS 9.13 introduces Dell Dynamic Licensing, a modern licensing framework that eliminates manual activation, supports appliance, hybrid, and cloud deployments, and enables seamless asset movement between clusters without licensing friction. This replaces traditional key-based licensing for PowerScale.

Hardware Innovations: On the hardware front, OneFS 9.13 adds support for 400Gb Ethernet connectivity on the all-flash PowerScale F910 platform and introduces in-field backend NIC swap capabilities for previous-generation all-flash platforms, including F900, F600, and F200 storage nodes, plus the B100 and P100 accelerator nodes.

With these advancements, OneFS 9.13 delivers enhanced security, operational simplicity, and performance scalability, making PowerScale an ideal solution for organizations managing diverse and demanding unstructured data workloads across hybrid and multi-cloud environments.

In summary, OneFS 9.13 brings the following new features and functionality to the Dell PowerScale ecosystem:

We’ll be taking a deeper look at OneFS 9.13’s new features and functionality in future blog articles over the course of the next few weeks.

Meanwhile, the new OneFS 9.13 code is available on the Dell Support site, as both an upgrade and reimage file, allowing both installation and upgrade of this new release.

For existing clusters running a prior OneFS release, the recommendation is to open a Service Request with to schedule an upgrade. To provide a consistent and positive upgrade experience, Dell is offering assisted upgrades to OneFS 9.13 at no cost to customers with a valid support contract. Please refer to Knowledge Base article KB544296 for additional information on how to initiate the upgrade process.

As we saw in the prior article in this series, the auto-remediation capability introduced in OneFS 9.12 implements an automated fault recovery mechanism to enhance cluster availability and reduce administrative overhead. Leveraging the HealthCheck framework, the system continuously monitors for OneFS anomalies and triggers corrective workflows without operator intervention. Upon detection of a known failure signature, auto-remediation executes a predefined repair procedure—such as service restarts or configuration adjustments—to restore operational integrity. By orchestrating these recovery actions programmatically, the feature minimizes manual troubleshooting and reduces mean time to repair (MTTR).

Auto-remediation is disabled by default in OneFS 9.12 to ensure administrators maintain full control over automated repair operations. Cluster administrators have the ability to enable or disable this functionality at any time through system settings. When a cluster is upgraded to OneFS 9.12, the WebUI HealthCheck page provides a notification banner informing users of the newly available auto-remediation capability:

This banner serves as an entry point for administrators to activate the feature if desired. If selected, the following enablement option is displayed:

Checking the ‘Enable Repair’ box provides the option to select either ‘Auto-repair’ or ‘Manual Repair’ behavior, with ‘Manual’ being the default:

Selecting ‘Auto-Repair’ will allow OneFS to automatically trigger and perform remediation on regular healthcheck failures:

Or from the CLI:

# isi repair settings view

Repair Behavior: manual

Repair Enabled: No

# isi repair settings modify --repair-enabled True --repair-behavior Auto

# isi repair settings view

Repair Behavior: auto

 Repair Enabled: Yes

# isi services -a | grep -i repair

   isi_repair           Repair Daemon                            Enabled

Once enabled, auto-remediation can automatically trigger repair actions in response to HealthCheck failures, reducing the need for manual intervention and improving operational efficiency.

The following repair actions are currently available in OneFS 9.12:

These can also be viewed from the WebUI under Cluster management > HealthCheck > HealthChecks, and filtering by ‘Repair Available’:

This will display the following checklist, each of which contain ‘Repair Available’ healthchecks:

For example, the ‘basic’ checklist can be expanded to show its seven health checks which have the ‘repair available’ option. In addition to the description and actions, there’s a ‘repair behavior’ field which indicates a check’s repair type – either ‘auto’ or ‘manual’:

Under the hood, these repair actions are defined in the ‘/usr/libexec/isilon/repair-actions/mapping.json’ file on each node. For example, the repair action for the job engine, which has a ‘cluster’ level scope and ‘auto’ repair behavior:

"job_engine_enabled": {

            "script": "fix_job_engine_enabled.py",

            "script_type": "python",

            "enabled": true,

            "behavior": "auto",

            "risk": "low",

            "description": "This repair will enable isi_job_d service",

            "scope": "cluster"

        }

Note that any repair actions which are considered ‘high-risk’ will still need to be manually initiated, even with the repair behavior configured for ‘auto’. The ‘smartconnect_enabled’ repair action is an example of a high-risk action, so it has a ‘manual’ behavior type, as well as a node-level repair type, or scope, as can be seen in its definition:

“smartconnect_enabled”: {

      “script”: “fix_smartconnect_enable.py”,

      “script_type”: “python”,

      “enabled”: true,

      “behavior”: “manual”,

      “risk”: “high”,

      “description”: “This repair will enable the smartconnect service”,

      “scope”: “node”

},

The principal auto-remediation configuration, definition, and results files can typically be found in the following locations:

Auto-remediation is disabled when the PowerScale cluster operates in certain restricted modes. These include Compliance, Hardening, Read-Only, Upgrade Pre-Commit, and Root Lockdown Mode (RLM). These restrictions are in place to maintain security and compliance during critical operational states, ensuring that automated changes do not compromise system integrity.

Since auto-remediation is implemented as a new ‘isi_repair’ service within OneFS 9.12, the first step in troubleshooting any issues is to verify that the service is running and to check its operational status.

The service status can be checked using the following CLI command syntax:

# isi services -a isi_repair

Service 'isi_repair' is enabled.

Additionally, repair logs are available for review in the following location on each node:

/var/log/isi_repair.log

These utilities help to provide visibility into repair operations and assist in diagnosing issues related to auto-remediation.