Unstructured Data Quick Tips

The previous two articles in this series examined what Parallel NFS (pNFS) is, how it is implemented in OneFS 9.15, and how to enable, configure, and manage pNFS on a PowerScale cluster. The recurring theme was parallelism on the cluster side: By separating the metadata path from the data path, pNFS lets a single NFS client read and write file data directly and concurrently across many nodes, rather than funneling everything through the one node behind the mount IP. Every node in the SmartConnect pool can act as a Metadata Server (MDS) or a Data Server (DS), so the cluster presents a genuinely parallel data path.

However, there is a second half to that equation, and it resides with the pNFS client. While the data path can be opened across many nodes, if a Linux client sends every one of those data connections out of a single network interface, the client itself becomes the bottleneck.

In this final article of the series, we look at client-side NIC trunking, a lightweight, fully reversible technique that spreads a single pNFS mount’s data-server connections across two client NICs so that one mount can drive the aggregate bandwidth of both.

With pNFS flex-files, defined in RFC 8435 and carried over NFSv4.1 or NFSv4.2, a file system’s metadata path is separated from its data path. In a flex-files deployment, the PowerScale cluster fills two roles at once:

Role	Description
Metadata Server (MDS)	The IP address you mount. It handles namespace operations and issues data layouts to the client through the LAYOUTGET operation.
Data Server (DS)	The addresses the client reads and writes file data to directly. Every node in the network pool can act as a DS, so DS addresses span the entire pool range.

When the client performs I/O, the MDS returns a flex-files layout listing the DS addresses for the data, and the client opens transport connections directly to those addresses. The problem is what happens next on the client. By default, Linux resolves all of those DS addresses to a single egress interface (the route with the lowest metric wins), so every data connection leaves through one NIC. A client with two NICs attached to the same PowerScale data network therefore uses only half of its available bandwidth, while the second NIC sits idle.

The technique pins half of the pool’s addresses to each NIC using host-specific (/32) routes. Data-server connections then fan out across both interfaces, so a single mount can drive the aggregate bandwidth of both NICs. The approach works identically for RDMA (RoCE/InfiniBand) and TCP; the RDMA-only steps are called out where they apply. Importantly, all configuration is performed on the Linux client, and the cluster requires no change beyond having NFS over RDMA and pNFS flex-files enabled on the export.

Consider a Linux client with two NICs sitting on separate subnets, each with its own gateway to a PowerScale cluster’s front-end network. Host routes split the pool so that, in our example, IP addresses .81 to .100 egress NIC 1 and .101 to .120 egress NIC 2. Whichever PowerScale cluster node answers the mount request acts as the MDS, and the flex-files layouts it returns still reference DS addresses drawn from across the whole pool, so both NICs are exercised once the routes are split evenly.

One caveat is worth noting up front: Split the data pool you actually mount through, not the management pool, and not a different data pool. OneFS returns roughly one DS address per node drawn from the pool your mount uses. If the cluster has several data pools on different ranges, mounting through one pool returns only that pool’s DS addresses, so splitting a different range accomplishes nothing.

Prerequisites

Before making any changes, confirm that all the following client, cluster, and networking prerequisites are met.

On the Linux client

Client Prerequisite	Details
Operating system	A recent enterprise Linux distribution with NFS/RDMA, pNFS flex-files, and nconnect support, for example RHEL 8.x/9.x, Ubuntu 20.04+, or SLES 15 SP3+.
Installed packages	The ‘rdma-core’ and ‘nfs-utils’ packages installed.
Network interfaces	Two client NICs, each with its own IP address and a working route to the cluster data subnet.
Access control	Root (or sudo) access.
RDMA-only configuration	Confirm each NIC’s RDMA device mapping with rdma link show (for example, mlx5_0 maps to eth0).

On the PowerScale cluster:

Cluster Prerequisite	Details
OneFS version	OneFS 9.15 or later.
Network interfaces	RDMA-capable front-end interfaces.
NFS configuration	NFS over RDMA enabled and pNFS flex-files enabled on the export.

Verify that NFS over RDMA is enabled on the cluster:

# isi nfs settings global view | grep -i rdma

On the network:

Network Prerequisite	Details
Transport	A RoCEv2 fabric with lossless Ethernet (PFC/ECN) configured end-to-end.
Routing	One subnet per client NIC is recommended for clean routing, with a reachable gateway on each NIC’s subnet toward the cluster pool..

The commands throughout this article use the example values below. Substitute your own environment’s IP addresses and paths consistently, as appropriate.

The pool is split in half: For example, .81 to .100 via NIC 1, and .101 to .120 via NIC 2.

Item	Value	Notes
Cluster data subnet	192.168.100.0/24	Where the NFS pool lives
Network pool range	192.168.100.81 to .120  (40 IPs)	Any one IP is the mount target; all are potential DS IPs
NIC 1	eth0, 10.10.1.11, gw 10.10.1.254, mlx5_0	First client NIC
NIC 2	eth1, 10.10.2.11, gw 10.10.2.254, mlx5_1	Second client NIC
Export path	/ifs/data	NFS export on the cluster
Mount point	/mnt/pscale	Local directory on the client

The following NFS mount options are used throughout:

Option	Value	Purpose
proto	rdma	Use RDMA transport (use tcp for the TCP variant)
port	20049	Standard NFS-over-RDMA port
vers	4.1	NFSv4.1, required for pNFS flex-files
rsize / wsize	1048576	1 MB I/O buffers (a good default for RDMA)
nconnect	4	Parallel connections per DS IP; start at 4 and raise only if a single DS connection bottlenecks

Deployment procedure

The PowerScale pNFS NIC trunking deployment procedure comprises eight steps, of which all are performed on the Linux client, except Step 1, which queries the cluster.

The recommended path is to validate first with temporary routes (Step 3, Option A), confirm the split is working (Steps 4 through 8), and only then make the routes permanent (Step 3, Option B). Specifically:

Step 1 – Confirm the cluster pool

On the cluster, examine the SmartConnect static network pool to obtain the exact IP range and gateway that the client will route across its two NICs:

# isi network pools list

# isi network subnets list

You will mount one address from this range (or the pool’s SmartConnect DNS name, which resolves to one of them) and route the entire range across your two NICs. Use the NFS data pool, not the management pool. A management or pool0 address may answer ping and even accept a TCP connection on port 2049, but mounts through it hang and never produce data layouts.

Step 2 – Back up the current network configuration

Before changing anything on the Linux pNFS client, capture the runtime routing state so you can restore it (all distributions):

mkdir -p /root/net-backup-$(date +%F)

ip route show > /root/net-backup-$(date +%F)/routes.txt

ip rule show  > /root/net-backup-$(date +%F)/rules.txt

Also back up the persistent network configuration for your distribution. This is only required if using Option B (permanent routes) in Step 3. For NetworkManager (RHEL, Rocky, SLES), profiles live in /etc/NetworkManager/system-connections/ (keyfile) or /etc/sysconfig/network-scripts/ (ifcfg); for netplan (Ubuntu), back up /etc/netplan:

# Ubuntu / netplan example

cp -a /etc/netplan /root/net-backup-$(date +%F)/

Step 3 – Add the split routes

Add a host-specific (/32) route for every pool address, sending half to each NIC. There are two ways to accomplish this, so select one of the following methods as appropriate:

Option A: Temporary Configuration (runtime only, for testing).

These routes take effect immediately but to not persist a reboot, which makes them ideal for validating the configuration before committing. Run as root on the Linux client:

# .81–.100 via NIC 1

for id in {81..100}; do

  ip route replace 192.168.100.$id via 10.10.1.254 dev eth0 src 10.10.1.11

done

# .101–.120 via NIC 2

for id in {101..120}; do

  ip route replace 192.168.100.$id via 10.10.2.254 dev eth1 src 10.10.2.11

done

ip route flush cache

The ‘src’ attribute above pins the connection’s source address to the egress NIC. It is required for RDMA (the RoCE source GID must belong to the egress device) and keeps the path symmetrical. The commands are safe to re-run because the ‘ip route replace’ command is idempotent,.

Option B: Permanent Configuration.

Store the routes declaratively so the network stack reapplies them on boot or NIC bounce. Do not combine methods for the same addresses.

NetworkManager (RHEL, Rocky, SLES): add the routes with the ‘nmcli’ CLI command, using the connection name (from ‘nmcli connection show’), which often differs from the device name:

nmcli connection show    # note the NAME column (may differ from DEVICE)

# NIC 1 connection

for id in {81..100}; do

  nmcli connection modify eth0 \
+ipv4.routes "192.168.100.$id/32 10.10.1.254 src=10.10.1.11"

done

# NIC 2 connection (name may be e.g. "Wired connection 1")

for id in {101..120}; do

  nmcli connection modify "Wired connection 1" \
+ipv4.routes "192.168.100.$id/32 10.10.2.254 src=10.10.2.11"

done

nmcli connection up eth0

nmcli connection up "Wired connection 1"

Note that the ‘nmcli connection up’ command briefly bounces the interface and drops any SSH session running over that NIC. Run the connection activation from the console/BMC, or detached (for example, wrap it in ‘nohup … &’), and do not chain the two commands with ‘&&’ over a live SSH session on the first NIC.

For Ubuntu linux using the ‘netplan’ CLI command, add routes under each interface in the netplan YAML, then run the ‘sudo netplan apply’ command. A route placed under an interface egresses that interface, and uses its IP as the source automatically, so no explicit ‘src’ is required. Validate with the ‘sudo netplan generate’ command before applying:

network:

  version: 2

  ethernets:

    eth0:

      routes:

        - to: 192.168.100.81/32

          via: 10.10.1.254

        # ... one entry per .81–.100

    eth1:

      routes:

        - to: 192.168.100.101/32

          via: 10.10.2.254

        # ... one entry per .101–.120

An alternative, Linux OS-agnostic option uses the ‘systemd oneshot’ service. For mixed client fleets, or when a single approach across all hosts is preferred, this small oneshot service replays the ‘Option A’ commands at boot. It is the most transparent method, and the safest to apply remotely, because it never reconfigures a NIC’s address and so never bounces a link:

# /usr/local/sbin/nfs-trunk-routes.sh

#!/bin/bash

set -e

for id in {81..100}; do

  ip route replace 192.168.100.$id via 10.10.1.254 dev eth0 src 10.10.1.11

done

for id in {101..120}; do

  ip route replace 192.168.100.$id via 10.10.2.254 dev eth1 src 10.10.2.11

done

ip route flush cache



# /etc/systemd/system/nfs-trunk-routes.service

[Unit]

Description=Split NFS pool routes for NIC trunking

After=network-online.target

Wants=network-online.target

[Service]

Type=oneshot

RemainAfterExit=yes

ExecStart=/usr/local/sbin/nfs-trunk-routes.sh

[Install]

WantedBy=multi-user.target



chmod +x /usr/local/sbin/nfs-trunk-routes.sh

systemctl daemon-reload

systemctl enable --now nfs-trunk-routes.service

systemctl status nfs-trunk-routes.service --no-pager  # expect: active (exited)

An ‘on-link pool’ (no gateway) is a special case. If the cluster pool is in the same subnet as the client NICs, shown by the ‘ip route show’ CLI command as a proto kernel scope link route with no via, omit the gateway (for example, ‘ip route replace 192.168.100.81 dev eth0 src 10.10.1.11’, or, in netplan, use ‘scope: link’ with no via). When the pool is reached through a router, keep the gateway.

Step 4 – Verify the split

On the client, confirm that the boundary addresses resolve to the expected interfaces:

ip route get 192.168.100.81    # expect: dev eth0  src 10.10.1.11

ip route get 192.168.100.100   # expect: dev eth0  src 10.10.1.11

ip route get 192.168.100.101   # expect: dev eth1  src 10.10.2.11

ip route get 192.168.100.120   # expect: dev eth1  src 10.10.2.11

Or check the whole pool at once:

for id in {81..120}; do ip route get 192.168.100.$id | head -1; done

Using this example configuration, addresses .81 to .100 should all show eth0, and .101 to .120 should all show eth1.

Step 5 – Connectivity check (optional)

A quick reachability check before mounting. Skip it if you prefer to go straight to Step 6, since the mount plus the verification in Steps 7 and 8 is the definitive end-to-end test:

# RDMA only: both devices should be ACTIVE / LINK_UP

rdma link show

# reach one IP from each half over its intended NIC

ping -c2 -I eth0 192.168.100.81      # path via NIC 1

ping -c2 -I eth1 192.168.100.101     # path via NIC 2

A ping failure on one NIC means that path is down, so check link state, switch/VLAN configuration, and gateway reachability (for example, ‘arping -I eth1 10.10.2.254’).

Step 6 – Mount

Mount one pool address as the MDS, using nconnect to set the number of connections opened per DS address. For RDMA:

mount -v -o proto=rdma,port=20049,vers=4.1,rsize=1048576,wsize=1048576,nconnect=4 \
192.168.100.81:/ifs/data /mnt/pscale/

For the TCP variant, replace ‘proto=rdma,port=20049’ with ‘proto=tcp’. The mount confirms the source address it selected, and the reported ‘clientaddr’ should match your NIC 1 IP (10.10.1.11). Immediately after mounting, and before any I/O, only the MDS address appears in the transport table, which is expected, because DS connections are created once I/O triggers LAYOUTGET.

To remount automatically on boot, add an ‘fstab’ entry on the client (_netdev’ defers the mount until the network is up):

192.168.100.81:/ifs/data  /mnt/pscale  nfs4  \
proto=rdma,port=20049,vers=4.1,rsize=1048576,wsize=1048576,nconnect=4,_netdev  0  0

Step 7 – Verify trunking: transport connections

Generate a small amount of I/O (for example, a ‘find’, or a short ‘fio’ or ‘dd’ job) to trigger layout retrieval, then count the transport connections per address:

grep -h 'addr:' /sys/kernel/debug/sunrpc/rpc_xprt/*/info \

  | awk '{print $2}' | sort -t. -k4 -n | uniq -c

Healthy trunking shows every participating address with ‘nconnect’ (here, 4) connections: the MDS plus DS addresses from both halves of the pool:

  4 192.168.100.81     <- MDS (NIC 1 range)

  4 192.168.100.90     <- DS  (NIC 1 range)

  4 192.168.100.101    <- DS  (NIC 2 range)

  4 192.168.100.118    <- DS  (NIC 2 range)

  ...

If only .81 to .100 appear, the NIC 2 routes are wrong, so recheck Step 4. Entries for the cluster management address (port 2049) and the local rpcbind socket are unrelated to data I/O and can be ignored.

Step 8 – Verify trunking: RDMA queue pairs (RDMA only)

For RDMA configurations, confirm that queue pairs (QPs) exist on both RDMA devices:

rdma resource show qp link mlx5_0    # NIC 1

rdma resource show qp link mlx5_1    # NIC 2

Both devices must return QPs, and each should carry roughly half of the total. For a 40-address pool with nconnect=4, that is about 20 × 4 = ~80 QPs per device. QPs on only one device mean all traffic is on one NIC, so recheck the routes for the other half. If you validated with temporary routes (Option A) and everything checks out, return to Step 3, Option B to make the routes permanent.

Rollback and restore

To return to the original configuration, roll back using the same method you applied (replace the backup date with the date of your Step 2 backup directory).

Temporary routes (Option A, all distributions): delete the host routes, or simply reboot:

for id in {81..120}; do ip route del 192.168.100.$id; done

ip route flush cache

NetworkManager: remove exactly the routes you added, using -ipv4.routes:

# NIC 1 connection

for id in {81..100}; do

  nmcli connection modify eth0 \
-ipv4.routes "192.168.100.$id/32 10.10.1.254 src=10.10.1.11"

done

# NIC 2 connection (use the same NAME you used to add the routes)

for id in {101..120}; do

  nmcli connection modify "Wired connection 1" \
-ipv4.routes "192.168.100.$id/32 10.10.2.254 src=10.10.2.11"

done

nmcli connection reload

nmcli connection up eth0

nmcli connection up "Wired connection 1"

Do not rely on restoring the keyfile backup alone: If a NIC had no saved profile when you backed up, Option B created a new keyfile for it, and a plain cp -a restore never deletes that new file, so those routes survive the rollback. Use the -ipv4.routes method above, then verify none remain with ‘ip route show | grep /32’.

Netplan (Ubuntu): Restore the backed-up netplan, then apply. If you added a separate file, delete it first (the backup will not contain it):

rm -f /etc/netplan/90-pnfs-trunking.yaml   # only if you created a separate file

cp -a /root/net-backup-YYYY-MM-DD/netplan/* /etc/netplan/

sudo netplan apply

 

Systemd service (all distributions): Disable the unit, remove its files, then clear the live routes (or reboot):

systemctl disable --now nfs-trunk-routes.service

rm -f /etc/systemd/system/nfs-trunk-routes.service \

      /usr/local/sbin/nfs-trunk-routes.sh

systemctl daemon-reload

for id in {81..120}; do ip route del 192.168.100.$id; done

ip route flush cache

Troubleshooting

Symptom	Likely cause	Resolution
Ping or mount hangs on one NIC’s path	No working path via that NIC’s gateway	Check link, switch/VLAN, and gateway (arping -I eth1 10.10.2.254)
Mount hangs though ping and port are reachable	Mounting the wrong pool (management, not data)	Confirm the IP is in the NFS data pool (isi network pools list); mount a data-pool IP
Only NIC 1 addresses in the transport table	NIC 2 host routes missing or wrong	Recheck ip route get 192.168.100.101
No QPs on mlx5_1	RDMA device not mapped to NIC 2	rdma link show; confirm device-to-netdev mapping
Transport table shows only the MDS IP after I/O	pNFS flex-files inactive, or no real I/O performed	Confirm flex-files is enabled on the export; perform real reads/writes
ip route get shows the wrong NIC	Route cache stale	ip route flush cache
SSH session drops during Option B apply/rollback	nmcli connection up bounced the NIC carrying your session	Run the activation detached (nohup … &) or from console/BMC
Routes still present after Option B rollback	cp -a keyfile restore did not delete an NM-created profile	Use the -ipv4.routes rollback method; verify with ip route show
ip route replace permission error	Not root	Use sudo

The following recommended practices can help ensure a reliable, maintainable NIC-trunking deployment, regardless of the persistence method or transport (RDMA or TCP):

• Gateways: reuse the existing gateway each NIC already uses for the cluster subnet (from ip route show) rather than creating new ones.
• Source pinning: the src attribute pins the source address to the egress NIC. It is required for RDMA and keeps the path symmetric.
• Uneven pools: adjust the pool ranges so roughly half of the addresses go to each NIC. The split need not be exactly even.
• nconnect: start at nconnect=4 and raise it only if a single DS connection becomes a bottleneck.
• Verify before permanence: validate with temporary routes, confirm with the checks in Steps 7 and 8, and only then make the configuration permanent. Always keep a tested rollback option.

So, in summary, client-side NIC trunking is an efficient, fully reversible way to unlock the aggregate bandwidth of two client interfaces for pNFS flex-files on a PowerScale cluster running OneFS 9.15 or later. Because the technique relies only on host-specific routes and an nconnect mount, it requires no cluster-side change and applies equally to RDMA and TCP. Following the procedure above (confirm the correct data pool, back up first, split the routes, verify with the transport and queue-pair checks, and persist with a tested rollback in place) delivers a single mount that drives data traffic across both NICs.

Since OneFS 9.10, MetadataIQ has provided PowerScale clusters with a global metadata namespace, by exporting file system metadata to an off-cluster Elasticsearch database. The ability to index and query metadata across geo-distributed clusters, without resorting to time-consuming tree walks, has proven invaluable for data discovery, lifecycle management, and increasingly for AI-driven workflows such as intelligent chunking and retrieval-augmented generation (RAG).

However, in OneFS 9.14 and earlier, when a directory was renamed on the cluster, MetadataIQ dutifully recorded the new directory entry in Elasticsearch, but every child path beneath that directory remained indexed under the old prefix. This resulted in stale entries: the database reported the files living under /ifs/data/A/B/C, but the file system had them under /ifs/data/A/Bx/C. Until the release of OneFS 9.15, the only remedy was a full resync — a brute-force re-indexing of the entire dataset that is neither performant nor automation-friendly.

The new OneFS 9.15 releases addresses this MetadataIQ shortcoming with automatic directory rename propagation. Specifically, when a directory rename is detected, the system now tracks both the old and new paths and runs a dedicated rename phase that updates all descendant entries in the Elasticsearch index, eliminating stale data without a full resync. Additionally, the schema encoding for hardlinks has been redesigned to remove path sensitivity entirely.

To understand the solution, it helps to be clear about the problem. MetadataIQ’s Elasticsearch index uses a primary key (_id) for each document. In releases prior to 9.15, this ‘_id’ was encoded as a combination of the file’s LIN (logical inode number) and a hash of its full path: <LIN>#<path_hash>.

This encoding has two consequences when a parent directory is renamed:

• Every child entry in the database retains its old path. A directory rename from /ifs/data/projects to /ifs/data/projects_2026 leaves every file and subdirectory under it still indexed with the /ifs/data/projects prefix. The database and the file system disagree, and queries return incorrect results.
• For hardlinks, the path_hash component of the _id changes when a parent is renamed, even though the file itself has not changed. This creates duplicate or orphaned entries in the index.

Pre-OneFS 9.15 this required an expensive resync, effectively re-indexing the full dataset.

However, to avoid this overhead, the directory rename feature in OneFS 9.15 operates within MetadataIQ’s existing transfer agent (‘isi_metadataiq_transfer’), adding a post-processing rename phase that runs after the normal changelist transfer is complete. As such, the 9.15 workflow has two distinct phases:

Phase 1: Primary Processing

During normal MetadataIQ processing, the transfer agent iterates through the changelist entries and pushes them to the Elasticsearch database. When a directory rename event is detected, the following occurs:

• The changelist entry is pushed to the database as usual, with the old path and path depth preserved.
• The new directory path is recorded in a special rename_path field on the renamed directory’s document in Elasticsearch.
• Metadata fields such as timestamp, disk pool, and node pool are updated to reflect the current file system state.

At this stage, the children of the renamed directory still have their old paths in the database. The rename_path field is a breadcrumb that tells Phase 2 which directories have pending renames.

Phase 2: Rename (Post-processing)

After all primary changelist processing is complete, the transfer agent enters the rename phase. This is where the descendant path updates happen:

• The script queries Elasticsearch for all documents that have a non-empty rename_path field — these are the directories with pending renames.
• For each rename candidate, the script identifies all descendant entries whose path begins with the old directory prefix. These are processed in descending order by path depth, ensuring that deeply nested renames are handled correctly before their parents.
• The actual path update is performed atomically using an Elasticsearch Painless script execution. A single Painless script replaces the old path prefix with the new directory path and recomputes associated metadata for all descendants. This atomic update includes:

–  Path depth: adjusted by calculating the depth difference between the new and old paths.

–  Snapshot generation number: updated atomically so that parent-level renames can skip entries already modified by a child-level rename.

–  Other metadata fields: all relevant metadata is refreshed for every descendant entry.

• Once the root rename document and all its descendants are updated, the rename_path field is cleared, marking that rename as complete.
• This process iterates until all pending renames have been resolved.

Above, Phase 1 (blue) records rename candidates during normal changelist processing, while phase 2 (green) iterates through candidates and atomically updates all descendant paths.

Note that the descendant prefix search in Elasticsearch requires the ‘allow_expensive_queries’ setting to be enabled, which it is by default. This setting should not be disabled on MetadataIQ’s Elasticsearch instance.

The MetadataIQ transfer agent uses batched, checkpointed processing throughout both phases. An interrupted transfer resumes from the last recorded checkpoint rather than restarting from the beginning. The various stages of the transfer workflow are as follows:

Stage	Description
Fetch batch	Fetch changelist entries batchwise from the PAPI server (maximum 2,048 entries per fetch)
Detect rename	Detect directory rename events within the batch
Record rename_path	Record the new path in the rename_path field; preserve the old path and path depth
Checkpoint	Refresh object metadata, update ‘tail_cookie’, and record a checkpoint at batch end
Iterate candidates	In the rename phase, iterate over each recorded directory rename candidate
Find descendants	Identify descendants by old path prefix, processed in descending depth order
Update child paths	Replace the old prefix with the new directory path and update child metadata
Reconcile	Clear the rename_path field, marking the subtree as fully reconciled
Failure exit	On crash or ElasticSearch connection failure, exit and resume from the last checkpoint

If the transfer script exits during the rename phase, for example, in the event of a node failover, service restart, or a network interruption, the ‘rename_path’ fields in Elasticsearch remain set on any unprocessed rename candidates. When the transfer agent is next invoked, it detects these pending rename entries, skips the primary processing phase entirely, and proceeds directly to the rename phase to complete the outstanding work. No manual intervention is required. This is a significant improvement over the old full-resync approach, where an interrupted resync would simply need to be restarted from the beginning.

The second component of the directory rename feature addresses the hardlink problem at the schema level. In pre-9.15 releases, the Elasticsearch document ‘_id’ is encoded as ‘<LIN>#<path_hash>’. Since a hardlink is an additional directory entry pointing to the same inode, renaming a parent directory changes the ‘path_hash’ for every hardlink entry beneath it, creating duplicates or orphans in the index.

In OneFS 9.15, the ‘_id’ encoding for hardlinks is changed to use the parent LIN and filename rather than a full path hash. Because the filename itself does not change when a parent directory is renamed, hardlink entries are no longer sensitive to parent path changes. This eliminates an entire class of stale-entry scenarios.

The OneFS 9.14 and earlier encoding includes a hash of the full path, which changes when any ancestor is renamed. Conversely, the 9.15 encoding uses only the filename, making hardlinks immune to parent renames.

Architecturally, the rename phase is built directly into the existing MetadataIQ transfer agent. No new daemons, services, or ports are introduced. The transfer agent (‘isi_metadataiq_transfer’) already handles the mechanics of reading changelist entries, batching them, and uploading them to Elasticsearch. The rename phase simply adds a post-processing step after the draining loop completes.

There are two paths into the rename logic:

Path	Details
Normal path	After the primary processing phase completes and all changelist entries have been drained to Elasticsearch, the rename phase is triggered automatically.
Recovery path	If the transfer script was interrupted during a previous rename phase (leaving unprocessed rename_path entries in Elasticsearch), the next invocation skips primary processing and goes directly to the rename phase.

No separate installation or configuration is required for the directory rename feature. It is built into OneFS 9.15 and follows the same installation, configuration, and scheduling as the existing MetadataIQ capability. If MetadataIQ is already configured and running on a pre-9.15 cluster, upgrading to 9.15 enables directory rename propagation automatically after the upgrade commit and subsequent resync.

The existing MetadataIQ CLI commands remain unchanged:

# isi metadataiq settings view

# isi metadataiq settings modify --schedule "every day every 5 minutes"

# isi metadataiq resync

When it comes to upgrading OneFS and MetadataIQ, the directory rename feature introduces a change to the ‘_id encoding’ in the main Elasticsearch index in order to eliminate the path dependency for hardlinks. This encoding change is not backward compatible with pre-9.15 indices, which means that a full resync is required after the NDU (non-disruptive upgrade) commit.

The important details:

• The resync is triggered automatically by the system after the upgrade commit when incompatible versions are detected. Administrators do not need to initiate it manually.
• During the resync, do not interact with or interrupt the MetadataIQ service. Manual intervention during this process may cause the resync to fail, potentially requiring a manual reset and restart.
• The resync duration depends on the size of the indexed dataset. Plan accordingly when scheduling the upgrade window.

In the event of an upgrade rollback, the producer and consumer services return to their pre-upgrade behavior automatically, resuming operation on the older version with the original schema.

Understanding how MetadataIQ’s services behave during each stage of a non-disruptive upgrade is helpful for planning:

It’s also worth understanding how the producer and consumer components behave through the upgrade lifecycle:

Phase	Details
Pre-upgrade	Both producer and consumer run on the older version. The producer continues generating changelists as normal, and the consumer transfers data using the existing pre-upgrade schema.
Post-upgrade (mixed version):	The producer is disabled for the duration of the mixed-version window. The consumer adapts per node version — old schema on not-yet-upgraded nodes, updated logic on upgraded nodes — and transfers may pause or continue based on compatibility checks.
Post-commit	After the automatic resync completes, the producer is re-enabled and resumes generating changelists, while the consumer updates the database mapping and begins transferring under the new schema.
Post-rollback	If the upgrade is rolled back, both components simply return to their pre-upgrade behavior on the older version and the original schema.

To illustrate the behavior, consider two scenarios that demonstrate how the rename phase handles common directory operations.

Scenario 1: Simple nested renames

Starting from a nested directory structure:

# mkdir -p /ifs/data/A/B/C/D/E/F/G/H

After the initial MetadataIQ cycle, all eight directories are indexed in Elasticsearch with their full paths. Now, two renames are performed:

# mv /ifs/data/A/B /ifs/data/A/Bx

# mv /ifs/data/A/Bx/C/D/E /ifs/data/A/Bx/C/D/Ex

In prior releases, the descendants of those directories would have retained their old paths in the index. Instead, in OneFS 9.15, when the next MetadataIQ cycle runs, the transfer agent detects both rename events during primary processing and records the new paths in the rename_path field. Then, in the rename phase, it processes the renames in depth-descending order:

• First, E → Ex is processed: /ifs/data/A/Bx/C/D/Ex/F, /ifs/data/A/Bx/C/D/Ex/F/G, and /ifs/data/A/Bx/C/D/Ex/F/G/H are all updated.
• Then, B → Bx is processed: all remaining descendants under the old /ifs/data/A/B prefix are updated to /ifs/data/A/Bx.

The outcome is that all child paths are correctly updated in the database, and no stale entries remain.

Scenario 2: Subtree moves within the same tree

Starting from the same structure, this time with more complex operations:

# mkdir -p /ifs/data/A/B/C/D/E/F/G/H

# mv /ifs/data/A/B/C/D/E /ifs/data/A/

# mv /ifs/data/A/E/F/G /ifs/data/A/B/C/D/

# mv /ifs/data/A/B /ifs/data/A/Bx

Here, subtree E is moved up to become a direct child of A, subtree G is moved into D, and B is renamed to Bx. After the MetadataIQ cycle, the database reflects all three operations correctly:

• E and its child F are now under /ifs/data/A/E and /ifs/data/A/E/F.
• G and its child H are now under /ifs/data/A/Bx/C/D/G and /ifs/data/A/Bx/C/D/G/H.
• B’s rename to Bx is propagated to all remaining descendants: /ifs/data/A/Bx/C, /ifs/data/A/Bx/C/D.

So, this results in no stale entries, nor a need for manual resync. The depth-descending processing order and the snapshot generation number atomicity ensure that even compound operations within the same tree are resolved correctly.

After a MetadataIQ cycle involving directory renames has completed, the following CLI utilities can be used to monitor and verify operation. This includes:

Checking the MetadataIQ services:

# isi services -a | grep -i metadataiq

Monitoring the ChangelistCreate job:

# isi job jobs list | grep -i ChangelistCreate

Confirming the MetadataIQ snapshot count (which should be <= 2 under healthy operating conditions):

# isi snapshot snapshots list | grep -i metadataiq

Additionally, to verify that rename propagation has completed successfully, the Elasticsearch index can be queried for any documents with a non-empty ‘rename_path’ field. In normal operation, the following query should return zero results after the transfer agent completes:

GET isi_metadata_index/_search

{

  "query": {

    "exists": {

      "field": "rename_path"

    }

  }

}

A non-zero result count indicates that rename processing was interrupted and will be completed on the next transfer agent invocation.

Data replication is only half the equation. The other part, which often gets tested at the worst possible moment, is what happens when the primary cluster goes dark and storage administrators need to redirect clients to the secondary site with minimum disruption and maximum confidence in data integrity.

OneFS 9.15 introduces SmartSync failover and failback capabilities to ensure uninterrupted operations and fast recovery when a primary PowerScale cluster becomes unavailable. Rather than relying on time‑consuming restores from traditional backups, SmartSync enables rapid redirection of client access to a secondary cluster. By keeping data and metadata immediately available, it significantly reduces Recovery Time Objectives (RTO).

When the primary cluster is restored, SmartSync efficiently performs failback by syncing only the incremental changes generated during the failover period, preserving data integrity and enabling a seamless return to normal replication operations.

Together, these capabilities deliver a robust, high‑availability data mobility solution that strengthens business continuity and lowers operational risk.

SmartSync Failover/Failback provides the following capabilities:

·         Converts a read‑only SmartSync target path into a writeable path.

·         Switches a writeable SmartSync source path to read‑only.

·         Restores a SmartSync base path to a previous dataset state.

·         Enables reverse incremental synchronization to transfer client‑written data from a failover cluster back to the original source.

With OneFS SyncIQ, the allow-writes > resync-prep > redirect > reverse flow is generally well understood. In contrast, OneFS SmartSync has some notable variations to this theme, and it’s worth understanding how the SmartSync process differs before it’s actually required. This article walks through the full SmartSync failover and failback lifecycle, including what the commands do, why the sequence matters, and where the behavior diverges from what seasoned SyncIQ administrators will expect.

SmartSync replication is built on the Datamover service (isi_dm_d) and operates exclusively on Global Snapshots, referred to in its vernacular as ‘datasets’.

A dataset is an immutable, point-in-time view of the file system created via SnapshotIQ. Unlike a SyncIQ-replicated path, where the target is a live, writable mirror held read-only by convention, a SmartSync target path is backed by a series of versioned, lineage-aware datasets. This lineage is what makes efficient failback possible without re-baselining.

First, there are a few caveats to consider before getting into the workflow:

• Failover and failback are supported only by file-to-file replication (PowerScale cluster to PowerScale cluster). File-to-object cloud copy does not support either operation.
• The prerequisite state is a working repeat-copy relationship: a dataset creation policy running on the source and a repeat-copy policy replicating incrementally to the target. The baseline run must have completed successfully.
• Both clusters must have mutual TLS trust established and replication accounts configured in both directions. Trust is symmetric by design, which is what makes role reversal possible without reconfiguring certificates.
• The required privilege is ISI_PRIV_DATAMOVER. SmartSync also requires active SyncIQ and SnapshotIQ licenses on both clusters.

Workflow-wise, SmartSync failover and failback entails a seven-step process:

Steps 1 and 2 constitute the failover. Steps 3 through 5 constitute the failback. Steps 6 and 7 restore the original replication direction.

Step	Action	Details
1	Failover (part i)	Make the target writeable; optionally revert to a clean dataset.
2	Failover (part ii)	Client writes occur on the target; create a dataset to capture changes.
3	Failback (part i)	Prepare the original source by reverting to the matching dataset.
4	Failback (part ii)	Perform the reverse incremental sync from target back to source.
5	Failback (part iii)	Make the original source writeable and redirect clients.
6	Reset (part i)	Prepare the original target for forward replication (revert and make read-only).
7	Reset (part ii)	Resume the original-direction repeat-copy policy.

In practice, a planned failover may compress some steps. Similarly, an unplanned failover may skip others depending on the state of the source cluster.

Step 1: Making the Target Writeable (and Optionally Reverting)

Under normal repeat-copy operation, the target base path is read-only. SmartSync enforces this to prevent data written on the target from conflicting with incoming incremental updates from the source. The failover action unlocks the path with a single command:

target-1# isi dm setup-dir client-access --base-path /ifs/tgt

Are you sure you would like to make base path /ifs/tgt writable? (yes/[no]): yes

Created job: 1.

Path will become writable.

In an unplanned failover where the source is simply unavailable, take the last replicated dataset and make it available for writes. However, in a ransomware or data-corruption scenario, the latest dataset on the target may itself be compromised. SmartSync addresses this with the ‘–dataset-revert’ option, which simultaneously makes the path writeable and rolls it back to a specific earlier dataset:

target-1# isi dm setup-dir client-access --base-path /ifs/tgt --dataset-revert 1

Are you sure you would like to revert base path /ifs/tgt to dataset ID 1? (yes/[no]): yes

Created job: 1.

Dataset will revert and path will become writable.

The dataset ID to revert to is identified beforehand using the ‘isi dm datasets list’ command, inspecting creation timestamps to find the last known-clean point in time. Once the revert completes, the target dataset that was reverted away from is removed from the dataset list, and the base path reflects the contents of the chosen dataset, ready for clients.

Step 2: Capturing Changes on the Failover Cluster

With clients now writing to the former target, a dataset creation policy is needed to capture those changes as Global Snapshots. This is necessary for Step 4 (failback sync) to have a reference point to sync from, and is created with a one-shot ‘run-now’ execution to immediately capture the current state, then configure recurrence for ongoing snapshots:

target-1# isi dm policies create fo_creation CREATION \

    --priority NORMAL \

    --run-now true \

    --creation-account-id local \

    --creation-base-path /ifs/tgt

Created policy: 1.

At this point, if the original source is to be permanently decommissioned, the workflow stops here. The former target is now a fully independent source cluster. A new repeat-copy policy can be defined pointing at a fresh target, and normal forward replication resumes from this cluster with no re-baselining required anywhere downstream.

Step 3: Preparing the Original Source for Failback

Failback mandates that the original source be in a state the reverse-direction incremental sync can build upon. Specifically, the source must contain no datasets under the base path that post-date the dataset the target was using when it became writeable. If the source was still receiving new datasets from a running dataset creation policy at the moment of failover, those newer datasets must be removed before failback can proceed.

The ‘isi dm setup-dir replication-target’ CLI command handles this. It always requires a dataset revert and makes the base path read-only, transforming the cluster from an active source into a replication target ready to receive data:

source-1# isi dm setup-dir replication-target --base-path /ifs/src --dataset-revert 2

Are you sure you would like to revert base path /ifs/src to dataset 2? (yes/[no]): yes

Created job: 1.

Dataset will revert and path will become read-only

Dataset IDs are cluster-local, not globally synchronized, so the dataset ID on the source cluster will generally differ from the corresponding ID on the target. The preferred approach is to match by dataset revision and tree ID, not by the local integer ID. Running ‘isi dm dataset view <id>’ on both clusters and comparing the Dataset Revision and Dataset Global ID fields allows the match to be confirmed before the revert is issued.

Note that, if the final dataset creation policy on the source is configured with the ‘–creation-dataset-make-read-only true’ argument, the source base path is automatically made read-only as soon as that dataset is created. Combined with a scheduled dataset creation immediately before the planned maintenance window, this eliminates the need to manually run replication-target on the source, and ensures no writes land between the last sync and the failover.

Step 4: Reverse Incremental Sync

This is where SmartSync’s re-baselining elimination pays off in the failback scenario. A reverse-direction repeat-copy policy is created from the failover cluster (now acting as source) back to the original source (now acting as target). The ‘–reconnect true’ flag instructs SmartSync to resume from the last shared dataset rather than initiating a full baseline copy. Without it, the first run would transfer the entire dataset unnecessarily.

target-1# isi dm policies create failback_rc REPEAT_COPY \

    --priority NORMAL \

    --run-now true \

    --repeat-copy-source-base-path /ifs/tgt \

    --repeat-copy-base-source-account-id local \

    --repeat-copy-base-target-account-id <source-cluster-account-id> \

    --repeat-copy-base-target-base-path /ifs/src \

    --reconnect true \

    --repeat-copy-base-target-dataset-type FILE

Created policy: 1025.

After the run completes, the job type can be verified in the ‘isi dm reports list’ command output. The type should be marked as ‘DATASET_INCREMENTAL’, confirming that SmartSync found the shared lineage and performed an incremental transfer rather than a full baseline. If it shows ‘DATASET_BASELINE’, the dataset revision match between the two clusters was incorrect and the revert (step 3) needs revisiting.

Step 5: Redirecting Clients Back to the Original Source

Once the failback sync has completed and the original source base path is current, the clients can be redirected. The ‘isi dm setup-dir client-access’ CLI command makes the path writeable without requiring a dataset revert, which is appropriate when the source is known-clean and the reverse sync is current:

source-1# isi dm setup-dir client-access --base-path /ifs/src

Are you sure you would like to make base path /ifs/src writable? (yes/[no]): yes

Created job: 1.

Path will become writable.

Following this, any pertinent DNS entries, SmartConnect zone configuration, or NFS mount points can be updated as appropriate to redirect client traffic. The original source is now serving live client workloads again.

Step 6: Prepare the Original Target for Forward Sync

With clients back on the original source, the final pair of steps (steps 6 & 7) restores the normal A-to-B replication direction. First, the former failover cluster (B) is prep’d to become a replication target again. As such, any datasets created on it since the last successful reverse sync are reverted away, and the path is made read-only.

target-1# isi dm setup-dir replication-target \

    --base-path /ifs/tgt \

    --dataset-revert <latest-shared-dataset-id>

Step 7: Resume the Original Replication Direction

With the original target read-only and aligned to the same dataset revision as the source, the original forward repeat-copy policy is resumed with a manual ‘run-now’ trigger. SmartSync uses the shared lineage to perform an incremental update, again with no re-baseline:

source-1# isi dm policies modify <original-policy-id> --run-now true

Once this job completes successfully, the relationship is back to its pre-failover state: Source writeable, target read-only, incremental replication running on schedule. As such, the full cycle is complete.

PowerScale cluster admins who are familiar with SyncIQ will notice both familiar patterns plus notable differences when it comes to managing SmartSync failover. The following table summarizes the key operational distinctions:

Aspect	SyncIQ	SmartSync
Failover command	isi sync policies modify –action allow-writes	isi dm setup-dir client-access
Source lock for failback	Implicit: resync-prep job prepares source as target	Explicit: isi dm setup-dir replication-target (always requires revert)
Re-baselining on failback	Not required (changelist-based incremental)	Not required (Global Snapshot lineage, –reconnect true)
Ransomware / data corruption recovery	Fail over to last clean sync snapshot via isi snapshot restore	Integrated: –dataset-revert in same failover command
Reverse sync direction	New SyncIQ policy, source becomes target	New REPEAT_COPY policy with –reconnect true from failover cluster
Restoring forward direction	Disable reverse policy; re-enable original policy	setup-dir replication-target on ex-target; modify original policy –run-now
Object / cloud replication	Not applicable (file only)	Cloud copy does not support failover or failback

 

SmartSync provides two views into replication state during and after a failover or failback operation. Running jobs appear in ‘isi dm jobs list’; completed jobs move to ‘isi dm reports list’. For granular job detail, the ‘isi dm job view <id>’ CLI command provides per-task progress, and ‘isi_datamover_job_status <id>’ gives lower-level internal state. Transfer failures write a job-specific log to ‘/ifs/data/Isilon_Support/datamover/transfer_failures/<job-id>/’, and the primary log file is ‘/var/log/isi_dm.log’.

For RPO visibility across replication relationships, particularly useful in the lead-up to a planned failover, the ‘isi dm rpo reports list’ CLI command provides a per-policy view of how current the replicated data is relative to defined recovery objectives.

SmartSync’s support for pull policies, where the target cluster initiates and pulls data from the source, adds a useful dimension to DR topology design that SyncIQ does not offer. In a resource-constrained source cluster, configuring the target to pull rather than the source to push offloads the replication compute burden to the DR site, leaving primary client I/O unaffected.

For the failback reverse sync specifically (Step 4), the policy is always issued from the cluster that holds the current writeable data, pushing to the recovering original source. There is no functional difference in the failover/failback sequence between clusters that used push versus pull in normal operations; both directions use the same commands. The key is that account IDs and base paths are swapped to reflect the reversed roles.

SmartSync failover and failback is a structured, seven-step workflow anchored on Global Snapshot lineage and three core commands: ‘isi dm setup-dir client-access’ (unlock a path for writes), ‘isi dm setup-dir replication-target’ (lock a path as a replication target and revert to a known state), and a reverse ‘REPEAT_COPY’ policy with ‘–reconnect true’ (incremental failback without re-baselining). The sequence handles both unplanned outage (fail over to last clean dataset, write locally, sync back) and planned maintenance (using the ‘–creation-dataset-make-read-only’ flag to cleanly quiesce the source before cutover).

Note the following important differences from SyncIQ: There is no equivalent of the ‘resync-prep job’, The ‘isi dm setup-dir replication-target’ CLI command, and its mandatory dataset revert, serve that function. Dataset ID matching between clusters is by revision and global ID, not by local integer. And –reconnect true on the reverse repeat-copy policy is non-negotiable for incremental failback.

Note also that SmartSync failover and failback is only supported for file-to-file replication. File-to-object cloud copy (replication to ECS, AWS, Azure, or GCP) does not support these operations and is intended for backup and archival workflows only. Verify the Job Type field in ‘isi dm reports list’ after a reconnect-mode failback sync: It must read ‘DATASET_INCREMENTAL’ to confirm that no re-baseline occurred.

As we saw in the previous article in this series, with OneFS 9.15, any node in a PowerScale cluster can now act as the OneFS metadata server (MDS), or serve data connections (DS). This separation of the metadata path from the data path allows file I/O traffic to be distributed across multiple nodes in the cluster, thereby maximizing available bandwidth and parallelism.

In this article, we turn our attention to the enablement, configuration, and management of pNFS.

When a pNFS‑capable client mounts an NFS export on a pNFS‑enabled OneFS cluster, each node functions as both a Metadata Server and a Data Server, with the Metadata Server role being implicitly assigned to the node the client initially contacts during the mount.

Data I/O operations are performed using NFSv3, as permitted by the pNFS Flexible File Layout specification, while NFSv4.1 or NFSv4.2 is used exclusively for layout management and metadata operations. Layouts are granted on a per‑file basis, with the client requesting a layout upon file open that specifies the appropriate data server or servers for that file. Different files may be assigned to different nodes, and traffic distribution behavior is influenced by server‑side alignment settings. The set of data server addresses provided to the client is determined by the SmartConnect pool associated with the mount IP address.

In order for pNFS to activated, a cluster must be running OneFS 9.15, with NFSv4.1 or NFSv4.2 enabled for metadata operations and NFSv3 enabled for data operations. The SmartConnect pool must also use static IP allocation.

Attribute	Details
OneFS version	OneFS 9.15 to provide pNFS support.
NFS protocol versions	NFSv4.1 or NFSv4.2 must be enabled (for the metadata path). NFSv3 must also be enabled (for the data path).
SmartConnect pool type	The SmartConnect network pool hosting pNFS must use Static IP allocation.
Client OS	Linux with kernel pNFS Flexible File Layout support (available in mainline Linux kernels 5.14+).
Client mount options	The client must mount the cluster export using NFSv4.1 (-o vers=4.1) or NFSv4.2 (-o vers=4.2). Transport protocol may be RDMA (proto=rdma) or TCP (proto=tcp). UDP is unsupported.

On the client side, systems must be running Linux kernels that support the pNFS Flexible File Layout, which is available in mainline kernels version 5.14 and later. Additionally, mounts must explicitly specify NFSv4.1 or NFSv4.2 using either TCP or RDMA transport, as UDP is not supported.

With OneFS 9.15, pNFS configuration can be performed via the OneFS CLI or platform API (pAPI), but the WebUI does not currently have an equivalent. Under the hood, the configuration is managed by the  ‘isi_gconfig’ registry keys under the NFS driver path. Client remounts are strongly recommended when enabling or disabling pNFS, as clients cache file attributes that determine layout eligibility, so may otherwise retain stale information. Activation requires unmounting NFS clients, enabling the pNFS feature from the OneFS CLI, and restarting the NFS service, with a corresponding process to disable the feature.

Several configuration parameters control pNFS behavior, including a master enablement flag for advertising Metadata Server and Data Server capabilities, alignment settings that determine how file data is distributed across nodes, options for mapping network interfaces to pNFS devices, and internal settings governing layout tracking scalability. In particular, the stripe alignment configuration has a substantial impact on I/O distribution patterns and must be carefully considered when tuning performance, as different alignment values favor different workload characteristics.

Facet	Details
Layout	Layouts are per-file. Each time a client opens a file, it requests a layout that tells it which data server to use. Different files may be assigned to different nodes. How the client spreads its traffic also depends on the server settings around alignment.
Network	SmartConnect determines the pool. The set of data server addresses returned to the client comes from the SmartConnect pool that the client’s mount IP belongs to.
Node roles	Every node in the cluster acts as both a Metadata Server and a Data Server. The ‘MDS’ role simply means the node the client happened to connect to for that mount.
Protocol version	Data I/O uses NFSv3. The pNFS Flexible File Layout standard allows data servers to speak NFSv3, which is well-supported and performant. The client uses NFSv4.1 only for metadata and layout management.

Every node in the cluster functions as both a Metadata Server (MDS) and a Data Server (DS), with the MDS role simply representing the node to which the client initially connects during the mount operation. Data I/O operations are performed using NFSv3, as permitted by the pNFS Flexible File Layout specification, while NFSv4.1 is used exclusively by the client for metadata handling and layout management. Layouts are granted on a per‑file basis, such that each time a file is opened, the client requests a layout identifying the appropriate data server, allowing different files to be serviced by different nodes, with overall traffic distribution further influenced by server‑side alignment configurations. The set of data server addresses provided to the client is determined by SmartConnect, with the addresses drawn from the SmartConnect pool associated with the client’s mount IP.

When it comes to enabling pNFS, ensure that the following prerequisites are met:

OneFS version	OneFS 9.15 or later.
NFS protocol versions	NFSv4.1 or NFSv4.2 must be enabled for the metadata path. Additionally, NFSv3 must also be enabled for the data path.
SmartConnect pool type	The pool must use static IP allocation. Dynamic IP pools are not currently supported for pNFS because device addresses must remain stable.
Client OS	Linux with kernel pNFS Flexible File Layout support (available in mainline Linux kernels 5.14+).
Client mount options	The client must mount using either NFSv4.1 (-o vers=4.1) or NFSv4.2 (-o vers=4.2). Transport protocol may be RDMA (proto=rdma) or TCP (proto=tcp). UDP is unsupported.

OneFS pNFS support can be enabled via the CLI as follows:

1. First, verify that the desired SmartConnect network pool is configured for static IPs:

# isi network pools list

2. Next, check that NFSv3 and NFSv4.1 and/or NFSv4.2 are enabled:

# isi nfs settings global view

3. When both the above prerequisites are confirmed, activate pNFS as follows:

# isi nfs settings global modify --pnfs-enabled 1

4. Once done, the cluster’s NFS exports can then be mounted from a Linux client running kernel 5.14+. For example:

# mount -t nfs4 -o vers=4.1 <pool-ip>:/ifs/data /mnt/pnfs

5. From the NFS client, after performing some I/O, the ‘mountstats’ file contents or CLI command can be used to verify pNFS is active. For example:

# grep -A 20 "pnfs" /proc/self/mountstats

Or:

# mountstats --nfs /mnt/pnfs

Specifically, the presence of non-zero ‘pNFS_read’ and ‘pNFS_write’ counters in the output confirms that the Linux client is using pNFS.

Optionally, as a second part to step 3 above, NIC-level device granularity configuration can also be applied. For example:

# isi_gconfig registry.Services.lwio.Parameters.Drivers.nfs.NFSV4LayoutDeviceUseNICAsUniqueDevice=1

Followed by an NFS service restart:

# isi services nfs disable && isi services nfs enable

Additionally, if pNFS does need to be disabled for any reason, first unmount any NFS clients, then run the following CLI command:

# isi nfs settings global modify --pnfs-enabled=true

The OneFS pNFS implementation (and the pNFS spec in general) does not require any changes to applications, as it is negotiated transparently between the NFS client and server. This allows applications to continue reading and writing files normally while the client kernel automatically manages layout negotiation and communication with data servers.

If a data server node becomes unavailable while a client holds a layout, the client may encounter I/O errors on the NFSv3 data path to that node. However, most Linux NFS clients will fall back to performing I/O through the metadata server (MDS) until the data server becomes available again, at which point they can re-request a layout, with the exact behavior varying by client kernel version and implementation.

OneFS pNFS is supported with both NFSv4.1 and NFSv4.2 and can coexist with non-pNFS clients on the same export, since pNFS is negotiated individually during session setup and clients that do not support it will continue using traditional NFS I/O paths while still accessing the same files.

Additionally, pNFS can work with RDMA when the server advertises both TCP and RDMA data server addresses and the environment includes RDMA-capable interfaces, enabling compatible clients to use RDMA for the data path.

Finally, pNFS requires NFSv3 to be enabled because the Flexible File Layout specification allows data servers to be accessed via NFSv3. The data path between clients and data server nodes uses NFSv3 for simplicity and compatibility, while the metadata operations, including layout management and file open or close operations, must use NFSv4.1 or NFSv4.2.

If you’ve ever witnessed a high-core-count compute node saturate a single NFS mount point while the rest of a PowerScale cluster still has plenty of resources to spare, you’ve experienced the fundamental limitation that Parallel NFS (pNFS) was designed to eliminate. Traditional NFS, whether NFSv3 or NFSv4.1 in conventional mode, binds all client I/O to the single node behind the mount IP. The cluster may have dozens of nodes, petabytes of aggregate flash bandwidth, and a 400GbE back-end fabric, but from the perspective of that NFS client, it’s talking to one node. With OneFS 9.15, PowerScale introduces native pNFS support, allowing a single NFS client to read and write data directly and concurrently across multiple nodes in a SmartConnect pool — turning the entire cluster into a parallel data path rather than routing everything through a single server. This article covers what pNFS is, how the metadata-server and data-server roles work in a OneFS environment, the prerequisites you need to meet before enabling it, and the CLI steps to activate and configure pNFS on your cluster.

The pNFS protocol is an extension of NFSv4.1, designed to significantly improve throughput for NFS workloads by enabling clients to perform data reads and writes directly and concurrently to multiple nodes across a cluster or distributed storage system, rather than routing all I/O through a single, node-bound NFS server.

In a traditional NFS deployment, a client mounts a share and directs all read and write operations to the single node associated with the mount IP address, which inherently limits performance and creates a bottleneck regardless of overall cluster size or capability.

All I/O for this mount is between the client and node over NFSv3 or NFSv4.1, while the remainder of the cluster’s nodes sit idle for this client.

In contrast, when pNFS is enabled, the client continues to communicate with a single node on the cluster for metadata operations such as file open, close, rename, and attribute queries, but is explicitly informed which other nodes to send the actual file data to.

With OneFS 9.15, any node in a PowerScale cluster can now act as the OneFS metadata server (MDS), or serve data connections (DS). This separation of the metadata path from the data path allows file I/O traffic to be distributed across multiple nodes in the cluster, thereby maximizing available bandwidth and parallelism.

In a pNFS-enabled PowerScale cluster, the client always talks to the MDS over NFSv4.1 or NFSv2 for namespace, attributes, and locking, and it talks directly to data servers for the actual I/O.

1. When a pNFS client opens a file, it sends an NFSv4.1 ‘OPEN’ and ‘GETATTR’ to the MDS, which returns a filehandle and any required state.
2. The client then issues a ‘LAYOUTGET’ to the MDS, which responds with a layout describing exactly which Data Servers (DSs) hold which byte ranges or stripes of the file, often using a ‘files’ layout with per-node NFS file-handles or device IDs. The client may also send a ‘GETDEVICEINFO’ request, to which the MDS responds.
3. For reads, once the client has this layout, it bypasses the MDS and issues NFS ‘READ’ RPCs directly to the Data Servers corresponding to the required byte ranges. The DSs return data straight to the client, while the MDS is only involved again if there is a layout error or a recall, in which case the client may perform ‘LAYOUTCOMMIT’, ‘LAYOUTRETURN’, and a fresh ‘LAYOUTGET’ to resynchronize.

For writes, the client similarly acquires a read-write layout via ‘LAYOUTGET’ from the MDS, then sends NFS ‘WRITE’ (and, if needed, ‘COMMIT’) operations directly to the DSs based on the stripe mapping in the layout.

4. After the Data Servers have durably stored the data, the pNFS client uses ‘LAYOUTCOMMIT’ to inform the MDS about changes such as the updated file size and modification time.
5. Finally, the client passes a ‘LAYOUTRETURN’ when it is finished with the layout.

Throughout this process, all metadata operations, including directory lookups, creates, deletes, renames, attribute changes, and locks, are handled solely by the MDS using standard NFSv4.1 operations (i.e. LOOKUP, CREATE, REMOVE, RENAME, GETATTR, SETATTR, LOCK, and LOCKU, etc). In contrast, the Data Servers never see namespace, simply serving or accepting data for extents identified by the layout. The net effect is that metadata and control traffic (opens, layouts, locks) flow between the client and the MDS, while bulk data reads and writes flow directly between the client and multiple DS nodes in parallel, allowing the data path to scale with the number of storage nodes rather than being bottlenecked by a single NFS server.

The primary benefits of pNFS include increased single‑client throughput by eliminating the single‑node bottleneck and distributing I/O across all nodes within a SmartConnect pool, as well as improved aggregate cluster throughput through more balanced I/O distribution and better utilization of existing caching and prefetch mechanisms. This functionality is transparent to applications, as pNFS negotiation occurs automatically between the Linux NFS client and the OneFS NFS server without requiring application‑level changes. Because OneFS provides a single distributed filesystem in which every node inherently has access to all data, pNFS simply authorizes the client to communicate directly with additional nodes for data transfer.

When a pNFS‑capable client mounts an NFS export on a pNFS‑enabled OneFS cluster, each node functions as both a Metadata Server and a Data Server, with the Metadata Server role being implicitly assigned to the node the client initially contacts during the mount. Data I/O operations are performed using NFSv3, as permitted by the pNFS Flexible File Layout specification, while NFSv4.1 or NFSv4.2 is used exclusively for layout management and metadata operations. Layouts are granted on a per‑file basis, with the client requesting a layout upon file open that specifies the appropriate data server or servers for that file. Different files may be assigned to different nodes, and traffic distribution behavior is influenced by server‑side alignment settings. The set of data server addresses provided to the client is determined by the SmartConnect pool associated with the mount IP address.

Several prerequisites must be met prior to enabling pNFS on a PowerScale cluster:

Attribute	Details
OneFS version	OneFS 9.15 to provide pNFS support.
NFS protocol versions	NFSv4.1 or NFSv4.2 must be enabled (for the metadata path). NFSv3 must also be enabled (for the data path).
SmartConnect pool type	The SmartConnect network pool hosting pNFS must use Static IP allocation.
Client OS	Linux with kernel pNFS Flexible File Layout support (available in mainline Linux kernels 5.14+).
Client mount options	The client must mount the cluster export using NFSv4.1 (-o vers=4.1) or NFSv4.2 (-o vers=4.2). Transport protocol may be RDMA (proto=rdma) or TCP (proto=tcp). UDP is unsupported.

In the next article in this series, we’ll look at the process to enable and configure pNFS on a PowerScale cluster.

Dell PowerScale is kicking off the summer with the release of OneFS 9.15, introduced on August 11, 2026, marking a significant evolution of its scale-out NAS platform. This release brings broad innovation across platform capabilities, performance, security, serviceability, and ease of use, delivering meaningful value for a wide range of enterprise workloads.

As the latest version of PowerScale’s unified software platform for both on-premises and cloud deployments, OneFS 9.15 is well suited for traditional file services, vertical industry use cases such as media and entertainment, healthcare, life sciences, and financial services, as well as modern workloads including generative AI, machine learning, deep learning, and analytics.

PowerScale continues to provide flexible deployment options across core, edge, and cloud environments, whether on-site, in colocation facilities, or through customer-managed deployments in AWS and Microsoft Azure. Its scale-out architecture ensures the performance and agility required to support increasingly complex unstructured data workflows. In today’s environment, where data security, threat detection, and operational visibility are more critical than ever, OneFS 9.15 introduces a slew of new capabilities and enhancements designed to strengthen performance, resiliency, and security while simplifying operations.

Performance enhancements are a central highlight of OneFS 9.15, including the general availability of pNFS, which enables parallel client data access and significantly improves throughput for AI and high-performance computing workloads.

With 9.15 also comes the introduction of software-defined OneFS on Dell Exascale, decoupling the PowerScale file system software from proprietary hardware, enabling flexible deployment on industry-standard servers and cloud infrastructure. This approach delivers the same enterprise-grade data services, scalability, and performance as traditional PowerScale appliances while providing greater deployment flexibility and cost efficiency for large scale AI environments.

Exascale, Dell’s rack-scale AI and data platform, uses AMD EPYC–based nodes and NVMe-rich storage behind 400Gb-class fabrics for massive AI training bandwidth. In addition to PowerScale, the Exascale platform can also be configured to natively run ObjectScale, Lightning file system, or PowerStore block.

OneFS 9.15 also delivers predictable, linear scaling across clusters ranging from 3 to 32 nodes, allowing organizations to grow efficiently without overprovisioning. SmartQoS has been enhanced with support for both bandwidth- and IOPS-based limits, separation of read and write workload classes, improved latency observability, and the ability to define system pools that isolate performance-sensitive nodes, providing administrators with granular control over resource allocation.

Security and compliance have also been strengthened through validation of cryptographic entropy sources for FIPS 140-3 compliance and updates to the secure development lifecycle, including removal of outdated software components and adoption of current packages to maintain a strong security posture. Reliability improvements are deeply embedded across the platform, with enhancements to the non-disruptive upgrade framework that improve stability, introduce retry mechanisms, and streamline upgrade operations. Additional resiliency gains include better service startup handling, continuous performance data collection through Always-On Performance Analysis, and optimizations for large file workflows and deduplication consistency.

Serviceability and observability see major gains with automated CELOG event aggregation, which correlates and enriches events to provide clearer insights and reduce troubleshooting time. Administrators also benefit from API-driven configuration management that eliminates the need for root access when adjusting advanced settings, along with automated diagnostic tools that generate actionable reports and proactive alerts for job failures or cancellations.

Finally, data mobility and user experience are further enhanced through the transition from SyncIQ to SmartSync, enabling more modern and flexible disaster recovery strategies while preserving existing investments. A new SmartSync web interface simplifies management, while improvements to SMB2 failover durability and expanded SmartQoS visibility enhance both resilience and operational clarity.

Theme	Key Features
Unlock Platform Power for Gen AI	·         Exascale – Performance, reliability, compression ·         NVIDIA CX8 VPI network controller support ·         PowerScale switch DNOS 10.6.1.1 support ·         Enable SONiC on S5224 for TOR
File/Object Handling Enhancements	·         New checksum support for S3 ·         Support for CRC64NVME default AWS checksum algorithm ·         Hadoop release 3.3.3 support ·         pNFS General Availability ·         Ability to force a dump of S3 server logs
Performance Improvements	·         Support for read, write and op classes as SmartQoS metrics ·         SmartQoS bandwidth limits ·         Linear Performance Scaling ·         Configurable system pool
Reliability	·         NDU hook framework stability, resilience, and performance enhancements ·         S3 service start up and non-disruptive config reload enhancements ·         Automated management for user configuration changes ·         Always On Performance Analysis ·         Cluster resiliency for large client workloads ·         Large file performance improvements ·         File metrics included in ‘isi statistics workload’ reporting
Serviceability	·         Aggregate CELOG event details for better customer experience ·         Remote Secure Credential (RSC) Access Enablement ·         Ability to alert when multiple jobs have cancelled/failed ·         Improved Performance diagnostics for Protocols
Data Mobility and Data Recovery	·         SmartSync DR – FO/FB – Moves across domains ·         Migration of SyncIQ policies to SmartSync
Useability	·         Enhanced WebUI Dashboard/Node Overview ·         Dynamic licensing subscription model for software-defined PowerScale on the Exascale platform ·         Directory Rename in PowerScale Cluster to reflect in ElasticSearch via Metadata IQ ·         SmartSync: WebUI for SmartSync ·         SMB 2 Durable handles for Dynamic Pools ·         Enhance latency metric for SmartQoS ·         Job engine reports.db size under control
Security	·         Cluster join rule enforcement update ·         OpenSSL support for FIPS 140-3

Overall, OneFS 9.15 reinforces Dell PowerScale’s position as a leading platform for unstructured data, delivering powerful advancements in scalability, performance, security, and operational efficiency, and equipping organizations to meet the growing demands of modern data-driven environments.

We’ll be taking a deeper look at the new OneFS 9.14 features and functionality in blog articles over the course of the next few weeks. Meanwhile, the new OneFS 9.15 code is available on the Dell Support site, as both an upgrade and reimage file, allowing both installation and upgrade of this new release.

For existing clusters running a prior OneFS release, the recommendation is to open a Service Request with to schedule an upgrade. To provide a consistent and positive upgrade experience, Dell Technologies is offering assisted upgrades to OneFS 9.15 at no cost to customers with a valid support contract. Please refer to this Knowledge Base article for additional information on how to initiate the upgrade process.

Standing up a new PowerScale cluster has always required a human armed with a serial cable. Before a node can join a cluster and receive a routable IP address through the standard network management path, an administrator must physically connect to the node’s serial console and manually configure the front-end network interface. For environments deploying a handful of nodes occasionally, this is a minor inconvenience. For large-scale datacenter deployments — where dozens or hundreds of nodes may need to be provisioned at once — the serial console dependency becomes a meaningful operational bottleneck, requiring scheduled on-site time from qualified engineers for what is essentially a repetitive, low-complexity task.

In addition to time, serial provisioning is inherently sequential, potentially error-prone, and tricky to automate. Plus it typically involves coordination between the storage admin and whoever has physical access to the racks. In a hyperscale or rapidly growing environment, the time spent cabling and typing IP addresses into consoles is time that scales linearly with node count.

OneFS 9.14 takes another decisive step in the journey to PowerScale Zero Touch Provisioning (ZTP) via native DHCP support for a cluster’s front-end network interfaces. This enables nodes to obtain their network configuration automatically from a remote DHCP server, rather than requiring manual IP assignment at the serial console.

To appreciate ZTP’s benefits in OneFS 9.14, it helps to understand how OneFS already handles dynamic IP assignment in certain contexts. For several release now, OneFS has used an externally managed allocation method for network pools, backed by two SmartConnect components: The IP Reporter and the IP Merger.

The IP Reporter parses external IP sources — such as DHCP lease files — and generates IP report files on the cluster filesystem (for example, /ifs/.ifsvar/modules/flexnet/ip_reports/DHCP/node.1). The IP Merger, which runs as a single instance per cluster, reads these reports and merges the discovered IP information into the Flexnet network configuration. Flexnet is the OneFS subsystem responsible for maintaining the authoritative view of which IPs are assigned to which interfaces on which nodes. Both components run on a fixed interval — roughly a one-minute cycle — so newly obtained leases are picked up automatically within a short window.

This SmartConnect-based path already exists and works in cloud deployments of OneFS, where DHCP is always enabled and the cloud provider’s DHCP server assigns IPs automatically. What was missing for on-premises deployments was the DHCP client management layer — the component responsible for actually running dhclient on the appropriate interfaces and producing the lease files that the existing IP Reporter can then parse. ZTP closes that gap with a new purpose-built daemon and a new cluster-level configuration surface, while deliberately reusing the proven IP Reporter/Merger pipeline rather than reinventing it.

The new component introduced in OneFS 9.14 is ‘isi_dhcp_manager_d’, a daemon that runs on every node in the cluster.

Under the purview of MCP, the DHCP manager acts as a mini service controller, spawning and supervising one dhclient process per configured front-end interface. It monitors these processes for health, and continuously compares which interfaces should have DHCP and which nodes are excluded against the actual state (i.e. which dhclient processes are currently running) and corrects any drift. If an interface is added to the included list, the manager starts a client for it, and if one is removed or a node is excluded, the manager stops the corresponding client. This declarative model is what allows the same configuration to be safely re-applied across reboots, upgrades, and transient interface flaps without the need for manual intervention.

The end-to-end flow works as follows. The network administrator uses the CLI or PAPI to configure DHCP for the cluster. The DHCP Manager reads that configuration from Flexnet and starts a dhclient process on each included interface — for example, interface class ext-2 mapped to the physical NIC on a given node. Each dhclient negotiates with the DHCP server and writes a lease file to /var/db/dhclient/lease.<nic>. The DHCP Manager log consolidates the output from every dhclient it spawns into a single file, which makes debugging the DHCP protocol exchange considerably easier. The existing IP Reporter detects the lease, parses the assigned address, and writes an IP report; the IP Merger then merges it into the Flexnet configuration, and the Flexnet daemon performs the actual IP assignment to the interface.

Importantly, IPs remain assigned and tracked by Flexnet throughout, with ‘dhclient’ creating only the lease file. DHCP-obtained addresses show an allocation method of ‘externally_managed’, distinguishing them from statically assigned IPs. The DHCP Manager does not start or terminate dhclient for interfaces that are not link-up, and it automatically terminates any orphaned dhclient processes it finds running at startup.

One of the more useful behaviors demonstrated in ZTP is automatic network pool management. When a DHCP-assigned IP arrives, OneFS checks for an existing network pool covering that IP range. If one exists, the new IP is added to it, otherwise a pool is created automatically, along with the DHCP-obtained IP range is associated with it. The naming convention is of the general ‘groupnet0.subnet0.pool_dhcp’ format. As additional nodes obtain leases, their addresses are merged into the same pool’s IP ranges.

Critically, DHCP coexists with existing allocation methods on the same interface. An interface class such as ext-1 that already carries a statically assigned IP can also be enrolled in DHCP and receive an additional DHCP-assigned address. The result is an interface that holds both a static IP and a DHCP IP simultaneously, associated with multiple pools. This flexibility means DHCP can be layered onto an existing statically configured cluster without displacing any current addressing. This is an important property for brownfield environments, where removing a working static configuration to adopt DHCP would likely be a non-starter.

Zero Touch Provisioning in OneFS 9.14 introduces three configurable dimensions for DHCP management:

Capability	Description
Enable or disable DHCP cluster-wide	A single toggle governs whether DHCP is active across the cluster.
Specify which interface classes participate	Admins define an ‘included’ list of interface classes that should be managed via DHCP.
Exclude specific nodes	Individual nodes can be added to an ‘excluded’ list by Logical Node Number (LNN). assigned IPs remain untouched. This is useful when specific nodes have static requirements or when rolling DHCP out incrementally.

A single toggle governs whether DHCP is active across the cluster. DHCP is disabled by default on both fresh installs and upgrades from earlier OneFS versions, so existing deployments are never affected unless an administrator explicitly opts in. When disabled, the ‘isi_dhcp_manager_d’ service is present but not running.

Administrators define an ‘included’ list of interface classes (ie. ext-1, ext-2, mgmt-1) that should be managed via DHCP. Interfaces not in the list keep their existing allocation method.

Individual nodes can be added to an ‘excluded’ list by Logical Node Number (LNN). An excluded node will not run dhclient for any interface class regardless of the included list, and any DHCP-obtained IPs on that node are removed — while its statically assigned IPs remain untouched. This is useful when specific nodes have static requirements or when rolling DHCP out incrementally.

All DHCP configuration lives under the ‘isi network dhcp’ CLI command family, which supports both ‘view’ and modify ‘options’. The general configuration flow is as follows:

The following CLI commands can be used to configure and manage DHCP. For example, to view current state:

# isi network dhcp view

To enable DHCP cluster-wide:

# isi network dhcp modify --enabled true

To enroll interface classes, for example external and management interfaces:

# isi network dhcp modify --add-interfaces ext-1,ext-2

To remove an interface class:

# isi network dhcp modify --remove-interfaces ext-2

To exclude a node (for example LNN 4):

# isi network dhcp modify --add-excluded-nodes 4

To disable DHCP entirely:

# isi network dhcp modify --enabled false

Any modification triggers an interactive confirmation prompt — a safeguard against accidentally severing front-end connectivity if DHCP is the only IP source:

# isi network dhcp modify --enabled false
You are modifying DHCP settings. Please ensure you have front-end
connectivity via other statically/dynamically assigned IPs as you
may lose connectivity. Would you like to proceed? (yes/[no]):

The ‘–force’ flag can be added to suppress this prompt for scripted workflows. The same functionality is exposed through the PAPI at /platform/25/network/dhcp for orchestration tooling, and the DHCP configuration is captured hourly as telemetry via that endpoint.

When DHCP is disabled, the DHCP Manager terminates every dhclient process, removes the DHCP-obtained leases, and gracefully shuts down the daemon itself. Two pieces of state intentionally persist, however. First, the included-interfaces and excluded-nodes configuration is not cleared — so re-enabling DHCP later reapplies the previous configuration without re-entry. Second, any auto-created pool is not deleted; it remains (with no IP ranges) and is reused under the same name when DHCP IPs return within the same subnet. This persistence is a deliberate convenience: it means disabling DHCP is a reversible, low-friction operation rather than a destructive one.

Aspect	Cloud	On-Premises
Default state	Always enabled, not configurable	Disabled by default, admin-configurable
DHCP management	MCP manages dhclient directly	isi_dhcp_manager_d daemon manages dhclient
DHCP server	Cloud provider managed	Customer network managed
Node exclusion	Not applicable	Supported via LNN excluded list

Note that while cloud deployments already rely on DHCP, they do not yet take advantage of this new configuration surface or daemon — bringing the two models together is planned for a follow-up release.

When DHCP-assigned IPs are not appearing as expected, the following systematic diagnostic sequence will usually cover the most common failure points:

Step	CLI Command	Expected Result
DHCP enabled?	isi network dhcp view	Enabled: True
Node not excluded?	isi network dhcp view	Node LNN absent from Excluded Nodes list
Service running?	isi services -a isi_dhcp_manager_d	Service reported as enabled
dhclient active?	ps aux | grep dhclient	One dhclient: <nic> per configured interface
Lease file present?	ls /var/db/dhclient/	lease.<nic> present per interface
IP bound to interface?	ifconfig <nic> | grep inet	inet <ip-address> present

A good approach is generally to work through the above checklist from top to bottom, with each step gating the next. If DHCP shows disabled or the node appears in the excluded list, nothing downstream will happen by design. If the service is enabled but no dhclient is running, the interface’s link may not be up, or the interface class may not be in the included list. If dhclient is running but no lease file appears, the issue is almost certainly between the node and the DHCP server (reachability, scope exhaustion, or relay configuration). And if a lease exists but no IP is bound, the bottleneck is typically the IP Reporter/Merger cycle or a pool conflict during the merge.

The principal log file locations include:

Log file	Detail
/var/log/isi_dhcp_manager_d.log	Primary daemon log, including consolidated dhclient output.
/var/log/isi_smartconnect	IP Reporter and IP Merger logs covering lease parsing and pool merging.
/var/db/dhclient/lease.<nic>	Raw lease files, confirming what the DHCP server offered.
/ifs/.ifsvar/modules/flexnet/flx_config.xml	Stored DHCP configuration as seen by Flexnet

 

There are also three new DHCP-related CELOG events flags introduced in OneFS 9.14 in support of ZTP:

Event	Severity	Meaning / Action
SW_SC_DHCP_CLIENT_RESTART	Warning	dhclient restarted 5+ times in 15 min — check network stability and DHCP server availability
SW_SC_IPMERGE_FAILED	Warning	DHCP IP could not be merged into Flexnet (e.g., address already in a static pool) — check for IP pool conflicts or invalid pool configuration
SW_SC_DHCP_LEASE_REBIND	Critical	Lease renewal failed, IP at risk of expiring — verify DHCP server is reachable

With OneFS 9.14’s ZTP capability, on-prem PowerScale nodes can now receive their front-end network addresses from a DHCP server automatically, without serial console access during provisioning. Built on the existing SmartConnect IP Reporter and Merger infrastructure, the new isi_dhcp_manager_d daemon slots cleanly into the Flexnet pipeline, auto-creating network pools, allowing static and DHCP addresses to coexist on the same interface, and preserving configuration across enable/disable cycles — all without displacing any existing IP allocation mechanism. Removing the requirement for serial console access from the provisioning path helps set the stage for broader zero-touch deployment automation in future releases.