In the first article in this series, we looked at the architecture and considerations of the new OneFS 9.5’s SmartPools Transfer Limits. Now, we turn our attention to the configuration and management of this feature.
From the control plane side, OneFS 9.5 contains several WebUI and CLI enhancements to reflect the new SmartPools Transfer Limits functionality. Probably the most obvious change is in the ‘local storage usage status’ histogram, where tiers and their child nodepools have been aggregated, for a more logical grouping. Also blue limit-lines have been added above each of the storagepools, and a red warning status displayed for any pools that have exceeded the transfer limit.
Similarly, the storage pools status page now includes transfer limit details, with the 90% limit displayed for any storagepools using the default setting.
From the CLI, the ‘isi storagepool nodepools view’ command reports the transfer limit status and percentage for a pool. The used SSD and HDD bytes percentages, in the command output indicate where the pool utilization is relative to the transfer limit.
# isi storagepool nodepools view h5600_200tb_6.4tb-ssd_256gb ID: 42 Name: h5600_200tb_6.4tb-ssd_256gb Nodes: 77, 78, 79, 80, 81, 82, 83, 84 Node Type IDs: 10 Protection Policy: +2d:1n Manual: No L3 Enabled: Yes L3 Migration Status: l3 Tier: - Transfer Limit: 90% Transfer Limit State: default Usage Avail Bytes: 1.13P Avail SSD Bytes: 0.00 Avail HDD Bytes: 1.13P Balanced: No Free Bytes: 1.18P Free SSD Bytes: 0.00 Free HDD Bytes: 1.18P Total Bytes: 1.41P Total SSD Bytes: 0.00 Total HDD Bytes: 1.41P Used Bytes: 229.91T (17%) Used SSD Bytes: 0.00 (0%) Used HDD Bytes: 229.91T (17%) Virtual Hot Spare Bytes: 56.94T
The storage transfer limit can be easily configured from the CLI as for either a specific pool, as a default, or disabled, using the new –transfer-limit and –default-transfer-limit flags.
The following CLI command can be used to set the transfer limit for a specific storagepool:
# isi storagepool nodepools/tier modify --transfer-limit={0-100, default, disabled}
For example, to set a limit of 80% on an A200 nodepool:
# isi storagepool a200_30tb_1.6tb-ssd_96gb modify --transfer-limit=80
Or to set the default limit of 90% on tier ‘perf1’:
# isi storagepool perf1 --transfer-limit=default
Note that setting the transfer limit of a tier automatically applies to all its child nodepools, regardless of any prior child limit configurations.
The global ‘isi storage settings view’ CLI command output shows the default transfer limit, which is 90%, but can be configured between 0 to 100% if desired.
# isi storagepool settings view Automatically Manage Protection: files_at_default Automatically Manage Io Optimization: files_at_default Protect Directories One Level Higher: Yes Global Namespace Acceleration: disabled Virtual Hot Spare Deny Writes: Yes Virtual Hot Spare Hide Spare: Yes Virtual Hot Spare Limit Drives: 2 Virtual Hot Spare Limit Percent: 0 Global Spillover Target: anywhere Spillover Enabled: Yes Default Transfer Limit: 90% SSD L3 Cache Default Enabled: Yes SSD Qab Mirrors: one SSD System Btree Mirrors: one SSD System Delta Mirrors: one
This default limit can be reconfigured from the CLI with the following syntax::
# isi storagepool settings modify --default-transfer-limit={0-100, disabled}
For example, to set a new default transfer limit of 85%:
# isi storagepool settings modify --default-transfer-limit=85
And the same changes can be made from the SmartPools WebUI, too, by navigating to Storage pools > SmartPools settings:
Once a SmartPools job has completed in OneFS 9.5, the job report contains a new field that reports any ‘files not moved due to transfer limit exceeded’.
# isi job reports view 1056 ... ... Policy/testpolicy/Access changes skipped 0 Policy/testpolicy/ADS containers matched 'head’ 0 Policy/testpolicy/ADS containers matched 'snapshot’ 0 Policy/testpolicy/ADS streams matched 'head’ 0 Policy/testpolicy/ADS streams matched 'snapshot’ 0 Policy/testpolicy/Directories matched 'head’ 0 Policy/testpolicy/Directories matched 'snapshot’ 0 Policy/testpolicy/File creation templates matched 0 Policy/testpolicy/Files matched 'head’ 0 Policy/testpolicy/Files matched 'snapshot’ 0 Policy/testpolicy/Files not moved due to transfer limit exceeded 0 Policy/testpolicy/Files packed 0 Policy/testpolicy/Files repacked 0 Policy/testpolicy/Files unpacked 0 Policy/testpolicy/Packing changes skipped 0 Policy/testpolicy/Protection changes skipped 0 Policy/testpolicy/Skipped files already in containers 0 Policy/testpolicy/Skipped packing non-regular files 0 Policy/testpolicy/Skipped packing regular files 0
Additionally, the ‘SYS STORAGEPOOL FILL LIMIT EXCEEDED’ alert is triggered when a storagepool’s usage has exceeded its transfer limit. Raised at the INFO level. Each hour, CELOG fires off a monitor helper script which will measure how full each storagepool is relative to its transfer limit. The usage is gathered by reading from the diskpool database, and the transfer limits are stored in gconfig. If a nodepool has a transfer limit of 50% and usage of 75%, the monitor helper will report a measurement of 150%, triggering an alert.
# isi event view 126 ID: 126 Started: 11/29 20:32 Causes Long: storagepool: vonefs_13gb_4.2gb-ssd_6gb:hdd usage: 33.4, transfer limit: 30.0 Lnn: 0 Devid: 0 Last Event: 2022-11-29T20:32:16 Ignore: No Ignore Time: Never Resolved: No Resolve Time: Never Ended: -- Events: 1 Severity: information
And from the WebUI:
And there you have it: Transfer Limits, and the first step in the evolution towards a smarter SmartPools.