The largest file size that OneFS currently supports was raised to 16TB in the OneFS 8.2.2 release – a fourfold increase over the previous maximum of 4TB.
This helps enable additional applications and workloads that typically deal with large files, for example videos & images, seismic analysis workflows, as well as a destination or staging area for backups and large database dumps.
Firstly, large file support is available for free. No special license is required to activate large file support and, once enabled, files larger than 4TiB may be written to and/or exist on the system. However, large file support cannot be disabled once enabled.
In order for OneFS to support files larger than 4TB, adequate space is required in all of a cluster’s disk pools in order to avoid a potential performance impact. As such, the following requirements must be met in order to enable large file support:
| Large File Support Requirement | Description |
| Version | A cluster must be running OneFS 8.2.2 in order to enable large file support. |
| Disk Pool | A maximum sized file (16TB) plus protection can consume no more than 10% of any disk pool. This translates to a minimum disk pool size of 160TB plus protection. |
| SyncIQ Policy | All SyncIQ remote clusters must be running OneFS 8.2.2 and also satisfy the restrictions for minimum disk pool size and SyncIQ policies. |
Note that the above restrictions will be removed in a future release, allowing support for large (>4TiB) file sizes on all cluster configurations.
The following procedure can be used to configure a cluster for 16TiB file support:
Once a cluster is happily running OneFS 8.2.2 or later, the ‘isi_large_file -c’ CLI utility will verify that the cluster’s disk pools and existing SyncIQ policies meet the requirements listed above. For example:
# isi_large_file -c Checking cluster compatibility with large file support... NOTE: Isilon requires ALL clusters in your data-center that are part of any SyncIQ relationship to be running on versions of OneFS compatible with large file support before any of them can enable it. If any cluster requires upgrade to a compatible version, all SyncIQ policies in a SyncIQ relationship with the upgraded cluster will need to resync before you can successfully enable large file support. * Checking SyncIQ compatibility... - SyncIQ compatibility check passed * Checking cluster disk space compatibility... - The following disk pools do not have enough usable storage capacity to support large files: Disk Pool Name Members Usable Required Potential Capable Add Nodes ----------------------------------------------------------------------------- h500_30tb_3.2tb-ssd_128gb:2 2-3,6,8,10-11,13-16,18-19:bay3,6,9,12,15 107TB 180TB 89T N X h500_30tb_3.2tb-ssd_128gb:3 2-3,6,8,10-11,13-16,18-19:bay4,7,10,13,16 107TB 180TB 89T N X h500_30tb_3.2tb-ssd_128gb:4 2-3,6,8,10-11,13-16,18-19:bay5,8,11,14,17 107TB 180TB 89T N X h500_30tb_3.2tb-ssd_128gb:9 1,4-5,7,9,12,17,20-24:bay5,7,11-12,17 107TB 180TB 89T N X h500_30tb_3.2tb-ssd_128gb:10 1,4-5,7,9,12,17,20-24:bay4,6,10,13,16 107TB 180TB 89T N X h500_30tb_3.2tb-ssd_128gb:11 1,4-5,7,9,12,17,20-24:bay3,8-9,14-15 107TB 180TB 89T N X The cluster is not compatible with large file support: - Incompatible disk pool(s)
Here, the output shows that none of the pools meet the 10% disk pool rule above and contain insufficient storage capacity to allow large file support to be enabled. In this case, additional nodes would need to be added.
The following table explains the detail of output categories above:
| Category | Description |
| Disk Pool Name | Node pool name and this disk pool ID. |
| Members | Current nodes and bays in this disk pool . |
| Usable | Current usable capacity of this disk pool. |
| Required | Usable capacity required for this disk pool to support large files. |
| Potential | The max usable capacity this disk pool could support at the target node count |
| Capable | Whether this disk pool has the size of disk and number per node to support large files |
| Add Nodes | If this disk pool is capable, how many more nodes need to be added |
Once the validation confirms that the cluster meets the requirements, the following CLI command can then be run to enable large file support:
# isi_large_file -e
Upon successfully enabling large file support, the ‘cluster full’ alert threshold is automatically lowered to 85% from the OneFS default of 95%. This is to ensure that adequate space is available for large file creation, repair, and restriping. Additionally, any SyncIQ replication partners must also be running OneFS 8.2.2 or later, adhere to the above minimum disk pool size, and have the large file feature enabled.
Any disk pool management commands that violate the large file support requirements are not allowed. Once enabled, disk pools are periodically checked for compliance and OneFS will alert if a disk pool fails to meet the minimum size requirement.
If Large File Support is enabled on a cluster, any SyncIQ replication policies will only succeed with remote clusters that are also running 8.2.2 or later and have Large File Support enabled. All other SyncIQ policies will fail until the appropriate remote clusters are upgraded and have large file support switched on.
Be aware that, once enabled, large file support cannot be disabled on a cluster – regardless of whether it’s a SyncIQ source or target, or not participating in replication. This may impact future expansion planning for the cluster and all of its SyncIQ replication partners.
Also note that, after enabling large file support, the ‘cluster full’ alert threshold is automatically lowered to 85% from the default of 95%. This helps ensure that adequate space is available for large file creation, repair, and restriping.
When the maximum filesize is exceeded, OneFS typically returns an ‘EFBIG’ error. This is translated to an error message of “File too large”. For example:
# dd if=/dev/zero of=16TB_file.txt bs=1 count=2 seek=16384g dd: 16TB_file.txt: File too large 1+0 records in 0+0 records out 0 bytes transferred in 0.000232 secs (0 bytes/sec)