In this article, we’ll take a quick peek at the new PowerScale F910 hardware platform that was released last week. Here’s where this new node sits in the current hardware hierarchy:
The PowerScale F910 is the high-end all-flash platform that utilizes a dual-socket 4th gen Zeon processor with 512GB of memory and twenty four NVMe drives, all contained within a 2RU chassis. Thus, the F910 offers a generational hardware evolution, while also focusing on environmental sustainability, reducing power consumption and carbon footprint, and delivering blistering performance. This makes the F910 and ideal candidate for demanding workloads such as M&E content creation and rendering, high concurrency and low latency workloads such as chip design (EDA), high frequency trading, and all phases of generative AI workflows, etc.
An F910 cluster can comprise between 3 and 252 nodes. Inline data reduction, which incorporates compression, dedupe, and single instancing, is also included as standard to further increase the effective capacity.
The F910 is based on the 2U R760 PowerEdge server platform, with dual socket Intel Sapphire Rapids CPUs. Front-End networking options include 100/25 GbE and with 100 GbE for the Back-End network. As such, the F910’s core hardware specifications are as follows:
Attribute | F910 Spec |
Chassis | 2RU Dell PowerEdge R760 |
CPU | Dual socket, 24 core Intel Sapphire Rapids 6442Y @2.6GHz |
Memory | 512GB Dual rank DDR5 RDIMMS (16 x 32GB) |
Journal | 1 x 32GB SDPM |
Front-end network | 2 x 100GbE or 25GbE |
Back-end network | 2 x 100GbE |
Management port | LOM (LAN on motherboard) |
PCI bus | PCIe v5 |
Drives | 24 x 2.5” NVMe SSDs |
Power supply | Dual redundant 1400W 100V-240V, 50/60Hz |
These node hardware attributes can be easily viewed from the OneFS CLI via the ‘isi_hw_status’ command. Also note that, at the current time, the F910 is only available in a 512GB memory configuration.
Starting at the business end of the node, the front panel allows the user to join an F910 to a cluster and displays the node’s name once it has successfully joined:
As with all PowerScale nodes, the front panel display provides some useful current node environmentals telemetry. The ‘check’ button activates the panel and the ‘arrow’ buttons scroll to navigate, with the initial options being ‘Setup’ or View’, as below:
After selecting ‘View’, the menu presents ‘Power’ or ‘Thermal’:
Available thermal stats include BTU/hour:
Node temperature:
Air flow in cubic ft per minute (CFM):
Removing the top cover, the internal layout of the F910 chassis is as follows:
The Dell ‘Smart Flow’ chassis is specifically designed for balanced airflow, and enhanced cooling is primarily driven by four dual-fan modules. These fan modules can be easily accessed and replaced as follows:
Additionally, the redundant power supplies (PSUs) also contain their own air flow apparatus and can be easily replaced from the rear without opening the chassis. In the event of a power supply failure, the iDRAC LED on the rear panel of the node will turn orange:
Additionally, the front panel LCD display will indicate a PSU or power cable issue:
And the amber fault light on the front panel will illuminate at the end corresponding to the faulty PSU:
For storage, each PowerScale F910 node contains ten NVMe SSDs, which are currently available in the following capacities and drive styles:
Standard drive capacity | SED-FIPS drive capacity | SED-non-FIPS drive capacity |
3.84 TB TLC | 3.84 TB TLC | |
7.68 TB TLC | 7.68 TB TLC | |
15.36 TB QLC | Future availability | 15.36 TB QLC |
30.72 TB QLC | Future availability | 30.72 TB QLC |
Note that 15.36TB and 30.72TB SED-FIPS drive options are planned for future release.
Drive subsystem-wise, the PowerScale F910 2RU chassis is fully populated with twenty four NVMe SSDs. These are housed in drive bays spread across the front of the node as follows:
The NVMe drive connectivity is across PCIe lanes, and these drives use the NVMe and NVD drivers. The NVD is a block device driver that exposes an NVMe namespace like a drive and is what most OneFS operations act upon, and each NVMe drive has a /dev/nvmeX, /dev/nvmeXnsX and /dev/nvdX device entry and the locations are displayed as ‘bays’. Details can be queried with OneFS CLI drive utilities such as ‘isi_radish’ and ‘isi_drivenum’. For example:
# isi_drivenum Bay 0 Unit 15 Lnum 9 Active SN:S61DNE0N702037 /dev/nvd5 Bay 1 Unit 14 Lnum 10 Active SN:S61DNE0N702480 /dev/nvd4 Bay 2 Unit 13 Lnum 11 Active SN:S61DNE0N702474 /dev/nvd3 Bay 3 Unit 12 Lnum 12 Active SN:S61DNE0N702485 /dev/nvd2 <snip>
Moving to the back of the chassis, the rear of the F910 contains the power supplies, network, and management interfaces, which are arranged as follows:
The F910 nodes are available in the following networking configurations, with a 25/100Gb ethernet front-end and 100Gb ethernet back-end:
Front-end NIC | Back-end NIC | F910 NIC Support |
100GbE | 100GbE | Yes |
100GbE | 25GbE | No |
25GbE | 100GbE | Yes |
25GbE | 25GbE | No |
Note that, like the F710 and F210, an Infiniband backend is not supported on the F910 at the current time. Although this option will be added in due course.
These NICs and their PCI bus addresses can be determined via the ’pciconf’ CLI command, as follows:
# pciconf -l | grep mlx mlx5_core0@pci0:23:0:0: class=0x020000 card=0x005815b3 chip=0x101d15b3 rev=0x00 hdr=0x00 mlx5_core1@pci0:23:0:1: class=0x020000 card=0x005815b3 chip=0x101d15b3 rev=0x00 hdr=0x00 mlx5_core2@pci0:111:0:0: class=0x020000 card=0x005815b3 chip=0x101d15b3 rev=0x00 hdr=0x00 mlx5_core3@pci0:111:0:1: class=0x020000 card=0x005815b3 chip=0x101d15b3 rev=0x00 hdr=0x00
Similarly, the NIC hardware details and drive firmware versions can be view as follows:
# mlxfwmanager Querying Mellanox devices firmware ... Device #1: ---------- Device Type: ConnectX6DX Part Number: 0F6FXM_08P2T2_Ax Description: Mellanox ConnectX-6 Dx Dual Port 100 GbE QSFP56 Network Adapter PSID: DEL0000000027 PCI Device Name: pci0:23:0:0 Base GUID: a088c20300052a3c Base MAC: a088c2052a3c Versions: Current Available FW 22.36.1010 N/A PXE 3.6.0901 N/A UEFI 14.29.0014 N/A Status: No matching image found Device #2: ---------- Device Type: ConnectX6DX Part Number: 0F6FXM_08P2T2_Ax Description: Mellanox ConnectX-6 Dx Dual Port 100 GbE QSFP56 Network Adapter PSID: DEL0000000027 PCI Device Name: pci0:111:0:0 Base GUID: a088c2030005194c Base MAC: a088c205194c Versions: Current Available FW 22.36.1010 N/A PXE 3.6.0901 N/A UEFI 14.29.0014 N/A Status: No matching image found
Compared with its F900 predecessor, the F910 sees a number of hardware performance upgrades. These include a move to PCI Gen5, Gen 4 NVMe, DDR5 memory, Sapphire Rapids CPU, and a new software-defined persistent memory file system journal ((SPDM). Also the 1GbE management port has moved to Lan-On-Motherboard (LOM), whereas the DB9 serial port is now on a RIO card. Firmware-wise, the F910 and OneFS 9.8 require a minimum of NFP 12.0.
In terms of performance, the new F910 provides a considerable leg up on the previous generation F900. This is particularly apparent with NFSv3 streaming writes, as can be seen here:
OneFS node compatibility provides the ability to have similar node types and generations within the same node pool. In OneFS 9.8 and later, compatibility between the F910 nodes and the previous generation F900 platform is supported.
Component | F900 | F910 |
Platform | R740 | R760 |
Drives | 24 x 2.5” NVMe SSD | 24 x 2.5” NVMe SSD |
CPU | Intel Xeon 6240R (Cascade Lake) 2.4GHz, 24C | Intel Xeon 6442Y (Sapphire Rapids) 2.6GHz, 24C |
Memory | 736GB DDR4 | 512GB DDR5 |
This compatibility facilitates the addition of individual F910 nodes to an existing node pool comprising three of more F900s if desired, rather than creating a F910 new node.
In compatibility mode with F900 nodes containing the 1.92TB drive option, the F910’s 3.84TB drives will be short stroke formatted, resulting in a 1.92TB capacity per drive. Also note that, while the F910 is node pool compatible with the F900, a performance degradation is experienced where the F910 is effectively throttled to match the performance envelope of the F900s.