Tucked amongst the array of new functionality within OneFS 9.7’s payload is the debut of a networking feature called externally managed network pools. In layman’s terms, this is essentially the introduction of a front-end dynamic host control protocol (DHCP) client for the PowerScale cluster.
The context and motivation behind implementing this new functionality is predicated on the fact that cloud networking differs substantially from on-prem infrastructure. This is largely because the cloud hyperscalers typically require a primary IP to be configured on a specific interface that they dictate. Normally, systems operating within an off-prem environment obtain their network configuration via the DHCP protocol. But as you’re likely aware, until OneFS 9.7, DHCP has not been supported on a cluster’s front end network previously. To support APEX file services for AWS, OneFS 9.6 implemented a manual work-around for this, which had its limitations. However, with OneFS 9.7, the desire was to make the system smarter and by adding proper support for IPv4 primary IP addresses on AWS deployments, thereby negating the need for configuring manual work-arounds, with their inherent risks.
This new externally managed IP addresses feature is automatically enabled upon committing and upgrade to OneFS 9.7. To support this feature, a new network allocation method has been added to OneFS’ network pools called ‘externally managed’. This is actually managed by an external service such as AWS, which dictates where these primary IPs live. So they are in charge of IP allocation, rather than the cluster’s Flexnet or SmartConnect services, which has been the case up to now. It’s worth noting that OneFS 9.7 only includes (and enforces) limited DHCP support, strictly for cloud deployments currently. That said, on-prem DHCP support may be added in a future release but this is currently not on the near-term roadmap. Additional work was also included in OneFS 9.7 to prevent IP oversubscription.
So let’s take a look under the hood… Architecturally, there are three main components to the externally managed IP addresses feature:
- DHCP Service
- IP Reporter Module
- IP Merger Module
OneFS 9.7 actually talks DHCP by leveraging the FreeBSD ‘dhclient’ implementation. Dhclient is modified so it does not actually configure the network interfaces like it would normally, in order to avoid conflicts with the OneFS Flexnet network config daemon. Instead, dhclient just persists the leases to the following files:
-
/ifs/.ifsvar/modules/flexnet/flx_config.xml
-
/ifs/.ifsvar/modules/flexnet/pool_members/groupnet.1.subnet.1.pool.1.
Additionally, SmartConnect sees the addition of two new modules, IP Reporter and IP Merger.
| Component | Details |
| DHCP service | Adds new MCP-controlled DHCP service: dhclient-ext-1
– Uses modified FreeBSD dhclient implementation – Does not configure network interfaces – Persists leases to /var/db/dhclient/ |
| IP Merger | Adds new cluster-wide module to SmartConnect, IP Merger:
– Coordinates ownership of the role by taking locks on files on /ifs – Loads all files from IP Reports directory – Verifies network pool is configured correctly and generates IP Assignments – Updates the following files: ▪ /ifs/.ifsvar/modules/flexnet/flx_config.xml ▪ /ifs/.ifsvar/modules/flexnet/pool_members/groupnet.1.subnet.1.pool.1 |
| IP Reporter | Adds new module to each node’s SmartConnect service:
– Parses DHCP leases – Converts to a generic format – Saves to /ifs/.ifsvar/modules/flexnet/ip_reports/DHCP/node. |
These modules are still part of the overarching isi_smartconnect_d, and just new components within that same daemon. The IP Reporter module will parse the above lease files and then save the information to /ifs/.ifsvar/modules/flexnet/ip_reports/DHCP/node.
In contrast, the IP Merger is a single cluster-wide instance that loads the files from the IP Reports directory, verifies the network pool configuration, generates the IP assignments, and updates the config files. The ip_merger file contains the devID of the node that has been elected as responsible for IP merging. The full path is as follows:
/ifs/.ifsvar/modules/smartconnect/resource/workers/ip_merger
The following CLI syntax can be used to determine which node is acting as the merger: For example:
# isi_for_array 'grep "Taking ownership of the IPMerger role" /var/log/isi_smartconnect’ TME-4: 2024-02-07T16:26:20.946863+00:00 <3.6> GLaDOS-4(id4) isi_smartconnect_d[3626]: Taking ownership of the IPMerger role
In this case, the command output indicates that node ID4 has taken ownership of the IPMerger role.
The underlying process is very similar to how OneFS manages SSIPs in that all nodes attempt to lock a file under /ifs, and one granted that lock, they own that responsibility. So OneFS takes the files from under /ifs/.ifsvar/modules/flexnet/ip_reports and merges the IP information into the Flexnet config and the pool members file, as follows:
The above graphic illustrates how data flows through the system from the cloud provider’s DHCP server, to dhclient, and then into isi_smartconnect_d. The modular, extensible architecture requires only a small portion of OneFS to be made aware of this new type of network pool. This all happens on the side until the data is merged into the Flexnet config and the associated state files, so it is low risk to everything else.
In OneFS 9.7, this new DHCP allocation method is now set as ‘externally managed’ for subnet0.pool0. This can be seen even on network pools that have been upgraded from an earlier OneFS release. Additionally, the CLI output also reports the type of external manager for this network pool – for instance AWS in the example below:
The ‘isi network interfaces’ CLI syntax is also updated in OneFS 9.7 to allow filtering by ‘externally managed’ pools. For example below, again showing that the owner is AWS:
As a quick reminder, there are a currently a couple of instances of unsupported networking functionality in the APEX file services for AWS offering, as compared to on-prem OneFS, and these include:
- IPv6 support
- VLANs
- Link aggregation
- NFSoverRDMA
In the next article in this series we’ll turn our attention to the management, monitoring, and security of OneFS 9.7 externally managed network pools.