OneFS Namespace API (RAN)

 Among the array of HTTP services, and in addition to the platform API, OneFS also provides a namespace API.

This RESTful Access to Namespace, or RAN, provides HTTP access to any of the files and directories within a cluster’s /ifs filesystem hierarchy.

RAN can be accessed by making an HTTP call that references the /namespace/ API, rather than the /platform/ API that we’ve seen in the previous articles in this series. For example:

namespace == “http share”

This provides HTTPS access to the files & directories on the filesystem, but more as a data management API rather than object.

As you would expect, by default, the root of a cluster’s RAN namespace is the top level /ifs directory:

Namespace resources are accessed through a URL, such as:

Where:

Attribute Description
Access point Access Point is the root path of the file system endpoint (RAN share), ie. /ifs.
Authority Authority is the IP address or FQDN for the cluster.
Container Container is a directory or folder.
Data object Data object contains content data, such as a file on the system.
Endpoint Endpoint is the targetable URL.
File File is the data object you wish to query/modify.
Namespace Namespace is the file system structure on the cluster.
Object Object is either a container or data object.
Path Path is the file path to the object you want to access.
Port Port is the number of the port; the default port number is 8080
Scheme Scheme is the access protocol method

For example, using the RAN API to access the file ‘file1’ which resides in the ‘dir1’ directory under the access point and path ‘/ifs/data/dir1/’.

From the OneFS CLI, the ‘curl’ utility can be used to craft a ‘GET’ request for this file:

# curl -u <user:password> -k https://10.1.10.20:8080/namespace/ifs/data/dir1/file1

Or from a browser:

Also, using ‘curl’ via the CLI to view the RAN access point:

# curl -X GET https://10.1.10.20:8080/namespace --insecure --basic --user root:a

{"namespaces":[{

   "name" : "ifs",

   "path" : "/ifs"

}

]}#

Additionally, you can append a pre-defined keyword to the end of the URL when you send a request to the namespace. These keywords must be placed first in the argument list and must not contain any value. If these keywords are placed in any other position in the argument list, the keywords are ignored. Pre-defined keywords include: ‘acl’, ‘metadata’, ‘worm’, and ‘query’.

For example:

https://10.1.10.20:8080/namespace/ifs/data/dir1?acl

Or for metadata. For example:

https://10.1.10.20:8080/namespace/ifs/data/dir1/file1?metadata

A cluster’s files and directories can be accessed programmatically through RAN, similarly to the way they’re accessed through SMB or NFS protocols, as well as limited by filesystem permissions. As such RAN enables the following types of file system operations to be performed.

Operation Action Description
Access points CREATE, DELETE Identify and configure access points (shares) and obtain protocol information.
Directory CREATE, GET, PUT, LIST, DELETE List directory content.; get and set directory attributes; delete directories from the file system.
File CREATE, GET, PUT, LIST, DELETE View, move, copy, and delete files from the file system.
Access control GET/SET ACLs Manage user rights; set ACL or POSIX permissions for files and directories. Set access list on access points (RAN Share Permissions).
Query QUERY

 

Search system metadata or extended attributes, and tag files.
SmartLock GET, SET, Commit Allow retention dates to be set on files; commit files to a WORM state.

Additionally, applications or external clients can be built to access RAN in any major programming or scripting language, such as C++, Java, .net, Python, etc.

Note, though, that RAN access in general is disabled for clusters running in ‘hardened’ mode. In such cases a warning will be displayed notifying that HTTP browse is disabled, similar to the following:

OneFS Platform API Configuration, Management, and Monitoring

In addition to the platform API (pAPI) and RESTful access to a cluster’s namespace (RAN), OneFS makes extensive use of HTTP for a variety of services and client protocols.

As such, OneFS also supports the following HTTP-based services:

Service Description Ports
PlatformAPI OneFS platform API service, for remote cluster management. TCP 8080
PowerScaleUI OneFS WebUI configuration and management console. TCP 8080
RAN RESTful Access to Namespace, allowing cluster data access via HTTP. TCP 8080
RemoteService Remote-Service API  handlers under the /remote-service/ namespace, managed by isi_rsapi_d. TCP 8080
S3 AWS S3 object protocol. TCP 9020 (http) TCP 9021 (https)
SWIFT SWIFT object protocol (deprecated in favor of S3). TCP 8083
WebHDFS WebHDFS over HTTP. TCP 8082

In OneFS 9.4 and later, the above HTTP services may be enabled or disabled independently via the CLI or platform API, by a user account with the ‘ISI_PRIV_HTTP RBAC’ privilege.

The ‘isi http services’ CLI command set can be used to view and modify the services HTTP services. For example, remote HTTP access to the platform API can easily be disabled as follows:

 # isi http services modify Platform-API-External --enabled=0

You are about to modify the service Platform-API-External. If you disable Platform-API-External then PowerScaleUI will also be disabled. Are you sure? (yes/[no]):

Similarly, a subset of the HTTP configuration settings, including WebDAV, can also be viewed and modified via the WebUI by navigating to Protocols > HTTP settings:

Similarly, the OneFS web services can be viewed and controlled from the CLI via the ‘isi http services’ command set. For example:

# isi http services list

ID                    Enabled

------------------------------

Platform-API-External Yes

PowerScaleUI          Yes

RAN                   Yes

RemoteService         Yes

------------------------------

Total: 4

The astute will have observed that both S3 and Swift are notably absent from the above list of OneFS HTTP services.

Since S3 has become the de facto object protocol, after a period of gradual deprecation the OpenStack Swift protocol & API has finally been completely removed in OneFS 9.9. That said, Swift will remain available and supported in OneFS 9.8 and earlier releases, until their respective end of support dates.

Also, while S3 service uses HTTP as its transport, it is considered as a tier-1 protocol, and as such is managed via its own ‘isi s3’ CLI command set, corresponding WebUI area, and platform API endpoints:

In the example above, the ‘?describe&list’ suffix provides all of the S3 pAPI endpoints.

Another useful facet is that the OneFS command line syntax provides a ‘—debug’ option, which displays the associated pAPI endpoint information for each CLI command entered. For example, when querying OneFS for a cluster’s storage pool info, the ‘GET [‘9’, ‘storagepool’, ‘storagepools’]’ endpoint is being used by the CLI command:

# isi --debug storagepool list

2024-08-14 07:33:01,652 DEBUG rest.py:72: >>>GET ['9', 'storagepool', 'storagepools']

2024-08-14 07:33:01,652 DEBUG rest.py:74:    args={}

   body={}

2024-08-14 07:33:01,752 DEBUG rest.py:96: <<<(200, {'status': '200 Ok', 'content-type': 'application/json', 'allow': 'GET, HEAD'}, '\n{\n"storagepools" : \n[\n\n{\n"can_disable_l3" : true,\n"can_enable_l3" : false,\n"health_flags" : [],\n"id" : 1,\n"l3" : false,\n"l3_status" : "storage",\n"lnns" : [ 1, 2, 3 ],\n"manual" 

<snip>

So the corresponding pAPI URL to the ‘isi storagepool storagepools list’ CLI command is:

Or via curl:

# curl --insecure --basic --user <uname:passwd> https://10.1.10.20:8080/platform/9/storagepool/storagepools

{

"storagepools" :

[

{

"can_disable_l3" : true,

"can_enable_l3" : false,

"health_flags" : [],

"id" : 1,

"l3" : false,

"l3_status" : "storage",

"lnns" : [ 1, 2, 3 ],

"manual" : false,

<snip>

In addition to curl, the OneFS API endpoints can also be incorporated into script languages such as bash, perl, powershell, python, etc. This provides a powerful option for automating routine cluster management tasks.

For example, a python script along the lines of the following can be used to view a cluster’s critical events:

#!/usr/bin/python
import requests
import json
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning) 
# Suppresses the self signed cert warning

CLUSTERIP = '10.1.10.20'
PORT=8080
USER='root'
PASSWD='$1cyL@wn'

uri = "https://%s:%s" % (CLUSTERIP, PORT)
papi = uri + '/platform'
headers = {'Content-Type': 'application/json'}

data = json.dumps({'username': USER, 'password': PASSWD, 'services': ['platform']})

# uri of the cluster used in the referrer header
uri = f"https://{CLUSTERIP}:{PORT}"

# url of Papi used for all further calls to Papi
papi = uri + '/platform'

# Set header as content will provided in json format
headers = {'Content-Type': 'application/json'}

# Create json dictionary for auth
data = json.dumps({'username': USER, 'password': PASSWD, 'services': ['platform']})

# create a session object to hold cookies
session = requests.Session()

# Establish session using auth credentials
response = session.post(uri + "/session/1/session", data=data, headers=headers, verify=False)

if 200 <= response.status_code < 299:
    # Set headers for CSRF protection. Without these two headers all further calls with be "auth denied"
    session.headers['referer'] = uri
    session.headers['X-CSRF-Token'] = session.cookies.get('isicsrf')
    print("Authorization Successful")
else:
    print("Authorization Failed")
    print(response.content)

endpoint = '/7/event/eventlists'
response = session.get(papi + endpoint, verify=False)
result = json.loads(response.content)

#iterate through each event in each eventlist and output only critical events
for eventlist in result['eventlists']:
    for event in eventlist['events']:
        if event['severity'] == 'critical':
            print("Event ID: %s -- Event Severity: %s -- Description: %s " % (event['event'], event['severity'], event['message']))

Note that the ‘CLUSTERIP’, ‘USER’, and ‘PASSWD’ fields in the above python script will need to be edited appropriately, to reflect a cluster’s settings.

There is also an extensive OneFS API portal and developer community:

https://www.delltechnologies.com/en-us/storage/storage-automation-and-developer-resources/index.htm

This portal provides a central location for all the Dell ecosystem integrations (plugins), including CSI drivers, VMware, Containers, DevOps, Infrastructure as Code (IaC), OpenStack, etc. It also provides community forums to collaborate, post questions, discuss ideas, share tips & tricks, etc. – in addition to code samples and ready to use integrations for developers.

PowerScale OneFS 9.9

Dell PowerScale is already powering up summer with the launch of the innovative OneFS 9.9 release, which shipped today (13th August 2024). This new 9.9 release is an all-rounder, introducing PowerScale innovations in performance, security, serviceability, protocols, and ease of use.

OneFS 9.9 delivers the next version of PowerScale’s common software platform for on-prem and cloud deployments. This can make it a solid fit for traditional file shares and home directories, vertical workloads like M&E, healthcare, life sciences, financial services, and next-gen AI, ML and analytics applications.

PowerScale’s scale-out architecture can be deployed on-site, in co-lo facilities, or as customer managed Amazon AWS and Microsoft Azure deployments, providing core to edge to cloud flexibility, plus the scale and performance and needed to run a variety of unstructured workflows on-prem or in the public cloud.

With data security, detection, and monitoring being top of mind in this era of unprecedented cyber threats, OneFS 9.9 brings an array of new features and functionality to keep your unstructured data and workloads more available, manageable, and secure than ever.

Monitoring and Alerting

On the monitoring and reporting front, OneFS 9.9 sees the debut of an automatic maintenance mode, allowing a maintenance window to be automatically triggered during an upgrade, node reboot, shutdown, or similarly disruptive event. In concert with this are the ‘noise reduction’ benefits provided by a new superfluous alerting and call-home suppression feature. Additionally, OneFS 9.9 also adds IPv6 networking support to its SupportAssist ‘phone home’ service.

Security

OneFS 9.9 now provides class of service (CoS) and quality of service (QoS) tagging via the introduction of Differentiated Service Code Point (DSCP) support within the OneFS firewall. This allows storage and network administrators to classify and separate network traffic, such as transactional data, management, bulk data, etc, in line with an organization’s security mandate and operational needs.

Upgrading

Non-disruptive cluster upgrades (NDU) get a boost in OneFS 9.9 from both enhanced SMB connection management and a new pre-upgrade health check system. The latter identifies potential issues before an upgrade can commence, improving the reliability and efficiency of system updates, and ensuring minimal disruption and optimal system performance post-upgrade.

Hardware Innovation

On the platform hardware front, OneFS 9.9 also unlocks dramatic performance and capacity enhancements – particularly for the all-flash F710, which sees the introduction of support for 61TB QLC SSDs, plus a 200Gb Ethernet backend network.

Performance

Performance is further boosted in OneFS 9.9 with the release of an NFS multipath client driver. For successful large-scale AI model customization and training, GPUs need data served to them quickly and efficiently. As such, compute and storage must be sized and deployed accordingly to eliminate potential bottlenecks in the infrastructure.

To meet this demand, the new multipath client driver enables performance aggregation of multiple PowerScale nodes through a single NFS mount point to one or many NFS clients, dramatically increasing the I/O to multiple PowerScale nodes through a single NFS mount point, for higher single-client throughput.

This can directly benefit generative AI and machine learning environments, plus other workloads involving highly concurrent streaming reads and writes of different files from individual, high throughput capable Linux servers. As such, the multipath client driver enables PowerScale to deliver the first Ethernet storage solution validated on the NVIDIA DGX SuperPOD.

In summary, OneFS 9.9 brings the following new features and functionality to the Dell PowerScale ecosystem:

Feature Info
Events and Alerts ·         Automatic maintenance mode.

·         Superfluous alerting suppression.

Upgrade ·         NDU improved pre-upgrade healthchecks

·         SMB disruption reduction.

Protocols ·         Support for a multipath NFS driver for Linux client systems.

·         Complete removal of deprecated SWIFT object protocol.

Networking ·         Nvidia Spectrum-4 switch support for F710 200GbE.

·         Support for in-field Ethernet adapter (NIC) swaps.

CoS / QoS Tagging ·         Class of service and quality of service tagging enabled via the introduction of  Differentiated Service Code Point (DSCP) support.
Platform ·         Infrastructure support for 61TB QLC drives and a 200Gb/s back-end Ethernet network on the all-flash F710 platform.
Support ·         IPv6 networking added for SupportAssist phone home service.

Additionally, since S3 has become the de facto object protocol due to its extensive features, robust performance, and wide industry adoption, while the OpenStack community’s focus has shifted significantly, OneFS 9.9 sees the complete removal of the OpenStack Swift protocol & API. Swift will remain available and supported in OneFS 9.8 and earlier, until their respective end of support dates.

We’ll be taking a deeper look at OneFS 9.9’s new features and functionality in future blog articles over the course of the next few weeks.

Meanwhile, the new OneFS 9.9 code is available on the Dell Support site, as both an upgrade and reimage file, allowing both installation and upgrade of this new release.

For existing clusters running a prior OneFS release, the recommendation is to open a Service Request with to schedule an upgrade. To provide a consistent and positive upgrade experience, Dell EMC is offering assisted upgrades to OneFS 9.9 at no cost to customers with a valid support contract. Please refer to Knowledge Base article KB544296 for additional information on how to initiate the upgrade process.

OneFS Platform API Architecture and Operation

Upon receiving an HTTP request sent through the OneFS API, the cluster’s web server (Apache) verifies the username and password credentials – either through HTTP Basic Authentication for single requests or via an established session to a single node for multiple requests.

Once the user has been successfully authenticated, OneFS role-based access control (RBAC) then verifies the privileges associated with the account and, if sufficient, enables access to either the /ifs file system, or to the cluster configuration, as specified in the request URL.

The request URL that calls the API is comprised of a base URL and end-point:

And the response is usually along the lines of the following:

Or from the CLI using the ‘curl’ utility:

# curl -X GET https://10.224.127.8:8080/platform/21/audit/settings/global --insecure --basic --user <username:password>

{
"settings" :
{
"audited_zones" : [],
"auto_purging_enabled" : false,
"cee_server_uris" : [],
"config_auditing_enabled" : false,
"config_syslog_certificate_id" : "",
"config_syslog_enabled" : false,
"config_syslog_servers" : [],
"config_syslog_tls_enabled" : false,
"hostname" : "",
"protocol_auditing_enabled" : false,
"protocol_syslog_certificate_id" : "",
"protocol_syslog_servers" : [],
"protocol_syslog_tls_enabled" : false,
"retention_period" : 180,
"system_auditing_enabled" : false,
"system_syslog_certificate_id" : "",
"system_syslog_enabled" : false,
"system_syslog_servers" : [],
"system_syslog_tls_enabled" : false
}
}

From the CLI, the ‘curl’ output can be parsed using standard shell commands and utilities. For example, to find the cluster’s OneFS version:

# curl -X GET https://10.224.127.8:8080/platform/21/cluster/config --insecure --basic --user root:a | grep -I release

"release" : "9.8.0.0",

In the event that authentication fails for some reason, a response similar to the following notification will be returned:

When it comes to finding the appropriate platform API endpoint, the following ‘describe’ suffix can be used at the base level (platform) within the URL to list all the available options:

https://<cluster_IP>:8080/platform?describe

For example:

As can be seen above, the endpoints are appropriately named to aid navigation of the API.

When used on an endpoint, rather than a path, the ‘describe’ option returns a collection of JSON of methods and fields. In addition to the supported methods (GET, POST, DELETE), the output also includes all support fields and types. Given the breadth of information, it is best viewed from a web browser using a JSON viewer add-on/extension, such as the popular ‘JSONview’ utility. For example:

In addition to the ‘describe’ syntax, the platform API also recognizes the ‘list’ suffix. This can be used at any point in the API hierarchy, in conjunction with ‘describe’, to report the available endpoint(s) for a particular OneFS feature or function. For example, to show the pAPI options for the S3 protocol:

https://<cluster_IP>:8080/platform/21/protocols/S3?describe&list

For example, we can see that OneFS currently provides seven API endpoints for the S3 protocol:

Note that the numerical ‘protocol version’ must be included in the URL – in this case version 21 (the most current).

Typically, new features and API primitives are added to new releases by incrementing the pAPI version number. As such, the endpoints functionality is consistent in each version. For example, the /1/cluster/config endpoint is not be changed to add new functionality, and any new calls and features are uprev’d and placed into the next version. That said, the version number is not guaranteed to be whole number. For example, an incremental version number (v5.1) was introduced back in OneFS 8.1.0.4 to accommodate the NDU rolling reboot endpoint. Querying a particular version will only report the API endpoints that were available at that point in time. For instance, since the S3 protocol was only added in pAPI v10, querying earlier versions will not return any of the seven current endpoints:

Additionally, if you already know the CLI command for something you can get the REST endpoint etc. by simply running:

# isi --debug <command>

The output from the debug flag contains the HTTP REST traffic in both directions, including both endpoint and payloads. For example, take the following ‘isi dedupe settings view’ CLI command output, both with and without the –debug flag:

# isi dedupe settings view

Dedupe Schedule: -

          Paths: /ifs/data

   Assess Paths: -

And:

# isi --debug dedupe settings view

2024-07-21 21:59:52,488 DEBUG rest.py:80: >>>GET ['1', 'dedupe', 'settings']

2024-07-21 21:59:52,488 DEBUG rest.py:81:    args={}

   body={}

2024-07-21 21:59:52,503 DEBUG rest.py:106: <<<(200, {'content-type': 'application/json', 'allow': 'GET, PUT, HEAD', 'status': '200 Ok'}, b'\n{\n"settings" : \n{\n"assess_paths" : [],\n"dedupe_schedule" : null,\n"paths" : [ "/ifs/data" ]\n}\n}\n')

Dedupe Schedule: -

          Paths: /ifs/data

   Assess Paths: -

The first line of output above shows that the equivalent endpoint for this CLI command is:

/1/dedupe/settings/

From this, the API URL can be inferred:

# curl -X GET https://10.1.10.20:8080/platform/1/dedupe/settings/ --insecure --basic --user <username>:<password>

{

"settings" :

{

"assess_paths" : [],

"dedupe_schedule" : null,

"paths" : [ "/ifs/data" ]

}

}

In addition to ‘curl’, OneFS also offers an ‘isi_papi_tool’ CLI utility for querying a node’s platform API endpoints. For example,  the following syntax can be used to view the status of a node’s SMB sessions via the ’16/protocols/smb/sessions’ endpoint:

# isi_papi_tool GET 16/protocols/smb/sessions | egrep "computer|client_type|encryption"

Or to see all the SMB sessions across the entire cluster with ‘isi_papi_tool’:

# for node in $(isi_nodes %{lnn}); do echo Sessions on Node $node:; isi_papi_tool GET 16/protocols/smb/sessions\?lnn=$node | egrep "computer|client_type|encryption";done

Sessions on Node 1:

Sessions on Node 2:

"client_type" : "SMB 3.1.1",

"computer" : "10.1.10.22",

"encryption" : false,

Sessions on Node 3:

Sessions on Node 4:

Sessions on Node 5:

"client_type" : "SMB 3.1.1",

"computer" : "10.1.10.25",

"encryption" : true,

Sessions on Node 9:

In the next article in this series, we’ll turn our attention to OneFS RAN, the RESTful namespace access API.