Tucked amongst the payload of the recent OneFS 9.11 release is new functionality that enables a seamless migration from EMC Secure Remote Services (ESRS) to Dell Technologies Connectivity Services (DTCS). DTCS, as you may recall from previous blog articles on the topic, is the rebranded SupportAssist solution for cluster phone-home connectivity.
First, why migrate from ESRS to DTCS? Well, two years ago, an end of service life date of January 2024 was announced for the Secure Remote Services version 3 gateway, which is used by the older ESRS, ConnectEMC, and Dial-home connectivity methods. Given this, the solution for clusters still using the SRSv3 gateway is to either:
- Upgrade Secure Remote Services v3 to Secure Connect Gateway v5.
- Upgrade to OneFS 9.5 or later and use the SupportAssist/DTCS ‘direct connect’ option.
The objective of this new OneFS 9.11 feature is to help customers migrate to DTCS so they can achieve their desired connectivity state with as little disruption as possible.
| Scenario | After upgrade to OneFS 9.11 |
| Clusters with ESRS + SCGv5 | Seamless migration capable |
| New cluster | DTC is the only way for connectivity.
“isi esrs”, “Remote Support” in the WebUI will either be unavailable or hidden. |
| Clusters without ESRS/SupportAssist/DTC configured | Same as above |
| Clusters with ESRS + SRSv3 | Retain ESRS + SRSv3
HealCheck warning triggered WebUI banner showed the migration did not happen Resolution is to upgrade to SCGv5 or use direct connection retry command is “isi connectivity provision start –retry-migration” |
So when a cluster that has been provisioned with ESRS using a secure connect gateway is upgraded to OneFS 9.11, this feature automatically attempts to migrate to DTCS. Upon successful completion, any references to ESRS will no longer be visible.
Similarly, on new clusters running 9.1 1, the ability to provision with ESRS is removed, and messaging is displayed to encourage DTCS provisioning and enablement.
Under the hood, the automatic migration architecture comprises the following core functional components:
| Component | Description |
| Upgrade commit hook | Starts the migration job.
|
| Healthcheck connectivity_migration checklist | A group of checks used to determine if automatic migration can proceed.
|
| Provision state machine | Updated to use the ESE API /upgradekey for provisioning in migration scenario.
|
| Job Engine Job | · Precheck: Runs the connectivity_migration checklist which must pass
· Migrate settings: Configure DTCS using ESRS and Cluster identity settings · Provision: Enables DTCS, starts a provision task using state machine |
There’s a new healthcheck checklist called ‘connectivity migration’, which contains a group of checks to determine whether its safe for an automatic migration to proceed.
There’s been an update to the provision state machine so that it now uses the upgrade key from the ESE API, so that we can provision in the migration scenario.
And the final piece is the migration job. Executed and managed by the Job Engine, this migration job has 3 phases.
The first, or pre-check, phase runs the connectivity migration checklist. All the checklist elements must to pass in order for the job to continue.
If the checklist fails, the results of those checks can be used to determine what remedial actions are needed in order to get the cluster to their desired connectivity state. When it does pass, the job progresses to the migration settings phase. Here, the required configuration data is extracted from ESRS and the cluster settings in order to configure DTCS. This includes items like the gateway host, customer contact info, telemetry settings, etc. Once the DTCS configuration data is in place, the job continues to its final phase, which spawns the actual provision task.
After enabling DTCS, the provisioning state machine uses the ESRS API key that was configured or paired with the configured gateway, which it uses and passes to the ESE API upgrade key, associate the key with the new ESE back end. Once that’s in place, DTCS provisioning via the upgrade hook background process.
A new CELOG alert has been added that will be triggered if DTCS provisioning fails during a seamless migration. This alert will automatically open a service request with a sev3 priority, and recommends contact Dell Support for assistance.
The connectivity CLI are minimal in OneFS 9.11, and essentially comprise providing messaging based on the state of the underlying system. The following example is from a freshly installed OneFS 9.11 cluster, where any ‘isi esrs’ CLI commands now display the following ‘no longer supported’ message:
# isi esrs view Secure Remote Services (SRS) is no longer supported. Use Dell Technologies connectivity services instead via 'isi connectivity'.
A cluster that’s been upgraded to OneFS 9.11, but fails to automatically migrate to DTCS will display a message stating that SRS is at the end of its service life.
# isi esrs view Warning: Secure Remote Service is at end of service life. Upgrade connectivity to Dell Technologies connectivity services now using ‘isi connectivity’ to prevent disruptions. See https://www.dell.com/support/kbdoc/en-us/0000152189/powerscale-onefs-info-hubs cluster administration guides for more information.
There’s also a new ‘–retry-migration’ option for the ‘isi connectivity provision start’ command:
# isi connectivity provision start --retry-migration SRS to Dell Technologies connectivity services migration started.
This can be used to rerun the migration process once any issues have been corrected, based on the results of the connectivity migration checklist.
Finally, upon successful migration, a message will inform that ESRS has been migrated to DTCS and that ESRS is no longer supported:
# isi esrs view Secure Remote Services (SRS) connectivity has migrated to Dell Technologies connectivity services. Use ‘isi connectivity’ to manage connectivity as SRS is no longer supported.
Similarly, the WebUI updates will reflect the state of the underlying system. For example, on a freshly installed OneFS 9.11 cluster, the WebUI dashboard will remind the administrator that Dell Technologies Connectivity Services needs to be configured:
On the general settings page, the tab for ‘remote support’ has been removed in OneFS 9.11:
In the diagnostics gather when the checkbox comes up, the option for ESRS uploads has been removed and replaced with the DTCS upload:
And on a fresh OneFS 9.11 cluster, the remote support channel is no longer listed as an option for alerts:
If a migration does not complete successfully, a warning is displayed on the remote support tab on the general settings page informing that the migration has failed. This warning also provides information on how to proceed:
The WebUI messaging prompts the cluster admin to resolve the failed migration by examining the results of that checklist, and provides a path forward.
The alert is also displayed on the licensing tab, because at this point the connectivity needs to be reestablished because the migration failed:
The WebUI messaging provides steps to help resolve any migration issues. Plus, if a migration has failed, the ESRS upload will still remain present and active until DTCS is successfully provisioned:
Once successfully migrated, the WebUI dashboard will confirm this status:
The dashboard will also confirm that DTCS is now enabled and connected via the SCG:
Additionally, the ‘remote support’ tab and page are no longer visible under general settings, and the former ESRS option is replaced by the DTCS option on the gather menu:
When investigating and troubleshooting connectivity migration issues, if something goes wrong with the migration job, examining the /var/log/isi_ job_d.log file and search for ‘EsrsToDtcsMigration’ can be a useful starting point. For additional detail, increasing the verbosity to ‘debug logging’ for the isi_job_d service and retrying the migration can also be helpful.
Additionally, the ‘isi healthcheck evaluations’ command line options can be used to query the status of the connectivity_migration checklist, to help determine which of the checks has failed and needs attention:
# isi healthcheck evaluations list # isi healthcheck evaluations view <name of latest>
Similarly, from the WebUI, navigating to Cluster management > Job operations displays the job status and errors. While Cluster Management > Healthcheck > Evaluations tab allows the connectivity_migration checklist details to be examined.
Note that ESRS to DTCS auto migration is only for clusters running ESRS that have been provisioned with and are using the Secure Connect Gateway (SCG) option. Post successful migration, the customer can always switch to using a direct connection rather than via SCG, if desired.