File System Administration#

Instructions for creating and configuring a file system for cloud storage on CONVERGE Horizon are included below.

Create a File System#

Follow these steps to create a new file system on CONVERGE Horizon:

In the CONVERGE Horizon web interface, go to the File Systems page.
Click Create File System in the upper right.
Fill out the file system options.
Review the configuration and pricing estimate on the right. When you are satisfied with the settings, click Create File System.

Your new file system will be added to the list on the File Systems page. When the status changes to Online, the file system is ready for use.

Your team can access the file system using the File Manager or using standard command-line utilities. The file system is automatically mounted to job clusters when you run jobs.

View the File System Log#

Each file system has a system log showing detailed information from the CONVERGE Horizon backend.

On the File Systems page, click on the name of the file system to open the File System Details page. Expand the Log section to view the system log. Only the most recent information from the system log is displayed.

../_images/system_log.png — An example system log for a file system.#

View File System Metrics#

You can view real-time metrics to keep tabs on file system performance. On the File Systems page, click on the name of a file system to open the File System Details page. Expand the Node Metrics section to see recent data for CPU utilization, memory utilization, and disk IOPS.

Edit a File System#

You can edit an existing file system if you need to change the size or other settings. There are two ways to open the Edit File System page:

On the File Systems page, click the edit () button in the row for the file system you want to modify.
On the File System Details page, click Edit in the lower right.

When you specify the new file system settings, the pricing estimate will be updated to show the new monthly cost. If you are satisfied with these details, click Update File System.

Hibernate a File System#

You can hibernate a file system if your team will not be using it for an extended period of time. During hibernation, the file system’s storage device is detached from the VM node and the node is deleted, while your data remains intact on the storage device. This allows you to continue storing data securely in the cloud without paying the hardware cost of the node (storage costs still apply).

Follow these steps to hibernate a file system:

In the CONVERGE Horizon web interface, go to the File Systems page.
Click on the file system name to open the File System Details page.
In the bottom right, click Hibernate.
At the prompt, enter the name of the file system to confirm that you want to continue.
Click Hibernate.

You can wake the file system when you are ready to start using it again:

In the CONVERGE Horizon web interface, go to the File Systems page.
Click on the name of the hibernating file system to open the File System Details page.
In the bottom right, click Wake.

CONVERGE Horizon will provision a new VM node and attach the storage device to the new node. You can log in to the file system when its status changes back to Online.

Mount a File System to a Local Device#

If you have added an SSH key to your account, you can use an SSH or SFTP client to mount a CONVERGE Horizon file system to a local device. This allows you to interact with directories and files stored in the cloud as if they were on your own computer. For example, you can load the output files from a job that is running in the cloud directly into post-processing tools that are running on your local system.

Various SSH/SFTP clients are widely available, and we do not officially support specific clients. If you are interested in this functionality, we recommend working with your IT team to identify and install a suitable client for your system.

The process for mounting a file system will vary depending on which client you are using. Below, we’ve provided example commands for mounting a file system with rclone.

Example: Mounting a File System With Rclone

In a Linux/macOS terminal or Windows Command Prompt (cmd), run rclone config to add the file system’s virtual machine to your rclone configuration file. Select the option to add a new remote and then answer the prompts as shown below. Any prompt not listed below can be left blank to accept the default value.

Recommended Rclone Configuration#
Prompt	Response
name	Enter any name for the remote (in the examples below, we use `remote_name` as a placeholder for the remote name)
Storage	sftp (option 51 on Linux and option 55 on Windows)
host	Enter the IP address or DNS name shown in File System Details > Connection Information (in the examples below, we use `x.x.x.x` as a placeholder for the host)
user	opc
key_file	Enter the full path to your private SSH key (e.g., `/home/user/.ssh/id_ed25519`)

After the configuration is completed, answer Yes (y) when prompted to keep this remote, and then quit the configuration (q). The resulting rclone configuration file (rclone.conf) looks something like this:

[remote_name]
type = sftp
host = x.x.x.x
user = opc
key_file = /home/user/.ssh/id_ed25519

If you want to confirm that your rclone.conf file has the correct format, you can find the file path with rclone config file.

Example commands for mounting the file system to your local disk are shown below. To mount only a specific directory within the file system, change /mnt/fs/files to /mnt/fs/files/dir.

On Linux systems, the mount point (tmp in these examples) must exist before you run rclone mount. We recommend creating an empty directory within your home directory for the mount point. On Windows systems, the mount point will be created by rclone and cannot already exist.

For Windows, we recommend using --sftp-skip-links to skip any soft links that exist in the file system.

# Linux
rclone mount remote_name:/mnt/fs/files /home/user/tmp

# Windows
rclone mount --sftp-skip-links remote_name:/mnt/fs/files C:\Users\user\tmp

Refer to rclone documentation for more options and details.

Disable File Manager#

You can disable the File Manager for a given file system if you do not want your team to use it. Only the file system owner and team managers have permissions to do this.

Ensure that your team members have SSH access before proceeding. That will be the only way to access the file system after the File Manager is disabled.

In the CONVERGE Horizon web interface, go to the File Systems page.
Click on the file system name to open the File System Details page.
In the bottom right, click next to the Open File Manager button.
Select Disable File Manager.

Delete a File System#

When your team is done using a file system and you have downloaded all of your data from the cloud, you can delete the file system.

Caution

All files on the file system will be permanently deleted when the file system is deleted. Before proceeding, confirm that your team has downloaded any files they want to keep.

In the CONVERGE Horizon web interface, go to the File Systems page.
Click on the file system name to open the File System Details page.
In the bottom right, click Delete.
At the prompt, enter the name of the file system to confirm that you want to continue.
Click Delete.