Disk Space and Data Storage

PDF View

Data lies at the heart of almost everything we do in the computing world, and there are myriad of attributes that are important in figuring out where to store data and how to access it. To enable SCI users to best manage their data, SCI IT provides a number of different data storage locations. This document details those locations and their features.

If you need more disk space in a centralized data store for your research project, would like to set up a new project directory, or need to recover lost data that you weren’t able to find in your snapshots, please contact SCI IT.

Centralized Data Stores

SCI IT provides centralized disk space that is fast, highly available, accessible from any SCI computer, and backed up regularly.  Please be aware that this is the only user data that is backed up.   Any data local to servers is not backed up.

There are two main types of centralized storage:

  • Home Directories–the home directory you have when you log into a SCI computer is meant for your personal storage, and may be used however you like. These are accessed via /home/<username>.
  • Research and Other Directories–these directories are provided for specific purposes, such as for a specific research project or group, departmental web pages, etc. Legacy data is accessed via /usr/sci which will be removed at the end of the fall semester of 2025, with current data stored in /uufs/{sci,eng,chpc}.utah.edu/ and convenience pathing via /sci-it/…

Directory Layout

The following directories are meant to provide locations (e.g. directories in a linux filesystem or SMB mount directories) for each of the data categories in a straightforward and intuitive manner: 

/home

Conforming with long-standing UNIX convention, user home directories will be available as /home/<username>.

/sci-it

In order to not conflict with any existing or future root-level directories provided by the system and provide a simple, easy-to-type path, /sci-it will be used as the base directory with several subdirectories underneath it: 

  • /sci-it/projects: each research project, dataset, etc will have a subdirectory in this directory (e.g. /sci-it/projects/cibc, /sci-it/projects/cedmav, etc).
  • /sci-it/pkgs: scripts and software provided to SCI users
  • /sci-it/scratch: NFS-provided scratch space
  • /sci-it/user-www: where user files accessed via www.sci.utah.edu/~<username> can be managed
  • /sci-it/home: where user home directories can be accessed
  • /sci-it/datasets: repositories for public datasets that are used by more than one research group and don’t count against quotas

There may be other directories that will come up over time, but the aim will be for /sci-it to remain a manageable and intuitive directory for all things SCI. 

/local

This directory will be for any data that is endemic to an individual machine (i.e. local disk on the machine), such as scratch space, local datasets, etc. 

/uufs/<domain>.utah.edu

A standard that has emerged at the UofU is to use /uufs/<domain> as a directory structure that allows for data from different computing facilities to coexist without conflicting with each other (e.g. /uufs/chpc.utah.edu).  This also clarifies which organization is providing the storage space and allows for cross mounting.

Local Data

A number of linux machines in SCI have local data stores, such as individual disks or RAID/zfs/mdadm partitions, which are used for a variety of needs such as fast data access (e.g. NVME drives) or large partitions (e.g. RAID, zfs, etc).  Sometimes, however, it is convenient to be able to access those data stores on other SCI computers as well. 

In order to provide a uniform directory structure to meet all of these ends, local data stores will be made available through the /sci-it/hosts/<hostname>/<automount directory>. This can easily be configured such that when accessed on <hostname>, it will be a local mount (for performance reasons) and NFS mount on all other hosts (for convenience). 

Quotas

Each of these directories is limited to a set size (a quota) in order to manage costs and assure availability. For home directories, quotas are:

  • Faculty/SCI Payroll Staff: 250GB
  • All other accounts: 100GB

Permissions

Access to files and directories, including shared directories such as those under /uufs/sci.utah.edu/projects, is managed using standard UNIX filesystem permissions. Each researcher or research group has their own methodology for how they manage access control– if you need access to a particular set of files/directories, please contact that person or group directly about being given access.

Snapshots

If you accidentally delete data in one of the centralized data stores, you may be able to recover it yourself by copying it out of a snapshot directory. Snapshots are automatically taken on an hourly, nightly, and weekly basis by the fileservers that provide our centralized disk space. Snapshots can be accessed via the .snapshot directory, which is a special subdirectory that exists in every directory in our centralized storage. Note that the .snapshot directory will not appear unless you explicitly reference it on the command line, as in the example below, where I remove a file and then get a backed up copy from the hourly snapshot taken at 5pm (17:00 in military time):

[@memphis] ../dev/desktop > ls -al
total 40K
drwxr-xr-x 2 clake sci-it 2.5K Oct 23 17:58 ./
drwxr-xr-x 4 clake sci-it 1.0K Oct 23 17:51 ../
-rw-r–r– 1 clake sci-it 6.1K Jun 26 14:04 50unattended-upgrades
-rwxr-xr-x 1 clake sci-it 582 Sep 04 17:58 dns_check*
-rwxr-xr-x 1 clake sci-it 4.0K May 31 17:09 harden_sshd*
-rwxr-xr-x 1 clake sci-it 286 Jul 01 15:13 set_mtu*
-rwxr-xr-x 1 clake sci-it 181 Apr 30 17:06 Ubuntu_24.04_customization*

[@memphis] ../dev/desktop > rm dns_check
rm: remove regular file ‘dns_check’? y

[@memphis] ../dev/desktop > ls -al
total 36K
drwxr-xr-x 2 clake sci-it 2.0K Oct 23 17:53 ./
drwxr-xr-x 4 clake sci-it 1.0K Oct 23 17:51 ../
-rw-r–r– 1 clake sci-it 6.1K Jun 26 14:04 50unattended-upgrades
-rwxr-xr-x 1 clake sci-it 4.0K May 31 17:09 harden_sshd*
-rwxr-xr-x 1 clake sci-it 286 Jul 01 15:13 set_mtu*
-rwxr-xr-x 1 clake sci-it 181 Apr 30 17:06 Ubuntu_24.04_customization*

[@memphis] ../dev/desktop > ls -ltr .snapshot
total 132K
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-14_18_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-14_19_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-14_20_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-14_21_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-14_22_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-14_23_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-15_00_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-15_01_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-15_02_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-15_03_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-15_04_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-15_05_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-15_06_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-15_07_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-15_08_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-15_09_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-15_10_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-15_11_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-15_12_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-15_12_45_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-15_13_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-15_13_15_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 10:47 snap_2025-11-15_13_30_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 01:24 snap_2025-11-14_14_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 01:24 snap_2025-11-14_15_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 01:24 snap_2025-11-14_16_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 14 01:24 snap_2025-11-14_17_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 13 21:31 snap_2025-11-14_06_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 11 22:20 snap_2025-11-12_06_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 11 22:20 snap_2025-11-13_06_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 10 16:57 snap_2025-11-11_06_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 9 13:38 snap_2025-11-10_06_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Nov 8 18:08 snap_2025-11-09_06_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Oct 30 17:40 snap_2025-11-02_06_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Oct 24 12:56 snap_2025-10-26_06_00_00_UTC/
drwxr-xr-x 2 clake sci-it 4096 Oct 17 22:32 snap_2025-10-19_06_00_00_UTC/

[@memphis] ../dev/desktop > cp .snapshot/snap_2025-10-19_06_00_00_UTC/dns_check .

[@memphis] ../dev/desktop > ls -al
total 40K
drwxr-xr-x 2 clake sci-it 2.5K Oct 23 17:58 ./
drwxr-xr-x 4 clake sci-it 1.0K Oct 23 17:51 ../
-rw-r–r– 1 clake sci-it 6.1K Jun 26 14:04 50unattended-upgrades
-rwxr-xr-x 1 clake sci-it 582 Oct 23 17:58 dns_check*
-rwxr-xr-x 1 clake sci-it 4.0K May 31 17:09 harden_sshd*
-rwxr-xr-x 1 clake sci-it 286 Jul 1 15:13 set_mtu*
-rwxr-xr-x 1 clake sci-it 181 Apr 30 17:06 Ubuntu_24.04_customization*

[@memphis] ../dev/desktop >

Local Disk Storage

Desktops

Practically all modern desktop and workstation computers come with locally installed hard disks that provide space well in excess of what is needed by the operating system. The standard build in SCI makes the extra space not needed by the operating system available for users to access and use as scratch space.

The benefits of using this scratch space are that there is generally a lot of space available, and when used locally on a machine, it is fast. However, this scratch space is not backed up, so in the event of a hardware failure or data deletion, SCI IT will be unable to recover data. As such, this is best used for data that can be recreated or recovered from elsewhere (i.e. kept in multiple places).

Laptops

Similarly to desktops, most modern laptops come with a large amount of available space on them. Since laptops are not backed up by SCI IT, any important information on your laptop should be backed up by copying it to a centralized data directory or by some other means of backing up your laptop (e.g., 3rd-party backup software, manual copying to an external disk, etc). If you do backup your laptop outside of SCI, please be mindful about sensitive data and where/how that data may be copied.

Non-SCI data stores

The University of Utah contains many other groups and departments that provide data storage services, in addition to those provided at SCI. Two of the most relevant ones are:

UIT

UIT has several options for storage, including personal and departmental storage via Box and OneDrive, departmental shares, Google Workspace, etc. For more information, see the UIT Service Catalog.

CHPC

When using resources in the Center for High Performance Computing, there are several options for data storage, including home directories, scratch space, etc. Additionally, they do provide storage in their Protected Environment if using sensitive data. For more information, see the CHPC page on Storage Services.

/uufs/sci.utah.edu/{projects,datasets} is available in CHPC so that data can be computed on in both environments.