File systems on the Cray XT4 (hexagon)

The following file systems exist on hexagon.

/home -- file system for user home directories on hexagon. This file system is currently very small, and not mounted on the compute nodes, so it should NOT be used for running jobs. Currently the quota is set to 10Gb for each user. Files are backed up daily, except for folders called "scratch" or "tmp" and their sub-folders.

/work/users -- large external storage shared by all compute nodes on hexagon. Files are not backed up. /work should be used when running jobs since it's much larger than /home and the only filesystem available on the compute nodes. For convenience, a symlink from /work/$USER to /work/users/$USER is created. /work is a Lustre file system where the user can specify stripe size or stripe count to optimize performance. Note: as of Oct. 1st 2009 the /work/users/* directories are subject to automatic deletion dependent on modification and access time and the total usage. Oldest files will be deleted first.

/work/shared -- part of the Lustre /work filesystem dedicated to spesific projects that need semi-permanent input data used by many across a group. Not part of the automatic deletion described above.

The hexagon nodes do not have internal disks.

/migrate -- the area for archiving data, i.e., for the automatic migration of data between disk and tape. Can be used upon request only. Transfer of data to or from /migrate is only meant for users that have large collections of data (tens of Gigabytes or more, typically the result of long simulations) that need to be archived and that cannot be stored in the home directories. See here for more details.

/bcmhsm -- the area for archiving data, i.e., for the automatic migration of data between disk and tape for Bjerknes. Can be used upon request only. Transfer of data to or from /bcmhsm is only meant for users that have large collections of data (tens of Gigabytes or more, typically the result of long simulations) that need to be archived and that cannot be stored in the home directories. See here for more details.

$HOME and $USER -- when your account on one of the machines is created, you get a personal home directory (called $HOME) and one called /work/user/$USER, where $USER is your account name. The processors on all compute nodes of hexagon have only access to your /work/users/$USER directory on hexagon.

Disk quota -- all users get by default a soft quota and a hard quota for their home directory. If the soft quota is exceeded for more than 7 days, or the hard quota is exceeded, you will not be able to create any more files in your home directory. You can check your disk usage (KB), soft quota (quota) and hard quota (limit) with the 'quota' command.