Lustre at VPAC

advancedcomputing

Lustre is a parallel distributed file system designed to deliver high performance input/output for cluster computing; it is the de facto standard parallel file system for advanced computing.

The VPAC Lustre file system resides on a 165 TB DDN S2A high performance array that can approach 2 gigabytes per second of I/O throughput. The S2A is a sophisticated enterprise class storage system with higher costs than commodity consumer storage, so Lustre capacity should only be used for active projects.

Each project using Trifid has access to the Lustre file system via the /lustre directory. If your project is 'project_ID' then you will have a Lustre project directory named '/lustre/project_ID'. Eg. project pVPAC001 will access lustre through directory ‘/lustre/pVPAC001’.

All members of each project share a single ‘/lustre/project_ID’ directory and the project team is responsible for managing the capacity used.. Users should not assume file permissions and each user is responsible for assuring all their file and directory permissions suit the needs of their project and their project members.

Storage quotas are initially set at 500GiB per active project. If you need more storage capacity please note this in your initial project application or contact the VPAC Help Desk.

If you have an inactive project at VPAC that will resume or your project does not have a ‘/lustre/project_ID’ directory, please contact the VPAC Help Desk to have Lustre capacity allocated.

Please note that the lustre filesystem is not backed up; it is configured with RAID 6 and several hot spares. VPAC's philosophy on user data has always been that the user should ensure their own data is backed up, while we will try to build systems that are as fault tolerant as possible.

Quota Issues for Moving files between /home and /lustre/project_ID

Do not:

  • move (mv) files between /home and /lustre/project_ID directories;
  • move (mv) files between /lustre/project_ID and /home directories;

Do:

  • copy (cp) and delete (rm) files across the file systems (and check permissions)