Research Cluster (Plato)

Plato is a Linux-based computing cluster designed to support your research projects. It can also be used for training highly qualified personnel on advanced computing techniques and applications.

The Plato research cluster best supports research projects which require mid-range power—those between the smaller projects that can be handled by your own computer and the larger projects which are better suited to more powerful Compute Canada/WestGrid resources.

Also, because Plato is more quickly configurable, you may prefer to use it instead of Compute Canada resources for certain problems or situations with tight timeframes.

Your access to Plato is managed by the “fairshare” scheduling algorithm.

Specifications


  • 1 head node: plato.usask.ca
    • 2 x eight-core Intel Xeon processors
    • 32 GB RAM
    • 4 TB RAIDed storage
    • TORQUE/Maui scheduling software
    • 10 Gb Ethernet to private network
  • 96 computational nodes:
  • Compute-0-0 to compute-0-31
    • 2 x eight-core Intel E5-2650L Xeon processors @1.8GHz (16 cores/node)
    • 32 GB RAM
    • 500GB 7200 RPM hard drive
    • 1 Gb Ethernet
  • Compute-0-32 to compute-0-95
    • 2 x eight-core Intel E5-2640L Xeon processors @2.0GHz (16 cores/node)
    • 32 GB RAM
    • 500GB 7200 RPM hard drive
    • 1 Gb Ethernet
  • 1Gb Ethernet private network (3 x 48 port GigE switch) between nodes, 10Gb between head node and switches.
  • Centos / ROCKS clustering software
  • 4 TB RAID storage exported to nodes
  • Local hard drives on computational nodes for high performance, non-network scratch

Software

Most of the software you will need to use the Plato research cluster can be found here:

/share/apps

Environmental variables are configured via the "module" software for given applications.

To see your currently configured modules, use

module list

To see the list of available modules, use

module avail

To load a module for one login shell/session:

module add <modulename>
or
module load <modulename>

To load a module for every future login shell/session:

module initadd <modulename>

For more information, see

man module

Compilers

Compilers

/usr/bin/gcc
/usr/bin/g77
/usr/bin/gfortran

Intel Compilers

In order to configure your session to use the Intel compilers for "mpif90" and "mpicc", you should ensure that you do not have another MPI configuration loaded:

module unload rocks-openmpi

you must use the following command to load the correct configurations:

module load intel/<version>
module load openmpi/<version>_intel

You will have to use the "initadd" command to use the correct libraries for running your programs in the batch system.

module initadd intel/<version>
module load openmpi/<version>_intel

Note that the Intel compilers include the optimised Math Kernel Library with BLAS, LAPACK, ScaLAPACK, LINPACK and FFT libraries. This utility can help you determine the library linking procedure.

Modules (Set up environment for MPI)

The "module" command controls which parallel processing environment is loaded at each shell invocation for the default installed versions of mpi on the cluster. Unless your software requires something else, use the OpenMPI environment.

module initadd openmpi/<version>_gnu
or 
module initadd openmpi/<version>_intel 

Policies

  1. ICT provides Plato for research use requiring High Performance Computing by University of Saskatchewan research groups working on U of S research projects.
  2. Groups eligible for Plato access are headed by a U of S faculty member, referred to as the Principle Investigator (PI) and include the PI and their associated students, staff and collaborators.
  3. Access to Plato is granted by ICT only on request of the PI, who manages membership of his or her group by requesting access for a list of students, staff and collaborators. The PI is responsible for the actions of the group's members on Plato, ensuring appropriate (e.g. related to U of S research projects and per university policies) work is done on Plato.
  4. There is currently no direct charge for the service to researchers.
  5. Each group is assigned an equal potential share of the system, regardless of number of members/accounts in a group. Scheduling occurs at the group level.
    1. Members of the same group are expected to coordinate their usage with each other. 
    2. Maximising a group’s actual usage in the “equal potential share” of the system requires regular submission of jobs over the long term (months and years). 
    3. Share of the system enforced by scheduling software; users shall use the job batching system, unless other arrangements have been made.
  6. Priority access for a faculty group or guaranteed allocations to Plato resources can be requested in service of faculty grant applications (and similar). For each request, ICT will determine the details of the request and make a recommendation to the HPC Advisory Committee.
  7. Non-traditional uses (e.g. custom images, web services for cluster nodes, etc.) are possible and feasibility is determined on a case-by-case basis by ICT.
  8. Limits on job parameters (e.g. maximum walltime or processor equivalents, limits to number of jobs submitted at a time, etc.) may be imposed to maintain the integrity of the system, optimize the use of the hardware and/or improve fairness of scheduling.
  9. Plato cannot by itself provide all needed HPC computational cycles for the entire university. Plato should be considered a stepping stone to shared resource usage (e.g. WestGrid/Compute Canada).
  10. For ICT to grant access to Plato for a PI and/or the members of his group, the PI must agree to:
    1. Provide a one paragraph abstract describing the research project:
      1. ICT intends to publish the abstracts on a webpage of ongoing research using Plato. ICT will ask the PI for permission (and approval of publication date, where needed) to publish the abstract.
    2. Report annually on research outcomes achieved using Plato:
      1. Provide (and allow ICT to publish on webpage or similar) a list of papers, theses and conference presentations published using results generated on Plato.
  11. Conditions of use:
    1. Runaway user jobs may be terminated with little or no warning to preserve the integrity of the system.
    2. Plato's uptime and network availability are on a best effort basis. Plato is not on emergency power or UPS.
    3. Users will arrange their own long term data storage. Some disk space is available for intermediate calculations and input data, but data on Plato in user directories is not backed up. ICT provides several data storage options. Users may be asked to remove data from Plato that is not immediately required as inputs for computation. Quotas may be enforced.
    4. There is no guarantee on availability of computational cycles to a PI Group.
    5. Development tools, parallel libraries, math libraries and compilers are provided on Plato; additional compiler/library purchase/installation requests will be handled on a case-by-case basis. ICT cannot commit in advance to the purchase or installation of all scientific libraries or software.
    6. Installation of scientific/research software, customization of Plato nodes and technical support is available on a best-effort basis from ICT.
    7. ICT/Plato administrators may make changes to the priorities of jobs, or request delay/termination of running jobs, to facilitate overall throughput.
Last modified on