Skip to main content

Purchasing Resources on Quest

Full Access users of the Quest cluster have dedicated access to purchased compute nodes for a period of five years from the date of installation of the hardware. The workload manager (MOAB) places a standing reservation to reserve a subset of compute resources for a full access allocation. 

The reservation is created by specifying a list of nodes, a time frame, and an access control list. The scheduler is thus constrained to ensure that only users that belong to the allocation can use those nodes. While basic access users can use these subset of resources through the scheduler-enabled backfill mechanism, such usage is limited to jobs submitted to short queue, which ensures that a full access user will have a maximum wait time of 4 hours to access their purchased resources.

The Procedural Guidelines governing the usage of Quest remain the same as basic access but the access criteria, classes or queues, job duration and job size limits and qualities of service are different. See Full Access Job Commands.

Request Full Access to Quest

If you are interested in purchasing resources on Quest, please review the guidelines, Purchase of High Performance Computing (HPC) Central Compute Resources by Northwestern Researchers pdf, and complete the request form.

How Quest Full Access Accounts Work

  1. Full Access accounts are treated as a user group on Quest with a project space (/projects/bxxxx) and an allocation (bxxxx). However, the service units are never charged; so in effect accounting is turned off. Users can use /software/scripts/checkproject command to check the file space usage of their account.
  2. Full Access resources have the Quest standard queues (short/normal/long) enabled. In addition, there is a buyin queue where there is no usage restriction on wall time or core usage. What this means is that, in this queue, users can execute jobs for as long as they want in accordance with planned and unplanned system downtimes.
  3. Backfill is a scheduling optimization that allows the scheduler to make better use of available resources. While Moab schedules primarily through a FIFO sorted list, backfill does allow the scheduler to prioritize jobs out of order to achieve significant scheduler performance improvement. For example, a job requesting low resources (nodes/cores) might be scheduled ahead of a high resource one as long as it improves turnaround time without delaying the higher-priority job.

Accessing Full Access Reservations

Full access users must use the following directive in their submission scripts to be able to access their reservation:

#MSUB -A bxxxx

where bxxxx is the account number. In addition, it is recommended to estimate the runtime of the job and set the walltime limit accordingly:

#MSUB -l walltime=hh:mm:ss

This helps the scheduler to optimally make use of the computing resources. As for basic access users, the job gets routed to the standard Quest queues based on the requested walltime and number of cores.

For full access users there is an additional buyin queue available that has no restrictions on number of cores or walltime. In order for the job to get routed to this queue, the following has to be added to the job script:

#MSUB -q buyin

This will also set the maximum runtime to walltime, which can be chosen freely in this case, and, if not specified, defaults to 2 days.

Note: Reservations b1001-b1003 use Intel Nehalem Quad Core Xeon processors (4 cores/CPU, 8 cores/node), b1004-b1009 use Intel Westmere based Xeon processors, (6 cores/CPU, 12 cores/node), and b1011 uses Intel Sandy Bridge based 8 Core Xeon processors (8 cores/CPU, 16 cores/node). For further details see Quest architecture.

Setting Job Priorities

Moab gives users an effective way to set priorities of jobs submitted by different users belonging to an existing allocation. Jobs submitted to Moab have a system priority, which is displayed by the showq command, and a user priority. While users do not have control over system priority, Moab does allow users to set job priority values relative to their own jobs. The valid priority values are -1024 to 0 with lower values indicating lower priority compared to other jobs.

Priority values can be set in multiple ways:

Recommendation: To control relative wait times within an existing allocation (including buy-ins), use a hierarchy of user priorities to fulfill multiple, independent goals that may include giving preference to users in specific projects. This approach can be used to assign weights to the various objectives of an allocation so that a priority value can be associated with each potential scheduling decision.

For example, jobs that involve the highest degree of importance can use a value of 0 and subsequent jobs can use lower priorities like -128, -256, -512, and so on until -1024 to ensure these jobs will fall back in queue to promote any critical jobs. To review user priorities after job submission, the command mdiag -p -v can be used to display a UserPrio field for running and eligible jobs.

See examples of commands for Full Access users of Quest.

Last Updated: 7 February 2019

Get Help Back to top