Minnesota Supercomputer Institute

Netfinity Linux Cluster Quickstart Guide

This guide will provide you with the basic information needed to get up and running on this machine. For detailed operations information, see the detailed user manual.

How to login

To connect:

ssh -l username nf.msi.umn.edu

You only need to use -l if your netfinity login name is different than the login name you use on your own workstation.

How to compile serial code

In the examples below "-options" refers to optional flags which can be used to specify various attributes you want the compiled code to have, such as optimization level, etc. See man pages for a detailed explanation of all the compiling options for pgf77, pgf90, pgcc, and pgCC.

Fortran 77:

pgf77 -options mycode.f

Fortran 90:

pgf90 -options mycode.f

C:

pgcc -options mycode.c

or

gcc -options mycode.c

C++:

pgCC -options mycode.C

How to compile parallel code

Fortran77:

mpif77 -options myMPIcode.f

Fortran90:

mpif90 -options myMPIcode.f

C:

mpicc -options myMPIcode.c

C++:

mpiCC -options myMPIcode.C

How to interactively run a serial executable

cd working_directory
a.out

How to submit jobs to the queue using Portable Batch System (PBS)

We have only one queue that consists of all the nodes on the machine.

The per job queue limits are:

Please note that there are three types of nodes in the cluster each with different amount of memory. PBS allows you to choose a particular node to meet the memory need. If you need memory greater than 1.2 GB, you should specify -l nodes=1:ppn=1:faster in your job submission script. If you want to run sequential jobs on the blade nodes, please specify -l nodes=1:blade:ppn=1. More examples are given in the detailed user manual.

We use the Maui scheduler on the cluster. This scheduler is better than the standard PBS scheduler and allows for backfill policies and will hopefully provide good turn-around time. Since the scheduler is distinct from the queuing system, the commands used to check job status, etc. are slightly different. While the standard PBS commands (qstat, etc.) will work, they may not provide the most accurate information about running/queued jobs.

How to monitor the job status

You can check job status using the showq command.

The output is quite detailed and has three sections:

Active jobs are those that are currently running. (For those familiar with the qstat output, these are jobs that have status "R"). Idle jobs are those that are being considered for scheduling but are not running yet, and non-queued jobs are those that have been submitted but are not being considered for scheduling (because the person who submitted them has 10 jobs already queued). The presence of non-queued jobs prevents one person from monopolizing the queuing system by submitting lots of small jobs. Both idle and non-queued jobs will show up with status "Q" in qstat.

How to cancel/kill/remove my job

Jobs are killed or removed from the queuing system by using the qdel command. There is a man page for qdel that lists the options you can use with it. If you wish to signal a running job, you may use the qsig command. Again, please see the man page for information.