Interactive queue use with isub

The command isub is an MSI-written wrapper to ssh and qsub, designed specifically for interactive use.

Notes on isub Use

  • When isub is run with default options, it will ssh to a compute node in a pool of nodes reserved for interactive use (not batch use, which is explained in MSI's guide to batch processing in the labs. These nodes are the lab back-end, so when your shell starts it will be using your standard MSI home directory, /home/msi/username.Command-line options can be used to submit to other queues; see isub --help for more details.
  • After you run isub, you may use the qsub command to submit batch jobs to the lab queue.
  • If you have logged in to login.msi.umn.edu with X11 forwarding enabled, isub will submit a job with X11 support. If you haven't enabled X11 forwarding, isub will submit a standard interactive job.
  • Because isub uses ssh to connect to the submission node, you will be prompted for your MSI password again. SSH keys, and optionally SSH agent forwarding, can be used to avoid needing password authentication.
  • Although you can submit a submission script, that script must already exist on the compute resource on which your interactive job will run, and you should specify a full path to the script. As such, MSI recommends that you specify no submission script at all and simply use the options to isub when submitting an interactive job.
  • The default walltime, cores, and memory that isub will request can be modified with command-line options. By default isub on the reserved interactive nodes requests 4000MB of memory, one processor core, and two hours of connectivity. If you need more than these, be sure to specify them when launching isub. In particular, if you do not request more time, your interactive session will be terminated after two hours. The maximum time you can request (in the default lab queue)  is 72 hours. The maximum number of simultaneous isub sessions you can open on the default interactive nodes is 6. Any options isub doesn't recognize are assumed to be options to qsub, and will be passed to qsub on the destination compute node.
  • isub expects that you will be using MSI's supported shell, bash. If you are using another shell, such as tcsh, we can change your shell to bash upon request. If you want to keep tcsh, you can use this:

isub --noprofile

isub Options

To view all command line options for isub,type:

isub --help

Some software may fail to operate or fail to operate correctly with the default 4GB of memory isub requests. You can increase the allocated memory by using a command line option when invoking isub. The following command would request 8GB of memory (in one node and 4 processors).

isub -n nodes=1:ppn=4 -m 8GB

If you would like an isub session with a larger amount of memory and wall clock time than in the default interactive queue, the command line could look like this:

isub -n nodes=1:ppn=4 -m 16GB -w 24:00:00

This command requests 16 GB of memory and wall clock execution time of 24 hours.

Important! Do not request more resources than you will actually use; the more you request, the longer it may take your isub session to start (particularly if your processor or memory needs mean you can only be scheduled on certain resources).

Example isub Session via login.msi.umn.edu

username@mydesktop$ ssh -X msiusername@login.msi.umn.edu
Last login: Tue May 10 15:16:06 2011 from mydesktop.mydept.umn.edu
-------------------------------------------------------------------------------
 University of Minnesota Supercomputing Institute
-------------------------------------------------------------------------------
For assistance please contact us at
https://www.msi.umn.edu/support/request,
help@msi.umn.edu, or 612-626-0802.
-------------------------------------------------------------------------------
This is a login host. Please avoid running resource-intensive tasks on 
this machine. Instead, use the isub command to be logged into a node 
appropriate for long-running interactive jobs. See 'isub --help' for options.
Or, for non-interactive tasks, submit your job to one of MSI's HPC 
resource; details at http://www.msi.umn.edu/hardware.
-------------------------------------------------------------------------------
username@login02$ isub
Password:
qsub: waiting for job 110714.elmom.msi.umn.edu to start
qsub: job 110714.elmom.msi.umn.edu ready

Directory: /home/msi/username
Tue May 10 15:16:41 CDT 2011
username@lab072$ 

At this point you can load modules, launch software including GUI software, run qsub to submit jobs, etc.

Example isub Session via nx.msi.umn.edu

First, connect via the NX client as usual. Then, type isub in a terminal window:

username@tern9$ isub
Password:
qsub: waiting for job 110714.elmom.msi.umn.edu to start
qsub: job 110714.elmom.msi.umn.edu ready

Directory: /home/msi/username
Tue May 10 15:16:41 CDT 2011
username@lab072$ 

At this point you can load modules, launch software including GUI software, run qsub to submit jobs, etc.