Page not found

CRYSTAL

Software Support Level: 
Secondary Support
Software Description: 

The CRYSTAL program computes the electronic structure of periodic systems within Hartree Fock, density functional or various hybrid approximations. The Bloch functions of the periodic systems are expanded as linear combinations of atom centred Gaussian functions. Powerful screening techniques are used to exploit real space locality. The code may be used to perform consistent studies of the physical, electronic and magnetic structure of molecules, polymers, surfaces and crystalline solids. 

 

Software Access Level: 
Open Access
Software Categories: 
Software Interactive/GUI: 
No

This tutorial provides an introduction on how to write a parallel program using OpenMP, and will help researchers write better and more portable parallel codes for shared memory Linux nodes.

netcdf

Software Support Level: 
Secondary Support
Software Description: 

NetCDF (network Common Data Form) is an interface for array-oriented data access and a library that provides an implementation of the interface. The netCDF library also defines a machine-independent format for representing scientific data. Together, the interface, library, and format support the creation, access, and sharing of scientific data.

Load one of the netcdf modules to add the netcdf bin directory to your path and set NETCDF_INCLUDE and NETCDF_LIB environmental variables appropriately. You can also use full paths to the lib and include directories to build your code. Use "module avail netcdf" to find supported builds on each system.

Software Access Level: 
Open Access
Software Categories: 
Software Interactive/GUI: 
No
General Linux Documentation: 

To load this software interactively in a Linux environment run the command:

module load netcdf

Several versions of netcdf are available, but the versions may be different on different platforms. To list all versions of netcdf available on the machine, type 

module avail netcdf

To load specific version of this software interactively in a Linux environment run the command:

module load netcdf/[software version]

antiSMASH

Software Description: 

antiSMASH - antibiotics & Secondary Metabolite Analysis SHell

From the web site:

The secondary metabolism of bacteria and fungi constitutes a rich source of bioactive compounds of potential pharmaceutical value, comprising biosynthetic pathways of many chemicals that have been and are being utilized as e.g. antibiotics, cholesterol-lowering drugs or antitumor drugs.
Interestingly, the genes encoding the biosynthetic pathway responsible for the production of such a secondary metabolite are very often spatially clustered together at a certain position on the chromosome; such a compendium of genes is referred to as a 'secondary metabolite biosynthesis gene cluster'.
This genetic architecture has opened up the possibility for straightforward detection of secondary metabolite biosynthesis pathways by locating their gene clusters. In recent years, the costs of sequencing bacterial and fungi has dropped dramatically, and many genome sequences have become available. Based on profile hidden Markov models of genes that are specific for certain types of gene clusters, antiSMASH is able to accurately identify the gene clusters encoding secondary metabolites of all known broad chemical classes.
antiSMASH not only detects the gene clusters, but also offers detailed sequence analysis.
Software Support Level: 
Secondary Support
Software Interactive/GUI: 
No
General Linux Documentation: 

The antiSMASH application cannot be run as a loaded module. It must be installed and run from local user space.

An antiSMASH install script is available on the MSI Lab cluster, Itasca and Mesabi.

Steps for a local installation:

1) Login to MSI Lab cluster, Itasca orMesabi.

2) Create an install directory within your local directory (for instance /home/YOUR_GROUP/YOUR_USER_NAME/antiSMASH)

3) CD to the directory created in step 2 and run the following command:

>/home/support/public/antiSMASH/msi_antismash_install.sh 

This command will load modules, download and untar code, download and prepare needed databases all within the directory created in step 2. The install will take a few minutes.

4) To run, put the 'run_antismash_wrapper' script into your path.

Do not call run_antismash.py directly. Use the wrapper since it invokes the virtaul environment.

4) The command

./run_antismash_wrapper.sh -help

will provide a list of available antiSMASH commands.

Help documentation is available from the antiSMASH web site.

PacBio SMRT Analysis Portal

Software Support Level: 
Primary Support
Software Description: 

The PacBio Single Molecule Real Time (SMRT) analysis portal is an easy-to-use web-based platform for analyzing 3rd generation sequencing data generated from the PacBio SMRT platform.  Currently, workflows for microbial whole genome assembly, resequencing analysis, transcriptome analysis and various data processing steps are available through the portal.  For more information on the analysis portal itself, see http://www.pacb.com/devnet/and the tutorial materials. The software must be run from a browser in the MSI network.  This can be achieved via connection through the NICE interface, or by working directly in one of the MSI laboratories. Due to limits in RAM the portal does not run reliably on the lab queue, so execution is supported for Mesabi only. Genomes up to 100 Mbp in size can be successfully run on Mesabi.

 
 

 

Software Access Level: 
Open Access
Software Categories: 
Software Interactive/GUI: 
No
General Linux Documentation: 

Instructions for SMRT Link version 3

Initial setup (only needed once unless re-installing)

  1. Get a MSI account (https://www.msi.umn.edu/content/eligibility-getting-access)
  2. Setup your NICE client. (https://www.msi.umn.edu/support/faq/how-do-i-obtain-graphical-connection-using-nice-system). You will need to download the DCV client first.  You must use NICE to access the PacBio analysis portal at MSI remotely or you can come use the computer lab in Walter 575 or Cargill 138.
  3. SSH to MSI, then in the Terminal type: "/home/support/public/smrtlink311/install.sh" then hit return.  This will set up the pacbio portal files in your home directory under the folder name "smrtlink".

Running the PacBio portal (Mesabi queue for genomes < 100 Mbp)

Note: you must request a service unit (SU) allocation on Mesabi before proceeding with these instructions.

  1. Open an NICE session (non-GPU session, with more RAM and time for larger genomes). 
  2. Within the NICE session open a terminal
  3. In the Terminal:
    Type: "ssh -Y login" then enter your MSI account password and hit return to enter the gateway login node.
    Type: "ssh -Y mesabi" then enter your MSI account password and hit return to enter the HPC system.
    Type: "qsub -I -l nodes=1:ppn=8,walltime=24:00:00 -X" then hit return. [NOTEthe first -I is a capitol i and the second -l is a lowercase L.] 
    [NOTE2: if you have a larger genome, e.g.,  > ~30 Mbp, see advanced user tips below]
    When prompted enter your MSI account password then hit return
    Wait for job to start
    Type: "/home/support/public/smrtlink311/start.sh" then hit return.  This will start the portal server going.  Copy down the URL for use in the next step.
  4. In the same Terminal window and isub session:
    Type: "firefox &" then hit return
    When Firefox opens, enter the URL you copied down in the previous step into the browser address bar.  It will look something like this "http://cn0575:9090/".
  5. Note: If you receive an error message about needing to use Chrome, please follow these steps:
    • In the address bar type "about:config" and hit return.
    • In the list that appears, right-click, and select New -> String from the pop-up menu.
    • For "Enter the preference name", enter "general.useragent.override" (without quotes) and click "OK".
    • For the string value, paste "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2895.0 Safari/537.36" (without quotes) and click "OK".
  6. Do not exit the browser until the job is complete.  You can always close out the NICE server and Save your session, reconnecting later as you please.  The job will run in the background.
  7. When you are all complete be sure to clean up the session by running the following script: Type: "/home/support/public/smrtlink311/stop.sh" then hit return. Then type "exit" to exit your isub session.

 

Changing from local jobs to PBS

Edit `userdata/config/preset.xml`, change "False" to "True" for pbsmrtpipe.options.distributed_mode.
<!-- Enable Distributed Mode -->
<option id="pbsmrtpipe.options.distributed_mode">
    <value>True</value>
</option>
 
Edit `userdata/config/smrtlink.config`, change "NONE" to "PBS" for jmsselect__jmstype.
jmsselect__jmstype='PBS';

Instructions for SMRT Link version 2

Initial setup (only needed once unless re-installing)

  1. Get a MSI account (https://www.msi.umn.edu/content/eligibility-getting-access)
  2. Setup your NICE client. (https://www.msi.umn.edu/support/faq/how-do-i-obtain-graphical-connection-using-nice-system). You will need to download the DCV client first.  You must use NICE to access the PacBio analysis portal at MSI remotely or you can come use the computer lab in Walter 575 or Cargill 138.
  3. Open a NICE session. Choose one of the non-GPU options that meets your needs for time and RAM usage.
  4. Within the NICE session open a terminal
  5. In the Terminal:
    Type: "isub -m 8gb -w 24:00:00" then hit return
    When prompted enter your MSI account password then hit return
    Wait for job to start
    Type: "/home/support/public/smrtanalysis230v2/pacbio_user_setup_230.sh" then hit return.  This will set up the pacbio portal files in your home directory under the folder name "smrtanalysis".

Running the PacBio portal (Mesabi queue for genomes < 100 Mbp)

Note: you must request a service unit (SU) allocation on Mesabi before proceeding with these instructions.

  1. Open an NICE session (non-GPU session, with more RAM and time for larger genomes). 
  2. Within the NICE session open a terminal
  3. In the Terminal:
    Type: "ssh -Y login" then enter your MSI account password and hit return to enter the gateway login node.
    Type: "ssh -Y mesabi" then enter your MSI account password and hit return to enter the HPC system.
    Type: "qsub -I -l nodes=1:ppn=8,walltime=24:00:00 -X" then hit return. [NOTEthe first -I is a capitol i and the second -l is a lowercase L.] 
    [NOTE2: if you have a larger genome, e.g.,  > ~30 Mbp, see advanced user tips below]
    When prompted enter your MSI account password then hit return
    Wait for job to start
    Type: "/panfs/roc/pacbio/start_user_portal.sh" then hit return.  This will start the portal server going.  Copy down the admistrator username/password and URL for use in the next step.
  4. In the same Terminal window and isub session:
    Type: "firefox -no-remote &" then hit return
    When Firefox opens, enter the URL you copied down in the previous step into the browser address bar.  It will look something like this "http://cn0575:8080/smrtportal/".
    When prompted for your username and password, enter the administrator username/password you copied in the previous step.
  5. Do not exit the browser until the job is complete.  You can always close out the NICE server and Save your session, reconnecting later as you please.  The job will run in the background.
  6. When you are all complete be sure to clean up the session by running the following script: Type: "/panfs/roc/pacbio/stop_user_portal.sh" then hit return. Then type "exit" to exit your isub session.

Advanced users hints

Queue speedups

If you installed your PacBio portal prior to September 25, 2015, your portal is probably set up to use the PBS system, which tends to experience serious delays when running on Mesabi.  The portal works much better in multi-threaded mode, rather than in cluster mode.  So, you'll need to change a few things in a couple of config files.  Edit the following 2 files:

$HOME/smrtanalysis/install/smrtanalysis_2.3.0.140936/analysis/etc/user.smrtpipe.rc

$HOME/smrtanalysis/install/smrtanalysis_2.3.0.140936/analysis/etc/smrtpipe.rc

change CLUSTER_MANAGER = PBS to CLUSTER_MANAGER = BASH in both of those files.

You can continue to follow the instructions above (mesabi section).  When you run 'top', you should now see many processes happening in your local node, and qstat -u USERNAME should only show your single interactive batch job.

Adding more processor cores and memory for genomes > 30 Mbp

By default, we've set up the configuration files to use only 8 processor cores and 24 hours of walltime.  But if you have a large genome, you will greatly benefit from increasing these limits, and you may need more memory.  On Mesabi, you may request up to 96h of walltime and 32 processor cores on the ram1t nodes (See queue table specs here).  To take advantage of these increases, edit the 2 files:

$HOME/smrtanalysis/install/smrtanalysis_2.3.0.140936/analysis/etc/user.smrtpipe.rc

$HOME/smrtanalysis/install/smrtanalysis_2.3.0.140936/analysis/etc/smrtpipe.rc

change MAX_THREADS = 8 to MAX_THREADS = 32 in both of those files (assuming 32 is the number of cores you want to use).

change TMP = /tmp to TMP = /scratch.global/<your-user-name> to avoid overflowing the memory in /tmp for large genomes.

Then login to mesabi or itasca by ssh-ing to one of those machines and submit a request for an interactive queue submission: "qsub -I -l nodes=1:ppn=32,walltime=96:00:00 -q ram1t -X", for example.  Then follow the normal instructions.

Troubleshooting errors

If you get an error in the setup or running of the PacBio server, try the steps once more.  If it still fails, try the following:

  1. Open an NICE session. 
  2. Within the NICE session open a terminal
  3. In the Terminal:
    Type: "isub -m 8gb -w 24:00:00" then hit return
    When prompted enter your MSI account password then hit return
    Wait for job to start
    If you wish to save data from previous runs, move or make a copy of your current ~/smrtanalysis directory before proceding to the next step.
    Type: "/panfs/roc/pacbio/delete_user_portal.sh" then hit return.  This will delete your existing portal data and pending jobs.  Exit the current isub session by typing: "exit" and retry the steps for running the portal above.
  4. If you continue to have problems, send a email to "help@msi.umn.edu", being careful to include "trouble running PacBio portal" in the subject line.
 

Developer to Teach Hands-On PETSc Tutorial on Sep 30

posted on September 4, 2013 Dr. Matthew G. Knepley , University of Chicago , will present a hands-on tutorial for PETSc at MSI on September 30. Dr. Knepley is one of the authors of the widely used PETSc library for scientific computing from Argonne National Laboratory , and is a principal designer...

HyPhy

Software Description: 

HyPhy stands for hypothesis testing using phylogenies. It is an open-source software package for the analysis of genetic sequences using techniques in phylogenetics, molecular evolution, and machine learning. It also features a complete graphical user interface (GUI) and a rich scripting language for limitless customization of analyses. Additionally, HyPhy features support for parallel computing environments (via message passing interface) and it can be compiled as a shared library and called from other programming environments such as Python or R.

Software Support Level: 
Primary Support
Software Access Level: 
Open Access
Itasca Documentation: 

To launch this software interactively in a Linux environment run the commands:

module load hyphy
HYPHYMPI

It requires a batch file to run the job.  For more detailed instruction on the program, please refer to:

http://hyphy.org/w/index.php/Main_Page

Software Categories: 
Lab Documentation: 

To launch this software interactively in a Linux environment run the commands:

module load hyphy
HYPHYMPI

It requires a batch file to run the job.  For more detailed instruction on the program, please refer to:

http://hyphy.org/w/index.php/Main_Page

Software Interactive/GUI: 
No

Python

Software Support Level: 
Primary Support
Software Description: 

Python is a high level programming language that aims to combine remarkable power with very clear syntax. Its success in these areas has led to strong adoption by the scientific community, resulting in numerous math, physics, chemistry, and biology libraries being contributed from the community of users.

Software Updates and Version Consistency

MSI must make periodic updates to our existing Python installations in order to support the latest features available in the language, and for compatibility with other Python-supported software. In order to maintain a consistent Python environment, users are encouraged to use the conda tool available in the Anaconda Python distribution to clone the desired environment into their home directory. Environments cloned in this way will not be updated by MSI and are therefore suitable for use in applications that require a static Python environment. Instructions for cloning a Python environment can be found on MSI’s Anaconda Python software page.

 

Software Access Level: 
Open Access
PBS Example: 

Programs can be submitted to a queue using PBS script such as the one below:

#PBS -l nodes=1:ppn=1,mem=1gb,walltime=4:00:00
#PBS -m abe
cd /location/of/the/script
module load python-epd
python myscript.py
Software Categories: 
Software Interactive/GUI: 
No
General Linux Documentation: 

MSI maintains installations of Python optimized for scientific and development use on all MSI computational resources. These include additional tools and libraries such as IPython, SciPy, NumPy, BioPython, Django, and many others. MSI supports the Enthought Python Distribution/Canopy and Continuum Analytics Anaconda Python as well as the PyPy alternative Python implementation.

To enable our default version of Python 2.7 (currently a version of the Enthought distribution), type

module load python

To enable our recommend version of Python 3 (currently a version of the Anaconda distribution), type

module load python3

Several other versions of Python are available, but the versions may be different on different platforms. To list all versions of Python available on the machine, type

module avail python

Assistance with Python programming is available by contacting help@msi.umn.edu.

Python Usage

The following information applies to all supported versions of Python. Additional information relevant to a particular environment can be found at the following links:

In the examples below we use module load python. They will also work if you select a specific environment version (python3/3.4) or implementation (python-epd, python-pypy). Loading a module for Python will make that Python environment your default for all Python-related commands.

To run Python interactively in a Linux environment run the commands:

module load python
ipython

If you would like to run a Python script, myscript.py, use the following command:

module load python
python myscript.py

Installing Packages

Our recommended environments for Python 2.7 (module load python) and Python 3.4 (module load python3) include a wide variety of useful scientific, mathematical, and programming packages already installed. You can usually determine if a package is installed by attempting to import its module from Python:

module load python
ipython

> import somepackage

If this succeeds without printing an error, the package is installed and ready to use. Otherwise, in most cases we recommend that you use the provided tools to install a local copy of the package in your home directory.

The simplest way to accomplish this is to use pip to install from the Python Package Index. Once you have determined the PyPI name of the package you want, you can install or upgrade the package with the commands:

module load python
pip install --user --upgrade packagename

For more complex installations you may want to create a Python virtual environment in your home directory. The commands to do so are not the same across all Python environments. Please see the software documentation page for the environment you are using for instructions.

AMDIS NIST

Software Description: 

The Automated Mass Spectral Deconvolution and Identification System (AMDIS) is a computer program that extracts spectra for individual components in a GC/MS data file and identifies target compounds by matching these spectra against a reference library. It was developed at NIST with support from the United States Department of Defense and is freely available

Software Support Level: 
Secondary Support
Software Access Level: 
Open Access
Citrix Documentation: 

To run this software under Windows, connect using instruction provided in our Windows systems page. Once logged in, navigate to

Start > All Programs > AMDIS.

Software Categories: 
Software Interactive/GUI: 
No

ddt

Software Support Level: 
Secondary Support
Software Interactive/GUI: 
No
General Linux Documentation: 

To load this software in a Linux environment run the command(s):

module load ddt

Pages