How do I get started with Jupyter Notebooks?

What is the Jupyter Notebooks service?

MSI has opened to all users as a new MSI Beta service. This interactive computing environment requires only a web browser, and enables data analysis and visualization on our HPC resources in a shareable, reproducible notebook format. Notebooks currently supports Python 2.7, Python 3.4, and R 3.2. MSI Python tutorial materials provide a useful overview.

There are many excellent tutorials and videos online that explain how to use Jupyter Notebooks.

How do I connect to Notebooks at MSI?

You can connect to the Jupyter Notebooks service using any modern web browser. You do not need to install any special software. If you are off-campus you will need to connect through the University VPN.

  1. To begin, visit in your browser. You will be prompted to log in with your University InternetID. You will be asked to select your group if you belong to more than one group.
  2. If you do not already have a running Jupyter session, you will be prompted to "Start My Server".
    If your Jupyter Notebooks server is already running, you will be redirected directly to the server dashboard.
  3. On the "Spawner Options" page, select a job type (see below for additional information) and click "Spawn" to start your session.
  4. Please be patient. Usually your Jupyter Notebooks server will start in under a minute, but during busy periods you may have to wait up to several minutes, especially if you have selected one of the larger job types.
  5. Once your session starts, you will be redirected to the Jupyter Notebooks server dashboard. From here you can browse, edit, or upload files. You can open a ".ipynb" saved notebook by clicking on it. You can launch a new notebook or command line terminal from the "New->" menu in the upper right corner.
  6. When you are done working, you can end your session from the "Control Panel" button in the upper right corner. Unless you end your session, your server will continue running until the requested time runs out, even if you log out or close your web browser.

What do the "Spawner Options" mean?

Your session is provided by a Jupyter Notebook server running on your behalf on a scheduled HPC resource. Behind the scenes a PBS job is created and scheduled to run for you. The "Spawner Options" form allows you to select the resources requested for that job. The currently supported options include:

  • Interactive Cluster - the job will run in the "lab" queue, which provides rapid access to interactive compute resources. This resource has access to the Internet for downloading data or installing software modules. There is only one job type for this resource:
    • 2 CPU cores, 4 GB RAM, 8 hour run time limit
  • Mesabi - the job will run on the Mesabi cluster, which is our primary HPC resource. For on-demand parallel applications, it is possible to submit additional jobs to the Mesabi cluster from inside this type of session. This resource is blocked from accessing the Internet. There are several available job types:
    • 2 CPU cores, 4 GB RAM, 8 hour run time limit (a basic job that will usually start very quickly)
    • 12 CPU cores, 128 GB RAM, 4 hour run time limit (uses the high-memory queue)
    • 2 CPU cores, 4 GB RAM, 24 hour run time limit