Skip to content

Jupyter Hub

Jupyter Hub[1] allows for running Jupyter notebooks as Slurm jobs on compute nodes.

Danger

As this is the first deployment of Jupyter Hub on Picotte, there are and wil be unforseen problems with Jupyter Hub as more and more users attempt to use the system. Please be patient and if you have any feedback or encounter any issue, send us a detailed email describing the issue at urcf-support@drexel.edu

Overview

The JupyterHub server is located at https://picottemgmt.urcf.drexel.edu:8000/. It is only accessible after you have connected to Drexel University's VPN. You will need to use your Picotte account information to log in.

Jupyter

Jupyter Server Launch

This JupyterHub deployment utilizes BatchSpawner to launch a Slurm job on Picotte. Users' actual Jupyter server, its related Python kernels, and computation activities will be carried out on this Slurm allocation. At the moment, the Slurm job options allow users to choose the Slurm account associated with the job (--account), compute partition (--partition), number of CPUs (--cpu-per-task), amount of memory (--mem), number of GPUs (--gres=gpu:v100:1), and the number of hours (--time).

Server Options

After your job is launched successfully, you will be redirected to your Jupyter Server (Jupyter Lab format)

Jupyter Lab

You are now ready to use Jupyter Lab on Picotte!

Custom Python Kernel

  • Start a new terminal session from your Launcher (the blue button with the plus icon on the top left of the Jupyter Hub interface)

  • By default, the virtual environment for your Python kernel will use the Python executables from module pyhon/anaconda3.

Base Python

  • To switch to a different Python, you can run the following command to remove the python/anaconda3 module:
module remove python/anaconda3
  • Next, you can use module load to switch to a Python version that you prefer.
  • To create a virtual environment for your Jupyter kernel and add it to JupyterHub, run the following commands:
  • Replace testKernel with your preferred name for your kernel
python -m venv /home/${USER}/venv/testKernel
source /home/${USER}/venv/testKernel/bin/activate
pip install ipykernel
pip -m ipykernel install --user --name=testKernel
python -m ipykernel install --user --name=testKernel
  • After completion, refresh your JupyterHub page, and the new kernel will show up on your launcher:

New kernel

  • To add packages to your new kernel, you need to do the followings:
  • Open a terminal,
  • Load the preferred Python module if you did when create the virual environment,
  • Activate the virtual environment
  • Use pip install to install the packages
source /home/${USER}/venv/testKernel/bin/activate
pip install pandas numpy

Custom R Kernel

  • Start a new terminal session from your Launcher (the blue button with the plus icon on the top left of the Jupyter Hub interface)
  • Load your R module. In this example, we are using R/4.2.2
module load R/4.2.2
  • Launch R and install the IRkernel package.
  • If this is the first time you are installing packages, you will be asked to select a default non-root location and a CRAN repository.
  • Quit R when you are done. 
install.packages("IRkernel")
IRkernel::installspec()
quit()
  • Update your R kernel
cp /ifs/opt/R/4.2.2/bin/kernel.json /home/${USER}/.local/share/jupyter/kernels/ir/
  • Refresh your JupyterHub page, the new R kernel will show up in the launcher.

R kernel

  • Here is an example of how R notebook launched inside Jupyter

R notebook example

Using Jupyter Lab

More detailed documentation will be added here!