In this example, we’ll see how to run Jupyter as a SLURM job, and access the notebook remotely from your laptop.
Do not run Jupyter notebooks on the login node.
Doing so will adversely affect other users.
Please use the approache explained on this page to use Jupyter notebooks.
We suggest you use the following SLURM script jupyter.sh to run Jupyter notebooks on the cluster.
This requires you have configured password-less SSH from one CS machine to the other. If you have not configured this already, refer to this page for instructions.
#!/bin/bash
#SBATCH --job-name="Jupyter-GPU-Demo" # a name for your job
#SBATCH --partition=peregrine-gpu # partition to which job should be submitted
#SBATCH --qos=gpu_debug # qos type
#SBATCH --nodes=1 # node count
#SBATCH --ntasks=1 # total number of tasks across all nodes
#SBATCH --cpus-per-task=1 # cpu-cores per task (>1 if multi-threaded tasks)
#SBATCH --mem=4G # total memory per node
#SBATCH --gres=gpu:nvidia_a100_3g.39gb:1 # Request 1 GPU
#SBATCH --time=00:15:00 # total run time limit (HH:MM:SS)
module purge
module load python/anaconda
port=8888
ssh -N -f -R $port:localhost:$port falcon
jupyter-notebook --no-browser --port=$port
Here we are using the port 8888 as an example.
Please edit the value of port in this example and use a random port (>8888) which is free, instead of 8888.
Submit the job as
sbatch jupyter.sh
Monitor your job via squeue and once the job is in running state, wait a minute for Jupyter notebook to start.
Then, in the output file (named slurm-####.out) look for the notebook URL.
Copy the URL (including the token), it should look something like
http://localhost:8888/?token=208hjb2e66fckdkjkwu83268597c8083efbbcd204fbb57e989a
Now create a SSH tunnel on your laptop, using the command:
ssh -N -f -L 8888:localhost:8888 <your-cs-username>@falcon.cs.colostate.edu
Finally, open a browser window on your laptop and paste the URL. You browser should now load the jupyter notebook.
Your Jupyter notebook will run only for the duration which you define in the SLURM job script (15min in this example).
Make sure you choose the wall time appropriate for your job.