Um Jupyter-Notebooks effizient und ohne Port-Forwarding im Cluster zu nutzen gibt es einen JupyterHub unter https://jupyter.hpc.rz.uni-duesseldorf.de
Dort können Jobs mit vordefinierten Ressourcen abgeschickt werden, innerhalb derer dann das Notebook gestartet wird.
Derzeit werden dort nur Shells und Python 3 - Notebooks angeboten, aber die Liste soll erweitert werden.
Install packages
Install individual Python kernel
Jupyter allows you to work in your own virtual environment using conda. Start by creating a new conda environment:
module load Miniconda/3.1 conda create -p /gpfs/project/$USER/py310 python=3.10 conda activate /gpfs/project/$USER/py310
Hint: this only works if you have defined a .condarc with channels pointing to our repo server (see also Conda)
Install the programs that you need with conda install
, at least ipykernel
must be installed:
conda install ipykernel
Create a new file "kernel.sh" in the main directory of your environment and make it executable
cd /gpfs/project/$USER/py310 vi kernel.sh kernel.sh ---------------------------------------------------- #!/bin/bash export PYTHONHOME=/gpfs/project/$USER/py310/lib/python3.10/site-packages export PATH=/gpfs/project/$USER/py310/bin:$PATH exec python -m ipykernel $@ ---------------------------------------------------- chmod a+x kernel.sh
Create a new directory for your kernel in your /home/.local/share folder
mkdir -p /home/$USER/.local/share/jupyter/kernels/py310 cd /home/$USER/.local/share/jupyter/kernels/py310
Create a new file "kernel.json" with contents (!!replace $USER with your explicit username here!!)
{ "argv": [ "/gpfs/project/$USER/py310/kernel.sh", "-f", "{connection_file}" ], "display_name": "Python 3.10 (conda)", "language": "python", "metadata": { "debugger": true } }
In your next jupyterhub session a new kernel with the name "Python 3.10 (conda)" will then be available.
Hint: This seems to only work with Python versions < 3.11 !