Um Jupyter-Notebooks effizient und ohne Port-Forwarding im Cluster zu nutzen gibt es einen JupyterHub unter https://jupyter.hpc.rz.uni-duesseldorf.de
Dort können Jobs mit vordefinierten Ressourcen abgeschickt werden, innerhalb derer dann das Notebook gestartet wird.
Derzeit werden dort nur Shells und Python 3 - Notebooks angeboten, aber die Liste soll erweitert werden.
Install packages
Install individual Python kernel (Wait ... under construction .....)
Jupyter allows you to work in your own virtual environment using conda. Start by creating a new conda environment:
module load Miniconda/3.1 conda create -p /gpfs/project/$USER/py310 python=3.10 conda activate /gpfs/project/$USER/py310
Hint: this only works if you have defined a .condarc with channels pointing to our repo server (see also Conda)
Install the programs that you need with conda install
, at least ipykernel
must be installed:
conda install ipykernel
Create a new file "kernel.sh" in the main directory of your environment and make it executable
cd /gpfs/project/$USER/py310 vi kernel.sh kernel.sh ---------------------------------------------------- #!/bin/bash export PYTHONHOME=/gpfs/project/$USER/py310/lib/python3.10/site-packages export PATH=/gpfs/project/$USER/py310/bin:$PATH exec python -m ipykernel $@ ---------------------------------------------------- chmod a+x kernel.sh
Create a new directory for your kernel in your /home/.local/share folder
mkdir -p /home/$USER/.local/share/jupyter/kernels/py310 cd /home/$USER/.local/share/jupyter/kernels/py310
Create a new file "kernel.json" with contents (!!replace $USER with your explicit username here!!)
{ "argv": [ "/gpfs/project/$USER/py310/kernel.sh", "-f", "{connection_file}" ], "display_name": "Python 3.10 (conda)", "language": "python", "metadata": { "debugger": true } }
In your next jupyterhub session a new kernel with the name "Python 3.10 (conda)" will then be available.
Hint: This seems to only work with Python versions < 3.11 !