Seitenhistorie

...

https://jakevdp.github.io/blog/2017/12/05/installing-python-packages-from-jupyter/#How-to-use-Pip-from-the-Jupyter-Notebook

Install individual Python

...

kernel

Jupyter allows you to work in with your own virtual environment using the Python modules installed on the HPC. Start by loading the Python interpreter/module you want to work with, . You can use e.g. conda for this task. Start by creating a new conda environment:

module purge   ##just in case you are using jupyter terminal
module load PythonMiniconda/3.10.5

...

1
conda create -p /gpfs/project/$USER

...

python -m venv --prompt py310 --system-site-packages/py310 python=3.10
conda activate /gpfs/project/$USER/py310

PIP_CONFIG_FILE=/software/python/pip.conf pip install --user --upgrade pip setuptools wheel

Hint: it may be neccessary to install virtualenv as a user if this fails:  PIP_CONFIG_FILE=/software/python/pip.conf pip install --user virtualenv

Activate this environment and install all packages you want to work with - at least ipykernel must be installed

Hint: this only works if you have defined a .condarc with channels pointing to our repo server (see also Conda)

Install the programs that you need with conda install , at least  ipykernel must be installed:

conda 

source /gpfs/project/$USER/py310/bin/activate

PIP_CONFIG_FILE=/software/python/pip.conf pip install ipykernel

Create a new file "kernel.sh" in the main directory of your environment and make it executable

cd /gpfs/project/$USER/py310
vi kernel.sh

Contents of the file kernel.sh:

 
----------------------------------------------------
#!/bin/bash
module load Python/3.10.5
source export PYTHONPATH=/gpfs/project/$USER/py310/lib/python3.10/site-packages
export PATH=/gpfs/project/$USER/py310/bin/activate:$PATH
exec python -m ipykernel $@
----------------------------------------------------

Make the file executable with

chmod a+x kernel.sh

Create a new directory for your kernel in your /home/.local/share folder

...

Create a new file "kernel.json" with contents (!!replace $USER with your explicit username here!!)

{
"argv": [
  "/gpfs/project/$USER/py310/kernel.sh",
  "-f",
  "{connection_file}"
 ],
 "display_name": "VenvPython 3.10 (py310conda)",
 "language": "python",
 "metadata": {
  "debugger": true
 }
}

In your next jupyterhub session a new kernel with the name "venv (py311Python 3.10 (conda)" will then be available.

Hint: This seems to only work with Python versions < 3.11 !