Jupyter Notebooks

Jupyter Notebook Setup

Jupyter Notebooks are essential for interactive deep learning development and experimentation.

Installation

venv + pip

# Activate your environment
source ml-env/bin/activate

# Install Jupyter
pip install jupyter notebook jupyterlab

# Launch Jupyter
jupyter notebook
# or
jupyter lab

Conda

# Activate your environment
conda activate ml-env

# Install Jupyter
conda install jupyter notebook jupyterlab

# Launch Jupyter
jupyter notebook

Adding Environment as Jupyter Kernel

To use your Python environment in Jupyter notebooks:

venv + pip

# Activate your environment
source ml-env/bin/activate

# Install ipykernel
pip install ipykernel

# Add environment as kernel
python -m ipykernel install --user --name ml-env --display-name "Python 3.11 (ML)"

# Verify it's added
jupyter kernelspec list

Conda

# Activate your environment
conda activate ml-env

# Install ipykernel
conda install ipykernel

# Add environment as kernel with descriptive name
python -m ipykernel install --user --name ml-env --display-name "Python 3.11 (ML - Conda)"

# Verify it's added
jupyter kernelspec list

Recommended naming convention:

# Include Python version and environment purpose
python -m ipykernel install --user --name tfmain --display-name "Python 3.10 (TensorFlow)"
python -m ipykernel install --user --name pytorch-env --display-name "Python 3.11 (PyTorch)"

Managing Kernels

List All Kernels

jupyter kernelspec list

Output example:

Available kernels:
  ml-env       /home/user/.local/share/jupyter/kernels/ml-env
  python3      /usr/share/jupyter/kernels/python3
  pytorch-env  /home/user/.local/share/jupyter/kernels/pytorch-env

Remove Unused Kernel

# Remove a kernel
jupyter kernelspec uninstall unwanted-kernel

# Example
jupyter kernelspec uninstall old-tf-env

Update Kernel Display Name

# Remove old kernel
jupyter kernelspec uninstall old-name

# Re-add with new display name
python -m ipykernel install --user --name myenv --display-name "New Display Name"

Jupyter Notebook Tips

Timing Cell Execution

%%time (Quick)

%%time
# Your code here
model.train()

Shows total execution time for the cell.

%%timeit (Benchmark)

%%timeit
# Your code here - runs multiple times for accurate measurement
result = model(data)

Runs code multiple times and provides statistics.

ExecuteTime Extension

# Install extension for persistent timing
pip install jupyter_contrib_nbextensions
jupyter contrib nbextension install --user
jupyter nbextension enable execute_time/ExecuteTime

# Restart Jupyter to see execution times in cells automatically

Shows execution time for every cell automatically.

GPU Monitoring in Notebooks

PyTorch

# Check GPU availability
import torch
print(f"CUDA available: {torch.cuda.is_available()}")
print(f"GPU: {torch.cuda.get_device_name(0)}")
print(f"GPU count: {torch.cuda.device_count()}")

# Monitor GPU memory
print(f"Memory allocated: {torch.cuda.memory_allocated(0) / 1e9:.2f} GB")
print(f"Memory reserved: {torch.cuda.memory_reserved(0) / 1e9:.2f} GB")

TensorFlow

# Check GPU availability
import tensorflow as tf
gpus = tf.config.list_physical_devices('GPU')
print(f"GPUs available: {len(gpus)}")
for gpu in gpus:
    print(f"  - {gpu}")

# Get GPU memory info
gpu_devices = tf.config.experimental.list_physical_devices('GPU')
for device in gpu_devices:
    tf.config.experimental.set_memory_growth(device, True)

nvidia-smi

# Monitor all GPUs in real-time
!nvidia-smi

# Watch GPU usage continuously
!watch -n 1 nvidia-smi  # Updates every second

Auto-Reload Modules

# Add to first cell - automatically reload changed modules
%load_ext autoreload
%autoreload 2

Better Output Display

# Display all outputs (not just last one)
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"

Common Issues

Issue: Empty Browser Page on Jupyter Start

Problem: Jupyter launches but shows blank page

Solution:

# Hard refresh browser
Ctrl + Shift + R  # Windows/Linux
Cmd + Shift + R   # Mac

# Or clear browser cache and restart Jupyter
jupyter notebook --no-browser
# Then manually open the URL shown

Issue: Kernel Not Found / “Dead Kernel”

Problem: Notebook can’t connect to kernel

Solution:

venv + pip

# 1. Check if kernel exists
jupyter kernelspec list

# 2. Reinstall ipykernel in environment
source ml-env/bin/activate
pip install --force-reinstall ipykernel
python -m ipykernel install --user --name ml-env --display-name "Python (ML)"

# 3. Restart Jupyter

Conda

# 1. Check if kernel exists
jupyter kernelspec list

# 2. Reinstall ipykernel in environment
conda activate ml-env
conda install --force-reinstall ipykernel
python -m ipykernel install --user --name ml-env --display-name "Python (ML)"

# 3. Restart Jupyter

Issue: Wrong Python Environment

Problem: Notebook uses system Python instead of your environment

Solution:

# Verify kernel Python path
jupyter kernelspec list
cat ~/.local/share/jupyter/kernels/ml-env/kernel.json

# Should point to your environment, e.g.:
# "/home/user/ml-env/bin/python"

# If wrong, remove and re-add kernel
jupyter kernelspec uninstall ml-env
source ml-env/bin/activate
python -m ipykernel install --user --name ml-env

Issue: ImportError for Packages Installed in Environment

Problem: Package installed but import fails in notebook

Check you’re using the right kernel:

# In notebook cell
import sys
print(sys.executable)
# Should show: /home/user/ml-env/bin/python

# If wrong, change kernel:
# Kernel → Change Kernel → Select correct environment

Issue: Jupyter Lab Extensions Not Loading

# Rebuild JupyterLab
jupyter lab build

# Or clear cache and rebuild
jupyter lab clean
jupyter lab build

Issue: “Charset Normalizer” Error

Error:

AttributeError: partially initialized module 'charset_normalizer'
has no attribute 'md__mypyc' (most likely due to a circular import)

Fix:

pip install -U --force-reinstall charset-normalizer

JupyterLab vs Jupyter Notebook

Jupyter Notebook (Classic)

• Simpler interface
• Lighter weight
• Better for single-notebook work

jupyter notebook

JupyterLab (Modern)

• Multi-tab interface
• Built-in terminal
• File browser
• Extension system

jupyter lab

Recommendation: Use JupyterLab for most work, Notebook for simplicity.

Remote Jupyter Access

SSH Tunneling

On remote machine:

jupyter notebook --no-browser --port=8888

On local machine:

# Create SSH tunnel
ssh -L 8888:localhost:8888 user@remote-server

# Open browser to http://localhost:8888

JupyterHub (Multi-user)

For team environments, consider JupyterHub:

• Multi-user support
• Resource management
• Authentication

See JupyterHub docs for setup.

Productivity Extensions

Recommended Extensions

# Install extension manager
pip install jupyter_contrib_nbextensions
jupyter contrib nbextension install --user

# Enable useful extensions
jupyter nbextension enable execute_time/ExecuteTime
jupyter nbextension enable toc2/main  # Table of contents
jupyter nbextension enable varInspector/main  # Variable inspector
jupyter nbextension enable codefolding/main  # Code folding

JupyterLab Extensions

# GPU monitoring dashboard
pip install jupyterlab-nvdashboard
jupyter labextension install jupyterlab-nvdashboard

# Git integration
pip install jupyterlab-git

Best Practices

Notebook Organization

# Cell 1: Imports and setup
import torch
import numpy as np
%load_ext autoreload
%autoreload 2

# Cell 2: Configuration
BATCH_SIZE = 32
LEARNING_RATE = 1e-3
DEVICE = 'cuda' if torch.cuda.is_available() else 'cpu'

# Cell 3+: Your work
...

Save Notebooks as Scripts

# Convert notebook to Python script
jupyter nbconvert --to script notebook.ipynb

# Generates notebook.py

Version Control

# Add to .gitignore
*.ipynb_checkpoints/
.ipynb_checkpoints

# Clean output before committing
jupyter nbconvert --ClearOutputPreprocessor.enabled=True --inplace notebook.ipynb

Next Steps

• Set up Python environments for isolated dependencies
• GPU troubleshooting if GPU not detected
• Monitor training from notebooks

Kernel Best Practices

Create one kernel per project environment. Use descriptive names like “Python 3.11 (CV Project)” to easily identify which kernel to use.