Quick Steps to Install/Upgrade CUDA on Ubuntu 20.04

Hopefully, this saves you some time

Step 0: Stop anything running on your GPUs

This includes XServer, etc.

Step 1: Remove/Uninstall current drivers

You can use the following command mentioned on this website:

$ sudo dpkg -P $(dpkg -l | grep nvidia-driver | awk '{print $2}')
$ sudo apt autoremove

It might be a good idea to do a system restart at this point.

Step 2: Stop the GDM service

$ sudo systemctl stop gdm

Step 3: Download CUDA + Nvidia drivers

Step 4: Stop Nouveau drivers

Follow this documentation to do so per your OS.

Step 5: Install Driver

Note that the desired CUDA version might have requirements in terms of your driver version.

Step 6: Install CUDA

Make sure to uncheck the driver installation before you proceed.