Nvidia Driver and Driver Unlock on Debain 11 Bullseye
If you have a Nvidia GPU and would like to use the NVENC hardware for applications such as Plex or Handbrake on your Debian 11 server, chances are you will need to install both the Nvidia linux driver and the Nvidia NVENC patches that removes the restriction Nvidia places on consumer GPU that limits the number of simultaneous NVENC video encoding sessions.
You can find details of the driver patch at https://github.com/keylase/nvidia-patch
Install Nvidia Linux Driver
First, head over to the official Nvidia driver website https://www.nvidia.com/Download/index.aspx and enter your GPU information. This will let us search for the latest supported driver version.
For my Nvidia Quadro P400, the latest supported driver is version 515.76.
Then, head over to the the driver patch github https://github.com/keylase/nvidia-patch and check the supported Nvidia Driver version.
If the driver is listed on the github page, you can right-click the Driver Link to copy the link. Then, SSH into your Debian 11 Server. Use wget to download the Nvidia Driver.
wget https://international.download.nvidia.com/XFree86/Linux-x86_64/515.76/NVIDIA-Linux-x86_64-515.76.run
Install the Linux headers for your system:
sudo apt install linux-headers-`uname -r`
Run the Nvidia Driver install:
#bash nvidia_driver_version_name.run
sudo bash NVIDIA-Linux-x86_64-515.76.run
By default, Debian 11 will come with the open-source Nouveau driver for Nvidia GPUs. The official Nvidia driver will give you the option to attempt disabling the Nouveau driver.
Reboot your system, and run the installer again. This time, the installer will proceed.
Once the install finishes, you can run the nvidia-smi tool to check the driver version and the GPU model.
nvidia-smi
Run Unlock Patch to allow unlimited NVENC sessions
Install Git if your Debian system does not already have it.
sudo apt install git
Clone the Driver Patch Github repo https://github.com/keylase/nvidia-patch
git clone https://github.com/keylase/nvidia-patch.git
Change directory into the nvidia-patch folder
cd nvidia-patch
Run the patch-fbc.sh and patch.sh shell script to apply the patch.
sudo bash patch.sh
sudo bash patch-fbc.sh
That's it!