nvidia gpu driver extension failed to provision

The extension error text is misleading here. The important part is the exit code from the NVIDIA GPU Driver Linux extension and the extension logs inside the VM.

For this VM size and OS, use the built‑in troubleshooting path for the NvidiaGpuDriverLinux extension:

1. Check the extension provisioning state
   • From Azure CLI:

          az vm extension list \
            --resource-group <your-rg> \
            --vm-name <your-vm-name> -o table
     

   • Confirm that the extension name is NvidiaGpuDriverLinux and note its provisioning state and any status message.
2. Inspect the extension log inside the VM
   • SSH into the VM.
   • Open the extension status log:

          sudo less /var/log/azure/nvidia-vmext-status
     

   • This file contains the detailed output of the driver installation and the real failure reason for the “Installation failed. Exit code 0” message.
3. Interpret the exit code
   • The extension defines specific exit codes:
     • 0 – Operation successful
     • 1 – Incorrect usage of extension
     • 10 – Linux Integration Services for Hyper‑V and Azure not available or installed
     • 11 – NVIDIA GPU not found on this VM size
     • 12 – Image offer not supported
     • 13 – VM size not supported
     • 14 – Operation unsuccessful
   • The portal message is generic; use the value and message in /var/log/azure/nvidia-vmext-status to see which of these actually occurred and follow the suggested action (for example, use a supported VM size/OS if you see 11, 12, or 13).
4. Re‑deploy the extension with a supported version and optional settings
   • The current documented version for Linux is 1.6. When deploying via CLI:

          az vm extension set \
            --resource-group <your-rg> \
            --vm-name <your-vm-name> \
            --name NvidiaGpuDriverLinux \
            --publisher Microsoft.HpcCompute \
            --version 1.6
     

   • If the issue is related to GRID/CUDA driver version compatibility (for example, kernel/driver mismatch), pass a specific driver version in --settings, such as:

          az vm extension set  \
            --resource-group <your-rg> \
            --vm-name <your-vm-name>  \
            --name NvidiaGpuDriverLinux \
            --publisher Microsoft.HpcCompute \
            --settings "{'driverVersion':'535.161'}"
     

   • If the log indicates kernel/driver issues (for example, GRID 16.x/17.x on newer kernels), follow the guidance in the log and known‑issues section (for example, downgrade the kernel to 6.8, then reinstall the extension).
5. If the extension still fails
   • Run VM assist for Linux (https://aka.ms/vmassistlinux) to validate VM Guest Agent and VM health.
   • If the log shows image/size not supported or other conditions that cannot be changed, use a supported N‑series image/size combination as documented in the N‑series driver setup article referenced from the extension docs.

If, after checking /var/log/azure/nvidia-vmext-status and adjusting driver/kernel or VM size/OS as needed, the extension continues to fail, collect that log and open a support case so the extension team can review the specific failure.

References:

• NVIDIA GPU Driver Extension for Linux
• Troubleshoot GPU extension issues for GPU VMs on Azure Stack Edge Pro GPU
• Graphics processing unit (GPU) virtual machine (VM) on Azure Stack Hub