When I run nvidia-smi
I get the following message:
Failed to initialize NVML: Driver/library version mismatch
An hour ago I received the same message and uninstalled my cuda library and I was able to run nvidia-smi
, getting the following result:
After this I downloaded cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64.deb
from the official NVIDIA page and then simply:
sudo dpkg -i cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64.deb sudo apt-get update sudo apt-get install cuda export PATH=/usr/local/cuda-8.0/bin${PATH:+:${PATH}}
Now I have cuda installed, but I get the mentioned mismatch error.
Some potentially useful information:
Running cat /proc/driver/nvidia/version
I get:
NVRM version: NVIDIA UNIX x86_64 Kernel Module 378.13 Tue Feb 7 20:10:06 PST 2017 GCC version: gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.4)
I'm running Ubuntu 16.04.2 LTS.
Kernel release is: 4.4.0-66-generic.
Thanks!
Solution 1: Drain and reboot the worker Rebooting the node is the easiest way to fix the issue. Rebooting the node will make sure that the drivers are properly initialized after the upgrade.
A C-based API for monitoring and managing various states of the NVIDIA GPU devices. It provides a direct access to the queries and commands exposed via nvidia-smi.
You can install Nvidia drivers either using GUI or CLI method. Open “Software and Updates” app to install install Nvidia driver using GUI. OR type “ sudo apt install nvidia-driver-510 nvidia-dkms-510 ” at the CLI. Reboot the computer/laptop to load the drivers.
Surprise surprise, rebooting solved the issue (I thought I had already tried that).
The solution Robert Crovella mentioned in the comments may also be useful to someone else, since it's pretty similar to what I did to solve the issue the first time I had it.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With