How do you run a ONNX model on a GPU?

Question

I'm trying to run an ONNX model

import onnxruntime as ort
import onnxruntime.backend
model_path = "model.onnx"

#https://microsoft.github.io/onnxruntime/
ort_sess = ort.InferenceSession(model_path)


print( ort.get_device()  )

This prints out

cpu

How can I make it run on my GPU? How can I confirm it's working?

Answer 1

You probably installed the CPU version. Try uninstalling onnxruntime and install GPU version, like pip install onnxruntime-gpu.

Then:

>>> import onnxruntime as ort
>>> ort.get_device()
'GPU'