RuntimeError: Expected object of type torch.DoubleTensor but found type torch.FloatTensor for argument #2 'weight'

Question

My input tensor is torch.DoubleTensor type. But I got the RuntimeError below:

RuntimeError: Expected object of type torch.DoubleTensor but found type torch.FloatTensor for argument #2 'weight'

I didn't specify the type of the weight explicitly(i.e. I did not init my weight by myself. The weight is created by pytorch). What will influence the type of weight in the forward process?

Thanks a lot!!

Answer 1

The default type for weights and biases are torch.FloatTensor. So, you'll need to cast either your model to torch.DoubleTensor or cast your inputs to torch.FloatTensor. For casting your inputs you can do

X = X.float()

or cast your complete model to DoubleTensor as

model = model.double()

You can also set the default type for all tensors using

pytorch.set_default_tensor_type('torch.DoubleTensor')

It is better to convert your inputs to float rather than converting your model to double, because mathematical computations on double datatype is considerably slower on GPU.

Answer 2

I was also receiving exact same error. The root cause turned out to be this statement in my data loading code:

t = t.astype(np.float)

Here np.float translates to 64-bit float which maps to DoubleTensor. So changing this to,

t = t.astype(np.float32)

solved the issue.