PyTorch on M1 Mac: RuntimeError: Placeholder storage has not been allocated on MPS device

Question

I'm training a model in PyTorch 1.13.0 (I have also tried this on the nightly build torch-1.14.0.dev20221207 to no avail) on my M1 Mac and would like to use MPS hardware acceleration. I have the following relevant code in my project to send the model and input tensors to MPS:

device = torch.device("mps" if torch.backends.mps.is_available() else "cpu") # This always results in MPS

model.to(device)

... And in my Dataset subclass:

class MyDataset(Dataset):
    def __init__(self, df, window_size):
        self.df = df
        self.window_size = window_size
        self.data = []
        self.labels = []
        for i in range(len(df) - window_size):
            x = torch.tensor(df.iloc[i:i+window_size].values, dtype=torch.float, device=device)
            y = torch.tensor(df.iloc[i+window_size].values, dtype=torch.float, device=device)
            self.data.append(x)
            self.labels.append(y)
    def __len__(self):
        return len(self.data)
    def __getitem__(self, idx):
        return self.data[idx], self.labels[idx]

This results in the following traceback during my first training step:

Traceback (most recent call last):
  File "lstm_model.py", line 263, in <module>
    train_losses, val_losses = train_model(model, criterion, optimizer, train_loader, val_loader, epochs=100)
  File "lstm_model.py", line 212, in train_model
    train_loss += train_step(model, criterion, optimizer, x, y)
  File "lstm_model.py", line 191, in train_step
    y_pred = model(x)
  File "miniconda3/envs/pytenv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "lstm_model.py", line 182, in forward
    out, _ = self.lstm(x, (h0, c0))
  File "miniconda3/envs/pytenv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "miniconda3/envs/pytenv/lib/python3.10/site-packages/torch/nn/modules/rnn.py", line 774, in forward
    result = _VF.lstm(input, hx, self._flat_weights, self.bias, self.num_layers,
RuntimeError: Placeholder storage has not been allocated on MPS device!

I've tried creating tensors in my Dataset subclass without a device specified and then calling .to(device) on them:

x = torch.tensor(df.iloc[i:i+window_size].values, dtype=torch.float)
x = x.to(device)
y = torch.tensor(df.iloc[i+window_size].values, dtype=torch.float)
y = y.to(device)

I've also tried creating the tensors without a device specified in my Dataset subclass and sending tensors to device in both the forward method of my model and in my train_step function.

How can I resolve my error?

Answer 1

A possible issue with your code may be that you are not sending the inputs to the device inside your training loop. You should send both the model and the inputs to the device, as you can read about in this blog post.

An example code would be the following:

def train(model, train_loader, device, *args):
    model.train()

    for it, batch in tqdm(enumerate(train_loader), desc="Epoch %s: " % (epoch), total=train_loader.__len__()):
        batch = {'data': batch['data'].to(device), 'labels': batch['labels'].to(device)}

        # perform training
        ...

# set model and device
model = MyWonderfulModel(*args)
device = torch.device("mps" if torch.backends.mps.is_available() else "cpu")
model.to(device)

# call training function
train(model, train_loader, device, *args)

Running such training function on my M1 Mac works using MPS.