Lightning + hugging face, TypeError: linear(): argument 'input' (position 1) must be Tensor, not str

1

Hi,

I’m new with pytorch in general. I’m following a tutorial using pytorch lightning and hugging face transformers. Deep-Learning-with-PyTorch-Lightning/Github_BERT_transfer_learning.ipynb at main · PacktPublishing/Deep-Learning-with-PyTorch-Lightning · GitHub

I have this issue when I run trainer.fit(model):
TypeError: linear(): argument ‘input’ (position 1) must be Tensor, not str

I try to force encode_id to be a tensor, but it didn’t work.

# part of the code

    def forward(self, encode_id, mask):
        _, output= self.pretrain_model(encode_id, attention_mask=mask)
        output = self.new_layers(output)
        return output

# ...
    def training_step(self, batch, batch_idx):
      encode_id, mask, targets = batch
      outputs = self(encode_id, mask) 
      preds = torch.argmax(outputs, dim=1)
      train_accuracy = accuracy(preds, targets)
      loss = self.loss(outputs, targets)
      self.log('train_accuracy', train_accuracy, prog_bar=True, on_step=False, on_epoch=True)
      self.log('train_loss', loss, on_step=False, on_epoch=True)
      return {"loss":loss, 'train_accuracy': train_accuracy}

error:

/var/folders/nj/yb9tqq1s2l915mvsll1_5ky40000gn/T/ipykernel_40651/1007654021.py:69: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  encode_id = torch.tensor(encode_id)
Output exceeds the size limit. Open the full output data in a text editor
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[119], line 4
      1 model = HealthClaimClassifier()
      3 trainer = pl.Trainer(fast_dev_run=True)
----> 4 trainer.fit(model)

File ~/miniconda3/envs/cs197lec2/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py:520, in Trainer.fit(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
    518 model = _maybe_unwrap_optimized(model)
    519 self.strategy._lightning_module = model
--> 520 call._call_and_handle_interrupt(
    521     self, self._fit_impl, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path
    522 )

File ~/miniconda3/envs/cs197lec2/lib/python3.9/site-packages/pytorch_lightning/trainer/call.py:44, in _call_and_handle_interrupt(trainer, trainer_fn, *args, **kwargs)
     42         return trainer.strategy.launcher.launch(trainer_fn, *args, trainer=trainer, **kwargs)
     43     else:
---> 44         return trainer_fn(*args, **kwargs)
     46 except _TunerExitException:
     47     _call_teardown_hook(trainer)

File ~/miniconda3/envs/cs197lec2/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py:559, in Trainer._fit_impl(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
    549 self._data_connector.attach_data(
    550     model, train_dataloaders=train_dataloaders, val_dataloaders=val_dataloaders, datamodule=datamodule
    551 )
...
File ~/miniconda3/envs/cs197lec2/lib/python3.9/site-packages/torch/nn/modules/linear.py:114, in Linear.forward(self, input)
    113 def forward(self, input: Tensor) -> Tensor:
--> 114     return F.linear(input, self.weight, self.bias)

TypeError: linear(): argument 'input' (position 1) must be Tensor, not str

Thank you