Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

New posts in huggingface-transformers

NameError: name 'PartialState' is not defined error while training hugging face wave2vec model

How does one set the pad token correctly (not to eos) during fine-tuning to avoid model not predicting EOS?

How to use Huggingface Trainer with multiple GPUs?

Transformers PreTrainedTokenizer add_tokens Functionality

Using HuggingFace pipeline on pytorch mps device M1 pro

Why does the ProtBERT model generate identical embeddings for all non-whitespace-separated (single token?) inputs?

How to install pip install torch==2.1.2+cu118 in linux?

what is the difference between len(tokenizer) and tokenizer.vocab_size

The last dimension of the inputs to a Dense layer should be defined. Found None. Full input shape received: <unknown>

How can I make sentence-BERT throw an exception if the text exceeds max_seq_length, and what is the max possible max_seq_length for all-MiniLM-L6-v2?

Huggingface MarianMT translators lose content, depending on the model

How to efficiently mean-pool BERT embeddings while excluding padding?

HuggingFace's linear scheduler with warmup parameters

Tokenizer.from_file() HUGGINFACE : Exception: data did not match any variant of untagged enum ModelWrapper

Input type into Linear4bit is torch.float16, but bnb_4bit_compute_type=torch.float32 (default). This will lead to slow inference or training speed

How to Load a 4-bit Quantized VLM Model from Hugging Face with Transformers?

Loading checkpoint shards takes too long

Huggingface AutoTokenizer can't load from local path

what is so special about special tokens?