Asking gpt-2 to finish sentence with huggingface transformers

Question

I am currently generating text from left context using the example script run_generation.py of the huggingface transformers library with gpt-2:

$ python transformers/examples/run_generation.py \
  --model_type gpt2 \
  --model_name_or_path gpt2 \
  --prompt "Hi, " --length 5

=== GENERATED SEQUENCE 1 ===
Hi,  could anyone please inform me

I would like to generate short complete sentences. Is there any way to tell the model to finish a sentence before length words?

---

Note: I don't mind changing model, but would prefer an auto-regressive one.

Answer 1

Unfortunately there is no way to do so. You can set the length parameter to a greater value and then just discard the incomplete part at the end.

Even GPT3 doesn't support completing a sentence before a specific length. GPT3 support "sequences" though. Sequences force the model to stop when certain condition is fulfilled. You can find more information about in thi article