I'd like to send the OpenAI's API, the text from various PDF's. Specifically, the Summarize for a 2nd grader or the TL;DR summarization API's.
I can extract the text from PDF's using PyMuPDF and prepare the OpenAI prompt.
Question: How best to prepare the prompt when the token count is longer than the allowed 2049?
- Do I just truncate the text?
- Or is there a way to sample the text to "compress" it to lose key points?
Reactions are currently unavailable
I'd like to send the OpenAI's API, the text from various PDF's. Specifically, the Summarize for a 2nd grader or the TL;DR summarization API's.
I can extract the text from PDF's using PyMuPDF and prepare the OpenAI prompt.
Question: How best to prepare the prompt when the token count is longer than the allowed 2049?