Back in April, we released an initial version of our Summarization endpoint, which allowed for summarizing text using our language models.
We quickly noticed however, as did some of you, that it had a few problems when we tried integrating it into projects:
- Document length was limited to the context size of the model (only ~2000 tokens available for the document).
- As the documents grew larger, the endpoint became much more likely to just return the first sentence of the document.
In the meantime, we released some improvements to our Q&A endpoint that resolved similar problems for that use case. Particularly for Q&A, we can:
- efficiently process really large documents, returning the best answers across the entire document.
- process Docx and Text documents, not just Prompts.
With these improvements to our document handling, we were able to come back to Summarization with new approaches and better results.