Mastodon Feed
jsonstein@masto.deoan.org ("Jeff Sonstein") wrote:
a side-facing summarization tool will be essential for what I am working on. dealing with a GPT is like dealing with a dementia patient, in that they do not remember the conversation, so one has to send the existing conversational context/history along with each and every request. this starts getting slow and expensive after a while, so summarization of context is needed at periodic breakpoints. rust-bert looks like a useful toolset...
https://guillaume-be.github.io/2020-11-21/generation_benchmarks