LLMs should be aware of context window limit and have guards against overloading it
Prokop Hapala
I often encounter unspecified cascade error when I give large files into context window. But it should not be difficult inform LLMs that the files are large and reading it all at once is not an option. LLMs should be able to work knowing the limits of their context window and cascade should make sure it is not overloaded.
P
Paul Wade
A good lesson from Kiro IMHO is to have a simple button labeled "Summarize conversation and continue in a new chat." This button could appear when reading a large file or when other responses might overload the context, perhaps beyond a certain limit. It would first generate a summary, then start a new session where it can read the file in a fresh, condensed, history-aware context.