idea: add a tool for searching the full context of the main session

[IndianBoy42]

For really long sessions the hard token limit can be a problem. Solution: ai knows how to use grep, let it grep through the whole transcript and then retrieve sections by line number

[karamanliev]

Hey, I'm going to explore that idea later, but currently I'm not sending the full context to the mini session - only user queries, llm answers (without reasoning) and just the names of the tools that were called. The default 50k limit is a lot of this, from my rough estimations this should cover a 500-750k session. Changing it to 100k in the config should be enough for 1m+ sessions easily.

At least in my experience, most of the tokens in a session come from reasoning and results from the called tools, like reading files, fetching pages and etc. I still want to send some raw context to the mini session, because it happens instantly and we don't have to wait for a summarization or querying the db.

Maybe a better idea is to not change this behavior, but instead let the agent to query the sqlite db, which will make it possible to get additional context if needed, similar to this SKILL. This will help even in the cases when messages are cut off, because of the limit.

Does that make sense, what do you think?

[IndianBoy42]

Oh I see, actually I was hoping reasoning traces would be part of it. Asking "why did you choose this" is one of my main use cases, for which reasoning is important. Many models don't really 'say' much until right at the end so it would be hard to ask about an in progress task

[karamanliev]

In that case the session isn't mini anymore :)

I'll implement the db query suggestion soon, I think that is a good middle ground in the cases in which you need more context from the main session.