tool "read_file" response (150947 tokens) exceeds maximum allowed tokens (25000)

[tinovyatkin]

We should keep all our files inside the max tokens size

[tinovyatkin]

There is maximum allowed size also - 256 KB

[tinovyatkin]

❯ tc --model Quivr/claude-tokenizer crates/cribo/src/code_generator.rs
  127034 crates/cribo/src/code_generator.rs

[tinovyatkin]

https://github.com/ophidiarium/cribo/wiki/Enforcing-token-limits