Before you start
What problem does this solve?
Currently, our VectorLint evaluation logic relies on a base evaluator abstraction (base-evaluator) that gets extended for specific use cases like technical accuracy (accuracy-evaluator). Because of this, we also maintain dedicated integrations like Perplexity (perplexity-provider) just for accuracy checking.
As we are merging agentic capabilities into VectorLint, this rigid base evaluator model is no longer needed. We want to transition to a more flexible model where the agent just receives search functionality as a tool it can use for review, instead of maintaining an explicit "technical accuracy" evaluator logic and generic extensions.
Desired outcome
- Remove the base evaluator idea entirely so there is no more extendable evaluator pattern.
- Remove the technical accuracy evaluator.
- Remove the dedicated perplexity provider implementation that was supporting technical accuracy.
- In future work, we will just implement search natively as a tool for the agent.
Before you start
What problem does this solve?
Currently, our VectorLint evaluation logic relies on a base evaluator abstraction (base-evaluator) that gets extended for specific use cases like technical accuracy (accuracy-evaluator). Because of this, we also maintain dedicated integrations like Perplexity (perplexity-provider) just for accuracy checking.
As we are merging agentic capabilities into VectorLint, this rigid base evaluator model is no longer needed. We want to transition to a more flexible model where the agent just receives search functionality as a tool it can use for review, instead of maintaining an explicit "technical accuracy" evaluator logic and generic extensions.
Desired outcome