feat: integrate TOON format for data encoding and add dependency#1609
Conversation
There was a problem hiding this comment.
Pull request overview
This PR integrates the TOON (Token-Oriented Object Notation) format for encoding data returned from database queries and table structure requests. TOON is a compact, human-readable format designed to reduce token usage in AI interactions while maintaining readability.
Changes:
- Added
@toon-format/toonpackage dependency (v2.1.0) - Replaced
JSON.stringify()calls withencodeToToon()for successful database query results and table structure responses - Updated AI system prompt to inform the model about TOON format encoding
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| yarn.lock | Added lockfile entry for @toon-format/toon v2.1.0 dependency |
| backend/package.json | Added @toon-format/toon v2.1.0 to dependencies |
| backend/src/ai-core/utils/toon-encoder.ts | Created utility module with encodeToToon and encodeError functions |
| backend/src/ai-core/utils/index.ts | Exported toon-encoder utilities for use across the application |
| backend/src/entities/ai/use-cases/request-info-from-table-with-ai-v7.use.case.ts | Replaced JSON.stringify with encodeToToon for query results and encodeError for error cases |
| backend/src/ai-core/tools/prompts.ts | Added documentation about TOON format to the AI system prompt |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| } | ||
|
|
||
| export function encodeError(error: { error: string }): string { | ||
| return JSON.stringify(error); |
There was a problem hiding this comment.
The encodeError function still uses JSON.stringify for errors while successful results use TOON encoding. This creates an inconsistency where the AI model receives responses in two different formats (TOON for success, JSON for errors). Consider using encodeToToon for errors as well to maintain consistency, or document why errors should remain in JSON format.
| return JSON.stringify(error); | |
| return encodeToToon(error); |
No description provided.