Skip to content

feat: integrate TOON format for data encoding and add dependency #1609

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Artuomka merged 1 commit into from
Feb 17, 2026
Merged

feat: integrate TOON format for data encoding and add dependency #1609

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions backend/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@
"@rocketadmin/shared-code": "workspace:*",
"@sentry/minimal": "^6.19.7",
"@sentry/node": "10.39.0",
"@toon-format/toon": "^2.1.0",
"@types/crypto-js": "^4.2.2",
"@types/jsonwebtoken": "^9.0.10",
"@types/multer": "^2.0.0",
Expand Down
2 changes: 2 additions & 0 deletions backend/src/ai-core/tools/prompts.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ Table name: "${tableName}".
${schema ? `Schema: "${schema}".` : ''}
Current date and time: ${currentDatetime}

Tool responses are encoded in TOON (Token-Oriented Object Notation) format - a compact, human-readable format similar to YAML with CSV-style tabular arrays. Parse it naturally.

Please follow these steps EXACTLY:
1. First, always use the getTableStructure tool to analyze the table schema and understand available columns
2. If the question requires data from related tables, note their relationships
Expand Down
1 change: 1 addition & 0 deletions backend/src/ai-core/utils/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
export * from './message-builder.js';
export * from './toon-encoder.js';
9 changes: 9 additions & 0 deletions backend/src/ai-core/utils/toon-encoder.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
import { encode as toonEncode } from '@toon-format/toon';

export function encodeToToon(data: unknown): string {
return toonEncode(data);
}

export function encodeError(error: { error: string }): string {
return JSON.stringify(error);

Copilot AI Feb 17, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The encodeError function still uses JSON.stringify for errors while successful results use TOON encoding. This creates an inconsistency where the AI model receives responses in two different formats (TOON for success, JSON for errors). Consider using encodeToToon for errors as well to maintain consistency, or document why errors should remain in JSON format.

Suggested change
return JSON.stringify(error);
return encodeToToon(error);

Copilot uses AI. Check for mistakes.
}
12 changes: 7 additions & 5 deletions backend/src/entities/ai/use-cases/request-info-from-table-with-ai-v7.use.case.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ import {
AIToolDefinition,
createDatabaseQuerySystemPrompt,
createDatabaseTools,
encodeError,
encodeToToon,
isValidMongoDbCommand,
isValidSQLQuery,
MessageBuilder,
Expand Down Expand Up @@ -243,7 +245,7 @@ export class RequestInfoFromTableWithAIUseCaseV7
userEmail,
foundConnection,
);
result = JSON.stringify(structureInfo);
result = encodeToToon(structureInfo);
break;
}

Expand All @@ -259,7 +261,7 @@ export class RequestInfoFromTableWithAIUseCaseV7
}
const wrappedQuery = wrapQueryWithLimit(query, foundConnection.type as ConnectionTypesEnum);
const queryResult = await dataAccessObject.executeRawQuery(wrappedQuery, inputTableName, userEmail);
result = JSON.stringify(queryResult);
result = encodeToToon(queryResult);
break;
}

Expand All @@ -274,15 +276,15 @@ export class RequestInfoFromTableWithAIUseCaseV7
);
}
const pipelineResult = await dataAccessObject.executeRawQuery(pipeline, inputTableName, userEmail);
result = JSON.stringify(pipelineResult);
result = encodeToToon(pipelineResult);
break;
}

default:
result = JSON.stringify({ error: `Unknown tool: ${toolCall.name}` });
result = encodeError({ error: `Unknown tool: ${toolCall.name}` });
}
} catch (error) {
result = JSON.stringify({ error: error.message });
result = encodeError({ error: error.message });
}

results.push({ toolCallId: toolCall.id, result });
Expand Down
8 changes: 8 additions & 0 deletions yarn.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5231,6 +5231,13 @@ __metadata:
languageName: node
linkType: hard

"@toon-format/toon@npm:^2.1.0":
version: 2.1.0
resolution: "@toon-format/toon@npm:2.1.0"
checksum: 79d3595d4806cf16ce7a4961fdcbb8a9afe4583ecd6297e0d4cbcd07c5ef2a7ee60665a5d82ce07cbe3dd36c22ee54fa515968b12afa04c74df449baee28fb6a
languageName: node
linkType: hard

"@tsconfig/node10@npm:^1.0.7":
version: 1.0.12
resolution: "@tsconfig/node10@npm:1.0.12"
Expand Down Expand Up @@ -6382,6 +6389,7 @@ __metadata:
"@rocketadmin/shared-code": "workspace:*"
"@sentry/minimal": ^6.19.7
"@sentry/node": 10.39.0
"@toon-format/toon": ^2.1.0
"@types/bcrypt": ^6.0.0
"@types/body-parser": ^1.19.6
"@types/cookie-parser": ^1.4.10
Expand Down
Loading