HITL Implementation for action tools

[fentrexsolutions]

Description

This PR implements Human-in-the-Loop (HITL) tool-level confirmation for high-risk action tools. The system requires user confirmation before executing potentially destructive operations, providing safety guardrails while allowing granular bypass rules for trusted workflows.

fixes #5

Key Features:

• Tool-level confirmation metadata: Added confirmation field to tool definitions with level (high/medium), reason, and showArgs
• Confirmation modes: off, high-only, all - configurable via hitl.confirmationMode
• Bypass rules: Allow specific tools, categories, or risk levels to bypass confirmation via hitl.bypass config
• "Allow Always" support: SDK helper functions addBypassRule() and removeBypassRule() for programmatic bypass management
• CLI integration: Interactive confirmation modal with keyboard shortcuts (Y=Confirm, A=Allow Always, N=Deny, Esc=Cancel)
• Dynamic config reloading: Toolpack.reloadConfig() method to apply config changes without restart

Tools with HITL Confirmation:

• High risk: exec.run, exec.run-shell, exec.run-background, fs.write-file, fs.delete-file, fs.delete-dir, fs.move, fs.replace-in-file, fs.batch-write, http.delete, http.post, http.put, db.delete, db.update, coding.multi-file-edit, coding.refactor-rename, diff.apply
• Medium risk: db.insert

Bug Fixes:

• Fixed HITL bypass logic: hitl.enabled is now treated as true by default when undefined (previously bypassed all tools)
• Improved tool descriptions for write_file and delete_file to prevent misuse

Fixes # (issue)

Type of change

• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• This change requires a documentation update

How Has This Been Tested?

• Manual testing (describe below)

Manual Testing:

1. Tested confirmation modal appears for high-risk tools (exec.run, fs.write-file, fs.delete-file)
2. Tested "Allow Always" persists bypass rules to config and applies immediately via reloadConfig()
3. Tested bypass rules work correctly - bypassed tools execute without confirmation
4. Tested deny/cancel actions properly reject tool execution
5. Tested tool selection: AI now correctly uses fs.delete_file for "delete/remove" requests vs fs.write_file with empty content
6. Tested config file location: bypass rules correctly write to .toolpack/config/toolpack.config.json
7. Tested HITL enabled default: when hitl.enabled is undefined, confirmation is active (not bypassed)

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings (run npm run lint)
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[fentrexsolutions]

@sajeerzeji please review our changes

[sajeerzeji]

Missing Documentation (High Priority)

• No README updates explaining HITL configuration
• Create a PR for https://github.com/toolpack-ai/toolpack-documentation to document all hitl implementation including config structure, API Documentation for addBypassRule(), removeBypassRule(), and reloadConfig(), and bypass rule patterns and best practices, and all available tools with their descriptions.

[sajeerzeji]

No Automated Tests (High Priority)

• No unit tests for isBypassed() logic
• No tests for bypass rule matching
• No tests for config loading/merging
• No integration tests for confirmation flow

[sajeerzeji]

Type Safety Concerns (Medium Priority)

In @/packages/toolpack-sdk/src/toolpack.ts:575:

const { loadConfig, discoverConfigPath } = require('./providers/config.js');

Using require() loses type safety. Should use ES6 imports.

Recommendation:

import { loadConfig, discoverConfigPath } from './providers/config.js';

[fentrexsolutions]

Error Handling Gaps (Medium Priority)

• What happens if addBypassRule() fails to write the file?
• No error handling in reloadConfig() if config file is malformed
• No validation that tool names in bypass rules are valid

Recommendation: Add error handling and validation:

export async function addBypassRule(options: AddBypassRuleOptions): Promise<void> {
    try {
        // ... existing code ...
        fs.writeFileSync(configPath, JSON.stringify(config, null, 4), 'utf-8');
    } catch (error) {
        throw new SDKError(`Failed to add bypass rule: ${error.message}`, 'CONFIG_WRITE_ERROR');
    }
}

[fentrexsolutions]

Potential Race Condition (Low Priority)
In CLI's onAllowAlways:

await addBypassRule({ ... });
toolpack.reloadConfig(activeConfigPath || undefined);

If multiple confirmations happen simultaneously, config writes could conflict.

Recommendation: Consider adding a config write lock or queue.

6. Inconsistent Naming (Low Priority)

• hitl.bypass.tools uses tool names like "fs.write_file"
• But tool display names use spaces: "Write File"
• Could be confusing for users manually editing config

Recommendation: Document that bypass rules use tool.name not tool.displayName

[fentrexsolutions]

Inconsistent Naming (Low Priority)

• hitl.bypass.tools uses tool names like "fs.write_file"
• But tool display names use spaces: "Write File"
• Could be confusing for users manually editing config

Recommendation: Document that bypass rules use tool.name not tool.displayName

[fentrexsolutions]

Missing Edge Cases (Low Priority)

• What if a tool has no confirmation metadata but is in bypass rules?
• What if confirmationMode is set to an invalid value?

Current code handles this well at line 262:

if (tool.confirmation && bypass.levels?.includes(tool.confirmation.level))

But could add validation warnings.

[fentrexsolutions]

Github Workflows are failing, take a look

[fentrexsolutions]

All PR review items have been fixed (excluding documentation):

Completed Fixes:

• Type Safety - Changed require() to ES6 imports in toolpack.ts
• Error Handling - Added try-catch with SDKError in addBypassRule() and reloadConfig()
• Race Condition Prevention - Added file locking mechanism (acquireConfigLock) for config writes
• Unit Tests - Created comprehensive test suite (hitl.test.ts) covering:
• isBypassed() logic with various config states
• Bypass rule matching (tools, categories, levels)
• Confirmation mode filtering (off/high-only/all)
• Dynamic config updates via updateHitlConfig()

All tests passing ✅

[fentrexsolutions]

Added a PR for Documentation: toolpack-ai/toolpack-documentation#3