Open
Conversation
Copied wholesale from https://github.com/github/awesome-copilot/blob/main/hooks/tool-guardian - see the README there to understand the purpose. Copilot hooks docs: https://docs.github.com/en/copilot/concepts/agents/coding-agent/about-hooks https://docs.github.com/en/copilot/reference/hooks-configuration
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #264 +/- ##
=======================================
Coverage 99.26% 99.26%
=======================================
Files 201 201
Lines 5820 5820
Branches 974 974
=======================================
Hits 5777 5777
Misses 42 42
Partials 1 1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Added a warning about safety limitations of Tool Guardian.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Copied wholesale from https://github.com/github/awesome-copilot/blob/main/hooks/tool-guardian
See the README there to understand more, but the purpose is extra guardrails around Copilot, for when running it more autonomously.
I think the main thing I'd be worried about with yolo-ing (allowing copilot to run any command) is it messing up my computer or uncommitted work in irreparable ways. It could do this directly, by modifying/deleting files or uninstalling things, or indirectly by installing a malicious package. There is also a very specific risk of it posting private keys to github, which I don't address here.
My general approach thus far is to whitelist safe commands e.g.
ls,npx vitestas they come up; but this is perhaps over-cautious and prevents a lot of the benefits of using AI agents, namely being able to delegate and not have to supervise (so that working in parallel is made possible). In order to feel safe doing that I'd like to do something like run copilot inside a Docker sandbox: https://docs.docker.com/ai/sandboxes/. (If using Docker sandbox I would choose the 'balanced' policy with regard to network access: https://docs.docker.com/ai/sandboxes/security/policy/#network-policies). Another way to sandbox is to run Copilot in the cloud through the GitHub website, but then it's less able to be interacted with. Further research (not yet read): https://www.reddit.com/r/devops/comments/1q4pvy6/wrote_a_deep_dive_on_sandboxing_for_ai_agents/The bash script added in this PR is not me-authored, it's copied from that tool-guardian repo. That repo is trusted enough by GitHub to be linked in the Copilot docs.
Copilot hooks docs: https://docs.github.com/en/copilot/concepts/agents/coding-agent/about-hooks
https://docs.github.com/en/copilot/reference/hooks-configuration