Skip to content

Add Devin Playbook Sync GitHub Action#2

Open
devin-ai-integration[bot] wants to merge 3 commits into
mainfrom
devin/1762305531-playbook-sync-action
Open

Add Devin Playbook Sync GitHub Action#2
devin-ai-integration[bot] wants to merge 3 commits into
mainfrom
devin/1762305531-playbook-sync-action

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Nov 5, 2025

Copy link
Copy Markdown
Contributor

Add Devin Playbook Sync GitHub Action

Summary

This PR implements a GitHub Action that scans directories for Markdown files and syncs them as Devin playbooks using the Devin API. The action is implemented as a composite action with bash scripts and supports all 5 Devin playbook API operations (create, list, get, update, delete).

Key components added:

  • action.yml - Composite action definition with 9 inputs and 3 outputs
  • scripts/sync-playbooks.sh - Main bash script (312 lines) handling all API operations
  • .github/workflows/test-action.yml - CI workflow with 3 test jobs (all passing ✅)
  • Comprehensive documentation (README.md updated, DEPLOYMENT.md added)
  • Example playbook markdown files for testing
  • MIT License

Implementation approach:

  • Uses curl for Devin API calls with Bearer token authentication
  • Uses jq for JSON payload construction and response parsing
  • Logs redirected to stderr to prevent stdout contamination
  • Supports dry-run mode for testing without API calls
  • Proper error handling with set -e and exit codes

Review & Testing Checklist for Human

⚠️ CRITICAL - This PR has NOT been tested with real API calls. All CI tests run in dry-run mode.

  • Test with real Devin API key - Run the action with actual API credentials to verify end-to-end functionality. The dry-run tests pass, but real API behavior may differ.

    # Test locally or in a workflow
    export DEVIN_API_KEY="your-real-key"
    export DRY_RUN="false"
    ./scripts/sync-playbooks.sh
  • Test edge cases with markdown content - Verify the action handles:

    • Markdown files with special characters (quotes, backticks, newlines, unicode)
    • Very large files (>10KB)
    • Files with code blocks, tables, and complex formatting
    • Empty or whitespace-only files
  • Verify error handling - Test failure scenarios:

    • Invalid API key (should fail with clear error message)
    • Network timeouts or API unavailability
    • API rate limiting
    • Missing required inputs for operations (get/update/delete without playbook-id)
  • Review bash script security - Check scripts/sync-playbooks.sh for:

    • Potential command injection vulnerabilities
    • Proper handling of file paths and user inputs
    • API key exposure in logs or error messages
  • Repository structure discussion - This action is at the root level, but the repo already has devin-action/ in a subdirectory. Is this the desired organization? Consider if this action should also be in a subdirectory for consistency.

Recommended Test Plan

  1. Create a test repository with a few markdown files
  2. Add your Devin API key as a GitHub secret
  3. Create a workflow using this action (see README.md for examples)
  4. Run the workflow and verify:
    • Playbooks are created successfully in Devin
    • Output variables (playbook-ids, playbooks-count) are correct
    • Logs are clear and helpful
  5. Test the list, get, update, and delete operations
  6. Verify the action works with the tagged version: samfert-codeium/DEVIN-GITHUB-ACTIONS@v1

Notes

CI fixes applied during development:

  1. Fixed log functions mixing stdout/stderr - All log output now goes to stderr (>&2) to prevent JSON parsing errors
  2. Fixed arithmetic increment operations - Changed from ((var++)) to var=$((var + 1)) to avoid exit code 1 with set -e
  3. Resolved README merge conflict - Combined documentation for both actions in the repository

Limitations:

  • The action requires bash, curl, and jq (all pre-installed on GitHub runners)
  • Update and delete operations require the ManageOrgPlaybooks permission in Devin
  • The Devin API is currently in alpha and may change

Session info:

- Create composite action with full Devin API integration
- Support all 5 playbook operations (create, list, get, update, delete)
- Add comprehensive README with usage examples
- Add deployment documentation
- Include test playbooks and test workflow
- Integrate with Devin API endpoints for playbook management
- Add dry-run mode for testing without API calls
- Implement recursive directory scanning for .md files

Features:
- Automatic discovery of .md files in directories
- Full API support for all Devin playbook operations
- Secure API key handling via GitHub secrets
- Detailed outputs (playbook IDs, counts, results)
- Comprehensive error handling and validation

Documentation includes:
- Usage examples for all operations
- Input/output reference
- API documentation links
- Troubleshooting guide
- Deployment and release process

Co-Authored-By: Sam Fertig <sam.fertig@codeium.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

devin-ai-integration Bot and others added 2 commits November 5, 2025 01:29
- Redirect all log function output to stderr to prevent mixing with stdout data
- Change arithmetic increments from ((var++)) to var=$((var + 1)) to avoid exit code 1 with set -e
- Ensures script completes successfully when processing multiple files

Fixes:
- Test Dry Run Mode failure - script now processes all files and exits with code 0
- Test List Operation failure - log messages no longer interfere with JSON parsing
- Both test playbooks are now successfully processed in dry-run mode

Co-Authored-By: Sam Fertig <sam.fertig@codeium.com>
- Updated README to describe repository as containing multiple actions
- Added section for Playbook Sync Action (at root)
- Kept existing Devin API Action section (in devin-action/ subdirectory)
- Both actions are now documented in the root README
- Resolved merge conflict by combining documentation from both branches

Co-Authored-By: Sam Fertig <sam.fertig@codeium.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants