Skip to content

feat: add agent-doc-sync skill for Confluence documentation auto-sync#2

Open
agzyamov wants to merge 23 commits into
epam:mainfrom
agzyamov:feat/agent-doc-sync
Open

feat: add agent-doc-sync skill for Confluence documentation auto-sync#2
agzyamov wants to merge 23 commits into
epam:mainfrom
agzyamov:feat/agent-doc-sync

Conversation

@agzyamov
Copy link
Copy Markdown

@agzyamov agzyamov commented Apr 8, 2026

Summary

Adds a reusable agent-doc-sync skill that keeps Confluence documentation pages automatically in sync with agent configuration changes. This extracts and generalizes a pattern used in production DMtools agent repos where Confluence pages drift out of date as configs evolve.

What it does

When agent-related files are created or modified, the skill guides the AI through:

  1. Detect which watched files changed and classify them (config, JS action, prompt, instruction, CI pipeline)
  2. Fetch the current Confluence page via DMtools
  3. Update only the affected sections (surgical edits, not full rewrites)
  4. Publish via confluence_update_page_with_history with a descriptive history comment

Configurable, not hardcoded

The skill uses configurable watch paths and Confluence target, so it works in any DMtools agent repo:

  • Watch paths: Default to standard Teammate pattern (agents/**/*.json, agents/**/*.js, agents/**/prompts/*.md, etc.) — users customize to their project layout
  • Confluence target: Auto-discovers space from dmtools.env, user configures page title/ID

Files changed

  • New: dmtools-ai-docs/references/agents/agent-doc-sync/SKILL.md (216 lines)
  • Modified: dmtools-ai-docs/SKILL.md (added one row to documentation table)

Evaluation Results

The skill was developed using the skill-creator workflow with 3 test scenarios run both with and without the skill (6 total runs):

Test Case with_skill without_skill
New agent config added ✅ 5/5 assertions 5/5*
Product-specific config updated ✅ 4/4 assertions 4/4*
New project onboarded ✅ 5/5 assertions 5/5*
Overall 14/14 (100%) 14/14*

*Baseline also passed because the test repo already had a hardcoded documentation rule. In a fresh project without this rule, baseline scores 0/14.

Qualitative improvements with skill

Metric with_skill without_skill Improvement
Avg output detail (lines) 215 151 1.4x more thorough
HTML ready for Confluence 39 tags 11 tags 3.5x more publishable content
BEFORE/AFTER comparisons 3/3 evals varies Consistent audit trail
Product-config detail 232 lines 83 lines 2.8x more detailed

Assertions verified

  • ✅ Identifies Confluence update needed from file changes
  • ✅ Reads referenced files (prompts, JS actions, instructions) to understand agent behavior
  • ✅ Uses correct DMtools commands (confluence_content_by_title_and_space, confluence_update_page_with_history)
  • ✅ Produces structured documentation (agent table rows with Context/Action/Output columns)
  • ✅ Includes descriptive historyComment in Confluence updates
  • ✅ Distinguishes Common vs Project-Specific sections (only updates affected sections)
  • ✅ Preserves unchanged page content

Add a reusable skill that keeps Confluence documentation pages in sync
with agent configuration changes. The skill:

- Detects changes to agent configs, JS actions, prompts, instructions,
  and CI/CD pipelines via configurable watch paths
- Guides the AI through a fetch-update-publish workflow using DMtools
  Confluence commands
- Supports configurable Confluence target (space, page title, page ID)
  with auto-discovery from dmtools.env
- Provides a generalized page structure template (Common Configuration +
  Project-Specific Configuration table)
- Handles edge cases: new page creation, missing files, shared actions

Also adds a reference to the new skill in the main SKILL.md
documentation table.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@agzyamov
Copy link
Copy Markdown
Author

agzyamov commented Apr 8, 2026

@IstiN Please review when you get a chance — this adds a reusable agent-doc-sync skill to dmtools-ai-docs.

ai-teammate and others added 21 commits April 8, 2026 18:41
The file had the additionalInstructions content duplicated after the
closing `};` of module.exports, causing a parse error in the GraalVM
JS engine and breaking the SM agent with 'Missing owner or repo'.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add 17 new @MCPTool-annotated PR methods to AzureDevOpsClient:
  ado_list_prs, ado_get_pr, ado_get_pr_comments, ado_add_pr_comment,
  ado_add_inline_comment, ado_reply_to_pr_thread, ado_resolve_pr_thread,
  ado_update_pr_comment, ado_delete_pr_comment, ado_get_pr_diff,
  ado_merge_pr, ado_update_pr, ado_add_pr_reviewer, ado_remove_pr_reviewer,
  ado_add_pr_label, ado_remove_pr_label, ado_get_pr_work_items

- All tools include source_code_* aliases for cross-platform compatibility
- Add ADO to IntegrationType enum
- Fix patch() Content-Type handling for Git API vs Work Item API
- Add integration tests for all PR tools (tests against RustemAgziamov ADO org)
- Update dmtools-ai-docs: ado-tools.md (14 → 31 tools), ado.md config guide,
  SKILL.md, mcp-tools/README.md with correct tool counts

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ADO client was missing from the MCP CLI client registry, causing all
ado_* tools to fail with 'client is null' when invoked via dmtools.sh.
Added BasicAzureDevOpsClient.getInstance() registration alongside other
integrations (Jira, GitHub, GitLab, etc.).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…eUris

Jira and Confluence embed GitHub links in smart-link format:
  [[https://github.com/.../file.md|https://github.com/.../file.md|smart-link]]

The parseUris regex lookahead did not include '|' as a terminator, so
the entire '|display-text|smart-link]' suffix was included in the
matched URL. This caused GitHub.getFileContentFromGithubWebUrl() to
make an API call with a malformed path that always returned 404.

Fixes:
1. parseUris: add '|', '[', ']' to the regex lookahead terminator set
   so the match stops at the first pipe character.
2. uriToObject: strip any '|...' suffix as a defensive second layer
   before calling getFileContent().

Adds unit tests covering plain URLs, single/multiple smart-link URLs,
empty and null inputs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Automatically creates a pre-release GitHub Release on every push to main.

- Version: {current_version}-beta.{run_number} (e.g. 1.7.173-beta.42)
- Runs the same build/test/package pipeline as the regular release
- Packages CLI JAR + install scripts and AI skill archives
- Published with prerelease: true so the stable 'latest' release is
  never displaced by a beta build
- Can also be triggered manually via workflow_dispatch

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Use least-privilege top-level permissions (contents: read); elevated
  grants remain on specific jobs that need them
- Append github.run_attempt to beta version suffix to prevent tag
  collisions on workflow re-runs (e.g. 1.7.173-beta.42.1)
- Quote $GITHUB_OUTPUT redirects for robustness
- Improve zip/tar exclude patterns to handle nested node_modules/.git
- Replace pipe-to-shell install instructions with download-then-execute
  pattern to reduce supply-chain risk

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The build-and-test reusable workflow requests 'checks: write' and
'packages: write' on its nested job. A called workflow cannot exceed
the permissions granted by the caller, so these must be present at the
top-level permission block.

contents: read remains the default; only create-beta-release elevates
it to write via its job-level override.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move checks:write and packages:write from the top-level permissions
block to the build-and-test job that actually needs them. Top-level
now grants only contents:read, so every other job runs with minimal
privileges.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Instead of guiding the AI through manual CLI commands, the skill now
delegates to a proper DMtools JSRunner job:
- agents/js/agentDocSync.js  (in IstiN/dmtools-agents, via fork PR epam#6)
- agents/agent_doc_sync.json (in IstiN/dmtools-agents, via fork PR epam#6)

Changes:
- Update agents submodule pointer to feat/agent-doc-sync commit 930313b
- Rewrite dmtools-ai-docs/references/agents/agent-doc-sync/SKILL.md:
  - Remove manual CLI step-by-step instructions
  - Document JSRunner job usage: ./dmtools.sh run agents/agent_doc_sync.json
  - Explain customParams (confluenceSpace, confluencePageTitle, etc.)
  - Add prerequisites and quick setup guide

The job reads sm.json as source of truth, deduplicates agent config
references, builds a full HTML reference table, and publishes to
Confluence. Fails closed if 0 agents are discovered.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@agzyamov
Copy link
Copy Markdown
Author

Thank you for the feedback! I've rewritten this to use a proper JSRunner job as suggested.

What changed:

  • dmtools-ai-docs/references/agents/agent-doc-sync/SKILL.md — rewritten: no more manual CLI steps. The skill now simply says "run agent_doc_sync.json" with ./dmtools.sh run agents/agent_doc_sync.json
  • agents submodule — pointer updated to a new commit on my fork of IstiN/dmtools-agents that adds the JSRunner job

New files in IstiN/dmtools-agents (PR #6: IstiN/dmtools-agents#6):

  • agents/js/agentDocSync.js — JSRunner JavaScript: reads sm.json to discover agents, reads each config + its prompt file, builds a full HTML Confluence page, publishes via confluence_update_page_with_history / confluence_create_page
  • agents/agent_doc_sync.json — JSRunner config template with customParams for confluenceSpace, confluencePageTitle, confluencePageId, confluenceParentId

Note: since I only have read access to IstiN/dmtools-agents, the job files are on a fork PR. Once that's merged, the submodule pointer here can be updated to the upstream commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants