fix: enable GitLab productivity rate metrics#19
Open
agzyamov wants to merge 27 commits into
Open
Conversation
Add a reusable skill that keeps Confluence documentation pages in sync with agent configuration changes. The skill: - Detects changes to agent configs, JS actions, prompts, instructions, and CI/CD pipelines via configurable watch paths - Guides the AI through a fetch-update-publish workflow using DMtools Confluence commands - Supports configurable Confluence target (space, page title, page ID) with auto-discovery from dmtools.env - Provides a generalized page structure template (Common Configuration + Project-Specific Configuration table) - Handles edge cases: new page creation, missing files, shared actions Also adds a reference to the new skill in the main SKILL.md documentation table. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The file had the additionalInstructions content duplicated after the closing `};` of module.exports, causing a parse error in the GraalVM JS engine and breaking the SM agent with 'Missing owner or repo'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add 17 new @MCPTool-annotated PR methods to AzureDevOpsClient: ado_list_prs, ado_get_pr, ado_get_pr_comments, ado_add_pr_comment, ado_add_inline_comment, ado_reply_to_pr_thread, ado_resolve_pr_thread, ado_update_pr_comment, ado_delete_pr_comment, ado_get_pr_diff, ado_merge_pr, ado_update_pr, ado_add_pr_reviewer, ado_remove_pr_reviewer, ado_add_pr_label, ado_remove_pr_label, ado_get_pr_work_items - All tools include source_code_* aliases for cross-platform compatibility - Add ADO to IntegrationType enum - Fix patch() Content-Type handling for Git API vs Work Item API - Add integration tests for all PR tools (tests against RustemAgziamov ADO org) - Update dmtools-ai-docs: ado-tools.md (14 → 31 tools), ado.md config guide, SKILL.md, mcp-tools/README.md with correct tool counts Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ADO client was missing from the MCP CLI client registry, causing all ado_* tools to fail with 'client is null' when invoked via dmtools.sh. Added BasicAzureDevOpsClient.getInstance() registration alongside other integrations (Jira, GitHub, GitLab, etc.). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…eUris Jira and Confluence embed GitHub links in smart-link format: [[https://github.com/.../file.md|https://github.com/.../file.md|smart-link]] The parseUris regex lookahead did not include '|' as a terminator, so the entire '|display-text|smart-link]' suffix was included in the matched URL. This caused GitHub.getFileContentFromGithubWebUrl() to make an API call with a malformed path that always returned 404. Fixes: 1. parseUris: add '|', '[', ']' to the regex lookahead terminator set so the match stops at the first pipe character. 2. uriToObject: strip any '|...' suffix as a defensive second layer before calling getFileContent(). Adds unit tests covering plain URLs, single/multiple smart-link URLs, empty and null inputs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Automatically creates a pre-release GitHub Release on every push to main.
- Version: {current_version}-beta.{run_number} (e.g. 1.7.173-beta.42)
- Runs the same build/test/package pipeline as the regular release
- Packages CLI JAR + install scripts and AI skill archives
- Published with prerelease: true so the stable 'latest' release is
never displaced by a beta build
- Can also be triggered manually via workflow_dispatch
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Use least-privilege top-level permissions (contents: read); elevated grants remain on specific jobs that need them - Append github.run_attempt to beta version suffix to prevent tag collisions on workflow re-runs (e.g. 1.7.173-beta.42.1) - Quote $GITHUB_OUTPUT redirects for robustness - Improve zip/tar exclude patterns to handle nested node_modules/.git - Replace pipe-to-shell install instructions with download-then-execute pattern to reduce supply-chain risk Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The build-and-test reusable workflow requests 'checks: write' and 'packages: write' on its nested job. A called workflow cannot exceed the permissions granted by the caller, so these must be present at the top-level permission block. contents: read remains the default; only create-beta-release elevates it to write via its job-level override. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move checks:write and packages:write from the top-level permissions block to the build-and-test job that actually needs them. Top-level now grants only contents:read, so every other job runs with minimal privileges. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fix four bugs that silently suppressed all GitLab commit and approval
data in the productivity reporting pipeline:
1. GitLabCommit.getAuthor() — GitLab commits API returns author_name /
author_email as flat top-level strings, not a nested 'author' object.
Build a synthetic GitLabUser from those fields instead.
2. GitLabCommit.getCommiterTimestamp() — Use top-level 'committed_date'
(ISO 8601) via DateUtils.smartParseDate() instead of the non-existent
nested 'committer.date' path that caused JSONException at runtime.
3. GitLabCommit.getUrl() — Return 'web_url' from the response instead
of the hardcoded placeholder string.
4. GitLab.getCommitsFromBranch() — Add 'since' date filter, per_page=100,
and a page loop so all commits in the requested window are fetched
(previously fetched only the first 20 with no date filtering).
5. GitLab.pullRequestActivities() — Replace heuristic text-matching with
a real call to the GitLab MR Approvals API
(GET /projects/:id/merge_requests/:iid/approvals). Each entry in
'approved_by' becomes an IActivity with action=APPROVED and the
actual approver user set. Discussion notes are still included as
COMMENTED activities.
6. New model: GitLabMRApproval wrapping the 'approved_by' entry shape
{ "user": { ... } } returned by the Approvals API.
Tests updated / added:
- GitLabCommitTest: fix old wrong-shape JSON in testGetAuthor; add tests
for getAuthor() flat-field parsing, null-guard, getCommiterTimestamp(),
getUrl().
- GitLabMRApprovalTest: new test class for the new model.
- GitLabTest: replace old heuristic-approval test with two new tests
covering the real Approvals API path; add tests for
getCommitsFromBranch() pagination and null-response handling.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
agzyamov
force-pushed
the
feature/gitlab-rate-metrics
branch
from
db93f99 to
15ff3b0
Compare
Instead of fetching all MR pages and stopping when we hit an old one, pass created_after=startDate directly to the API. GitLab filters server-side — only MRs in the reporting window are returned. For INS with 36 repos × 150 avg MRs, this reduces API calls from ~10,800 to only MRs created in the last 3 months (~hundreds). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
agzyamov
force-pushed
the
feature/gitlab-rate-metrics
branch
from
365c2f0 to
ad883c0
Compare
MetricFactory was constructed once with the global SourceCode (GitHub default), so all SourceCollectors (PullRequestsMetricSource etc.) always called api.github.com regardless of sourceType in the datasource config. Fix: - DataSourceFactory.resolveSourceCode() made public - MetricFactory.createMetric() overload accepts resolvedSourceCode - createSourceCollector() overload uses resolvedSourceCode when provided - ReportGenerator.collectDataFromAllSources() resolves sourceCode per datasource and passes it to both DataSourceFactory and MetricFactory Verified locally: CON report now calls gitlab.com exclusively. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
With sort=asc (oldest first), page 1..N-1 contain stable old MRs that never change between daily runs → permanent HTTP cache hits. Only the last page (newest MRs) changes daily → single cache miss per repo. Combined with DMTOOLS_CACHE_ENABLED=true this gives true delta fetching. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- getCommitsFromBranch: add with_stats=true so each commit carries additions/deletions inline (no extra per-commit API calls) - GitLabCommit.getStats(): implement IDiffStats.getStats() reading the 'stats' JSON field returned by GitLab commits API - PullRequestsLOCMetricSource: use inline stats when commit already implements IDiffStats (avoids N getCommitDiffStat API calls); fall back to getCommitDiffStat only when stats are not inline - PullRequestsLinesOfCodeMetricSource: read inline IDiffStats weight (defensive fix for the old implementation path) Result: Lines Of Code (K) went from 0 to ~220K for CON project. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes 5 bugs in the GitLab client that silently prevented all commit and approval data from appearing in productivity reports. Teams using GitLab instead of GitHub can now get meaningful rate metrics.
Bugs Fixed
1.
GitLabCommit.getAuthor()— wrong JSON shapeGitLab commits API returns
author_name/author_emailas flat top-level strings, not a nestedauthorobject. The old code triedgetJSONObject("author")and always threw/returned null. Fixed by building a syntheticGitLabUserfrom the flat fields.2.
GitLabCommit.getCommiterTimestamp()— wrong field + wrong date parserGitLab uses
committed_date(ISO 8601 with milliseconds) at the top level. The old code triedgetJSONObject("committer").getString("date")which always threwJSONException. Fixed: usegetString("committed_date")withDateUtils.smartParseDate()(notparseIsoDate, which doesn't handle the milliseconds variant).3.
GitLabCommit.getUrl()— hardcoded stubThe method returned the literal string
"url is not supported"instead of reading the field. Fixed:getString("web_url").4.
GitLab.getCommitsFromBranch()— only first 20 commits, no date filterThe old implementation fetched a single page of 20 commits with no date range, so the 90-day productivity window always showed ≤ 20 commits. Fixed: added
per_page=100,since=ISO-date param, and a pagination loop.5.
GitLab.pullRequestActivities()— heuristic note-matching instead of Approvals APIThe old code scanned MR notes for the word "approved" — fragile, language-dependent, and wrong for system notes. Fixed: call the real GitLab Approvals API (
GET /projects/:id/merge_requests/:iid/approvals), parse theapproved_by[]array. A newGitLabMRApprovalmodel wraps each entry.New File
gitlab/model/GitLabMRApproval.java— models the{ "user": {...} }shape from the Approvals API response.Tests
All existing tests pass. New tests added for:
GitLabCommitTest— corrected JSON fixture + 4 new tests (getAuthor, getCommiterTimestamp, getUrl, correct field names)GitLabTest— replaced heuristic-approval test with 4 tests covering: pagination loop, since-filter param, Approvals API call, approved-by extractionGitLabMRApprovalTest— 3 tests for the new modelReferences