Skip to content

feat(uipath-automation-discovery): estimation accelerator — opportunity → pack-hours → contingency (PILOT-6044)#1815

Open
RaduAna-Maria wants to merge 1 commit into
mainfrom
fix/uipath-automation-discovery/estimation-accelerator
Open

feat(uipath-automation-discovery): estimation accelerator — opportunity → pack-hours → contingency (PILOT-6044)#1815
RaduAna-Maria wants to merge 1 commit into
mainfrom
fix/uipath-automation-discovery/estimation-accelerator

Conversation

@RaduAna-Maria

Copy link
Copy Markdown
Contributor

Summary

Adds an optional estimation accelerator to uipath-automation-discovery (new Phase 4.5) that sizes each prioritized opportunity: opportunity → complexity band → pack-hours → adjustment factors → contingency → total. Resolves PILOT-6044.

Background

PILOT-6044 / UV-14914 — a solution architect sized 23 finance-automation opportunities entirely by hand and diverged ~6,000 hours from UiPath's official estimate. Per the reporter's own root-cause analysis, the miss was methodology, not base numbers:

  1. Core RPA matrix and Pack-Hours catalogue were disconnected artifacts with different grain.
  2. Tool vs Process-Automation pack grain was ambiguous when pricing decomposed units.
  3. The matrices cap at "High" — no guidance for clusters over the ceiling (>7 apps / >8 variations).
  4. No model for multi-entity redeploy or existing-automation rebuild.

uipath-automation-discovery already owns the opportunity register and ROI, but produced no pack-hours/contingency — the natural home for this.

What this adds

  • references/estimation-guide.md — the method and the four fixes: an above-ceiling/decompose rule (>7 apps / >8 variations → decompose and sum, never clamp to "High"), a multi-entity redeploy factor, an existing-automation rebuild discount, confidence-tiered contingency, and one unified band→hours mapping that resolves the Tool vs Process-Automation grain (ask feat(CodedAgents): single skill #3).
  • assets/templates/estimation-worksheet-template.md — per-opportunity estimate table, the unified mapping, an above-ceiling decomposition log, a calibration register, and a scenario roll-up + benchmark reconciliation.
  • SKILL.md — Phase 4.5, Critical Rule 6, discovery triggers (estimate/size/pack-hours), reference nav, anti-patterns.

Accuracy guardrail — why no numbers are baked in

The ~6k miss was an accuracy failure, so hardcoding a Pack-Hours table into the skill would recreate the exact bug. Instead (Critical Rule 6): the band→hours numbers and matrix thresholds are authoritative references the user supplies (Core RPA + Agentic complexity matrices, Pack-Hours catalogue) — never invented. If absent, the skill stops and asks. Adjustment-factor and contingency percentages are explicit [CALIBRATE] defaults the user tunes against actuals. Every band→hours value cites its catalogue source, and the math is shown per opportunity.

This mirrors the ASDD-crosswalk decision on the sibling planner PR: the skill supplies the method; the user supplies the authoritative artifact. It fixes the root cause instead of encoding numbers that go stale.

Verification

  • All relative links resolve; references/ + assets/ reachable from SKILL.md.
  • hooks/validate-skill-descriptions.sh exit 0 (description 541 chars).
  • CODEOWNERS already covers /skills/uipath-automation-discovery/.

Follow-ups

  • coder-eval coverage test — not yet added; a fixture-driven task (register + supplied catalogue/matrices → assert decompose rule fires, catalogue sources cited, worksheet structure) is the natural next step.
  • Embedding the authoritative Pack-Hours numbers is intentionally not done here (see guardrail).

Issues

Solved here:

Source feedback:

🤖 Generated with Claude Code

Sizing 23 finance opportunities by hand diverged ~6,000 h from the official
estimate. Per the source feedback (UV-14914) the miss was methodology, not
base numbers: disconnected matrix/catalogue artifacts, ambiguous Tool vs
Process-Automation grain, no above-ceiling handling, no redeploy/rebuild model.

Add an optional Phase 4.5 that sizes each prioritized opportunity:
opportunity → complexity band → pack-hours → adjustment factors → contingency.

- references/estimation-guide.md — the method + the four fixes: an
  above-ceiling/decompose rule (>7 apps / >8 variations), a multi-entity
  redeploy factor, an existing-automation rebuild discount, confidence-tiered
  contingency, and one unified band→hours mapping that resolves grain.
- assets/templates/estimation-worksheet-template.md — per-opportunity table,
  unified mapping, decomposition log, calibration register, scenario roll-up.
- SKILL.md — Phase 4.5, Critical Rule 6, triggers, nav, anti-patterns.

Accuracy guardrail (Critical Rule 6): band→hours numbers and matrix thresholds
are authoritative user-supplied references, never invented; if absent, stop and
ask. Factor/contingency percentages are [CALIBRATE] defaults tuned against
actuals. This fixes the root cause instead of baking in unverifiable numbers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@RaduAna-Maria RaduAna-Maria requested a review from uisherif as a code owner July 2, 2026 12:18
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Claude finished @RaduAna-Maria's task in 2m 54s —— View job


PR Review: estimation accelerator (PILOT-6044)

  • Read review criteria and project conventions
  • Read PR diff and full changed files
  • Validate skill structure, links, and cross-skill independence
  • Check for tests, CODEOWNERS, secrets
  • Post review findings

Summary

Adds an optional Phase 4.5: ESTIMATE to uipath-automation-discovery that sizes prioritized opportunities into pack-hours with adjustment factors and contingency. Changes span SKILL.md (new critical rule, phase, triggers, anti-patterns), a new estimation-guide reference, and a new worksheet template. The design intentionally avoids hardcoding pack-hours numbers — the agent must use user-supplied catalogues.

Change-by-Change Review

1. skills/uipath-automation-discovery/SKILL.md (+26/−1)

Severity: Low

Well-integrated into the existing skill structure. The new Phase 4.5 section, Critical Rule 6, trigger bullet, workflow-overview entry, reference-nav link, and two anti-patterns are all correctly placed and consistent with the existing pattern.

  • Description (line 2): 554 chars — well under the 1024-char cap. Front-loads identity, includes redirects. The new trigger terms (estimate/size a delivery) are well-chosen. OK.
  • Critical Rule 6 (line 28): Clear, prescriptive, includes the stop-and-ask behavior. Good guardrail. OK.
  • Phase 4.5 section (lines 195-212): Bridge content is slightly heavy — lines 205-209 enumerate five specific method features (above-ceiling rule, redeploy factor, rebuild discount, contingency, unified mapping) that are the guide's domain. A single sentence like "The method addresses the gaps identified in PILOT-6044" with a See link would follow reference-pointer discipline more tightly. However the current version is concise enough (~15 lines) that the duplication is minor. Low.
  • Anti-patterns (lines 258-259): Both new entries are actionable and cover real agent mistakes. OK.

2. skills/uipath-automation-discovery/references/estimation-guide.md (+104, new)

Severity: Medium

The guide is well-structured with a clear procedure, inputs table, and unified mapping. A few items:

  • Line 93-94"The source engagement delivered three (16.2k / 12.1k / 10.7k h)": These are specific numbers from what appears to be a real customer engagement. While not a secret per se, including real engagement-specific data in a public skill doc could be sensitive. Consider generalizing to something like "Produce ≥ 2 scenarios — the spread comes from these factors, not from different base numbers" and dropping the specific hour figures. Medium — potential data sensitivity.
    Fix this →

  • Section header "Unified band→hours mapping (ask feat(CodedAgents): single skill #3)" (line 76): The (ask #3) suffix is an internal reference to the PR description's problem enumeration — it won't make sense to a future reader or agent. Remove the parenthetical or replace with a self-contained label like (resolves grain ambiguity). Low.
    Fix this →

  • No error handling for partial inputs. The guide says "STOP and ask" if catalogues are missing entirely, but doesn't cover partial cases — e.g., catalogue supplied but missing the Agentic add-on column, or matrix supplied without variation thresholds. An agent might proceed with incomplete data. Consider adding a brief note: "If the supplied catalogue or matrix is missing columns or bands, treat missing dimensions as unsupplied — ask for clarification before proceeding." Low.

3. skills/uipath-automation-discovery/assets/templates/estimation-worksheet-template.md (+80, new)

Severity: OK

Clean template with correct structure. All seven sections are well-designed. Placeholder conventions are consistent (<UPPER_SNAKE_CASE> for user values, [FROM CATALOGUE] / [CALIBRATE] for tagged values). The relative link back to the guide (../../references/estimation-guide.md) resolves correctly.

No issues.

What's Missing

  • No test task for Phase 4.5. Existing tests (smoke_intake.yaml, e2e_full_discovery.yaml) don't mention estimation, pack-hours, or Phase 4.5. The PR description acknowledges this as a follow-up, which is acceptable — but a smoke test validating that the agent stops and asks for catalogues (the Critical Rule 6 guardrail) would be high-value coverage. Medium.
  • assets/skill-status.json not updated. The skill's status is preview, which is likely still correct — but if the estimation accelerator changes the maturity assessment, this should be reviewed. No action needed if status is intentionally unchanged.

Area Ratings

Area Status Notes
Frontmatter OK 554 chars, correct name, proper redirects, new trigger terms
E2E Tests Issue No test task for Phase 4.5 (acknowledged as follow-up) — Medium
Skill Body OK Phase 4.5 well-integrated, Critical Rule 6 is clear, anti-patterns added
References & Assets Issue Possible real engagement data in guide line 93-94 — Medium; stale internal reference (ask #3)Low
Repo Hygiene OK CODEOWNERS covers path, no secrets, no cross-skill deps, all links resolve

Issues for Manual Review

  • Engagement-specific numbers (estimation-guide.md:93-94): Verify whether 16.2k / 12.1k / 10.7k h from "the source engagement" is appropriate for a public repo, or if it should be generalized.
  • Domain accuracy of thresholds: The above-ceiling rule uses > 7 applications OR > 8 variations as the decompose trigger. Verify these match UiPath's actual complexity-matrix ceiling.
  • Adjustment-factor ranges: The [CALIBRATE] defaults (15-30% redeploy, 20-40% rebuild discount, 10-40%+ contingency) are reasonable but should be validated against UiPath's delivery benchmarks.

Conclusion

Clean, well-structured addition that follows repo conventions. The design decision to require user-supplied catalogues rather than hardcoding numbers is sound and well-justified. Two actionable items:

  1. Medium: Consider removing the specific engagement hour figures (16.2k/12.1k/10.7k) from estimation-guide.md:93-94 — generalize the guidance.
  2. Medium: Plan a follow-up smoke test for Phase 4.5 (the "stop and ask for catalogues" guardrail is the most valuable thing to test).

One low item: remove the (ask #3) internal reference from the section header in estimation-guide.md:76.

Overall: minor fixes requested, nothing blocking.


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants