Single-pass totals computation for Trial Balance Excel (Totaling accounts)#8565
Single-pass totals computation for Trial Balance Excel (Totaling accounts)#8565mynjj wants to merge 10 commits into
Conversation
…als for Totaling accounts
7bdcefa to
ced97cc
Compare
…g accounts) (#8568) Backport of #8565. Fixes [AB#638253](https://dynamicssmb2.visualstudio.com/1fcb79e7-ab07-432a-a3c6-6cf5a88ba4a5/_workitems/edit/638253). Co-authored-by: Joshua Martínez Pineda <diegojoshuam@microsoft.com>
…g accounts) (#8567) Backport of #8565. Fixes [AB#638254](https://dynamicssmb2.visualstudio.com/1fcb79e7-ab07-432a-a3c6-6cf5a88ba4a5/_workitems/edit/638254). Co-authored-by: Joshua Martínez Pineda <diegojoshuam@microsoft.com>
…g accounts) (#8569) Backport of #8565. Fixes [AB#638252](https://dynamicssmb2.visualstudio.com/1fcb79e7-ab07-432a-a3c6-6cf5a88ba4a5/_workitems/edit/638252). Co-authored-by: Joshua Martínez Pineda <diegojoshuam@microsoft.com>
… into bugs/tb-totaling-accs
# Conflicts: # src/Apps/W1/ExcelReports/Test/src/TrialBalanceExcelReports.Codeunit.al
Rename TrialBalanceData to TempTrialBalanceData in QueryPathDoesNotDoubleCountNestedTotals to match the temporary-variable naming convention enforced by AA0073. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Pull request was closed
…tb-totaling-accs
| ContainingTotals: List of [Code[20]]; | ||
| begin | ||
| if AccountToTotals.Get(PostingAccountNo, ContainingTotals) then | ||
| ContainingTotals.Add(TotalAccountNo) |
There was a problem hiding this comment.
Dictionary list update silently discarded on existing key
In AL, Dictionary.Get(Key, var Value) copies the stored List of [T] value into the local variable. The subsequent ContainingTotals.Add(TotalAccountNo) mutates only that local copy; the dictionary entry is never updated. Any posting account that belongs to more than one totaling range will therefore only aggregate into the first total account encountered, silently producing incorrect (incomplete) total rows for every subsequent overlapping total.
Recommendation:
- After the
Addcall in the if-branch, write the updated list back to the dictionary withAccountToTotals.Set(PostingAccountNo, ContainingTotals).
| ContainingTotals.Add(TotalAccountNo) | |
| if AccountToTotals.Get(PostingAccountNo, ContainingTotals) then begin | |
| ContainingTotals.Add(TotalAccountNo); | |
| AccountToTotals.Set(PostingAccountNo, ContainingTotals); // persist the mutation | |
| end else begin | |
| ContainingTotals.Add(TotalAccountNo); | |
| AccountToTotals.Add(PostingAccountNo, ContainingTotals); | |
| end; |
👍 useful · ❤️ especially valuable · 👎 wrong - reply with why
Record assignment across temporary instances may lose context
Recommendation:
Line mapping was unavailable, so this was posted as an issue comment. 👍 useful · ❤️ especially valuable · 👎 wrong - reply with why |
| exit; | ||
| repeat | ||
| if TotalAccount.Totaling <> '' then begin | ||
| PostingAccount.Reset(); |
There was a problem hiding this comment.
N+1 query pattern in total-account map build
Inside the repeat..until TotalAccount.Next() = 0 loop, PostingAccount.Reset() and FindSet() issue a new G/L Account database query for every Total/End-Total account found. A chart of accounts with many total headers will generate N separate round-trips, potentially causing noticeable latency when the report is run.
Recommendation:
- Load all Posting-type G/L accounts into a temporary record once before the loop, then use
SetFilter/FindSetagainst that temporary set instead of going back to the database on each iteration.
| PostingAccount.Reset(); | |
| // Before the TotalAccount loop, load all posting accounts into a temp record: | |
| TempPostingAccount.Reset(); | |
| TempPostingAccount.DeleteAll(); | |
| PostingAccount.SetRange("Account Type", "G/L Account Type"::Posting); | |
| if PostingAccount.FindSet() then | |
| repeat | |
| TempPostingAccount := PostingAccount; | |
| TempPostingAccount.Insert(); | |
| until PostingAccount.Next() = 0; | |
| // Inside the loop, query the temp table: | |
| TempPostingAccount.SetFilter("No.", TotalAccount.Totaling); | |
| if TempPostingAccount.FindSet() then | |
| repeat | |
| AddTotalForAccount(AccountToTotals, TempPostingAccount."No.", TotalAccount."No."); | |
| until TempPostingAccount.Next() = 0; |
👍 useful · ❤️ especially valuable · 👎 wrong - reply with why
|
|
||
| local procedure GetOpeningBalanceCutoff(StartDate: Date): Date | ||
| begin | ||
| // We return the date immediately before the starting date, considering BC's date ordering and the presence of closing dates |
There was a problem hiding this comment.
GetOpeningBalanceCutoff comment contradicts behaviour
When StartDate is already a closing date, the function returns NormalDate(StartDate) — the same calendar day, not the day before. The inline comment says "the date immediately before the starting date", which is misleading and risks future maintainers misreading the intent, potentially introducing an off-by-one regression when this function is modified.
Recommendation:
- Rewrite the comment to describe what the returned value means in terms of BC's date-filter semantics: it is the latest date that should fall inside the opening balance filter, keeping the closing-date entry (or the normal entries on StartDate when StartDate is a closing date) inside the reported period.
| // We return the date immediately before the starting date, considering BC's date ordering and the presence of closing dates | |
| local procedure GetOpeningBalanceCutoff(StartDate: Date): Date | |
| begin | |
| // Returns the upper boundary for the opening-balance filter ('..X'). | |
| // If StartDate is already a closing date, the normal date of that day is the boundary: | |
| // closing entries for that day fall INSIDE the period (not the opening balance). | |
| // If StartDate is a normal date, all closing entries of StartDate-1 belong to the opening balance. | |
| if StartDate = ClosingDate(StartDate) then | |
| exit(NormalDate(StartDate)); | |
| exit(ClosingDate(StartDate - 1)); | |
| end; |
👍 useful · ❤️ especially valuable · 👎 wrong - reply with why
| // [WHEN] Running the query-based trial balance for the current year | ||
| GLAccount.SetRange("Date Filter", DMY2Date(1, 1, Date2DMY(WorkDate(), 3)), DMY2Date(31, 12, Date2DMY(WorkDate(), 3))); | ||
| TrialBalance.ConfigureTrialBalance(false, false); | ||
| TrialBalance.InsertTrialBalanceReportData(GLAccount, TempDimension1Values, TempDimension2Values, TempTrialBalanceData); |
There was a problem hiding this comment.
Nested-totals test missing dimension and budget assertions
QueryPathDoesNotDoubleCountNestedTotals only verifies Balance on the default (empty) dimension combination. The AddRowToTotal function keys on (G/L Account No., Dimension 1 Code, Dimension 2 Code, Business Unit Code, Period Start), so a posting account with dimension values in multiple overlapping totaling ranges is untested. Additionally, the budget accumulation path in AddRowToTotal is not exercised by this test, meaning a regression there would go undetected.
Recommendation:
- Add a second scenario with at least one non-empty dimension combination and verify that both the child and parent total rows carry the correct dimension-specific balance. Add a parallel budget-data variant using
ConfigureTrialBalance(false, true)and assertBudget (Net)on the total rows.
| TrialBalance.InsertTrialBalanceReportData(GLAccount, TempDimension1Values, TempDimension2Values, TempTrialBalanceData); | |
| // Extend the test or add a new procedure: | |
| // [GIVEN] Post entries with Dimension 1 = 'DEPT1' | |
| // [THEN] Child End-Total row for Dimension 1 = 'DEPT1' has Balance = EntryAmount | |
| // [THEN] Parent End-Total row for Dimension 1 = 'DEPT1' also has Balance = EntryAmount (not doubled) | |
| // Similarly assert Budget (Net) after ConfigureTrialBalance(false, true) |
👍 useful · ❤️ especially valuable · 👎 wrong - reply with why
…tb-totaling-accs
| TotalAccount.SetFilter("Account Type", '%1|%2', "G/L Account Type"::"End-Total", "G/L Account Type"::Total); | ||
| if AccountNoFilter <> '' then | ||
| TotalAccount.SetFilter("No.", AccountNoFilter); | ||
| if not TotalAccount.FindSet() then |
There was a problem hiding this comment.
TotalAccount (G/L Account) is iterated via FindSet in BuildAccountToTotalsMap without a preceding SetLoadFields call.
Inside the loop only the Totaling field is read; No. and Account Type are included automatically as the primary-key and filter fields respectively. G/L Account is a wide table and the loop may iterate well above ten Total/End-Total accounts in a large chart of accounts, which is the threshold above which the best practice requires SetLoadFields. Adding SetLoadFields("Totaling") before FindSet limits each row transfer to just the one field the body needs.
Suggested fix (apply manually — could not be anchored as a one-click suggestion):
TotalAccount.SetLoadFields("Totaling");
if not TotalAccount.FindSet() thenKnowledge:
👍 useful · ❤️ especially valuable · 👎 wrong - reply with why
Copilot PR ReviewIteration 1 · Outcome: completed Knowledge source: https://github.com/microsoft/BCQuality@822cae1b2771ac25f665f73369f69093bd4fd630 Findings by domainFindings split into Knowledge-backed (cite a BCQuality article) and Agent (the agent's own judgement, no matching BCQuality rule).
Totals: 1 knowledge-backed · 0 agent findings. Orchestrator pre-filter (13 file(s) excluded)
Findings produced by the Copilot CLI agent against BCQuality at |
Summary
The query-based Trial Balance Excel report (
codeunit 4410 "Trial Balance") synthesized Total/End-Total rows by looping over every Total account and, for each one, re-filtering the buffer andCalcSums-ing once per distinct (Dimension 1, Dimension 2, Business Unit) combination. On charts of accounts with many totals this isO(totals × combinations)passes over the buffer and dominated the cost of large trial balances.This change computes all totals in a single sweep:
BuildAccountToTotalsMapmaps each posting account to the list of totals whoseTotalingrange contains it.DistributePostingRowsToTotalssweeps the posting rows once, accumulating each row into every containing total's in-memory buffer (keyed by account + dimensions + business unit + period).MergeTotalsIntoBufferruns the budget comparison / all-zero checks and inserts the resulting totals.Because the map is built only from posting accounts, a parent End-Total whose range spans a nested child End-Total's number no longer re-sums the child's already-inserted buffer row — fixing a latent double-counting bug for nested totals.
Fix to allow closing dates to be specified as starting dates is also included.
Fixes AB#638251
Fixes AB#638353
Fixes AB#640630