Skip to content

avm2: Normalize flash.globalization.Collator.compare to -1/0/1 - #24148

Open
falpi wants to merge 1 commit into
ruffle-rs:masterfrom
falpi:fix-collator-compare-normalization
Open

avm2: Normalize flash.globalization.Collator.compare to -1/0/1#24148
falpi wants to merge 1 commit into
ruffle-rs:masterfrom
falpi:fix-collator-compare-normalization

Conversation

@falpi

@falpi falpi commented Jul 7, 2026

Copy link
Copy Markdown

Summary

flash.globalization.Collator.compare() returned a raw character-code delta instead of a
normalized -1 / 0 / 1. Flash Player's Collator.compare always returns exactly the sign,
and the Flex framework's collection sorting relies on that: the mismatch drives
spark.collections.Sort.findItem()'s binary search into an infinite loop, freezing any app
that sorts a Spark DataGrid string column by clicking its header (CPU pegged, no error thrown).

Root cause

Ruffle's Collator.compare stub delegated to String.localeCompare, which returns a
character-code difference rather than a normalized result:

"Rossi".localeCompare("Bianchi") // 'R'(82) - 'B'(66) == 16, not 1

spark.collections.Sort.findItem() (the binary search used when the collection view resolves an
item's index) branches on the comparison result with an exact switch:

switch (direction) {
    case -1: upperBound = index - 1; break;
    case  0: /* found */            break;
    case  1: lowerBound = index + 1; break;
}

A delta such as 16 matches none of these cases, so neither bound is updated,
lowerBound <= upperBound stays true forever, and the comparator
(SortField.stringCompareCollator.compare) is called endlessly.

The path reached from a header click is:

DataGrid.sortByColumns → ListCollectionView.refresh
  → Grid.dataProvider_collectionChangeHandler → updateCaretForDataProviderChange
    → ListCollectionView.getItemIndex → Sort.findItem
      → Sort.internalCompare → SortField.stringCompare → Collator.compare   // spins here

Reproduction

A minimal Spark DataGrid with inline data. Compile with the Flex 4.6 SDK and open the SWF in
Ruffle, then click the Name column header to sort. Before this patch the player hangs; after
it, the column sorts normally.

<?xml version="1.0" encoding="utf-8"?>
<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
              xmlns:s="library://ns.adobe.com/flex/spark"
              xmlns:mx="library://ns.adobe.com/flex/mx">

    <fx:Declarations>
        <mx:ArrayCollection id="people">
            <fx:Object name="Rossi"/>
            <fx:Object name="Bianchi"/>
            <fx:Object name="Verdi"/>
        </mx:ArrayCollection>
    </fx:Declarations>

    <s:DataGrid dataProvider="{people}" width="300" height="120">
        <s:columns>
            <s:ArrayList>
                <s:GridColumn dataField="name" headerText="Name"/>
            </s:ArrayList>
        </s:columns>
    </s:DataGrid>

</s:Application>

Any string values whose comparison delta is not exactly ±1 reproduce it (i.e. almost all real
data — here 'R' - 'B' == 16).

Fix

Normalize the comparison result to its sign in Collator.compare, matching Flash Player:

var result:int = string1.localeCompare(string2);
return (result < 0) ? -1 : ((result > 0) ? 1 : 0);

Notes

  • Collator.compare is still a stub for locale-aware collation (it uses code-point ordering);
    this change only corrects the shape of the return value, which is what the framework depends on.
  • Custom sortCompareFunctions that already return -1/0/1 were unaffected and keep working; this
    fixes the default string-sort path that goes through the Collator.

Checklist

  • I, a human, have self-reviewed this PR and fully understand the changes within.
  • I have made or updated tests where possible.
  • All of my commits are properly scoped, compile successfully, and pass all tests.
  • This PR does not make sense to split up into smaller PRs.
  • An LLM was involved in the authoring of this code.

@falpi
falpi force-pushed the fix-collator-compare-normalization branch 2 times, most recently from 925ff17 to 490807d Compare July 7, 2026 11:03
@evilpie evilpie added needs-tests This PR needs regression tests to be added before it can be merged A-avm2 Area: AVM2 (ActionScript 3) labels Jul 7, 2026
@falpi
falpi force-pushed the fix-collator-compare-normalization branch from 490807d to 0f435fa Compare July 7, 2026 21:01
flash.globalization.Collator.compare delegated to String.localeCompare,
which returns a raw character-code delta (e.g. 16), not a normalized
comparison result. Flash Player's Collator.compare returns exactly -1, 0,
or 1.

mx and spark collections' Sort.findItem runs a binary search whose body is
`switch (compareResult) { case -1: ...; case 0: ...; case 1: ... }`. A raw
delta matches none of those cases, so neither bound is updated and the
search spins forever. This froze the app (CPU pegged, no error) when
sorting a Spark DataGrid by clicking a column header.

Normalize the result to the sign, matching Flash Player.
@falpi
falpi force-pushed the fix-collator-compare-normalization branch from 0f435fa to c0fe40d Compare July 7, 2026 23:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-avm2 Area: AVM2 (ActionScript 3) needs-tests This PR needs regression tests to be added before it can be merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants