[ISSUE #10284] Fix consumerOffset deserialization error

[ChineseTony]

Which Issue(s) This PR Fixes

• Fixes [Bug] JSONException: not support unquoted name #10284

Brief Description

Discussion about this fix: #10283

Problem Introduction

update fastjson2 version to avoid deserialization error

How Did You Test This Change?

[ChineseTony]

@yx9o @zhishengzhang

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 48.90%. Comparing base (c6fc39a) to head (1845df9).

Additional details and impacted files

@@              Coverage Diff              @@
##             develop   #10287      +/-   ##
=============================================
- Coverage      48.97%   48.90%   -0.08%     
+ Complexity     13467    13448      -19     
=============================================
  Files           1375     1375              
  Lines         100450   100452       +2     
  Branches       12973    12973              
=============================================
- Hits           49197    49122      -75     
- Misses         45258    45319      +61     
- Partials        5995     6011      +16     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ChineseTony]

ConsumerOffsetSerializeWrapper cannot generate setter lambda via JDK8 LambdaMetafactory when using fastjson2 2.0.61

[ChineseTony]

This issue is caused by a FastJSON bug. It will be completely fixed once FastJSON releases an official patch. We adopt a temporary workaround for immediate mitigation.

[yx9o]

@yx9o @zhishengzhang

Sorry for the late reply.

Just to confirm, this PR and #10283 are both trying to address #10284, but the approaches look different. #10283 changes the generic RemotingSerializable decode path, while this PR narrows the fix to the ConsumerOffset response shape.

If we keep this approach, could we add a regression test covering the mixed offsetTable + pullOffsetTable response returned by GET_ALL_CONSUMER_OFFSET?

[ChineseTony]

The root cause is that the newly added getPullOffsetTable method and the private final field pullOffsetTable in ConsumerOffsetManager cause FastJSON to serialize redundant extra fields.
To avoid compatibility bugs brought by FastJSON, we temporarily add the pullOffsetTable field to ensure backward compatibility.

#10283 modifies the general deserialization path, while this PR fixes only the ConsumerOffset response shape with smaller scope and lower risk.
I'll add the corresponding regression test for the mixed offset response of GET_ALL_CONSUMER_OFFSET soon.

[zhishengzhang]

@yx9o @zhishengzhang

Sorry for the late reply.

Just to confirm, this PR and #10283 are both trying to address #10284, but the approaches look different. #10283 changes the generic RemotingSerializable decode path, while this PR narrows the fix to the ConsumerOffset response shape.

If we keep this approach, could we add a regression test covering the mixed offsetTable + pullOffsetTable response returned by GET_ALL_CONSUMER_OFFSET?

Thanks @ChineseTony for the fix and @yx9o for the review.

I'd like to share some findings from my investigation in #10283. I had already provided detailed analysis and verification there, but unfortunately those points were not addressed before this separate PR was created.

1. The try-catch fallback in RemotingSerializable is always necessary

When upgrading RocketMQ from 4.9.x → 5.4.x → 5.5.x, the Slave node throws com.alibaba.fastjson2.JSONException: not support unquoted name or illegal number during consumerOffset synchronization. Regardless of whether pullOffsetTable and groupTopicMap are added to ConsumerOffsetSerializeWrapper, the try-catch fallback in RemotingSerializable is always necessary — because cross-version serialized data may contain undeclared fields that we cannot predict today, and a single deserialization failure should not crash the entire slave sync process.

2. Comprehensive check of all SerializeWrapper classes

I've completed checking all SerializeWrapper classes in the codebase:

ConsumerOffsetSerializeWrapper — has unquoted numeric keys and version mismatch (this PR fixes it)
TopicConfigAndMappingSerializeWrapper, TopicConfigSerializeWrapper, SubscriptionGroupWrapper
OffsetSerializeWrapper, TopicQueueMappingSerializeWrapper, MessageRequestModeSerializeWrapper
DelayOffsetSerializeWrapper, KVConfigSerializeWrapper, RocksDBOffsetSerializeWrapper
TransactionMetricsSerializeWrapper
All other wrappers use String keys and have consistent fields across versions, so no action needed for them.

3. Production-verified

My fix from #10283 has been deployed in our test environment and is running normally, confirming that both the wrapper field addition and the RemotingSerializable fallback are effective in practice.

That said, I agree this PR is the right immediate fix for #10284 — it's targeted and low-risk. The RemotingSerializable hardening can be considered as a follow-up improvement.