fix(api-call): 【to dev】Initialize the optimized index list and rebuild the index

[11000100111010101100111]

No description provided.

[augmentcode]

🤖 Augment PR Summary

Summary: This PR updates API-call monitoring to support a top-level clientId field and rebuilds MongoDB indexes to match the optimized query patterns.

Changes:

• Adds clientId to ApiCallDto, ApiCallEntity, and ApiCallField.
• Updates ApiCallService filtering to query by clientId directly (instead of user_info.clientId).
• Refactors the ApiCall aggregation pipeline into a dedicated aggregate(...) helper for page retrieval.
• Introduces patch V4_17_6_ResetApiCallIndex to drop/recreate ApiCall-related indexes and backfill clientId from existing documents.
• Adjusts task CPU/memory metric persistence to use bulk updateOne instead of upsert.
• Updates API metrics raw aggregation to skip deleted ApiCall records during in-memory processing.
• Adds/updates unit tests for the new patch and modified behaviors.

Technical Notes: The patch rebuilds multiple indexes (including TTL) on ApiCall and related collections to align with the new clientId-based access patterns.

_{🤖 Was this summary useful? React with 👍 or 👎}

[augmentcode]

Review completed. 4 suggestions posted.

Comment augment review to trigger a new review at any time.

[ply0011]

Code Review

Found 2 confirmed bugs — both independently flagged by two separate analysis passes.

Issue 1 — Wrong setter in ApiCallService.save() (setCallId → setClientId)

The save path calls setCallId(...) but should call setClientId(...). This means all newly saved records will have clientId = null, rendering the new top-level field and its index permanently useless for new data. The existing callId field also gets corrupted. See inline comment.

Issue 2 — MongoDB field reference treated as literal string in loadingClientId

collection.updateMany(query, update) with a plain Document does not interpret "$user_info.clientId" as a field reference — it stores the literal string "$user_info.clientId". The aggregation pipeline form collection.updateMany(query, List.of(update)) is required. The existing unit test does not catch this. See inline comment.

[HarsenLin]

SonarQube Quality Gate Status: ERROR

• Status: ERROR, MetricKey: new_coverage, ActualValue: 17.4

See Sonar Scan Result at: http://fileserver.tapdata.io:29000/dashboard?branch=fix-api-call-dev&id=tapdata

[HarsenLin]

SonarQube Quality Gate Status: ERROR

• Status: ERROR, MetricKey: new_coverage, ActualValue: 14.4

See Sonar Scan Result at: http://fileserver.tapdata.io:29000/dashboard?branch=fix-api-call-dev&id=tapdata

[HarsenLin]

SonarQube Quality Gate Status: ERROR

• Status: ERROR, MetricKey: new_coverage, ActualValue: 14.4

See Sonar Scan Result at: http://fileserver.tapdata.io:29000/dashboard?branch=fix-api-call-dev&id=tapdata

[HarsenLin]

SonarQube Quality Gate Status: ERROR

• Status: ERROR, MetricKey: new_coverage, ActualValue: 14.4

See Sonar Scan Result at: http://fileserver.tapdata.io:29000/dashboard?branch=fix-api-call-dev&id=tapdata

[HarsenLin]

SonarQube Quality Gate Status: ERROR

• Status: ERROR, MetricKey: new_coverage, ActualValue: 13.9

See Sonar Scan Result at: http://fileserver.tapdata.io:29000/dashboard?branch=fix-api-call-dev&id=tapdata

[ply0011]

Code Review

Found 2 bugs in the introduced code. Both are in the clientId backfill/save path and will cause data corruption silently.

See inline comments for details and suggested fixes.

[ply0011]

Bug: Invalid MongoDB aggregation pipeline update syntax — migration will silently fail

The $set value here is a List, but a standard update document requires the $set value to be a Document (object). More critically, the field path expression "$user_info.clientId" is only evaluated by MongoDB when the update is passed as an aggregation pipeline — i.e., the entire update argument must be a List<? extends Bson>, not a Document wrapping a list.

As written, the MongoDB driver uses the updateMany(Bson filter, Bson update) overload (single-document update), and the server rejects it because $set's value is an array. The exception is swallowed by the catch block and only logged as a warning, so the migration silently fails to backfill clientId for all pre-existing ApiCall records — defeating the primary purpose of this patch.

Suggested change

	Document update = new Document().append("$set", List.of(new Document().append("clientId", "$user_info.clientId")));
	try {
	collection.updateMany(query, update);
	try {
	collection.updateMany(query, List.of(new Document("$set", new Document("clientId", "$user_info.clientId"))));

The fix passes the update as List<Document>, which causes the driver to call the correct updateMany(Bson filter, List<? extends Bson> pipeline) overload and allows MongoDB to evaluate "$user_info.clientId" as a field path reference.

[ply0011]

Bug: String.valueOf(null) stores the literal string "null" in the database

When userInfo is non-null but does not contain a "clientId" key (e.g. anonymous calls, system calls, or any auth flow that doesn't populate this field), userInfo.get("clientId") returns null. String.valueOf(null) is specified by the JDK to return the 4-character string "null", not a null reference.

This causes clientId = "null" (as a string) to be persisted in MongoDB for every such API call. This poisons the new clientId index introduced in V4_17_6_ResetApiCallIndex and corrupts downstream filtering — for example, genericFilterCriteria now queries criteria.and(Tag.CLIENT_ID).is(clientId), which would erroneously match these "null"-string records.

Suggested change

	if (apiCallEntity.getUserInfo() != null) {
	apiCallEntity.setClientId(String.valueOf(apiCallEntity.getUserInfo().get("clientId")));
	}
	if (apiCallEntity.getUserInfo() != null) {
	Object cid = apiCallEntity.getUserInfo().get("clientId");
	if (cid != null) {
	apiCallEntity.setClientId(cid.toString());
	}
	}