4.x: Add public API to force-refresh tablet metadata for a table

[dkropachev]

Problem

The driver already exposes tablet metadata objects (Metadata.getTabletMap(), TabletMap, Tablet), but that map is currently a lazy routing cache. It starts empty and is populated from tablets-routing-v1 custom payloads returned by queries that touch unknown tablets.

That works for point-query routing, but it is not enough for bulk readers such as the Spark connector. A Spark full-table scan needs a complete, current snapshot of tablet ranges and replicas before it builds Spark partitions. It should not have to probe arbitrary tokens or wait for normal query traffic to gradually fill the cache.

Current behavior

Relevant existing API/code:

• Metadata.getTabletMap() exposes a TabletMap, but the API documents that it starts empty and gradually receives updates.
• TabletMap.getMapping() and TabletMap.getTablet(keyspace, table, token) expose only currently known tablets.
• Tablet already has the useful shape for consumers: firstToken, lastToken, replica nodes, and shard lookup.
• CqlRequestHandler updates the tablet map from tablets-routing-v1 response payloads.

There does not appear to be a public API that says: fetch all tablets for this table from Scylla now, populate/replace the relevant entries in TabletMap, and return the resulting table tablet snapshot.

Proposed change

Add a public API to force-refresh tablet metadata for one table, for example:

CompletionStage<List<Tablet>> refreshTablets(CqlIdentifier keyspace, CqlIdentifier table);

or a similar synchronous/asynchronous pair consistent with the driver API style.

The API should:

• Resolve the table ID from schema metadata (TableMetadata.getId()), or fail clearly if it cannot.
• Fetch the complete tablet map for that table from Scylla using a supported metadata source. system.tablets WHERE table_id = ? has the required information today, but an equivalent server-provided routing view/protocol endpoint would also be fine.
• Return a stable table-level snapshot ordered by token range.
• Populate or replace the relevant entries in Metadata.getTabletMap() so normal tablet-aware routing can use the refreshed information.
• Build Tablet objects with the same range contract used today: (firstToken, lastToken].
• Map tablet replica host IDs to driver Node objects through Metadata.getNodes().
• Handle or clearly document behavior for colocated/base-table metadata and tablet migration transitions (replicas, new_replicas, stage, transition). Prefer reusing a server routing view equivalent to tablets-routing-v1 rather than duplicating server transition rules in the client.
• Be safe to call at Spark job planning time and again on task/job retry to recover from tablet splits or migrations.

Also verify that prepared token-range scans can provide enough routing metadata for tablet-aware load balancing: keyspace, table, and routing token. If the driver cannot reliably infer the routing table for prepared token-range statements, add the missing statement/routing API or inference needed by the Spark connector.

Acceptance criteria

• A caller can force-load all tablets for a given table without issuing data queries against each tablet.
• The returned tablets include token boundaries, replica nodes, and shard information.
• The driver's TabletMap is updated consistently for that table.
• Repeated refreshes replace stale entries after tablet split/migration.
• Integration coverage verifies that a newly created tablet table can be force-refreshed from an empty tablet map.
• Integration coverage verifies that the API returns all tablets for the table and that TabletMap.getTablet(...) can find tablets for tokens inside those ranges.

Consumers / related work

• Spark connector consumer/tracker: RDD partitioner should take tablets distribution into account spark-scylladb-connector#15
• Scylla Migrator consumer/tracker: Support tablet-aware source scans and savepoints for tablet tables scylla-migrator#398