[mariadb-replication-and-ha] Missing: SST vs IST explanation and gcache sizing

[jana-dbaasnow]

Summary

The Galera section mentions the SST user account but never explains the
difference between State Snapshot Transfer (SST) and Incremental State
Transfer (IST). This gap is the root cause of the single most common
Galera operational incident, and agents have no way to warn operators
about it or help them avoid it.

The problem

When a Galera node that has been offline reconnects, one of two things
happens:

• IST — the donor transfers only the missing write sets from its
  gcache. Fast, low donor impact, no cluster disruption.
• SST — a full data copy from a donor node. Triggered when the
  missing write sets exceed what gcache holds. This copies the entire
  dataset, places heavy load on the donor, and can take hours on a
  large cluster.

Agents currently have no knowledge of this distinction. They treat node
rejoins as a routine background event. In practice, an unexpected SST
on a production cluster is a significant incident.

What agents should know

# IST occurs when: node gap in write sets <= gcache.size
# SST occurs when: node gap in write sets > gcache.size

# Default gcache size is 128MB — far too small for most production clusters
# Size gcache.size to cover the expected maximum node outage window:
# gcache.size = estimated_write_rate_per_second * max_expected_downtime_seconds

# Check SST method in use:
SHOW VARIABLES LIKE 'wsrep_sst_method';

# mariabackup is preferred — donor remains online during SST (non-blocking)
# rsync locks the donor — avoid on production clusters
wsrep_sst_method = mariabackup

Suggested addition

A short IST vs SST subsection covering:

• What triggers each transfer type
• The role of gcache.size and how to size it
• Why wsrep_sst_method = mariabackup is preferred over rsync for
  production clusters (donor remains online during mariabackup SST;
  rsync locks the entire donor)