Pipe: throttle prolonged async sink retries #30493

Job	Run time
unit-test (17, windows-latest, datanode)	34m 19s
unit-test (17, ubuntu-latest, datanode)	25m 2s
unit-test (17, ubuntu-latest, others)	17m 46s
unit-test (17, windows-latest, others)	25m 41s
	1h 42m 48s

Provide feedback