WithBatchRule on SliceRuleSet will behave like WithRule except that it takes a maximum batch size. The rule will then be validated with slices the same size or smaller than the batch. This is intended to be able to efficiently process larger data sets without having to hold the entire result set in memory.
Use cases include:
- Very large data sets.
- Rules that require external API calls that have a maximum batch size.
- Data sets that stream over longer periods of time.
We may also include a maximum duration as either another argument or through an option WithBatchTimeRule method. The purpose of the timeout would be to batch items by either the max time reached or the batch size. Whichever comes first. This will avoid batches not processing for extended periods of time. This is different than a timeout, which would cancel the whole validation.
Additionally, we would want WithBatchRuleFunc variants.
WithBatchRuleonSliceRuleSetwill behave likeWithRuleexcept that it takes a maximum batch size. The rule will then be validated with slices the same size or smaller than the batch. This is intended to be able to efficiently process larger data sets without having to hold the entire result set in memory.Use cases include:
We may also include a maximum duration as either another argument or through an option
WithBatchTimeRulemethod. The purpose of the timeout would be to batch items by either the max time reached or the batch size. Whichever comes first. This will avoid batches not processing for extended periods of time. This is different than a timeout, which would cancel the whole validation.Additionally, we would want
WithBatchRuleFuncvariants.