Reduction functors (cuco::static_reduction_map refactoring 1/N)

[sleeepyjack]

This PR is part 1/N of the refactoring effort for PR #98

New design for reduction functors that can be used by cuco::static_reduction_map.

Implements the following ideas from @jrhemstad (link):

Here's what I was thinking. A person has 3 options for the ReductionOp

1. Use one of the provided cuco::reduce_* types.

   • No additional work should be required. Partial specialization could/should remove the ReductionOp argument from the constructor

2. Provide a unsynchronized binary callable T F(T, T) and Identity value

• This needs to be wrapped by custom_op to apply F in a CAS loop
• Ideally we could detect this kind of callable and implicitly wrap it in custom_op

3. Provide a synchronized binary callable T F(atomic_ref<T, Scope>, T) and Identity value

• User responsible for correct synchronization through atomic_ref

Examples:

// 1.
// no need to provide `reduce_add{}` 
// No need to provide identity value
cuco::static_reduction_map<cuco::reduce_add<int>, int, int> add_map{capacity, empty_key, alloc}; 

// 2. Unsynchronized binary callable must be wrapped in `custom_op`
struct unsync_add{ 
   int identity = 0; // Must provide identity value
   int operator()(int a, int b){ return a + b; }
};

// internally should wrap `unsync_add` in `custom_op`
cuco::static_reduction_map<unsync_add, int, int> custom_unsync_add_map(capacity, empty_key, unsync_add{}, alloc);

// 3.
stuct sync_add{
   int identity = 0; // Must provide identity value
   template <thread_scope Scope>
   int operator()(atomic_ref<int, Scope> a, int b){ return a.fetch_add(b, memory_order_relaxed); }
};

cuco::static_reduction_map<sync_add, int, int> custom_sync_add_map(capacity, empty_key, sync_add{}, alloc);

Includes changes from PR #186

[PointKernel]

@sleeepyjack which version of clang-format are you using locally?

[sleeepyjack]

@sleeepyjack which version of clang-format are you using locally?

@PointKernel

$ clang-format --version
Ubuntu clang-format version 14.0.0-1ubuntu1

but I don't have pre-commit setup locally since I thought our CI does this task automatically.

[PointKernel]

@sleeepyjack which version of clang-format are you using locally?

@PointKernel

$ clang-format --version
Ubuntu clang-format version 14.0.0-1ubuntu1

but I don't have pre-commit setup locally since I thought our CI does this task automatically.

No worries. CI will indeed fix style issues. I'm just curious about which version introduces the formatting difference.

[sleeepyjack]

I marked this PR as a draft since I might need to add some more changes while developing the other parts of the static_reduction_map. It's good to go for review though.

[sleeepyjack]

Thanks @PointKernel for the review! I have incorporated your suggestions into the newest commits.

[PointKernel]

rerun tests

[sleeepyjack]

Listing all possible values for cuda::thread_scope is very ugly. Does anyone have a better solution?

[sleeepyjack]

Closing this for now as the design will change a lot with the new refactoring