Possible performance gain in injector

[sigvaldm]

I found another place with lots of pointer dereferencing (ExteriorFacet), which can be optimized if it is found necessary to improve the injector performance.

Mesh has an array (or vector) of fine-grained ExteriorFacet objects. That's a typical Array-of-Objects data structure. Although Array-of-Objects is perfectly in agreement with the OO-philosophy, it is often more efficient to flip the arrangment to an Object-of-Arrays structure, where you have one object ExteriorFacets, containing the data for all facets in long vectors which must be indexed correctly, possibly through inlined helper methods in the class. In this particular example, ExteriorFacets could even be removed altogether and its members put directly in Mesh.

Since the performance loss of the Array-of-Objects approach comes from dereferencing pointers*, another option is to make the ExteriorFacet object a contiguous number of bytes in memory by changing all the data to static C-style arrays, like we did for Particle. It would then become a template of the number of geometric dimensions.

* Possibly also constructor and destructor calls, but that shouldn't be a problem here.

[sigvaldm]

To clarify the helper functions with an example:

class Mesh {
public:
    size_t dim;
    const double *get_facet_normal(size_t facet_id) const {
        return &facet_normals.data()[facet_id*dim];
    };
private:
    std::vector<double> facet_normals;
}

I chose to return a C-style pointer to the part of the facet_normals vector holding the normal vector of the given facet. No copies are made. The const should prevent the user from writing to this adress. It can then be used as:

size_t my_facet = 123;
const double *facet_normal = mesh.get_facet_normal(my_facet);
for(size_t i=0; i<mesh.dim; i++) cout << facet_normal[i] << endl;

[diakodarian]

Improving the performance of injector is quite desirable. Especially for non-Maxwellian VDF's, where it takes almost 80% of the total run-time.

[sigvaldm]

Yes, that's unreasonably much. I'm not sure how much of it can be attributed to this Array-of-Objects structure, though. If I remember correctly I looked through the rejection sampler for bottlenecks once, but maybe there's more I didn't notice at the time. Perhaps it's not too much work to restructure this into vectors directly inside Mesh though?

[diakodarian]

No, it shouldn't take too much time. I can do it tomorrow evening.

[sigvaldm]

As we discovered today, it may be that the injector can get a more significant performance boost by removing vector construction within the inject_particles function.

[diakodarian]

inject_particles and rejection_sampler are now optimized. Lambda expressions have been removed. std::vector is only used for particle positions and velocities, which are created once for each species. Now, we are mostly operating with the underlying C-array of std::vector.

What remains is to be done is removing pointer dereferencing of ExteriorFacet, and implementing an efficient ORS. With these in place, I think we will see a significant performance gain.

[sigvaldm]

Did you notice any gain from removing the lambdas?