Performance: CCX coach grade report (ccx_grades_csv) reads persisted grades with an N+1, unlike CourseGradeReport

[sergivalero20]

Summary

The synchronous CCX coach grade report (ccx_grades_csv in lms/djangoapps/ccx/views.py) reads persisted course/subsection grades one student at a time, producing an N+1 query pattern that makes the download slow and worker-blocking for larger classes. The asynchronous course grade report
(CourseGradeReport in lms/djangoapps/instructor_task/tasks_helper/grades.py) already avoids this by bulk-prefetching grades for the whole cohort. This issue proposes bringing the same, existing optimization to the CCX report.

Affected component

• App: lms/djangoapps/ccx
• View/endpoint: ccx_grades_csv → GET /courses/{ccx_id}/ccx_grades.csv
  (CCX Coach dashboard → Student Admin → “Download student grades”)

Current behavior

ccx_grades_csv iterates enrolled students with CourseGradeFactory().iter(enrolled_students, course) and, for each student, reads their persisted subsection grades individually (via SubsectionGradeFactory → bulk_read_grades). Because grades are not prefetched for the whole class, each student triggers a separate DB read. There is also no modulestore().bulk_operations(...) wrapper around the iteration.

As a result, the number of grade-related queries grows linearly with the number of enrolled learners, and the report runs entirely inside the HTTP request, holding a web worker until it completes.

Steps to reproduce

1. Create a CCX with a non-trivial number of enrolled learners who have attempted graded content (so persisted grades exist).
2. As the CCX coach, open Student Admin and click “Download student grades” (or call GET /courses/{ccx_id}/ccx_grades.csv).
3. Observe (e.g. with Django Debug Toolbar / query logging) that the number of grade-related queries scales with the number of enrolled learners, and that response time grows with class size.

Expected behavior

The grade-related query count should be independent of the number of enrolled learners (a single bulk read), and the download should be fast for typical class sizes — matching how CourseGradeReport behaves.

Impact

• Slow, sometimes timing-out downloads for CCX coaches on larger classes.
• One web worker held for the full duration of each synchronous download.
• Inconsistent performance characteristics between the CCX grade report and the
  standard course grade report, despite reading the same persisted data.

Proposed solution

In ccx_grades_csv, before iterating over students:

1. Materialize the enrolled-students queryset to a list.
2. Call prefetch_course_and_subsection_grades(course_key, users) (lms/djangoapps/grades/api) to bulk-load persisted course and subsection grades once for the whole class.
3. Wrap the grade iteration in modulestore().bulk_operations(course_key).

This mirrors CourseGradeReport._rows_for_users / _CourseGradeBulkContext and reuses existing, already-blessed helpers. The CSV output is unchanged — this is a pure performance improvement (the report already reads from the persisted grade tables; only the read is made bulk instead of per-student).

Backward compatibility / risk

• No change to report contents or format; output is byte-for-byte identical.
• No schema/data migration.
• Uses the same prefetch/bulk-operations pattern already relied on by the
  asynchronous grade report.

Additional bug found: email and username columns contain a bytes repr

While working on this report I found a second, related defect in the same view that is worth fixing together with the performance issue.

Current behavior

The email and username columns of the downloaded CSV contain the Python bytes representation instead of the value, e.g.:

id,email,username,grade,HW 01,...
42,b'learner@example.com',b'learner',0.75,...

Cause

ccx_grades_csv builds each row with:

rows.append([student.id, student.email.encode('utf-8'),
             student.username.encode('utf-8'),
             course_grade.percent] + row_percents)

On Python 3, .encode('utf-8') returns bytes, and csv.writer serializes a bytes object using its repr() — hence the b'...' prefix. The .encode() calls are a Python 2 leftover; csv.writer handles unicode strings natively. Note that student.id and the header row are not encoded, which is why only these two columns are affected.

Steps to reproduce

As a CCX coach, go to the CCX Coach dashboard → Student Admin → "Download student grades". Open the CSV and inspect the email and username columns. Expected behavior

Both columns should contain the plain values (learner@example.com, learner), including for usernames containing non-ASCII characters.

Proposed fix

Drop the two .encode('utf-8') calls. This also covers the unicode-username case that originally motivated them, since csv.writer encodes correctly on its own.

Related (optional, could be a separate issue/PR)

ccx_grades_csv sets Content-Disposition: attachment without a filename, so every CCX downloads as a generic ccx_grades.csv. Using the platform’s existing course_filename_prefix_generator to produce a descriptive, timestamped filename (e.g. <course_prefix>_grade_report_<YYYY-MM-DD-HHMM>.csv) would make it consistent with CourseGradeReport downloads. Happy to include this or split it out.

Environment

• Open edX release: <e.g. Ulmo / master>
• Deployment: <devstack / tutor / production>

Willingness to contribute

We have implemented this change (including tests: an updated test_grades_csv and a new test asserting a single bulk prefetch covers all enrolled students) and are willing to open a PR.