IDEA: Improve Opencast API Performance

[snoesberger]

1. Aim/description
In a production environment, some Opencast API calls can be very slow. This has a significant impact on the admin UI and other applications, such as LMSs, Tobira, etc.
The aim of this would be to improve the overall performance of the Opencast API calls so that interfaces (like admin UIs, LMSs, etc.) can react quickly, smoothly, and responsively. Some of these problems may also be addressed with the new data model. However, this project mainly addresses events, where we are also facing performance issues with series.

Some examples of API calls that respond very slowly in larger environments:

• Modifying series metadata PUT /api/series/<series-id>/metadata?type=dublincore%2Fseries
  • Series with 100 Events, 50 user-roles: response time 1.3 min
  • Series with 100 Events, 1 user-roles: response time 26 sec
  • Series with 19 Events, 3 user-roles: response time 5 sec
• Modifying series ACL PUT /api/series/<series-id>/acl
  • Series with 100 Events, 50 user-roles: response time 1.4 min
  • Series with 100 Events, 1 user-roles: response time 28 sec
  • Series with 19 Events, 3 user-roles: response time 6 sec

2. After the issue has been created, please use project fields to the right to categorise your idea:

• Funding: additional funding welcome
• Scope: S/M?
• Area: Opencast API
• Begin/End: not defined

3. Additional information

• Responsible Person: Sascha Nösberger @snoesberger University of Bern
• Additional Stakeholders: TBD
• Related Feature Requests and Issues:
  • PROJECT: Improve event data model of Opencast #21
  • Experiment with new API for new data model (OpenAPI with dummy data) opencast#6968
  • Performance issue with metadata admin-interface#1230
  • Metadata dropdown takes too long to update due to large response payload admin-interface#1259

[mtneug]

Wow, crazy numbers to see in prod. So +1 for improving this. Note that ACL changes of series can be very expensive and probably require the new data model to fully solve. This is because a new snapshot is created for every event, which can be a very expensive operation in itself, especially when using remote asset stores like S3. To make this fast, ACLs / metadata would likely have to move to the database.

[dagraf]

In roadmap meeting:

• this should probably go away with the data model. fixing it before that seems hard, because of the need of re-publishing. ELAN did try it already and did not succeed.
• Maybe small improvements would be possible already now.
• But with the new data model these improvements would come automatically.
• Maybe see: Index Extended Metadata opencast#3274

Decision:

• Focus on speeding up the new data model.