Current state
Images representing the different tiles of the different zoom levels of the spectrograms are generated when the dataset is created and stored throughout the campaign. This results in a large amount of stored media that is not always used.
With this in mind, we want to reduce storage by storing only the highest resolution version of the spectrogram or, going even further, by not storing any spectrograms at all.
On the other hand, to keep the experience as smooth as possible, we need to maintain lightweight tiles rather than sending a high-resolution image directly: Tile-based operation has the advantage of allowing progressive and lighter loading by loading only the visible tiles at the resolution corresponding to the display level. The overhead associated with the larger number of requests is negligible compared to the weight of the images loaded.
Solutions
The proposed solution is to generate tile images on demand from the user, on the server side.
Image generation
A : Store an NPZ matrix at the finest granularity desired and generate all tiles from this matrix
B : Generate tiles directly from audio
C : Hybrid approach for large file
We can consider storing one spectrogram image for the largest zoom level (1 tile = the entire file) and generating all zoomed tiles. The zoomed tile generation can be made using one of the previous solution
Query
GET Request
WebSocket
We will first focus on solutions involving simple HTTP requests.
Current state
Images representing the different tiles of the different zoom levels of the spectrograms are generated when the dataset is created and stored throughout the campaign. This results in a large amount of stored media that is not always used.
With this in mind, we want to reduce storage by storing only the highest resolution version of the spectrogram or, going even further, by not storing any spectrograms at all.
On the other hand, to keep the experience as smooth as possible, we need to maintain lightweight tiles rather than sending a high-resolution image directly: Tile-based operation has the advantage of allowing progressive and lighter loading by loading only the visible tiles at the resolution corresponding to the display level. The overhead associated with the larger number of requests is negligible compared to the weight of the images loaded.
Solutions
The proposed solution is to generate tile images on demand from the user, on the server side.
Image generation
A : Store an NPZ matrix at the finest granularity desired and generate all tiles from this matrix
B : Generate tiles directly from audio
C : Hybrid approach for large file
We can consider storing one spectrogram image for the largest zoom level (1 tile = the entire file) and generating all zoomed tiles. The zoomed tile generation can be made using one of the previous solution
Query
GET Request
WebSocket
We will first focus on solutions involving simple HTTP requests.