Make tutorial data generation faster#1063
Closed
CodyCBakerPhD wants to merge 3 commits intoNeurodataWithoutBorders:support-neuroconv-latestfrom
Closed
Make tutorial data generation faster#1063CodyCBakerPhD wants to merge 3 commits intoNeurodataWithoutBorders:support-neuroconv-latestfrom
CodyCBakerPhD wants to merge 3 commits intoNeurodataWithoutBorders:support-neuroconv-latestfrom
Conversation
Collaborator
|
closed accidentally. Reopening elsewhere |
bendichter
added a commit
that referenced
this pull request
Mar 4, 2026
Remove expensive principal_components and spike_amplitudes computations from generate_test_data(). Pass compute_pc_features=False and compute_amplitudes=False to export_to_phy instead. This reduces generation time from ~10min to ~3s. Based on CodyCBakerPhD's PR #1063. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
bendichter
added a commit
that referenced
this pull request
Mar 4, 2026
Remove expensive principal_components and spike_amplitudes computations from generate_test_data(). Pass compute_pc_features=False and compute_amplitudes=False to export_to_phy instead. This reduces generation time from ~10min to ~3s. Based on CodyCBakerPhD's PR #1063. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2 tasks
Collaborator
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
While the caching of tutorial data in CI is not a terrible idea overall, it is a workaround to a broader issue
Prior to when I began the work on #917 ( to support modern versions of SpikeInterface for the tutorial generation), the tutorial data generation step usually took less than 30 seconds to complete, if not faster (this can be tested using the previous releases)
I was unable to finish the debugging on that PR before work stopped on the project. When others finished the work and merged, they introduced an immense slowdown by adding the lines that this PR removes involving PC estimation
This means all releases after v1.0.5 suffered from this slowdown for any users attempting to follow the tutorials from the docs