Skip to content

Release per-iteration mdata copies in compute_fake_perturbation_tests #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
adamklie wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Release per-iteration mdata copies in compute_fake_perturbation_tests #5

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9665129
Fix AttributeError in compute_fake_perturbation_tests
adamklie May 9, 2026
520cb15
Release per-iteration mdata copies in compute_fake_perturbation_tests
adamklie May 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion ...ibration/Slurm_version/U-test_perturbation_calibration/U-test_perturbation_calibration.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

import os
import sys
import gc
import yaml
import logging
import argparse
Expand Down Expand Up @@ -157,7 +158,7 @@ def compute_fake_perturbation_tests():
prog_key=args.prog_key,
collapse_targets=True,
pseudobulk=False,
reference_targets=args.reference_targets,
reference_targets=args.guide_annotation_key,
FDR_method=args.FDR_method,
n_jobs=-1,
inplace=False
Expand All @@ -171,6 +172,16 @@ def compute_fake_perturbation_tests():
test_stats_fake_dfs.append(test_stats_df) # combine all
test_stats_fake_dfs_temp.append(test_stats_df) # combine for each k and sel_thresh

# Release per-iteration deep-copy of mdata. Without this, full
# mdata.copy() instances stack up (~10 GB sparse rna matrix per copy
# on real datasets) and OOM around iteration 16/50 in 256 GB.
del _mdata, mdata_samp
gc.collect()

# Release the K-loop mdata before reloading for next K.
del mdata
gc.collect()

# Save results
test_stats_fake_dfs_temp = pd.concat(test_stats_fake_dfs_temp, ignore_index=True)
test_stats_fake_dfs_temp.to_csv(
Expand Down