Skip to content

fix(security): replace pickle-based torch.save/load with safetensors#9

Merged
gomezzz merged 3 commits intomainfrom
hotfix/safetensors-migration
Mar 27, 2026
Merged

fix(security): replace pickle-based torch.save/load with safetensors#9
gomezzz merged 3 commits intomainfrom
hotfix/safetensors-migration

Conversation

@gomezzz
Copy link
Copy Markdown
Collaborator

@gomezzz gomezzz commented Mar 25, 2026

Summary

  • Replace all torch.save/torch.load(weights_only=False) with safetensors to eliminate arbitrary code execution risk from pickle deserialization
  • New checkpoint_io.py module handles all checkpoint serialization: tensors in safetensors binary format, metadata as JSON in the header
  • No legacy .pth/.pkl loading retained — all torch.load calls removed entirely
  • Convert test_model.pth fixture to test_model.safetensors

Security

The safetensors format stores only raw tensor bytes and plain JSON strings — no pickle, no arbitrary object deserialization, no code execution possible. The _checkpoint_object_hook restores enum values via name lookup only (not eval/exec).

Test plan

  • 14 new unit tests for checkpoint_io.py (round-trips, optimizer state, security)
  • Updated fitsbolt config persistence integration tests (10/10 pass)
  • Updated model IO integration tests
  • All 287+ tests pass (failures are pre-existing Jupyter widget issues)

gomezzz added 2 commits March 25, 2026 10:33
Replace all torch.save/torch.load(weights_only=False) usage with the
safetensors library to prevent arbitrary code execution via pickle
deserialization of untrusted model files.

New checkpoint format stores tensors in safetensors binary format and
non-tensor metadata as JSON in the safetensors header. No legacy
.pth/.pkl loading is retained — all torch.load calls are removed.

- Add anomaly_match/data_io/checkpoint_io.py with save_checkpoint()
  and load_checkpoint() functions
- Convert test_model.pth fixture to test_model.safetensors
- Update all file extension references from .pth to .safetensors
- Add safetensors to dependencies
- Add unit tests for checkpoint_io (round-trip, security)
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 25, 2026

Overall Coverage

Coverage Report
FileStmtsMissCoverMissing
__init__.py80100% 
data_io
   SessionIOHandler.py3264187%111, 128–129, 173–174, 184–185, 215–216, 282–283, 286, 326–328, 370, 388–390, 455, 530, 553, 567–568, 570–571, 573–575, 719–724, 738, 742–743, 747–749
   checkpoint_io.py1482583%68, 74, 76, 78, 95–109, 120, 122, 161, 182, 228, 281
   find_images_in_folder.py210100% 
   load_images.py912869%51–53, 58–60, 81, 93, 110, 140–147, 150, 156–157, 189, 248, 256, 277, 285, 292–294
   metadata_handler.py80890%70–71, 117, 122–123, 170–172
   save_config.py33390%92–94
datasets
   AnomalyDetectionDataset.py2483984%126–127, 356, 362–363, 365, 369, 371–372, 374, 395–396, 398, 404–405, 409, 422–423, 484–485, 491–496, 498–499, 503–506, 508–510, 512–514, 525
   BasicDataset.py52492%59, 61, 97, 103
   Label.py50100% 
   SSL_Dataset.py68395%136, 139, 209
   __init__.py00100% 
   data_utils.py56296%80, 199
datasets/augmentation
   randaugment.py921188%222, 225–226, 245, 328–330, 332–335
   randaugment_multispectral.py772370%77–79, 92–93, 120–121, 148–149, 166, 170, 174, 248, 253, 270–272, 274–277, 280–281
image_processing
   transforms.py57394%46–47, 76
models
   FixMatch.py2142986%108, 201–202, 217, 222, 254, 281–282, 285, 288–290, 293, 301, 306–307, 366, 404, 430–434, 495, 497–498, 502, 519–520
pipeline
   SessionTracker.py122397%127, 219–220
   session.py5358584%131–134, 384–387, 432, 445, 597, 654, 656–657, 665, 668, 679, 687, 717, 721, 725–726, 730, 735–736, 743, 747–748, 760, 770–771, 773, 777, 791–792, 794–796, 798, 801–802, 860–861, 863–866, 868–872, 878, 884, 889–893, 895, 903, 906–907, 923, 964–965, 971–972, 976–978, 980, 1005, 1031, 1065–1066, 1074, 1085, 1093, 1097, 1101, 1106, 1109, 1153, 1156
utils
   accuracy.py130100% 
   consistency_loss.py180100% 
   create_model_string.py40100% 
   cross_entropy_loss.py90100% 
   cutana_stream_utils.py70987%64–65, 88, 108, 118–122
   get_cosine_schedule_with_warmup.py11190%39
   get_default_cfg.py610100% 
   get_net_builder.py49785%81–82, 89, 92, 146–147, 151
   get_optimizer.py21290%54, 58
   print_cfg.py45491%89, 95, 117, 119
   set_log_level.py150100% 
   set_seeds.py13284%25–26
   validate_config.py1311191%180, 199, 214, 228, 244, 248, 264, 290, 379–381
TOTAL269334387% 

Tests Skipped Failures Errors Time
375 0 💤 0 ❌ 0 🔥 1m 0s ⏱️

@gomezzz gomezzz requested a review from giusgal March 25, 2026 09:40
Comment on lines 187 to 223
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like this function is not used anymore (session.py only uses save_model)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Removed both save_model_checkpoint and load_model_checkpoint — neither had production callers (session.py only uses save_model/load_model). Also removed their tests and vulture whitelist entries. See adcf233.

Remove save_model_checkpoint and load_model_checkpoint from
SessionIOHandler — neither is called from production code (session.py
only uses save_model/load_model). Remove their tests and vulture
whitelist entries.
@gomezzz gomezzz requested a review from giusgal March 26, 2026 08:29
@gomezzz gomezzz merged commit d63b154 into main Mar 27, 2026
5 checks passed
@gomezzz gomezzz deleted the hotfix/safetensors-migration branch March 27, 2026 11:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants