Skip to content

Add cache and offload support for TracInAttributor#257

Open
brightXian wants to merge 3 commits intoTRAIS-Lab:mainfrom
brightXian:mingtao/train_cache_offload
Open

Add cache and offload support for TracInAttributor#257
brightXian wants to merge 3 commits intoTRAIS-Lab:mainfrom
brightXian:mingtao/train_cache_offload

Conversation

@brightXian
Copy link
Copy Markdown

@brightXian brightXian commented Mar 2, 2026

Description

1. Motivation and Context

Implements cache + offload for TracInAttributor as discussed in #255.

2. Summary of the change

Added offload parameter with three modes: "none" (in-memory), "cpu", "disk". Reuses existing offload manager from block_projected_if.

attributor = TracInAttributor(..., offload="none")  # or "cpu" / "disk"
attributor.cache(train_loader)
score = attributor.attribute(test_loader)

Also changed attribute(train, test)attribute(test, train=None) to support the cached path. All existing callers updated.

Benchmark

Here is a simple benchmark on MNIST LR, 10k train / 2k test, batch 100, 2 checkpoints, CUDA, projector dim 512. All three cache modes match no-cache baseline.

Mode Avg (s) Peak (MB)
No cache 61.81 829.93
Cache (none) 3.37 846.86
Cache (cpu) 5.93 787.52
Cache (disk) 6.05 787.52

The speedup comes from avoiding redundant train gradient recomputation. "none" is fastest by concatenating all cached grads for a single matmul; "cpu"/"disk" trade speed for lower memory by retrieving batch-by-batch.

3. Tests

  • Unit test: cache + offload tests for all three modes in test_tracin and test_tracin_proj

Copy link
Copy Markdown
Collaborator

@TheaperDeng TheaperDeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants