Feature inversion pipeline for modular iCNN construction

[ganow]

#77

TODO:

• add callbacks to enable additional features (e.g., save snapshots, log metrics on wandb, etc.)
  • Perhaps we should implement additional features after we validate core functions with some test codes
• write example codes that use modules implemented here → moduleを使用した再構成のサンプルコード作成 brain-decoding-cookbook-public#10
• test codes (I will work on this after Refactor test modules #66 is merged on the dev branch)
  • critic
  • encoder
  • generator
  • latent
  • task
  • domain
  • callback
  • dataset

[github-actions]

Coverage Report

File	Stmts	Miss	Cover	Missing
bdpy/bdata
bdata.py	400	320	20%	73–79, 86, 90, 95, 99, 104, 109, 113, 118, 122, 132–134, 155–172, 190, 206–207, 232–248, 252–262, 276–277, 293, 310, 314, 318–356, 388–477, 508, 539, 547–551, 560, 577–584, 589–598, 604–614, 618, 622–625, 628, 632–653, 656–665, 668–677, 683–691, 696–729, 735–744, 750–757, 761–767, 771–799, 803–824, 828–862, 866–868, 872–874, 878–887
featureselector.py	64	57	11%	43–47, 52–93, 98–124
metadata.py	67	48	28%	21–30, 34, 38, 42, 46, 50, 54, 74–115, 135–144, 149, 154
utils.py	113	103	9%	44–110, 127–173, 201, 217–253, 258, 263
bdpy/dataform
datastore.py	107	85	21%	59–75, 90–93, 97–98, 102–113, 116–119, 122–127, 131–132, 137–158, 190–197, 222–259, 262–265
features.py	298	233	22%	29–32, 41–47, 74–103, 107, 111, 115, 119, 137–163, 168–197, 213–238, 241–269, 272–275, 278–289, 305–319, 323, 327, 331, 335, 339, 343, 347, 351, 355, 359, 364–394, 398–418, 422–462, 465, 470–477, 491–493, 496–499, 502–505, 508–512, 515–516, 536–549
kvs.py	140	140	0%	4–272
pd.py	9	5	44%	25–27, 43–44
sparse.py	67	51	24%	19–29, 35–46, 52–58, 65–74, 78, 81–87, 90–93, 96–98, 101–126
utils.py	12	12	0%	3–18
bdpy/dataset
utils.py	45	45	0%	3–98
bdpy/distcomp
distcomp.py	92	78	15%	20–29, 32–49, 52–70, 73–93, 96–107, 111, 114–117, 121–127
bdpy/dl
caffe.py	60	60	0%	4–129
bdpy/dl/torch
base.py	43	24	44%	31–41, 48, 54, 60, 63, 73–83, 90, 96, 102, 105
dataset.py	74	74	0%	1–195
models.py	333	298	11%	28–84, 114–140, 148–169, 175–238, 249–253, 259–279, 288–292, 297–316, 327–331, 340–405, 427–431, 442–494, 515–517, 528–587, 611–614, 625–684, 708–711, 722–771, 790–793, 804–853, 872–875
torch.py	121	92	24%	43–60, 63, 80–95, 105–111, 116, 123, 126, 131, 138, 141, 188–225, 228, 231–243, 246–281
bdpy/dl/torch/domain
core.py	46	20	57%	47, 63, 74, 77, 89–90, 99, 102, 137–138, 141–143, 146–148, 164–165, 168, 173
feature_domain.py	24	13	46%	14, 18, 27–38, 41, 44
image_domain.py	64	37	42%	26, 31, 36, 41, 84–108, 111, 114, 121, 124, 137, 140–150, 179, 211–216, 223, 229
bdpy/evals
metrics.py	95	87	8%	12–34, 40–73, 82–112, 118–159, 172–179
bdpy/feature
feature.py	30	28	7%	33–74
bdpy/fig
__init__.py	4	4	0%	6–9
draw_group_image_set.py	88	88	0%	3–182
fig.py	88	88	0%	16–164
makeplots.py	336	336	0%	1–729
tile_images.py	59	59	0%	1–193
bdpy/ml
crossvalidation.py	59	54	8%	34–61, 104–128, 138, 164–196
ensemble.py	13	9	31%	33–46
learning.py	309	265	14%	43–44, 48, 52, 59, 91–104, 109–125, 128, 158–170, 184–209, 260–284, 290–433, 436–461, 465–504, 507–508, 524–536, 541–613
model.py	140	120	14%	29–39, 54–70, 86–144, 156–169, 184–222, 225, 230–250, 254–258, 271–285
regress.py	11	8	27%	29–38
searchlight.py	16	13	19%	32–51
bdpy/mri
fmriprep.py	497	452	9%	25–34, 38, 44–62, 65–75, 78–89, 92–160, 163–194, 230–360, 367–380, 384, 388–390, 394, 398–400, 410–434, 437–454, 457–464, 471–472, 475–491, 494, 498, 502–815, 819–831, 842–862, 866
glm.py	40	36	10%	46–95
image.py	24	19	21%	29–54
load_epi.py	28	18	36%	36–50, 56–63, 82–88
load_mri.py	19	16	16%	16–36
roi.py	248	234	6%	37–100, 122–148, 165–235, 241–314, 320–387, 399–466, 473–499
spm.py	158	139	12%	26–155, 162–166, 170, 174–179, 183–300
bdpy/opendata
__init__.py	1	1	0%	1
openneuro.py	210	210	0%	1–329
bdpy/pipeline
config.py	36	29	19%	15–64
bdpy/preproc
interface.py	52	44	15%	31–40, 60–69, 96–105, 111–123, 148–157, 208–217
preprocessor.py	129	107	17%	35, 43–65, 74–78, 85–97, 104–132, 138–189, 196–227, 234–239
select_top.py	23	18	22%	35–60
util.py	6	2	67%	14, 22
bdpy/recon
utils.py	55	55	0%	4–146
bdpy/recon/torch
icnn.py	161	161	0%	15–478
bdpy/recon/torch/modules
critic.py	44	20	55%	19, 36, 58, 64–65, 68, 71, 96–110, 132, 157, 184–185
encoder.py	29	11	62%	29, 44, 63, 66, 104–109, 124–125, 175
generator.py	72	36	50%	15–22, 28–38, 47, 52, 68, 83, 90, 93, 96, 122–124, 128, 143, 186–189, 193, 208, 247–248, 252, 256, 306–309
latent.py	34	13	62%	16, 21, 32, 42, 49, 52, 55, 80–83, 87, 97
bdpy/recon/torch/task
inversion.py	83	49	41%	19, 32, 37, 42, 47, 54, 59, 64, 69, 86–89, 92–95, 98, 101–102, 159–167, 185–214, 218–220
bdpy/stats
corr.py	43	38	12%	29–77, 96–112
bdpy/task
callback.py	71	44	38%	15–20, 53, 102–116, 154–156, 161, 166, 212–218, 233–247, 264–265
core.py	16	4	75%	41, 45, 50, 60
bdpy/util
info.py	47	36	23%	19–79
math.py	13	10	23%	23–38
utils.py	36	26	28%	48–66, 93–96, 116–121, 137–145
TOTAL	5644	4782	15%

Tests	Skipped	Failures	Errors	Time
1	0 💤	0 ❌	1 🔥	7.334s ⏱️

[ShuntaroAoki]

@ganow
ありがとうございます。
現状の Base クラスの実装では、__call__() を abstractmethod として具象クラス側で実装するよう強制していますが、この部分をそれぞれの基底クラスで (各々の役割に応じた名前の) パブリックメソッドとして定義しておき、__call__() でそれを呼ぶ、とするのはどう思われますか? 以下のようなイメージです。

class BaseGenerator(ABC):
    """Generator module."""

    @abstractmethod
    def reset_states(self) -> None:
        """Reset the state of the generator."""
        pass

    @abstractmethod
    def parameters(self, recurse: bool = True) -> Iterator[nn.Parameter]:
        """Return the parameters of the generator."""
        pass

    @abstractmethod
    def generate(self, latent: torch.Tensor) -> torch.Tensor:
        """Forward pass through the generator network.

        Parameters
        ----------
        latent : torch.Tensor
            Latent vector.

        Returns
        -------
        torch.Tensor
            Generated image. The generated images must be in the range [0, 1].
        """

    def __call__(self, latent: torch.Tensor) -> torch.Tensor:
        """Call self.generate().

        Parameters
        ----------
        latent : torch.Tensor
            Latent vector.

        Returns
        -------
        torch.Tensor
            Generated image. The generated images must be in the range [0, 1].
        """
        return self.generate(latent)

意図は、メソッドに名前をあたえることでそれぞれのメソッドの役割を明確化することです。あとこれはほとんど私情ですが、親クラスのプライベートメソッドをオーバーライドすることに気持ち悪さがあります。

[ganow]

なるほど。実装するメソッド名はモジュールごとに変えずにforwardで共通化するのはどう思いますか？nn.moduleと透過的に扱うことを意識したAPIです

[myaokai]

sample codeを作成してのコメント

• 画像のresizeを行うdomainが欲しい
• encoder_moduleのlayer名がfeatures[0]のような扱いに対し、今までのconfigなどではconv1_1のように指定しているので変換が面倒
• モジュール内部の挙動についてデバッグがしづらい
• FeatureInversionTaskのreset_statesでgeneratorのパラメータもresetされているがicnnの場合latentのみのresetでいいのではないでしょうか
• FeatureInversionTaskのgeneratorに画像生成後のドメイン変換が含まれていますが、最終的な出力用のドメイン変換も必要ではないでしょうか

[ShuntaroAoki]

Instead of directly overriding __call__, I prefer the implementation of having an abstract method (e.g., run) and calling it in __call__.

[ShuntaroAoki]

Could you remind me of the reason why reset_states is not called in the beginning of __call__ (i.e., why users need to explicitly call reset_states)?

[ShuntaroAoki]

The name "comapre" sounds a bit ambiguous (but I don't have any alternative yet).

[ShuntaroAoki]

See comments on bdpy/task/core.py