Layer visualization zoom

[gheinrich]

This appears to be working on popular browsers.

Caveat: zooming on 'small' layers (those that have very few weights/activations) feels slightly awkward. Perhaps we should set a minimum size for visualisation images e.g. 256x256?

[lukeyeager]

Works on my browser 👍

I agree this isn't a good solution for small visualizations. Blowing everything up to some minimum size might work. What about a pop-out like the "Basic Zoom" on this page? We want it to work and look good on big/small images and big/small browser windows.

[gheinrich]

I tried the "Basic/Mousewheel Zoom" scheme from the Elevateweb demo page however if the visualisation image is tiny in the first place it is hardly possible to control anything using the mouse. I have added a patch to upscale the visualisation image in case it is less than 200 pixels in height. This looked slightly better to me although one possible downside is all small visualisations would have the same image size so for example the effect of pooling layers is less visually obvious.

[lukeyeager]

Where did you get the *.min.js file from? That one doesn't come with the library version or copyright header.
https://github.com/elevateweb/elevatezoom/blob/master/jquery.elevateZoom-3.0.8.min.js#L1

[lukeyeager]

Yeah, this doesn't seem like a good solution for data with shape (8,):

If you leave the image small and change the CSS for your img to be:

<img style="max-width:100%;min-width:50px;image-rendering:pixelated;" />

It comes out like this:

I like that better, what do you think?

Still playing around with the zoom options ...

[lukeyeager]

Augh, I can't get this to work with 2.2.3 (latest release) or 3.0.8 (current version on master branch). Did you fix some things in the version you uploaded?

[gheinrich]

There was a bug in elevate zoom which caused zooming to fail on square images. I fixed it on elevateweb/elevatezoom#115 but the project seems to be frozen. How do you recommend I shall mention the origin of the file when adding it to the DIGITS github?

The pixelate trick is nice! However it does not work on Firefox... I found this article which explains how to fix this: https://builtvisible.com/image-scaling-in-css/. Besides I noticed elevateZoom gets slightly confused when dealing with images that are resized this way.
We might as well do the resizing on server side to make this more robust: it is quite straightforward to do with sci-py: you just need to set the interpolation to nearest instead of bilinear. However because it seems nothing is every simple in life, I stumbled on this bug in scipy:
scipy/scipy#2347
This affects 3xWx3 or 4xWx3 images, like the visualisations of the last inner product layers on 10-class classifiers and this causes the colors to be completely wrong (white becomes blue, etc.).
So I implemented a small function in tools/image.py to upscale an image. I found the result was OK. I have updated the PR. Let me know what you think. Thanks!

[lukeyeager]

How do you recommend I shall mention the origin of the file when adding it to the DIGITS github?

We need the copyright header as the author intended it, even if we make a small change. Start with this file or this file, make the fix, minimize it, and then copy the header from here or here, depending on which version you started with.

The pixelate trick is nice! However it does not work on Firefox.

Yeah, I was afraid of that. Oh well.

However because it seems nothing is every simple in life, I stumbled on this bug in scipy: scipy/scipy#2347

Hah! Figures.

[lukeyeager]

Oh yeah, this is starting to look good!

Zooming in on gabor filters in conv1:

Upscaling for small fully-connected layers:

@gheinrich I like this a lot, and it's working for me on both Chrome and Firefox. Anything else you want to change before merging? (Besides the copyright)

[gheinrich]

Ah, the javascript "minification" script I used removed all comments including the copyright. But my starting point was this file, as you suggested. I have squashed the commits and uploaded the Pull Request. I hope it's OK now.

[lukeyeager]

Sorry, one last nitpicky thing. If your raw visualization is 100x50 (WxH), then it doesn't get upsampled:

But if your raw visualization is 50x100 (WxH), then it gets upsampled to 100x200:

That's not particularly consistent. Can we use min_image_size or min_image_dim instead of min_image_height?

[gheinrich]

No problem, I have updated the patch. Which network is this by the way? Interesting to see non-square filters (?)

[lukeyeager]

Awesome, I'll merge as soon as TravisCI passes. Coveralls is having problems - ignore the stalled checks.

Which network is this by the way? Interesting to see non-square filters (?)

I just made a little noop network to generate activations with the size I wanted to test 😏.