I want to know whether the data set is real shot or synthesized.

[whattoshow]

Hello ,I'm training the original dataset, and I want to shot some pics for calibration, so I want to know whether the dataset is real shot or sythesized, and should the image be adjusted to look like the one in the data set?

[BingyaoHuang]

@whattoshow The images are real captured. You need to capture your own training data if you want to apply our method to your setup.

[whattoshow]

@whattoshow The images are real captured. You need to capture your own training data if you want to apply our method to your setup.

Thank you! I'll capture some images and train them with your mathod!~

[whattoshow]

@BingyaoHuang Hi, these days I tried to make a dataset with my own pro-cam system, I saw the images from cam/warp/train folder look like the surface is fixed and unique, but the reference textured surface folder contains 126 images, I feel a little bit confused why there are 126 images but use only one surface to project image, is it to improve model's generalization performance?

[BingyaoHuang]

@whattoshow cam/warp/ref are used by TPS method only, our CompenNet does not use them, except the gray image (i.e., the surface image).

[whattoshow]

@whattoshow cam/warp/ref are used by TPS method only, our CompenNet does not use them, except the gray image (i.e., the surface image).

Yesterday, I trained my own dataset with your Model, the train result was good, I'll try to compensate the projection, thank you so much for your explanation of the experiment details.~

[BingyaoHuang]

@whattoshow Anytime! Great to know you got good results!

When you finish compensation, would you like to share some compensation results and setups details (e.g., surface material, projector/camera specs)? Maybe we can discuss how to improve it.

[whattoshow]

@whattoshow Anytime! Great to know you got good results!

When you finish compensation, would you like to share some compensation results and setups details (e.g., surface material, projector/camera specs)? Maybe we can discuss how to improve it.

Ok~ But maybe I'll do this experiment for a few days, when I finished, I will share the result and setup details with you~

[whattoshow]

@BingyaoHuang The train result of my experiment was good, but I used a simpler stripe surface to project and capture images, the projector compensation image looks like a little bit blurry, so I want to do more experiments to check out the blurry problem.

[BingyaoHuang]

@whattoshow Sure. For the blurry issue, these discussions (#4, #6) may help.

[whattoshow]

@BingyaoHuang Hi, I met the same problem as sapoluri 's in ( #4) , when I used the prj/cmp image to project, I found the results look not good.
Gray image projected on 10 stripe surface:

Compensated images:

Two compensated images projected on a 10 stripe surface:


Training curve and training result:

I used the original code, train and test dataset to train model with my own surface. I used a Viewsonic PJD6353 projector and Canon 5DS R camera
Is it the compensation problem caused by too dark surface texture or other factor?

[BingyaoHuang]

@whattoshow

1. Your environment light may be too strong, in this case the compensation may not work so well. For example, when you input a plain dark image (i.e., RGB = 0,0,0) to the projector, the camera-captured image may not be (0,0,0) due to the strong environment light, because the environment light cannot be decreased by your projector light.
2. Your projector output light is too weak, you can adjust gamma, gain or brightness to bring it up.

In #4 I also mentioned that we have more detailed in our supplementary, I quote here "It is worth noting that the camera-projector parameters are coadjusted such that the brightest projected input image (plain white) slightly overexposes the camera captured image. Similarly, the darkest projected input image (plain black) slightly underexposes the camera captured image. This allows the projector dynamic range to cover the full camera dynamic range."

For example, please check "light2\pos4\curves\cam\warp\ref", img_0001 is almost dark (underexposed) and img_0125 is almost white (overexposed), which means that our projector's dynamic range can cover the camera's dynamic range.

Note that the dynamic range requirement is not so strict, and you can follow this example to capture new data and test the compensation first, then adjust your setup to find a satisfactory setting.

[whattoshow]

I have another question, when you project images on the texture surface, did you use the 256*256 images from the benchmark dataset? Cause when I use these train and test images, the camera captured images are all with mosaic of the projector's DMD lens. I did almost the same system configuration with the paper's.

[BingyaoHuang]

did you use the 256*256 images from the benchmark dataset?

Yes, my projector was set to 800x600, I displayed 256x256 image and made it full screen, so the display will automatically resize the 256x256 window to 600x600 (keep aspect ratio).

the camera captured images are all with mosaic of the projector's DMD lens

Yes, I can see the mosaic in your camera-captured images. Can you see the mosaic with your naked eyes? If so, maybe the camera is too close to the surface? What is your camera resolution?

[whattoshow]

Yes, I can see the mosaic in your camera-captured images. Can you see the mosaic with your bear eyes? If so, maybe the camera is too close to the surface? What is your camera resolution?

My camera resolution is 8688x5792 and I used a telephoto lens, the projector resolution is 1024*768, this might be the reason why the captured images with mosaic, I'll change the camera setting and try to coadjust the camera-projector parameters through computer tonight.
Thanks for your help, I greatly appreciate~And I'll try my best to do something helpful to improve the compensation effect.

[BingyaoHuang]

@whattoshow The camera resolution is too high, try a lower resolution e.g., 640x480.