Current limitations

[C0untFloyd]

Thanks for starting such an ambitious project and open-sourcing it!

I'm pretty sure you know that you're still missing a lot of the "magic" from the insightface swapper but I thought I should list the most apparent ones here anyway:

• Colors aren't blended well, the input affects the target way too much
• Black&white images come out as slightly colored
• Face shape isn't modified that much. Faces are usually much "rounder" and have less features with reswapper
• Likeness. It just isn't there most of the time
• Too smooth and blurry. This is very obvious because some face features seem to get smoothed away.

[diverswan9]

Garr - Connely

[somanchiu]

Thanks for reviewing this project!

I have used some grayscale images as the target images to train the model to mitigate the color issue and published a new checkpoint reswapper_256-1567500.pth.
Related commit: d9da7c6

[DJHanceNL]

@somanchiu

i coult not get the onnx convert to work. can you try converting this 256-1567500 to onnx for me?

[C0untFloyd]

Same problem here, shape sizes don't match to the previous one. Did you forget to include the new emap perhaps? Also there are of course warnings because of constant folding not supported on opset >= 10.

[somanchiu]

@Elise96nl @C0untFloyd I believe you both modified the model architecture in StyleTransferModel_128.py. There are no issues on my end with exporting ONNX using the ModelFormat.save_as_onnx_model function.

I have uploaded reswapper_256-1567500_originalInswapperClassCompatible.onnx to Hugging Face. It was exported with the following code:

ModelFormat.save_as_onnx_model("reswapper_256-1567500.pth", img_size=256)

[C0untFloyd]

Thank you, it's working now! 👍
Weird, I did change nothing in StyleTransfer or any other code. Oh but I think I forgot to set the emap parameter back to true after testing. I was using:
save_as_onnx_model("reswapper_256.pth", False, 256, True)
No surprise emap info was missing...
Initial quick testing seems to show a much better likeness conservation and b/w images work well. Good job!

Now if only that smoothness would disappear, are you training with the FFHQ_Celeb Dataset and/or some image enhancer?

[somanchiu]

To resolve the smoothness issue on the face, I have an idea. I assume that the ArcFace model embedding can represent the face texture. We can then train a model using the same architecture as InSwapper and using supervised learning from the main branch, but train with skin-smoothed Image A (target) + Image A embedding (source) = Image A (ground truth). As a result, the model should be able to add any face texture back to the face.
As an extension of this approach, maybe we just need to add another "data augmentation" to train.py?

[C0untFloyd]

Isn't the ArcFace embedding the "Eigenface"? I don't have enough knowledge to comment on your idea but it sounds reasonable.
Do you know these 2 projects?

https://github.com/ygtxr1997/ReliableSwap
(interesting because it especially cares about identity likeness)

https://github.com/ai-forever/ghost
(still a close runner-up to insightface in my opinion)

[poepoeng]

Isn't the ArcFace embedding the "Eigenface"? I don't have enough knowledge to comment on your idea but it sounds reasonable. Do you know these 2 projects?

https://github.com/ygtxr1997/ReliableSwap (interesting because it especially cares about identity likeness)

https://github.com/ai-forever/ghost (still a close runner-up to insightface in my opinion)

man , sorry out of topic, ur repo unleashed is disable ? i hope it will comeback, i still have issue on 4.4.1 .

[C0untFloyd]

man , sorry out of topic, ur repo unleashed is disable ? i hope it will comeback, i still have issue on 4.4.1 .

It took me by surprise too, there was no warning whatsoever. I guess I got reported because other face swapping tools are used to earn money, where my stuff always was and will be free...

[somanchiu]

man , sorry out of topic, ur repo unleashed is disable ? i hope it will comeback, i still have issue on 4.4.1 .

It took me by surprise too, there was no warning whatsoever. I guess I got reported because other face swapping tools are used to earn money, where my stuff always was and will be free...

The ReActor repository encountered the same issue.

https://www.reddit.com/r/comfyui/comments/1i3294w/comfyui_reactor_nodeby_gourieff_s_repo_has_been/

[DJHanceNL]

@C0untFloyd this is really bad. Disabling roop unleashed is where it starts. Big tech and gov telling us what we are allowed to do and achieve and what not.

So what now? are you in contact? can we continue somewhere else?

[awpeoith1254]

@C0untFloyd I too was quite disappointed to see the news and am curious if there is anything that can be done to help support your work if you elect to continue.

[etruscanmuffin909]

man , sorry out of topic, ur repo unleashed is disable ? i hope it will comeback, i still have issue on 4.4.1 .

It took me by surprise too, there was no warning whatsoever. I guess I got reported because other face swapping tools are used to earn money, where my stuff always was and will be free...

So sad about this and I'm desperate to find an up to date install as my copy was lost the same day by terrible coincidence. :( Unscrupulous people instantly tried to profiteer off of this removal on reddit and other places.