working with real-world images and complex backgrounds

[abbhinavvenkat]

Hi,

I'm looking at training and testing GeoCode from real world images with complex backgrounds.

1. Do you have any suggestions on how I should go about creating the ground truth for this? What all modalities do I need to train this model?
2. For testing the your existing model (say for chairs) from real world images, I presume I have to convert the image to sketches. What do you recommend for this? Do I need anything else apart from sketches to perform this step?

Thanks in advance!

Regards,
Abbhinav

[ofekp]

If you want to test the existing model with real images, then converting the images to sketches is the best option I can think of right now. To generate the sketches you could explore classical algorithms or recent papers related to the subject, perhaps a combination of the two. If these fail to produce good results, I would also think of a segmentation model to extract the object and then convert the object to a sketch.
Given a dataset, I would start with an image encoder to see how it performs before exploring anything else.
Creating such a dataset could be tricky. To preserve the labels of the shapes as much as possible, I would probably go the way of generating scenes involving the objects as synthetic data. You will have to explore ways to bridge the domain gap if such a dataset will not have a good performance on real images.