Imagen3 material inpaint

Google Cloud Build with AI Custom ML & MLOps
1

Hi,

Currently i’m playing with the python genai package for Imagen3 image editing.
Although I can apply prompts to masked areas, which looks good, I’m unable to add reference images for the material to inpaint.

For instance, I have an old leather material image:

subject_reference_image = SubjectReferenceImage(
    reference_id=3,
    reference_image=Image(image_bytes=material_bytes),
    config= SubjectReferenceConfig(
        subject_type="SUBJECT_TYPE_DEFAULT",
        subject_description="high detailed leather",
    )
)

And have a prompt:

high detailed leather on a green backpack[1] on the backpack cover

whereby the green backpack is a raw reference image.

I think I’m doing something wrong or don’t understand the documentation correctly.
Any help would be appreciated.

Thanks in advance, kind regards
Martijn

2

Hi Schouwemn,

Welcome to the Google Cloud Community!

You’re on the right track by using SubjectReferenceImage for the leather material. The issue might be how the model uses the reference image with the prompt. Even though you’re providing the leather image and calling it “high detailed leather,” the model might not be directly using the texture from that image. It’s more likely using the description (“high detailed leather”) as a guide, rather than copying the texture from the image. To fix this, you can try being more specific about how you want the leather to be applied. The more descriptive you are, the better the chances of the model understanding your intent.

You can refer to this documentation for additional insights and examples:

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.