Imagen's result is pure randomness

Google Cloud Build with AI Generative AI & Foundational Models
, , ,
1

Hello!

I need to replace the person in the scene with another model, leaving everything else (clothes, stage, lighting, etc.) unchanged.

So, I have a prompt like this in Imagen:

prompt = (
    "Replace the person in [1] with the person from [2], keeping the same pose, "
    "clothes, colors, lighting, and studio background. Produce a realistic photo."
)

data = {
    "instances": [
        {
            "prompt": prompt,
            "referenceImages": [
                {
                    "referenceType": "REFERENCE_TYPE_RAW",
                    "referenceId": 1,
                    "referenceImage": {"bytesBase64Encoded": scene_b64},
                },
                {
                    "referenceType": "REFERENCE_TYPE_SUBJECT",
                    "referenceId": 2,
                    "referenceImage": {"bytesBase64Encoded": face_b64},
                    "subjectImageConfig": {
                        "subjectDescription": "a person with natural face",
                        "subjectType": "SUBJECT_TYPE_PERSON",
                    },
                },
            ],
        }
    ],
    "parameters": {
        "sampleCount": 2,
        "seed": 2,
        "language": "en",
        "negativePrompt": (
            "low quality, distorted face, artifacts, blurry, unnatural lighting"
        ),
    },
}

Source files:

Scene and face to replace

image
image1920Ă—1169 98.6 KB

I got two results:

First

1_0
1_0896Ă—1280 251 KB

Second

1_1
1_1896Ă—1280 137 KB

So, the second one is what I need, and the first one is not what I need at all. And this happens all the time. How can I fix this so that I don’t have to rely on luck? I feel like this is some kind of casino. Thank you!

Another example:

Input

(I can’t add more then 3 images, so i uploaded it)

https://i.ibb.co/Jj2WkTLj/2025-11-02-183322.png

Result

https://i.ibb.co/vx6KbmRr/2025-11-02-183201.png

Here it changed the colors and hair in the first try.

1 Like
2

Your approach looks good overall — the structure and reference setup are correct. However, Imagen doesn’t yet fully support subject replacement with preserved pose and clothing in a single prompt. You may need to first mask the person in image [1] (using an image edit or inpainting API) and then insert the new person from [2] as the subject reference.

Also, make sure the "referenceType": "REFERENCE_TYPE_SUBJECT" section includes consistent subject metadata (e.g., "subjectInstanceLabel" if required). Try simplifying the prompt to emphasize “replace the person keeping the same pose and outfit”, then refine lighting and background preservation through parameters.

1 Like
3

Hi! Thanks a lot for your reply.
But i want to clarify: when you say I should “mask the person in image [1] (using an image edit or inpainting API)”, do you mean I should first create a black-and-white mask that covers only the person (excluding the clothes), and then pass that mask as something like "REFERENCE_TYPE_MASK" along with the image [1]?

Thanks again!

4

P.S. to my previous question to you
As far as I understand, there are two ways to use Imagen.

1. Edit: you can use a mask to select what to delete/replace, but only text, not images.

2. Customize: you can use other images as a reference, but there is no mask.

Is it correct?

5

P.P.S

In short, I need functionality similar to that of the weshop.ai website:

image
image1281Ă—1487 251 KB

image
image1321Ă—914 244 KB