For use case where Google project is created newly for Apigee hybrid Org and we understand that GCP project maps to Apigee Control Plane Org in 1:1 fashion, then in that situation, sill we have to create the Apigee Organization Admin role under IAM Roles ?
Having such role will enable can enable some one to update Apigee Control Plane manually or accidently , without using IaC (which can lead us out of sync) hence
If situation is to use Terraform with build process to create Apigee Control plane, then IAM Roles should be read only permission in IAM and should not have "Apigee Organization Admin " to avoid updating without IaC (accidently?) or still we require such role Apigee Organization Admin in IAM for any reasons?
I understand your concern following best practices creating custom roles with only a subset of required permissions is definitely a sensible thing to do for a production hardened setup.
For the control plane you should definitely include CRUD permissions on the following resources for your custom role:
Read-only permissions won’t be enough for the terraform SA.
The guidance here would be to limit access to the service account to prevent other users to use its permissions.