We need to test ‘Conversational Analytics’ before exposing the ‘Conversations’ menu to end users:
I don’t want our end users to be writing natural language prompts before we have tested the accuracy, latency and cost, and documented how to write effective prompts etc.
I raised a support ticket to ask Google how to enabled Conversational Analytics for a subset of users (Developers), while restricting from end users (Viewers and Explorers). Meanwhile, we still want users to have access to other Gemini integrations that we have enabled e.g. the ‘Expression Assistant’.
The answer I got back from Google was basically that the only way to achieve this was to create a new model set which only the Developers had access to, and ensure that no Viewers or Explorers have access to any of the models in the model set.
Obviously the above set up would mean that we can’t actually test Conversational Analytics on any of our real-life models.
How are other people testing Conversational Analytics? I suppose I could create duplicate models files called test_gemini_<name_of_model>, but I am wondering if there is an easier workaround.
The issue is that if I enable ‘Conversational Analytics’ in the admin setting, then all users get access to the ‘Conversations’ option in the sidebar menu, and can chat with any Explores they have access to. I only want Developers to have access to the ‘Conversations’ option while we are testing it. If I remove the gemini_in_looker permission for end users then they cannot see the ‘Conversations’ option (desired behaviour), but they also cannot access any of the other Gemini integrations that we have enabled (undesired behaviour).
Update: I’ve tried the solution suggested by Google (creating a copy of an existing model, and creating a new role where model set = copied model + permission set = Conversational Analytics Agent Manager, and all other users have the parent permission gemini_in_look enabled, but the child permissions chat_with_explore, chat_with_agent, save_agents and admin_agents disabled) but this results in the same undesired behaviour i.e. if I enabled ‘Conversational Analytics’ everyone could access it.
Google have confirmed there is no workaround. I managed to at least remove the ability for user to chat with Explores by following points 1 and 2:
The gemini_in_looker permission controls whether the Conversations menu is visible. Once this is enabled (which is required for features like Expression Assistant), the menu will appear for all users who have it.
The ability to actually use Conversational Analytics depends on a combination of:
access_data on the underlying models, and
specific permissions like chat_with_agent and/or chat_with_explore
Because your Explorers and Viewers need access_data on the same models for other workflows, restricting via model sets isn’t a visible option in this case.
To properly limit CA usage while keeping the menu visible, the key step is:
Ensure that for both Explorer and Viewer permission sets:
chat_with_agent = disabled
chat_with_explore = disabled
Additionally, it’s important to double-check that:
No users in these roles are also assigned any default roles (such as a Conversational Analytics user role) that might reintroduce these permissions indirectly.
With this setup:
Users will still see the Conversations menu (due to gemini_in_looker)
But they should not be able to initiate or interact with conversations, and will instead encounter permission errors if they try.
If the lack of granular access controls is bothering anyone else, please vote for my feature request HERE