Thanks for highlighting the issues of the tester with security filters. I’ll share that back with the team. My guess is that the tester is implemented outside of the bot and so it’s missing contextual information that the bot would have.
I’m not sure I have the data schema properly in my mind based on your description, so I’ll be curious to double-check that with you so I understand the use case better. I understand the general request though: you essentially want the formulas specified in the data schema (and in particular “Suggested Values”) be taken into account by the extraction AI task. You don’t want Gemini to have to consider all the rows from a table when extracting for a REF column as they are not all relevant and you’ve already done the work of filtering them out!
We don’t have that set up today. The formulas from the data schema are generally not passed to Gemini and the data itself is not pre-processed with these formulas before sending them to the AI task.
I’m glad you’re sharing this though. One of our concerns with allowing to pass any number of columns from a child table was the amount of data passed to Gemini. If your child table has thousands and thousands of records and we let you select a lot of columns, that quickly can become problematic. We limited that by mapping to the “Label” column. For other considerations tied to your app, probably to facilitate ease of use, you implemented the “Suggested Values” to filter out unnecessary (or invalid) records which would also limit the amount data we would pass.
I think what you are asking makes sense and it might be ok w.r.t data volume. We started though with the simplest approach which honestly is to not have to preprocess all the info with the various expressions in the data schema.