Audience: Intermediate
1. Introduction
In the modern enterprise landscape, the ability to process unstructured data i.e. information that does not reside in a traditional database, such as text within documents, images, and hand-written notes is a critical operational requirement. AppSheet addresses this need through AI Tasks, a suite of native automation components powered by Google’s Gemini large language models (LLMs).
Unlike standard automation rules that rely on static conditions, AI Tasks introduce a cognitive layer to the AppSheet engine. This allows applications to interpret the semantic meaning of text and the visual context of images. By incorporating these tasks into an automation workflow, creators can transform raw, unorganized inputs into structured, actionable data points. This functionality significantly reduces the latency associated with manual data entry and analysis.
Note: This feature is available to AppSheet Enterprise Plus accounts only.
2. The Four Primary AI Tasks in AppSheet
AppSheet provides four specialized AI Task types designed to handle different data processing requirements. Each task is configured within the Automation Bot editor and can be triggered by data changes or scheduled events.
1. Extract
The Extract task focuses on Entity Recognition from unstructured sources. It is used to pull specific, singular data points from a document or image and map them directly into individual columns in a single row.
-
Functionality: It extracts information from an uploaded image (such as a photo), a file (PDF), or text (such as a description, as Text or LongText) using AI.
-
Use Case Example: Identify a car’s make, model, year, color, and more from a car photo.
2. Categorize
The Categorize task is designed for classification and sentiment analysis. It evaluates an input (usually a block of text) and assigns it to a predefined category based on the options available in an Enum or Ref column.
-
Functionality: The AI interprets the intent, tone, or subject matter of the input and matches it to the most logical label provided in your app’s schema.
-
Use Case Example: Analyze the description in employee expense submissions and automatically categorize them by type (Travel, Meals, Software, or Training).
3. Extract Rows
While the standard Extract task creates a single record, Extract Rows is designed for Tabular Data Extraction. It identifies repeating patterns or line items within a document and generates multiple child records in a related table.
-
Functionality: It understands the structure of lists and tables within a file, allowing it to “shred” a document into its constituent parts.
-
Use Case Example: Upload PDF file and extract items in the receipt that are above fifty dollars and save them to the expenses table.
4. Summarize
The Summarize task is an information-density tool. It processes large volumes of text from multiple columns or long-form notes and condenses them into a concise, actionable narrative.
-
Functionality: It utilizes natural language generation to highlight the most relevant points of a text block, following any specific formatting instructions provided by the creator.
-
Use Case Example: Summarize notes from a meeting to review the main highlights.
3. Understanding the Workflow
AI Tasks live within AppSheet Automation. The general logic follows this flow:
-
Event: A trigger (e.g., a user uploads a PDF/Image or adds a new row).
-
AI Task (The Step): The bot sends the data to Gemini for analysis.
-
Action: The bot saves the AI’s output back into your table or uses it to trigger a notification.
The below gives detailed implementation steps for the Extract AI tasks using the Car identification with Gemini application.
4. Implementation Steps
Step 1: Prepare Your Data Table
Before creating the bot, ensure your table has “Input” columns and “Output” columns.
- Input Column: Should be type File, Image, Text, or LongText.
- Output Column: Should be Text or LongText for summaries/extractions, or Enum/Ref for categorization.
Step 2: Create the Automation Bot
The “Bot” is the brain of your automation; it listens for an event and triggers a task.
-
Log in to the AppSheet Editor.
-
In the left-hand navigation menu, click on the Automation icon.
-
Click the + button next to “Bots.”
- Select Create a new bot and give it a descriptive name (e.g., “When new Car record added”) or select from the Suggestions provided.
2. Configure the Event
The Event defines when the Extract AI task (Extract car details) will be triggered.
- Click Configure event.
- Add the Event Name and click on the Create a new event.
- Select an Data change Type (e.g., “Adds only” if you want to trigger the Extract AI task when a new car image is added).
- Select the Table that will trigger the Extract AI task.
- (Optional) Add a Condition if the Extract AI task will be triggered only when the Image column is not blank (e.g., ISNOTBLANK([Image])).
Step 3: Configure the AI Task
- Inside Run this process, click + Add a step.
- Add a Step Name and click on “Create a new step”.
- In the Task Type, select AI Task.
-
Choose your AI Task Type from the dropdown**:**
-
Categorize: Select the column to evaluate and the Enum/Ref column for the result.
-
Extract: Select the file/image source and the specific columns you want Gemini to fill.
-
Summarize: Select the source text columns.
-
Step 4: Configure Extraction Mapping
This is the technical core where you link the image source to your data columns:
- Input: Select the [Image] column as the data source.
-
Outputs: Map the output columns you prepared in Step 1. For example:
-
Set Make to be extracted.
-
Set Model to be extracted.
-
Set Year to be extracted.
-
Under Additional instructions, tell Gemini exactly what you want.
-
Example for Extraction: “From this receipt, find the total amount and the vendor name. If the vendor name is unclear, set it to ‘Unknown’.”
-
Example for Summarization: “Summarize these meeting notes into three bullet points focusing on action items.”
Step 5: Map the Output
You have two choices for where the AI’s answer goes:
- Save to table: Directly updates a row in your data source.
- Return value: Keeps the result in the bot’s memory to be used in a later step (like an email body).
Step 6: Save
- Save your application.
Output Preview:
For more information, Refer to the Support article or use the Car identification with Gemini sample application.